This article is dedicated to Professor Norbert Victor, former President of the ISCB, who passed away on April 18, 2011.
Special Issue Paper
Assessment of statistical significance and clinical relevance†
Article first published online: 28 SEP 2012
Copyright © 2012 John Wiley & Sons, Ltd.
Statistics in Medicine
Special Issue: Papers from the Joint EMA, ISBS and DR-IBS International Symposium on Biopharmaceutical Statistics: Bridging Drug Development from Research to Marketing
Volume 32, Issue 10, pages 1707–1719, 10 May 2013
How to Cite
Kieser, M., Friede, T. and Gondan, M. (2013), Assessment of statistical significance and clinical relevance. Statist. Med., 32: 1707–1719. doi: 10.1002/sim.5634
- Issue published online: 17 APR 2013
- Article first published online: 28 SEP 2012
- Manuscript Accepted: 13 AUG 2012
- Manuscript Received: 24 MAY 2011
- statistical significance;
- clinical relevance;
- responder analysis;
- effect size;
- relative effect;
- probabilistic index;
- sample size
In drug development, it is well accepted that a successful study will demonstrate not only a statistically significant result but also a clinically relevant effect size. Whereas standard hypothesis tests are used to demonstrate the former, it is less clear how the latter should be established. In the first part of this paper, we consider the responder analysis approach and study the performance of locally optimal rank tests when the outcome distribution is a mixture of responder and non-responder distributions. We find that these tests are quite sensitive to their planning assumptions and have therefore not really any advantage over standard tests such as the t-test and the Wilcoxon–Mann–Whitney test, which perform overall well and can be recommended for applications. In the second part, we present a new approach to the assessment of clinical relevance based on the so-called relative effect (or probabilistic index) and derive appropriate sample size formulae for the design of studies aiming at demonstrating both a statistically significant and clinically relevant effect. Referring to recent studies in multiple sclerosis, we discuss potential issues in the application of this approach. Copyright © 2012 John Wiley & Sons, Ltd.