Professor and Chairman.
Critical interpretation of Cochran's Q test depends on power and prior assumptions about heterogeneity
Article first published online: 28 OCT 2010
Copyright © 2010 John Wiley & Sons, Ltd.
Research Synthesis Methods
Volume 1, Issue 2, pages 149–161, April/June 2010
How to Cite
Pereira, T. V., Patsopoulos,, N. A., Salanti, G. and Ioannidis, J. P. A. (2010), Critical interpretation of Cochran's Q test depends on power and prior assumptions about heterogeneity. Res. Synth. Method, 1: 149–161. doi: 10.1002/jrsm.13
- Issue published online: 25 NOV 2010
- Article first published online: 28 OCT 2010
- Manuscript Accepted: 12 SEP 2010
- Manuscript Revised: 26 AUG 2010
- Manuscript Received: 19 OCT 2009
- Q test;
- Bayes theorem;
- power analysis;
We describe how an appropriate interpretation of the Q-test depends on its power to detect a given typical amount of between-study variance (τ2) as well as prior beliefs on heterogeneity. We illustrate these concepts in an evaluation of 1011 meta-analyses of clinical trials with ⩾4 studies and binary outcomes. These concepts can be seen as an application of the Bayes theorem. Across the 1011 meta-analyses, power to detect typical heterogeneity was low in most situations. Thus, usually a non-significant Q test did not change perceptibly prior convictions on heterogeneity. Conversely, significant results for the Q test typically augmented considerably the probability of heterogeneity. The posterior probability of heterogeneity depends on what τ2 we want to detect. With the same approach, one may also estimate the posterior probability for the presence of heterogeneity that is large enough to annul statistically significant summary effects; that is half the average within-study variance of the combined studies; and that is able to change the summary effect estimate of the meta-analysis by 20%. The discussed analyses are exploratory, and may depend heavily on prior assumptions when power for the Q-test is low. Statistical heterogeneity in meta-analyses should be cautiously interpreted considering the power to detect a specific τ2 and prior assumptions about the presence of heterogeneity. Copyright © 2010 John Wiley & Sons, Ltd.