A simulation study comparing properties of heterogeneity measures in meta-analyses
Article first published online: 21 SEP 2006
Copyright © 2006 John Wiley & Sons, Ltd.
Statistics in Medicine
Special Issue: Papers from the 26th Annual Conference of the International Society for Clinical Biostatistics
Volume 25, Issue 24, pages 4321–4333, 30 December 2006
How to Cite
Mittlböck, M. and Heinzl, H. (2006), A simulation study comparing properties of heterogeneity measures in meta-analyses. Statist. Med., 25: 4321–4333. doi: 10.1002/sim.2692
- Issue published online: 28 NOV 2006
- Article first published online: 21 SEP 2006
- Manuscript Received: 18 JUL 2006
- Manuscript Accepted: 18 JUL 2006
- measures of heterogeneity;
- between-study variance;
- within-study variance;
- fixed versus random effect model;
- total information
The assessment of heterogeneity or between-study variance is an important issue in meta-analysis. It determines the statistical methods to be used and the interpretation of the results. Tests of heterogeneity may be misleading either due to low power for sparse data or to the detection of irrelevant amounts of heterogeneity when many studies are involved. In the former case, notable heterogeneity may remain unconsidered and an unsuitable model may be chosen and the latter case may lead to unnecessary complex analyses strategies. Measures of heterogeneity are better suited to determine appropriate analyses strategies. We review two measures with different scaling and compare them with the heterogeneity test. Estimates of the within-study variance are discussed and a new total information measure is introduced. Various properties of the quantities in question are assessed by a simulation study.
Heterogeneity test and measures are not directly related to the amount of between-study variance but to the relative increase of variance due to heterogeneity. It is more favourable to base the within-study variance estimate on the squared weights of individual studies than on the sum of weights. A heterogeneity measure scaled to a fixed interval needs reference values for proper interpretation. A measure defined by the relation of between- to within-study variance has a more natural interpretation but no upper limit. Both measures are quantifications of the impact of heterogeneity on the meta-analysis result as both depend on the variance of the individual study effects and thus on the number of patients in the studies. Copyright © 2006 John Wiley & Sons, Ltd.