Testing for Homogeneity in Meta-Analysis I. The One-Parameter Case: Standardized Mean Difference
Article first published online: 1 JUN 2010
© 2010, The International Biometric Society
Volume 67, Issue 1, pages 203–212, March 2011
How to Cite
Kulinskaya, E., Dollinger, M. B. and Bjørkestøl, K. (2011), Testing for Homogeneity in Meta-Analysis I. The One-Parameter Case: Standardized Mean Difference. Biometrics, 67: 203–212. doi: 10.1111/j.1541-0420.2010.01442.x
- Issue published online: 14 MAR 2011
- Article first published online: 1 JUN 2010
- Received August 2009. Revised January 2010. Accepted March 2010.
- Cohen’s d-statistic;
- Fractional degrees of freedom;
- Heterogeneity test;
- Q statistic;
- Weighted ANOVA;
- Weighted sum of squares
Summary Meta-analysis seeks to combine the results of several experiments in order to improve the accuracy of decisions. It is common to use a test for homogeneity to determine if the results of the several experiments are sufficiently similar to warrant their combination into an overall result. Cochran’s Q statistic is frequently used for this homogeneity test. It is often assumed that Q follows a chi-square distribution under the null hypothesis of homogeneity, but it has long been known that this asymptotic distribution for Q is not accurate for moderate sample sizes. Here, we present an expansion for the mean of Q under the null hypothesis that is valid when the effect and the weight for each study depend on a single parameter, but for which neither normality nor independence of the effect and weight estimators is needed. This expansion represents an order O(1/n) correction to the usual chi-square moment in the one-parameter case. We apply the result to the homogeneity test for meta-analyses in which the effects are measured by the standardized mean difference (Cohen’s d-statistic). In this situation, we recommend approximating the null distribution of Q by a chi-square distribution with fractional degrees of freedom that are estimated from the data using our expansion for the mean of Q. The resulting homogeneity test is substantially more accurate than the currently used test. We provide a program available at the Paper Information link at the Biometrics website http://www.biometrics.tibs.org for making the necessary calculations.