A Welch-type test for homogeneity of contrasts under heteroscedasticity with application to meta-analysis
Article first published online: 8 NOV 2004
Copyright © 2004 John Wiley & Sons, Ltd.
Statistics in Medicine
Volume 23, Issue 23, pages 3655–3670, 15 December 2004
How to Cite
Kulinskaya, E., Dollinger, M. B., Knight, E. and Gao, H. (2004), A Welch-type test for homogeneity of contrasts under heteroscedasticity with application to meta-analysis. Statist. Med., 23: 3655–3670. doi: 10.1002/sim.1929
- Issue published online: 8 NOV 2004
- Article first published online: 8 NOV 2004
- Manuscript Accepted: MAY 2004
- Manuscript Received: FEB 2004
- Directorate of Health and Social Care. Grant Numbers: HSR 1100/8, RCC 33039
- weighted ANOVA;
- weighted sum of squares;
- Q statistic;
- pooled variances;
- Welch test;
- power in homogeneity testing
A common problem that arises in the meta-analysis of several studies, each with independent treatment and control groups, is to test for the homogeneity of effect sizes without the assumptions of equal variances of the treatment and the control groups and of equal variances among the separate studies. A commonly used test statistic, frequently denoted as Q, is the weighted sum of squares of the differences of the individual effect sizes from the mean effect size, with weights inversely proportional to the variances of the effect sizes. The primary contributions of this article are the presentation of improved and very accurate approximations to the distributions of the Q statistic when the effect size is a linear contrast such as the difference between the treatment and control means. Our improved approximation to the distribution of Q under the null hypothesis is based on a multiple of an F-distribution; its use yields a substantial reduction in the type I error rate of the homogeneity test. Our improved approximation to the distribution of Q under an alternative hypothesis is based on a shift of a chi-square distribution; its use allows for much greater accuracy in the computation of the power of the homogeneity test. These two improved approximate distributions are developed using the Welch methodology of approximating the moments of Q by the use of multivariate Taylor expansions. The quality of these approximations is studied by simulation.
A secondary contribution of this article is a study of how best to combine the variances of the treatment and control groups (needed for the calculation of weights in the Q statistic). Our conclusion, based on simulations, is that use of pooled variances can result in substantially erroneous conclusions. Copyright © 2004 John Wiley & Sons, Ltd.