Research Article

A Welch-type test for homogeneity of contrasts under heteroscedasticity with application to meta-analysis

  1. E. Kulinskaya1,*,
  2. M. B. Dollinger2,
  3. E. Knight3,
  4. H. Gao4

Article first published online: 8 NOV 2004

DOI: 10.1002/sim.1929

Issue

Statistics in Medicine

Statistics in Medicine

Volume 23, Issue 23, pages 3655–3670, 15 December 2004

Additional Information

How to Cite

Kulinskaya, E., Dollinger, M. B., Knight, E. and Gao, H. (2004), A Welch-type test for homogeneity of contrasts under heteroscedasticity with application to meta-analysis. Statist. Med., 23: 3655–3670. doi: 10.1002/sim.1929

Author Information

  1. 1

    Statistical Advisory Service, Faculty of Medicine, Imperial College, London, U.K.

  2. 2

    Department of Mathematics, Pacific Lutheran University, Tacoma, WA, U.S.A.

  3. 3

    Department of Paediatrics, University of Cambridge, Cambridge, U.K.

  4. 4

    Statistical Advisory Service, Faculty of Medicine, Imperial College, London, U.K.

*Statistical Advisory Service, Imperial College, Room G06, Sir Alexander Fleming Building, South Kensington Campus, London SW7 2AZ, U.K.

Publication History

  1. Issue published online: 8 NOV 2004
  2. Article first published online: 8 NOV 2004
  3. Manuscript Accepted: MAY 2004
  4. Manuscript Received: FEB 2004

Funded by

  • Directorate of Health and Social Care. Grant Numbers: HSR 1100/8, RCC 33039

SEARCH

SEARCH BY CITATION

ARTICLE TOOLS

Get PDF (118K)

Keywords:

  • weighted ANOVA;
  • weighted sum of squares;
  • Q statistic;
  • pooled variances;
  • Welch test;
  • power in homogeneity testing

Abstract

A common problem that arises in the meta-analysis of several studies, each with independent treatment and control groups, is to test for the homogeneity of effect sizes without the assumptions of equal variances of the treatment and the control groups and of equal variances among the separate studies. A commonly used test statistic, frequently denoted as Q, is the weighted sum of squares of the differences of the individual effect sizes from the mean effect size, with weights inversely proportional to the variances of the effect sizes. The primary contributions of this article are the presentation of improved and very accurate approximations to the distributions of the Q statistic when the effect size is a linear contrast such as the difference between the treatment and control means. Our improved approximation to the distribution of Q under the null hypothesis is based on a multiple of an F-distribution; its use yields a substantial reduction in the type I error rate of the homogeneity test. Our improved approximation to the distribution of Q under an alternative hypothesis is based on a shift of a chi-square distribution; its use allows for much greater accuracy in the computation of the power of the homogeneity test. These two improved approximate distributions are developed using the Welch methodology of approximating the moments of Q by the use of multivariate Taylor expansions. The quality of these approximations is studied by simulation.

A secondary contribution of this article is a study of how best to combine the variances of the treatment and control groups (needed for the calculation of weights in the Q statistic). Our conclusion, based on simulations, is that use of pooled variances can result in substantially erroneous conclusions. Copyright © 2004 John Wiley & Sons, Ltd.

Get PDF (118K)