Meta-analysis of binary data: which within study variance estimate to use?

Authors

  • Bei-Hung Chang,

    Corresponding author
    1. Center for Health Quality, Outcomes, and Economic Research, Bedford VA Medical Center, Boston University SPH, 200 Springs Road (Building 70), Bedford, MA 01730, U.S.A.
    • Center for Health Quality, Outcomes, and Economic Research, Bedford VA Medical Center, Boston University SPH, 200 Springs Road (Building 70), Bedford, MA 01730, U.S.A.
    Search for more papers by this author
  • Christine Waternaux,

    1. Biostatistics Division, New York State Psychiatric Institute, 1051 Riverside Drive, New York, NY 10032, U.S.A.
    Search for more papers by this author
  • Stuart Lipsitz

    1. Division of Biostatistics, Dana Farber Cancer Institute, 44 Binney Street, Boston, MA 02115, U.S.A.
    Search for more papers by this author

Abstract

We applied a mixed effects model to investigate between- and within-study variation in improvement rates of 180 schizophrenia outcome studies. The between-study variation was explained by the fixed study characteristics and an additional random study effect. Both rate difference and logit models were used. For a binary proportion outcome pi with sample size ni in the ith study, (i(1−i)ni)−1 is the usual estimate of the within-study variance σ2i in the logit model, where i is the sample mean of the binary outcome for subjects in study i. This estimate can be highly correlated with logit(i). We used (i(1−)ni)−1 as an alternative estimate of σ2i, where is the weighted mean of i's. We estimated regression coefficients (β) of the fixed effects and the variance (τ2) of the random study effect using a quasi-likelihood estimating equations approach. Using the schizophrenia meta-analysis data, we demonstrated how the choice of the estimate of σ2i affects the resulting estimates of β and τ2. We also conducted a simulation study to evaluate the performance of the two estimates of σ2i in different conditions, where the conditions vary by number of studies and study size. Using the schizophrenia meta-analysis data, the estimates of β and τ2 were quite different when different estimates of σ2i were used in the logit model. The simulation study showed that the estimates of β and τ2 were less biased, and the 95 per cent CI coverage was closer to 95 per cent when the estimate of σ2i was ((1−)ni)−1 rather than (i(1−p̂)ni)−1. Finally, we showed that a simple regression analysis is not appropriate unless τ2 is much larger than σ2i, or a robust variance is used. Copyright © 2001 John Wiley & Sons, Ltd.

Ancillary