Get access

Critical interpretation of Cochran's Q test depends on power and prior assumptions about heterogeneity

Authors

  • Tiago V. Pereira,

    1. Heart Institute (InCor), Laboratory of Genetics and Molecular Cardiology, São Paulo, 05403-000, Brazil
    2. Department of Biochemistry and Molecular Biology, Federal University of São Paulo, São Paulo, 05403-000, Brazil
    3. Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece
    Search for more papers by this author
  • Nikolaos A. Patsopoulos,,

    1. Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece
    2. Division of Genetics, Department of Medicine, Brigham & Women's Hospital, Harvard Medical School, Boston 2115, USA
    Search for more papers by this author
  • Georgia Salanti,

    1. Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece
    Search for more papers by this author
  • John P. A. Ioannidis

    Corresponding author
    1. Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece
    2. Biomedical Research Institute, Foundation for Research and Technology – Hellas, Ioannina 45110, Greece
    3. Tufts Clinical and Translational Science Institute and Center for Genetic Epidemiology and Modeling, Institute for Clinical Research and Health Policy Studies, Tufts Medical Center and Department of Medicine, Tufts University School of Medicine, Boston, MA 02111, USA
    4. Stanford Prevention Research Center, Stanford University School of Medicine, Stanford, CA 94305, USA
    • Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece
    Search for more papers by this author
    • Professor and Chairman.


Abstract

We describe how an appropriate interpretation of the Q-test depends on its power to detect a given typical amount of between-study variance (τ2) as well as prior beliefs on heterogeneity. We illustrate these concepts in an evaluation of 1011 meta-analyses of clinical trials with ⩾4 studies and binary outcomes. These concepts can be seen as an application of the Bayes theorem. Across the 1011 meta-analyses, power to detect typical heterogeneity was low in most situations. Thus, usually a non-significant Q test did not change perceptibly prior convictions on heterogeneity. Conversely, significant results for the Q test typically augmented considerably the probability of heterogeneity. The posterior probability of heterogeneity depends on what τ2 we want to detect. With the same approach, one may also estimate the posterior probability for the presence of heterogeneity that is large enough to annul statistically significant summary effects; that is half the average within-study variance of the combined studies; and that is able to change the summary effect estimate of the meta-analysis by 20%. The discussed analyses are exploratory, and may depend heavily on prior assumptions when power for the Q-test is low. Statistical heterogeneity in meta-analyses should be cautiously interpreted considering the power to detect a specific τ2 and prior assumptions about the presence of heterogeneity. Copyright © 2010 John Wiley & Sons, Ltd.

Ancillary