Get access

Meta-research: The art of getting it wrong


  • John P. A. Ioannidis

    Corresponding author
    1. Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece
    2. Stanford Prevention Research Center, Stanford University School of Medicine, Stanford, U.S.A.
    3. Department of Medicine, Tufts University School of Medicine, Boston, U.S.A.
    4. Center for Genetic Epidemiology and Modeling, ICRHPS, Tufts Medical Center, Boston, U.S.A.
    5. Genetics/Genomics, Tufts Clinical and Translational Science Institute, Botson, U.S.A.
    6. Department of Epidemiology, Harvard School of Public Health, Boston, U.S.A.
    • Stanford Prevention Research Center, Stanford University School of Medicine, Medical School Office Building, Room X306, 251 Campus Dr, Stanford, CA 94305, U.S.A.
    Search for more papers by this author

  • This paper is based on the 2010 Presidential Address in Cartagena, Spain, 2010.


Meta-analysis has major strengths, but sometimes it can often lead to wrong and misleading answers. In this SRSM presidential address, I discuss some case studies that exemplify these problems, including examples from meta-analyses of both clinical trials and observational associations. I also discuss issues of effect size estimation, bias (in particular significance-chasing biases), and credibility in meta-research. I examine the factors that affect the credibility of meta-analyses, including magnitude of effects, multiplicity of analyses, scale of data, flexibility of analyses, reporting, and conflicts of interest. Under the current circumstances, a survey of expert meta-analysts attending the SRSM meeting showed that most of them believe that the true effect is practically equally likely to lie within the 95% confidence interval of a meta-analysis or outside of it. Finally, I address the placement of meta-analysis in the wider current research agenda and make a plea for adoption of more prospective meta-designs. In many/most/all fields, all primary original research may be designed, executed, and interpreted as a prospective meta-analysis. Copyright © 2011 John Wiley & Sons, Ltd.