Detecting and adjusting for small-study effects in meta-analysis

Authors

  • Gerta Rücker,

    Corresponding author
    1. Institute of Medical Biometry and Medical Informatics, University Medical Center, Stefan-Meier-Strasse 26, D-79104 Freiburg, Germany
    • Phone: +49-761-203-6673, Fax: +49-761-203-6680
    Search for more papers by this author
  • James R. Carpenter,

    1. Institute of Medical Biometry and Medical Informatics, University Medical Center, Stefan-Meier-Strasse 26, D-79104 Freiburg, Germany
    2. Medical Statistics Unit, London School of Hygiene & Tropical Medicine, Keppel Street, London WC1E 7HT, UK
    Search for more papers by this author
  • Guido Schwarzer

    1. Institute of Medical Biometry and Medical Informatics, University Medical Center, Stefan-Meier-Strasse 26, D-79104 Freiburg, Germany
    Search for more papers by this author

Abstract

Publication bias and related types of small-study effects threaten the validity of systematic reviews. The existence of small-study effects has been demonstrated in empirical studies. Small-study effects are graphically diagnosed by inspection of the funnel plot. Though observed funnel plot asymmetry cannot be easily linked to a specific reason, tests based on funnel plot asymmetry have been proposed. Beyond a vast range of funnel plot tests, there exist several methods for adjusting treatment effect estimates for these biases. In this article, we consider the trim-and-fill method, the Copas selection model, and more recent regression-based approaches. The methods are exemplified using a meta-analysis from the literature and compared in a simulation study, based on binary response data. They are also applied to a large set of meta-analyses. Some fundamental differences between the approaches are discussed. An assumption common to the trim-and-fill method and the Copas selection model is that the small-study effect is caused by selection. The trim-and-fill method corresponds to an unknown implicit model generated by the symmetry assumption, whereas the Copas selection model is a parametric statistical model. However, it requires a sensitivity analysis. Regression-based approaches are easier to implement and not based on a specific selection model. Both simulations and applications suggest that in the presence of strong selection both the trim-and-fill method and the Copas selection model may not fully eliminate bias, while regression-based approaches seem to be a promising alternative.

Ancillary