Performance of a proportion-based approach to meta-analytic moderator estimation: results from Monte Carlo simulations
Article first published online: 15 APR 2012
Copyright © 2012 John Wiley & Sons, Ltd.
Research Synthesis Methods
Volume 3, Issue 1, pages 11–29, March 2012
How to Cite
Aguirre-Urreta, M. I., Ellis, M. E. and Sun, W. (2012), Performance of a proportion-based approach to meta-analytic moderator estimation: results from Monte Carlo simulations. Res. Synth. Method, 3: 11–29. doi: 10.1002/jrsm.1038
- Issue published online: 22 MAY 2012
- Article first published online: 15 APR 2012
- Manuscript Accepted: 6 MAR 2012
- Manuscript Revised: 9 JAN 2012
- Manuscript Received: 28 AUG 2011
- group proportions;
This research investigates the performance of a proportion-based approach to meta-analytic moderator estimation through a series of Monte Carlo simulations. This approach is most useful when the moderating potential of a categorical variable has not been recognized in primary research and thus heterogeneous groups have been pooled together as a single sample. Alternative scenarios representing different distributions of group proportions are examined along with varying numbers of studies, subjects per study, and correlation combinations. Our results suggest that the approach is largely unbiased in its estimation of the magnitude of between-group differences and performs well with regard to statistical power and type I error. In particular, the average percentage bias of the estimated correlation for the reference group is positive and largely negligible, in the 0.5–1.8% range; the average percentage bias of the difference between correlations is also minimal, in the −0.1–1.2% range. Further analysis also suggests both biases decrease as the magnitude of the underlying difference increases, as the number of subjects in each simulated primary study increases, and as the number of simulated studies in each meta-analysis increases. The bias was most evident when the number of subjects and the number of studies were the smallest (80 and 36, respectively). A sensitivity analysis that examines its performance in scenarios down to 12 studies and 40 primary subjects is also included. This research is the first that thoroughly examines the adequacy of the proportion-based approach. Copyright © 2012 John Wiley & Sons, Ltd.