The problem of natural funnel asymmetries: a simulation analysis of meta-analysis in macroeconomics
Article first published online: 12 SEP 2011
Copyright © 2011 John Wiley & Sons, Ltd.
Research Synthesis Methods
Volume 2, Issue 2, pages 84–102, June 2011
How to Cite
Callot, L. and Paldam, M. (2011), The problem of natural funnel asymmetries: a simulation analysis of meta-analysis in macroeconomics. Res. Synth. Method, 2: 84–102. doi: 10.1002/jrsm.39
- Issue published online: 29 SEP 2011
- Article first published online: 12 SEP 2011
- Manuscript Accepted: 5 JUL 2011
- Manuscript Revised: 27 JUN 2011
- Manuscript Received: 4 OCT 2010
- funnel asymmetry;
Effect sizes in macroeconomic are estimated by regressions on data published by statistical agencies. Funnel plots are a representation of the distribution of the resulting regression coefficients. They are normally much wider than predicted by the t-ratio of the coefficients and often asymmetric. The standard method of meta-analysts in economics assumes that the asymmetries are because of publication bias causing censoring and adjusts the average accordingly. The paper shows that some funnel asymmetries may be ‘natural’ so that they occur without censoring. We investigate such asymmetries by simulating funnels by pairs of data generating processes (DGPs) and estimating models (EMs), in which the EM has the problem that it disregards a property of the DGP. The problems are data dependency, structural breaks, non-normal residuals, non-linearity, and omitted variables. We show that some of these problems generate funnel asymmetries. When they do, the standard method often fails. Copyright © 2011 John Wiley & Sons, Ltd.