Reviewing the literature: A comparison of traditional methods with meta-analysis


  • 1

    Reprint requests should be sent to the first author, Department of Psychology, Northwestern University, Evanston, Ill. 60201. We would like to thank Drs. Albert Erlebacher, Kenneth Howard, and Daniel Romer for their assistance and suggestions. Partial support was received from the U.S. Department of Agriculture, Contract #53–3198-9-26.


Meta-analysis is the name given to a set of techniques for reviewing research in which the data from different studies are statistically combined. Meta-analysts have criticized the more traditional qualitative methods of review on three principal grounds: (1) that relevant information is ignored in favor of a simplistic box count of the number of studies in which a particular relationship is and is not statistically significant; (2) that the sample of studies for review often contains important biases; and (3) that box counts ignore statistical interactions. Our discussion suggests that these criticisms are not intrinsic to qualitative reviews, but rather represent poor practices by reviewers using traditional methods. Moreover, although meta-analysis has some advantages, it is not without its unique limitations. Our comparison of both methods is applied to the qualitative literature review of Zuckerman (1979) and the meta-analysis of Arkin, Cooper, and Kolditz (1980) which reached different conclusions about the “existence” of self-serving attributions in studies of interpersonal influence.