What differentiates meta-analysis from other forms of review1


  • 1

    This research was funded in part by a contract from the U.S. Department of Agriculture, #53-3198-9-26. Requests for reprints should be sent to Thomas D. Cook, Department of Psychology, Northwestern University, Evanston, Illinois 60201.


An issue of friendly disagreement between Cooper and Arkin (1981) and Cook and Leviton (1980) involves the definition of meta-analysis. Cooper and Arkin favor a definition stressing the degree of quantification in review studies, while we claim that studies labeled as “meta-analysis” are differentiated from other reviews in terms of the aggregation of effect sizes or probability values across studies for the purpose of reaching numerical estimates of the average magnitude of a relationship (as preferred by Glass) or the probability of reaching the average probability by chance (as originally preferred by Rosenthal). This definitional issue is important because the major objection to past meta-analytic studies raised by Cook and Leviton involved the possibility that meta-analysis might be used uncritically by some persons to derive “bottom line” estimates that are biased because confounds biasing the estimate in one direction are more prevalent than confounds biasing it in the opposite direction. Defining meta-analysis as more quantification takes much of the steam out of our critique, but it also reduces the saliency of what has been unique about past efforts labeled as meta-analysis–the generation of a single numerical estimate of a relationship derived from a set of many related studies.