A Statistical Method for Empirical Testing of Competing Theories

Authors


  • Kosuke Imai is Assistant Professor, Department of Politics, Princeton University, Corwin Hall 036, Princeton, NJ 08544 (kimai@princeton.edu, http://imai.princeton.edu). Dustin Tingley is Assistant Professor, Department of Government, Harvard University, 1737 Cambridge St., Cambridge, MA 02138 (dtingley@gov.harvard.edu, http://scholar.harvard.edu/dtingley).

    The replication code and data archive for this article are available at http://hdl.handle.net/1902.1/16378. We thank Mike Hiscox and Todd Allee for kindly sharing their data. Thanks to Will Bullock, Christina Davis, Marty Gilens, Michael Hiscox, Simon Jackman, Evan Lieberman, Helen Milner, Grigo Pop-Eleches, Brandon Stewart, Teppei Yamamoto, Carlos Velasco Rivera, Jaquilyn Waddell Boie, Robert Walker, and seminar participants at Harvard University, Princeton University, the University of California, Berkeley, and the University of Chicago, Harris School for helpful suggestions. We also thank the editor and the four anonymous reviewers for extensive comments that have significantly improved this article. Imai acknowledges the financial support from the National Science Foundation (SES–0918968).

Abstract

Empirical testing of competing theories lies at the heart of social science research. We demonstrate that a well-known class of statistical models, called finite mixture models, provides an effective way of rival theory testing. In the proposed framework, each observation is assumed to be generated either from a statistical model implied by one of the competing theories or more generally from a weighted combination of multiple statistical models under consideration. Researchers can then estimate the probability that a specific observation is consistent with each rival theory. By modeling this probability with covariates, one can also explore the conditions under which a particular theory applies.We discuss a principled way to identify a list of observations that are statistically significantly consistent with each theory and propose measures of the overall performance of each competing theory. We illustrate the relative advantages of our method over existing methods through empirical and simulation studies.

Ancillary