Earlier versions of this article were presented at the Robert Wood Johnson Health Policy Scholars annual meeting, the Society for Political Methodology Summer Conference, the annual meeting of the American Political Science Association, and seminars at the University of Michigan and Stanford University. This work has benefited from the comments of the participants at these presentations. Thanks to Martha Bailey, Elizabeth Bruch, Vincent Hutchings, Gary King, Skip Lupia, Walter Mebane, Gerardo Munck, Maisy Samuelson, Jas Sekhon, and Chris Zorn for helpful comments and conversations. Two anonymous reviewers also made helpful comments that improved the presentation of this work. Thanks to the Robert Wood Johnson Foundation and Stanford’s Institute for Research in the Social Sciences (IRiSS) for research support and to the Center for Political Studies and the Department of Health Management and Policy at the University of Michigan for their hospitality while this work was initially written. The data analysis uses the open-source R library anchors (†), which is available at the Comprehensive R Archive Network. The anchors library also contains the data and replication code for this study. I am solely responsible for any errors or omissions.
1540-5907/asset/bannerforeground.gif?v=1&s=e0afb5a0045e2170d7bf982f39b2e3d4a6d932cc)
Credible Comparisons Using Interpersonally Incomparable Data: Nonparametric Scales with Anchoring Vignettes
Article first published online: 25 JUN 2012
DOI: 10.1111/j.1540-5907.2012.00597.x
©2012, Midwest Political Science Association
Additional Information
How to Cite
Wand, J. (2013), Credible Comparisons Using Interpersonally Incomparable Data: Nonparametric Scales with Anchoring Vignettes. American Journal of Political Science, 57: 249–262. doi: 10.1111/j.1540-5907.2012.00597.x
Publication History
- Issue published online: 2 JAN 2013
- Article first published online: 25 JUN 2012
Comparisons of individuals based on their selections from an ordinal scale traditionally assume that all respondents interpret subjective scale categories in exactly the same way. Anchoring vignettes have been proposed as a method to replace this homogeneity assumption with individual-specific data about how each respondent uses the ordinal scale. However, improving interpersonal comparisons with anchoring vignettes also requires a new set of assumptions. In this article, I derive the assumptions needed to make credible nonparametric comparisons using anchoring vignettes, and propose a new nonparametric scale that does not assume homogeneity among respondents. I also provide methods for evaluating empirically whether a set of anchoring objects can produce credible nonparametric interpersonal comparisons. Two empirical studies illustrate the importance of accounting for differences in the use of ordinal scales by showing how our inferences about interpersonal comparisons may change as a function of the assumptions we accept.