This essay benefited immeasurably from the assistance and support of Ronald Mitchell and from suggestions by Ken Conca, Gary Goertz, Paul Hensel, Brett Asley Leeds, Craig Parsons, Priscilla Southwell, participants in the ‘‘Coding across the Discipline’’ (2006) conference directed by Stuart Shulman and the Shambaugh Conference, ‘‘Building Synergies’’ (2006), directed by Sara McLaughlin Mitchell. This essay is based in part upon work supported by the National Science Foundation under Grant No. 0318374 entitled ‘‘Analysis of the Effects of Environmental Treaties,’’ September 2003-August 2007 (Principal Investigator: Ronald Mitchell). Any opinions, findings, conclusions, or recommendations expressed in the material are those of the author and do not necessarily reflect the views of the National Science Foundation. A previous version of the piece was presented at the meeting of the International Studies Association in Chicago in February 2007.
1468-2486/asset/MISR_left.gif?v=1&s=68a8abc3a933df1f6d0692974b2bff372dbadc72)
1468-2486/asset/MISR_centre.gif?v=1&s=b6846b10f509657cc8f192e0e781e8869e22b93a)
1468-2486/asset/MISR_right.gif?v=1&s=dbf4823a6a3d8adfd53f319b7e99bdff105757e6)
Understanding Data Quality through Reliability: A Comparison of Data Reliability Assessment in Three International Relations Datasets†
Article first published online: 20 NOV 2007
DOI: 10.1111/j.1468-2486.2007.00698.x
Additional Information
How to Cite
Rothman, S. B. (2007), Understanding Data Quality through Reliability: A Comparison of Data Reliability Assessment in Three International Relations Datasets. International Studies Review, 9: 437–456. doi: 10.1111/j.1468-2486.2007.00698.x
- †
Publication History
- Issue published online: 20 NOV 2007
- Article first published online: 20 NOV 2007
- Abstract
- Article
- References
- Cited By
Although recent data creation efforts in international relations have begun to focus on issues of reliability and validity more explicitly than previously, current efforts still contain significant problems. This essay focuses on three recent data generation projects that study international relations (the ICOW, ATOP, and River Treaty datasets) and shows the successes and failures of each in assessing reliability when generating data from qualitative evidence. All three datasets attempt to generate reliable data, document the procedures used, and present indications of data reliability. However, their efforts face problems when assessing the reliability of their case selection variables, in the development of reliability indicators, and in the presentation of reliability statistics. In addition to evaluating these recent efforts to generate large-N databases, this essay clarifies the difference between generating data from qualitative and quantitative evidence, explains the importance of reliability when coding qualitative evidence, and provides ways to improve the assessment of the quality of one’s data.