A Scaling Model for Estimating Time‐Series Party Positions from Texts
We would like to thank Kathleen Bawn, Ken Benoit, Jim DeNardo, Tim Groseclose, James Honaker, Thomas König, Jeff Lewis, Will Lowe, Burt Monroe, George Tsebelis, several anonymous reviewers, and participants at the UCLA Methods Workshop and the 2007 annual meeting of the Midwest Political Science Association for their comments and suggestions. In addition, we thank the Zentralarchiv für Empirische Sozialforschung at the University of Cologne, Germany, for providing us with the German party manifestos in electronic format. The order of authors' names reflects the principle of rotation. Both authors have contributed equally to all work.
Abstract
Recent advances in computational content analysis have provided scholars promising new ways for estimating party positions. However, existing text‐based methods face challenges in producing valid and reliable time‐series data. This article proposes a scaling algorithm called WORDFISH to estimate policy positions based on word frequencies in texts. The technique allows researchers to locate parties in one or multiple elections. We demonstrate the algorithm by estimating the positions of German political parties from 1990 to 2005 using word frequencies in party manifestos. The extracted positions reflect changes in the party system more accurately than existing time‐series estimates. In addition, the method allows researchers to examine which words are important for placing parties on the left and on the right. We find that words with strong political connotations are the best discriminators between parties. Finally, a series of robustness checks demonstrate that the estimated positions are insensitive to distributional assumptions and document selection.
Citing Literature
Number of times cited according to CrossRef: 273
- Baekkwan Park, Kevin Greene, Michael Colaresi, How to teach machines to read human rights reports and identify judgments at scale, Journal of Human Rights, 10.1080/14754835.2019.1671174, 19, 1, (99-116), (2020).
- Martin Gross, Michael Jankowski, Lokale Wahlprogramme. Ein blinder Fleck der deutschen Kommunalpolitikforschung?, Neue Koalitionen – alte Probleme, 10.1007/978-3-658-28452-7_6, (101-126), (2020).
- Carina I. Hausladen, Marcel H. Schubert, Elliott Ash, Text Classification of Ideological Direction in Judicial Opinions, International Review of Law and Economics, 10.1016/j.irle.2020.105903, (105903), (2020).
- Jonathan Schoots, Benjamin Rohr, Katalin Prajda, John F. Padgett, Conflict and revolt in the name of unity: Florentine factions in the Consulte e Pratiche on the cusp of the Ciompi Revolt, Poetics, 10.1016/j.poetic.2019.101386, (101386), (2020).
- A. Lupia, S. Soroka, A. Beatty, What does Congress want from the National Science Foundation? A content analysis of remarks from 1995 to 2018, Science Advances, 10.1126/sciadv.aaz6300, 6, 33, (eaaz6300), (2020).
- Andreas Schmitz, Jakob Horneber, , Angst in Kultur und Politik der Gegenwart, undefined, (205), (2020).
- Anselm Hager, Hanno Hilbig, Does Public Opinion Affect Political Speech?, American Journal of Political Science, 10.1111/ajps.12516, 64, 4, (921-937), (2020).
- Abdullah Aydogan, Party systems and ideological cleavages in the Middle East and North Africa, Party Politics, 10.1177/1354068819894299, (135406881989429), (2020).
- Nicole Kalaf-Hughes, Representation by the Minute: The Influence of Ethnicity, Partisanship and District Fit on Legislator One-Minute Floor Speeches, Congress & the Presidency, 10.1080/07343469.2019.1666437, (1-38), (2020).
- Maoz Rosenthal, Strategic agenda setting and Prime Ministers’ approval ratings: the heresthetic and rhetoric of political survival, British Politics, 10.1057/s41293-020-00137-5, (2020).
- Sophie Balech, Wordfish : un modèle pour quantifier les antagonismes des discours, Management & Data Science, 10.36863/mds.a.10081., (2020).
- Michael Courtney, Michael Breen, Iain McMenamin, Gemma McNulty, Automatic translation, context, and supervised learning in comparative politics, Journal of Information Technology & Politics, 10.1080/19331681.2020.1731245, (1-10), (2020).
- Dana Siobhan Atzpodien, Die SPD im Spagat zwischen Regierungsverantwortung und Überzeugung – Migration im parlamentarischen ParteienwettbewerbThe SPD in a balancing act between government responsibility and conviction—Migration in parliamentary party competition, Zeitschrift für Vergleichende Politikwissenschaft, 10.1007/s12286-020-00445-6, (2020).
- Michael Škvrňák, You’ll never rule alone: how football clubs and party membership affect coalition formation, Local Government Studies, 10.1080/03003930.2020.1787167, (1-19), (2020).
- Arthur Dyevre, The promise and pitfall of automated text-scaling techniques for the analysis of jurisprudential change, Artificial Intelligence and Law, 10.1007/s10506-020-09274-0, (2020).
- Tomoki Kaneko, Taka-aki Asano, Hirofumi Miwa, Estimating Ideal Points of Newspapers from Editorial Texts, The International Journal of Press/Politics, 10.1177/1940161220935058, (194016122093505), (2020).
- Franziska Pradel, Biased Representation of Politicians in Google and Wikipedia Search? The Joint Effect of Party Identity, Gender Identity and Elections, Political Communication, 10.1080/10584609.2020.1793846, (1-32), (2020).
- Rory Costello, Rally around the EU flag: Irish party positions on the EU in the wake of Brexit, Journal of Contemporary European Studies, 10.1080/14782804.2020.1825352, (1-17), (2020).
- Richard Hanania, The Humanitarian Turn at the UNSC: Explaining the development of international norms through machine learning algorithms, Journal of Peace Research, 10.1177/0022343320929728, (002234332092972), (2020).
- Ion Pagoaga Ibiricu, Party competition during secession crises in advanced democracies, Regional & Federal Studies, 10.1080/13597566.2020.1824994, (1-25), (2020).
- Marius Sältzer, Finding the bird’s wings: Dimensions of factional conflict on Twitter, Party Politics, 10.1177/1354068820957960, (135406882095796), (2020).
- Justin Bonest Phillips, V. K. G. Woodman, Party system fragmentation, social cleavages, and social media: New Zealand’s 2017 election campaign on Facebook, Australian Journal of Political Science, 10.1080/10361146.2020.1774506, (1-18), (2020).
- Kasper Welbers, Michaël Opgenhaffen, Chapter 4. News through a social media filter, Journalism and Translation in the Era of Convergence, 10.1075/btl.146.04wel, (85-105), (2019).
- Michael Jankowski, Sebastian H. Schneider, Markus Tepe, „…Deutschland eben“. Eine Analyse zur Interpretation des Begriffs „rechts“ durch Bundestagskandidaten auf Grundlage von Structural Topic Models, Identität - Identifikation - Ideologie, 10.1007/978-3-658-25604-3, (141-179), (2019).
- Richard Hanania, Explaining the Development of International Norms: The Humanitarian Turn at the United Nations Security Council, SSRN Electronic Journal, 10.2139/ssrn.3199611, (2019).
- Evelyne Hübscher, The impact of coalition parties on policy output – evidence from Germany, The Journal of Legislative Studies, 10.1080/13572334.2019.1570599, 25, 1, (88-118), (2019).
- Agnes Orban, Christine Trampusch, Fallstudien und Methoden, Handbuch Sozialpolitik, 10.1007/978-3-658-22803-3, (361-381), (2019).
- Carsten Jentsch, Eun Ryung Lee, Enno Mammen, Time-dependent Poisson reduced rank models for political text data analysis, Computational Statistics & Data Analysis, 10.1016/j.csda.2019.106813, (106813), (2019).
- Simon Fink, Jan-Eric Bartels, Winona Hagendorf, Niklas Klinger, Determinants of subnational party positions on electricity grid expansion in Germany: Economic over political interests, Utilities Policy, 10.1016/j.jup.2019.05.007, 58, (145-157), (2019).
- Federico Maria Ferrara, The Battle of Ideas on the Euro Crisis: Evidence from ECB Inter-Meeting Speeches, SSRN Electronic Journal, 10.2139/ssrn.3323844, (2019).
- Maya Berinzon, Ryan C. Briggs, Measuring and explaining formal institutional persistence in French West Africa, The Journal of Modern African Studies, 10.1017/S0022278X19000077, 57, 2, (183-202), (2019).
- Eric Linhart, Wahlsystem, Wahlen und Parteiensystem in Schleswig-Holstein, Politik und Regieren in Schleswig-Holstein, 10.1007/978-3-658-25748-4, (191-214), (2019).
- Bryan Schonfeld, Trading Places, Trading Platforms: The Geography of Realignment, SSRN Electronic Journal, 10.2139/ssrn.3357785, (2019).
- Jan Philipp Thomeczek, Michael Jankowski, André Krouwel, Die politische Landschaft zur Bundestagswahl 2017, Die Bundestagswahl 2017, 10.1007/978-3-658-25050-8, (267-291), (2019).
- Selim Jürgen Ergun, M. Fernanda Rivas, The effect of social roles, religiosity, and values on climate change concern: An empirical analysis for Turkey, Sustainable Development, 10.1002/sd.1939, 27, 4, (758-769), (2019).
- Jan Fuhse, Oscar Stuhler, Jan Riebling, John Levi Martin, Relating social and symbolic relations in quantitative text analysis. A study of parliamentary discourse in the Weimar Republic, Poetics, 10.1016/j.poetic.2019.04.004, (2019).
- Josemar Faustino, Hugo Barbosa, Eraldo Ribeiro, Ronaldo Menezes, A data-driven network approach for characterization of political parties’ ideology dynamics, Applied Network Science, 10.1007/s41109-019-0161-0, 4, 1, (2019).
- Arthur Dyevre, András Jakab, Foreword: Understanding Constitutional Reasoning, German Law Journal, 10.1017/S2071832200002133, 14, 8, (983-1015), (2019).
- Jonathan Sullivan, Eliyahu V. Sapir, Ma Ying-jeou's Presidential Discourse, Journal of Current Chinese Affairs, 10.1177/186810261204100303, 41, 3, (33-68), (2019).
- Martijn Schoonvelde, Gijs Schumacher, Bert N. Bakker, Friends with text as data benefits: Assessing and extending the use of automated text analysis in political science and political psychology, Journal of Social and Political Psychology, 10.5964/jspp.v7i1.964, 7, 1, (124-143), (2019).
- Niels D. Goet, Measuring Polarization with Text Analysis: Evidence from the UK House of Commons, 1811–2015, Political Analysis, 10.1017/pan.2019.2, (1-22), (2019).
- Bastiaan Bruinsma, Kostas Gemenis, Validating Wordscores: The Promises and Pitfalls of Computational Text Scaling, Communication Methods and Measures, 10.1080/19312458.2019.1594741, (1-16), (2019).
- Christopher Wratil, Sara B Hobolt, Public deliberations in the Council of the European Union: Introducing and validating DICEU, European Union Politics, 10.1177/1465116519839152, (146511651983915), (2019).
- Susumu Shikano, Dominic Nyhuis, The effect of incumbency on ideological and valence perceptions of parties in multilevel polities, Public Choice, 10.1007/s11127-019-00659-7, (2019).
- Sven-Oliver Proksch, Christopher Wratil, Jens Wäckerle, Testing the Validity of Automatic Speech Recognition for Political Text Analysis, Political Analysis, 10.1017/pan.2018.62, (1-21), (2019).
- Carsten Schwemmer, Oliver Wieczorek, The Methodological Divide of Sociology: Evidence from Two Decades of Journal Publications, Sociology, 10.1177/0038038519853146, (003803851985314), (2019).
- Federica Genovese, Sectors, Pollution, and Trade: How Industrial Interests Shape Domestic Positions on Global Climate Agreements, International Studies Quarterly, 10.1093/isq/sqz062, (2019).
- Nuria Font, Paolo Graziano, Myrto Tsakatika, Varieties of Inclusionary Populism? SYRIZA, Podemos and the Five Star Movement, Government and Opposition, 10.1017/gov.2019.17, (1-21), (2019).
- Dennis Christopher Spies, Simon T. Franzmann, Party Competition and Vote ChoiceParteienwettbewerb und Wahlverhalten, KZfSS Kölner Zeitschrift für Soziologie und Sozialpsychologie, 10.1007/s11577-019-00611-z, (2019).
- Abdullah Aydogan, Tayfun Tuna, A. Kadir Yildirim, Ideological Congruence and Social Media Text as Data, Representation, 10.1080/00344893.2019.1592012, (1-20), (2019).
- Pedro Feliú Ribeiro, Amanda Silva Mendes, Ideologia e Discurso: o posicionamento do Partido dos Trabalhadores na oposição e no governo., Revista Brasileira de Ciência Política, 10.1590/0103-335220192806, 28, (161-194), (2019).
- Kayla N. Jordan, Joanna Sterling, James W. Pennebaker, Ryan L. Boyd, Examining long-term trends in politics and culture through language of political leaders and cultural institutions, Proceedings of the National Academy of Sciences, 10.1073/pnas.1811987116, (201811987), (2019).
- Nadjim Fréchet, Justin Savoie, Yannick Dufresne, Analysis of Text-Analysis Syllabi: Building a Text-Analysis Syllabus Using Scaling, PS: Political Science & Politics, 10.1017/S1049096519001732, (1-6), (2019).
- Valeria Resta, The effect of electoral autocracy in Egypt's failed transition: a party politics perspective, Italian Political Science Review/Rivista Italiana di Scienza Politica, 10.1017/ipo.2019.6, (1-17), (2019).
- Martin Gross, Michael Jankowski, Dimensions of political conflict and party positions in multi-level democracies: evidence from the Local Manifesto Project, West European Politics, 10.1080/01402382.2019.1602816, (1-28), (2019).
- Yan de Souza Carreirão, O debate metodológico nos estudos de congruência política: uma revisão da literatura internacional, Revista de Sociologia e Política, 10.1590/1678-987319276902, 27, 69, (2019).
- Ludovic Rheault, Christopher Cochrane, Word Embeddings for the Analysis of Ideological Placement in Parliamentary Corpora, Political Analysis, 10.1017/pan.2019.26, (1-22), (2019).
- Ulrich Hartung, Why Parties Take Neutral Positions on Policy Issues: Insights from the German Christian Democratic Union, German Politics, 10.1080/09644008.2019.1667978, (1-27), (2019).
- TANJA ARTIGA GONZÁLEZ, GEORG D. GRANIC, Spatial Voting Meets Spatial Policy Positions: An Experimental Appraisal, American Political Science Review, 10.1017/S0003055419000492, (1-6), (2019).
- Kamil Marcinkiewicz, Mary Stegmaier, Speaking up to stay in parliament: the electoral importance of speeches and other parliamentary activities, The Journal of Legislative Studies, 10.1080/13572334.2019.1697048, (1-21), (2019).
- Nara Salles, Programs and Parties: Rethinking Electoral Competition Through Analysis of Brazilian 'Grotões', Brazilian Political Science Review, 10.1590/1981-3821201900020003, 13, 2, (2019).
- Anna Calissano, Simone Vantini, Marika Arena, Monitoring rare categories in sentiment and opinion analysis: a Milan mega event on Twitter platform, Statistical Methods & Applications, 10.1007/s10260-019-00504-7, (2019).
- Nara Salles, Fernando Guarnieri, Estratégia eleitoral nos municípios brasileiros: componente programático e alinhamento partidário, Revista de Sociologia e Política, 10.1590/1678-987319277201, 27, 72, (2019).
- Valerio Vignoli, Where are the doves? Explaining party support for military operations abroad in Italy, West European Politics, 10.1080/01402382.2019.1667164, (1-25), (2019).
- Federico Maria Ferrara, The battle of ideas on the euro crisis: evidence from ECB inter-meeting speeches, Journal of European Public Policy, 10.1080/13501763.2019.1670231, (1-24), (2019).
- Gary E Hollibaugh, The Use of Text as Data Methods in Public Administration: A Review and an Application to Agency Priorities, Journal of Public Administration Research and Theory, 10.1093/jopart/muy045, 29, 3, (474-490), (2018).
- Benjamin E. Bagozzi, Patrick T. Brandt, John R. Freeman, Jennifer S. Holmes, Alisha Kim, Agustin Palao Mendizabal, Carly Potz-Nielsen, The Prevalence and Severity of Underreporting Bias in Machine- and Human-Coded Data, Political Science Research and Methods, 10.1017/psrm.2018.11, 7, 3, (641-649), (2018).
- Sven‐Oliver Proksch, Will Lowe, Jens Wäckerle, Stuart Soroka, Multilingual Sentiment Analysis: A New Approach to Measuring Conflict in Legislative Speeches, Legislative Studies Quarterly, 10.1111/lsq.12218, 44, 1, (97-131), (2018).
- Loren Collingwood, Stephen Omar El‐Khatib, Benjamin Gonzalez O'Brien, Sustained Organizational Influence: American Legislative Exchange Council and the Diffusion of Anti‐Sanctuary Policy, Policy Studies Journal, 10.1111/psj.12284, 47, 3, (735-773), (2018).
- Caleb Pomeroy, The Quantitative Analysis of Space Policy: A Review of Current Methods and Future Directions, Space Policy, 10.1016/j.spacepol.2018.08.001, (2018).
- Matthew J. Denny, Arthur Spirling, Text Preprocessing For Unsupervised Learning: Why It Matters, When It Misleads, And What To Do About It, Political Analysis, 10.1017/pan.2017.44, 26, 2, (168-189), (2018).
- Sven-Oliver Proksch, Computergestützte Textanalysen, Handbuch Methoden der Politikwissenschaft, 10.1007/978-3-658-16937-4, (1-19), (2018).
- Andrew Peterson, Arthur Spirling, Classification Accuracy as a Substantive Quantity of Interest: Measuring Polarization in Westminster Systems, Political Analysis, 10.1017/pan.2017.39, 26, 1, (120-128), (2018).
- In Song Kim, John Londregan, Marc Ratkovic, Estimating Spatial Preferences from Votes and Text, Political Analysis, 10.1017/pan.2018.7, 26, 02, (210-229), (2018).
- Emanuele Lapponi, Martin G. Søyland, Erik Velldal, Stephan Oepen, The Talk of Norway: a richly annotated corpus of the Norwegian parliament, 1998–2016, Language Resources and Evaluation, 10.1007/s10579-018-9411-5, 52, 3, (873-893), (2018).
- Justin H. Kirkland, Jonathan B. Slapin, , Roll Call Rebels, 10.1017/9781108568883, (2018).
- Nicola Martocchia Diodati, Bruno Marino, Benedetta Carlotti, Prime Ministers unchained? Explaining Prime Minister Policy Autonomy in coalition governments, European Political Science Review, 10.1017/S1755773918000085, 10, 4, (515-536), (2018).
- Andrea Cerrato, Federico Maria Ferrara, Francesco Ruggieri, Why Does Import Competition Favor Republicans? Localized Trade Shocks, Voting Behavior, and Scapegoating in the U.S., SSRN Electronic Journal, 10.2139/ssrn.3147169, (2018).
- Andrea De Angelis, Alessandro Vecchiato, Panem Et Circenses: The Persuasive Effect of Soft News in Berlusconi's Italy, SSRN Electronic Journal, 10.2139/ssrn.3245572, (2018).
- Michio Umeda, Dyadic Representation in the Japanese Parliament: Adaptation by Candidates and Selection by Constituencies, SSRN Electronic Journal, 10.2139/ssrn.3320439, (2018).
- undefined 김정연, Measuring policy orientations through the automated text analysis, JOURNAL OF FUTURE POLITICS, 10.20973/jofp.2018.8.2.59, 8, 2, (59-91), (2018).
- Kenneth Benoit, Kohei Watanabe, Haiyan Wang, Paul Nulty, Adam Obeng, Stefan Müller, Akitaka Matsuo, quanteda: An R package for the quantitative analysis of textual data, Journal of Open Source Software, 10.21105/joss.00774, 3, 30, (774), (2018).
- Christopher Buss, Benedikt Bender, Positions of Interest Groups Towards Labour Market Reforms in Germany: A Novel Content Analysis of Press Releases, Welfare State Reforms Seen from Below, 10.1007/978-3-319-63652-8, (81-105), (2018).
- Snorre Sylvester Frid-Nielsen, Human rights or security? Positions on asylum in European Parliament speeches, European Union Politics, 10.1177/1465116518755954, 19, 2, (344-362), (2018).
- Eitan Tzelgov, Petrus Olander, Economic Decline and Extreme‐Right Electoral Threat: How District‐Level Factors Shape the Legislative Debate on Immigration, Legislative Studies Quarterly, 10.1111/lsq.12215, 43, 4, (649-679), (2018).
- Pascal D. König, Sebastian Jäckle, Voting Advice Applications and the Estimation of Party Positions – A Reliable Tool?, Swiss Political Science Review, 10.1111/spsr.12301, 24, 2, (187-203), (2018).
- Anna Storz, Julian Bernauer, Supply and Demand of Populism: A Quantitative Text Analysis of Cantonal SVP Manifestos, Swiss Political Science Review, 10.1111/spsr.12332, 24, 4, (525-544), (2018).
- Baekkwan Park, Michael Colaresi, Kevin Greene, Beyond a Bag of Words: Using PULSAR to Extract Judgments on Specific Human Rights at Scale, Peace Economics, Peace Science and Public Policy, 10.1515/peps-2018-0030, 24, 4, (2018).
- Dominic Nyhuis, Pascal König, Estimating the Conflict Dimensionality in the German Länder from Vote Advice Applications, 2014–2017, Statistics, Politics and Policy, 10.1515/spp-2018-0007, 0, 0, (2018).
- Stefan Ewert, Michael Jankowski, Jochen Müller, Politikpositionen von Regierungen im LändervergleichComparing governments’ policy positions across states, Zeitschrift für Vergleichende Politikwissenschaft, 10.1007/s12286-018-0405-x, (2018).
- Mickael Temporão, Corentin Vande Kerckhove, Clifton van der Linden, Yannick Dufresne, Julien M. Hendrickx, Ideological Scaling of Social Media Users: A Dynamic Lexicon Approach, Political Analysis, 10.1017/pan.2018.30, (1-17), (2018).
- Varsha Jain, Meetu Chawla, B.E. Ganesh, Christopher Pich, Exploring and consolidating the brand personality elements of the political leader, Spanish Journal of Marketing - ESIC, 10.1108/SJME-03-2018-0010, (2018).
- Brice D.L. Acree, Justin H. Gross, Noah A. Smith, Yanchuan Sim, Amber E. Boydstun, Etch-a-Sketching: Evaluating the Post-Primary Rhetorical Moderation Hypothesis, American Politics Research, 10.1177/1532673X18800017, (1532673X1880001), (2018).
- Nicole Baerg, Will Lowe, A textual Taylor rule: estimating central bank preferences combining topic and scaling methods, Political Science Research and Methods, 10.1017/psrm.2018.31, (1-17), (2018).
- Luigi Curini, Airo Hino, Atsushi Osaka, The Intensity of Government–Opposition Divide as Measured through Legislative Speeches and What We Can Learn from It: Analyses of Japanese Parliamentary Debates, 1953–2013, Government and Opposition, 10.1017/gov.2018.15, (1-18), (2018).
- Camila Feix Vidal, Polarização Partidária e Política Externa nos Estados Unidos (1936-2012), Revista Brasileira de Ciência Política, 10.1590/0103-335220182505, 25, (167-204), (2018).
- Ben Curran, Kyle Higham, Elisenda Ortiz, Demival Vasques Filho, Look who’s talking: Two-mode networks as representations of a topic model of New Zealand parliamentary speeches, PLOS ONE, 10.1371/journal.pone.0199072, 13, 6, (e0199072), (2018).
- Pascal D. König, Dominic Nyhuis, Assessing the applicability of vote advice applications for estimating party positions, Party Politics, 10.1177/1354068818790111, (135406881879011), (2018).
- WILLIAM HOBBS, NAZITA LAJEVARDI, Effects of Divisive Political Campaigns on the Day-to-Day Segregation of Arab and Muslim Americans, American Political Science Review, 10.1017/S0003055418000801, (1-7), (2018).
- Royce Carroll, Hiroki Kubo, Measuring and comparing party ideology and heterogeneity, Party Politics, 10.1177/1354068817710222, 25, 2, (245-256), (2017).
- See more




