A Reassessment of “The Methods behind the Madness: Presidential Electoral College Strategies, 1988–1996”
Andrew Reeves is Ph.D. Candidate of government, Harvard University, Cambridge, MA 02138 (firstname.lastname@example.org).
Lanhee Chen is Ph.D. Candidate of government, Harvard University, Cambridge, MA 02138 (email@example.com).
Tiffany Nagano is undergraduate student of government, Harvard University, Cambridge, MA 02138 (firstname.lastname@example.org).
Daron Shaw (1999) offers evidence for the predictable and systematic effects of campaign strategy. This article has influenced other scholars who rely on its findings in support of key assumptions about the strategic behavior of presidential candidates (Damore 2002; Heppen 2001; Prior 2001; Stoker and Bowers 2002).
To conclude that candidates form campaign strategy based on predictable factors such as competitiveness, electoral vote share, and the cost of TV advertisements, Shaw (1999) claims to use “ordered probit” (1999, 906) instead of least-squares regression (LS) to avoid familiar problems with ordinal dependent variables. Then, to conclude that electoral college strategy is a strong exogenous predictor of campaign resource allocation, Shaw claims to use “two-stage least squares (2SLS)” (1999, 907) to avoid problems with endogeneity bias. Attempting to replicate Shaw's (1999) results, we discover that both analyses are actually LS regressions presented as if they were ordinal probit and 2SLS. When we perform ordinal probit and 2SLS analyses, all substantive findings in Shaw (1999) vanish.
In Reeves, Chen, and Nagano (2003), we compare the published tabular results in Shaw (1999), which are supposedly derived from 2SLS and ordered probit, with LS results. Here we offer a summary of these findings.
Shaw's (1999) analysis indicates that the formation of electoral college strategies is determined by competitiveness, electoral votes, the cost of television advertisements (TV Ad Cost), and interactions between competitiveness and electoral votes, as well as competitiveness and TV Ad Cost. The results are claimed to be statistically significant and substantively very important. The published “ordered probit” coefficients and standard errors for competitiveness, electoral votes, TV Ad Cost, dummy variables for 1992 and 1996, and the intercepts of the models, are identical to those from LS. The results presented in Shaw (1999) as “ordered probit” are in fact LS, a method that can, and in this case does, produce biased estimates of the model.
Shaw (1999) then claims to use “2SLS” to determine that electoral college strategy drives decisions about campaign resource allocation; however, the published results are almost identical to LS. Taking into account two apparent transposition errors, the coefficients and standard errors reported in Shaw (1999) for the quantities of interest are identical to LS estimators. Using LS, Reeves, Chen, and Nagano (2003) also precisely replicate the model's standard error and adjusted R2 for all four of the dependent variables that Shaw (1999) reports.
Both of the claimed methods in Shaw (1999) would be improvements over LS given the data, and we limit our criticism strictly to the published analyses although we acknowledge that others may have other methodological concerns with ordered probit and 2SLS regressions. We implement an ordered probit analysis and present the results in Table 1. We find that the LS results are indeed biased; electoral college strategy is not strongly predicted or explained by the factors identified by Shaw (1999). The article finds interactive effects between competitiveness and electoral votes in determining electoral college strategy; states that have a large number of electoral votes and are highly competitive will be targeted as more electorally important. An actual ordered probit analysis reveals a confidence interval so wide as to be uninformative—clearly rejecting the hypotheses in Shaw (1999). In implementing the ordered probit analysis, the substantive conclusions reached in Shaw (1999) are dashed; the model yields not one coefficient for a quantity of interest that reaches conventional levels of statistical significance. Electoral college strategy, as defined by Shaw (1999), using methods favored in the original article, is not shown by these data to be a strong function of competitiveness, electoral votes, or TV Ad Cost.
Table 1. Ordered Probit Regression: Factors Influencing Republican and Democratic Electoral Strategies, 1988–1996
|TV Ad Costb||.936||−1.962|
|Competitiveness × Electoral Votes||−.004||.001|
|Competitiveness × TV Ad Cost||−.028||.027|
A 2SLS analysis may be appropriate when an endogenous variable is present in a given equation, violating the exogeneity assumption of LS regression; as Shaw points out, “both strategy and the opposition's campaigning are endogenous” and are “dependen[t] on the error term” of the equation (1999, 907). The implementation of 2SLS as described in the text of Shaw (1999) is problematic. We are unable to ascertain the instrument for candidate strategy based on the description in the article or via communication with the author. The instrument for opponent's resource allocation is also problematic. If the implementation in Shaw (1999, 910) were to be followed, then the instrument in the first stage would be the dependent variable in the second stage, resulting in a correlation of 1 and an obviously biased instrument. We do not attempt to implement this method since its results would yield extremely biased estimators.
We consider competitiveness, electoral college vote share, and TV Ad Cost as instruments as a best alternative; however, this approach also comes with methodological shortcomings. We present this implementation of 2SLS in Table 2 only in an attempt to follow the procedures described in the text of Shaw (1999), which we are unable to do based on the text. Cyclical relationships between campaign resource allocation and the lack of a valid instrument prevent an appropriate analysis of the systematic effects of campaign strategy.
Table 2. 2SLS Regression: Electoral College Strategy's Influence on Republican and Democratic Campaign Resource Allocation, 1988–1996
|Residual Standard Error||1,547.286||2.320||1,204.580||2.09|
We are grateful to Shaw (2003), a paper written in response to Reeves, Chen, and Nagano (2003), for acknowledging all the points summarized in the first two sections of this paper. In addition, the only difference between the results in Tables 1 and 2, which demonstrate that the results in Shaw (1999) vanish, and those in the erratum by Shaw (2003), which seeks to support the conclusions in Shaw (1999), is new data. The analysis in Shaw (2003) is based on new data for competition, TV Ad Cost, and TV Ad Buys. These are the same variables used in Shaw (1999) but with different values. We are unable to assess the validity of the new measures of TV Ad Cost and TV Ad Buys. We find that the new data for competition are in error and that they do not match the sources claimed; moreover, these errors account for the different results.
In comparison to calculations based on data from CQ's Guide to U.S. Elections 4th Edition, Volume 1 (Moore, Preimesberger, and Tarr 2001, 682–86) the new formulation of competition in Shaw (2003) is frequently incorrect. States are portrayed as more competitive than they actually were, and with crucially larger errors in battleground states. This is not a matter of interpretation, it is a point that we verified in written correspondence with Daron Shaw. The new data are merely wrong. Since a key variable is the interaction between electoral vote size and TV Ad Cost, the data error in Shaw (2003) completely accounts for any difference between our findings.
While formal theorists three decades ago specified models with a candidate's resources fixed (Brams and Davis 1974), researchers today tend to believe that each candidate's strategy responds to the other, so that we observe not a perfect experiment with exogenous conditions but rather an equilibrium.1 We do not doubt the claim that candidates act strategically and that they target their resources where they will matter most, but neither the actual LS analyses nor the claimed ordinal probit and 2SLS analyses run in Shaw (1999) are reasonable ways to test this theory. Assuming one candidate's strategy stands frozen while the other adjusts to his heart's content is not an accurate description of American presidential politics. When we follow the methods as prescribed in Shaw (1999), we find that all substantive conclusions are unsupported by the data and methods. Future research should examine whether there are actually systematic aspects to campaign strategy formation and whether candidates stick to these strategies or instead whether the candidates strategically respond to each other in dynamic equilibrium.
We thank Daron Shaw for providing his data and generously responding to our inquiries. We also wish to thank Scott Ashworth, Jennifer Fitzgerald, Thomas Holbrook, Dan Hopkins, Kosuke Imai, Gary King, David Konisky, Olivia Lau, Sarah Love, Ryan Moore, Isaac Nakhimovsky, Jasjeet Sekhon, and Sarah Sled for their helpful comments. A replication archive of the data and methods used here can be found at http://www.people.fas.harvard.edu/~reeves.
Another possibility is that the electoral college strategy variable may simply be a bad measure of campaign strategy. Instead of a long-term, systematic statement of strategy, electoral college strategy may reflect a campaign's short-term expectations about opponent's resource allocation.