ACCURACY OF 2010 AND 2014 STATE LEGISLATIVE FORECASTS

This is the third consecutive midterm for which I have made state legislative forecasts. No other quantitatively based district level forecasts for state legislative elections have been made, at least publicly.

These models performed well in both 2010 and 2014, although both forecasts understated the extent of the Republican wave in both years. In 2010, the PS forecast made on July 22, 2010 predicted that the Republicans would pick up 11 chambers, while they actually picked up 21, with 82% of chambers being correctly called (see Klarner Reference Klarner2010; Reference Klarner2011). A later forecast, made on September 18 (Klarner Reference Klarner2010b) predicted the Republicans would gain 15 chambers, and called 89% of them correctly.

In 2014, the Republicans picked up 11 chambers, and the forecasts called 74 out of 86 chambers correctly (86.0%) (Klarner Reference Klarner2014). Humorously, an error in the simulation code identified after the election actually made the pre-election forecasts more accurate—the publicly made forecasts based on the error called 91.9% of chambers correctly. This is a noteworthy lesson about why not much should be inferred about the quality of a model based on how well it does for one prediction. The mean absolute value of error in percentage of seats held by the Democrats was 4.14%. A scatterplot of the forecast change in Democratic seats with the actual change in Democratic seats appears in figure 4, which depicts the unpublicized and less accurate forecasts.

Figure 4

2014 Forecasts Actual vs. Predicted Democratic Seat Loss

PREDICTION MODEL

Aspects of the prediction models used in 2010 (Klarner Reference Klarner2010) that are the same as the current model are as follows. State legislative elections conducted between 1968 and the present were used to assess the impact of predictor variables on election outcomes at the district level. Only regularly scheduled partisan elections conducted in even-numbered years were included in the analysis.

Predictor variables appear at three levels: district-election year, state-election year, and nation-election year.

District predictors include lagged vote share, incumbency, whether a candidate held office in the other chamber of the legislature immediately before running, and whether a candidate held legislative office in the past, but not the immediate past. Other variables tracking how long incumbents have been in office were also included. Each independent variable’s lagged component also appears in the model. Controls for free-for-all multimember districts being under-contested and the level of contestation of the prior election are also included.

State level variables include a state midterm penalty (the party of the governor tends to lose seats when the governor is not up for reelection) while change in state real disposable income was omitted because tests indicated these did not bring down forecast error.

National level variables intended to capture the national partisan wave include presidential approval, a national midterm penalty variable, and change in real disposable income. The specific way these variables are coded is explained in Klarner (Reference Klarner2010).

“Drop-1 analysis” was utilized to assess whether including predictor variables reduced prediction error. In this context, “drop-1 analyses” means that the dependent variable (Democratic percentage of the vote) was turned to system missing for each election year in turn, for all state legislative elections in the country that year, coefficients were estimated using the remaining years, and forecast error in the omitted biennium was assessed.

When data were missing—most often lagged variables, missing because of redistricting—Stata’s missing data imputation algorithm was used. Drop-1 analyses indicated that prediction error goes down when missing data imputation is used, even for districts that do not have missing lagged values.

Beyond the shared attributes of the 2010 and 2018 models, numerous improvements have been made (see appendix). As explained in the appendix, these include interactions for the type of contest (such as a contest between the same winner and loser from last time), modeling the declining incumbency effect, and the addition of “supplemental” lagged variables, such as lagged values from the other chamber of the legislature when there are identical districts.

Perhaps the most noteworthy published finding about state legislative elections to date is the interaction between how much is spent on the state legislature and the electoral boost that incumbents get (Berry, Berkman, and Schneiderman Reference Berry, Berkman and Schneiderman2000). However, analyses indicated that including this interaction did not reduce prediction error, and so was removed from the model.

How the national wave is modeled has also been markedly altered. First, congressional vote intention has been added as a predictor variable, as the goal currently is to maximize forecast accuracy. Second, the four national level predictor variables have been reduced to just one variable.

Because the model presented here is a model of change, lagged independent variables for all variables are included, as was done in prior models.

A hierarchical linear model is used to estimate the impact of the predictor variables on election outcomes, and is reported in table 2. The same amount of error at higher levels is more damaging to chamber predictions, as district level predictions cancel out to an extent, so the hierarchical model allows the magnitude of these different sources of error to be computed. The findings of the model were largely consistent with theoretical expectations.

Table 2

Determinants of Democratic Vote Percentage in State Legislative Elections, 1968–2016

Dependent variables is the Democratic percent of the two-party vote in each contest. Uncontested elections excluded. The figures for the error terms above represent their standard deviations. The standard deviation of the district level error from the drop-1 analyses is the mean of those estimated standard deviations. The standard deviation of error at the district level is contingent on the type of contest (i.e., repeat challengers, etc.) and how much information is available regarding the prior election (i.e., whether redistricting or an uncontested election occurred, and other scenarios).

2018 VALUES

Candidates were collected from each state as filing deadlines passed. The last filing deadline passed on July 12, 2018, in New York, which is also a key state. Candidates were integrated into the State Legislative Election Returns database, and name matching was done to establish the identity of these candidates, after which incumbency, and other factors were also coded. When two or more candidates filed for one primary for one seat, it was assumed that incumbents would beat candidates from the other chamber, candidates from the other chamber would beat candidates with past legislative service, and candidates with past legislative service would beat candidates with no legislative service. As primaries have been conducted, 2018 incumbency scores have been altered appropriately based on primary outcomes.

SIMULATIONS

After point predictions were produced from the prediction model, simulations were conducted to see how uncertainty about each election would influence uncertainty about winning a particular chamber. The probability of ties in even seat chambers and veto-proof majorities are also displayed. For free-for-all multimember districts, an intermediate step where the number of seats a party wins based on the predicted percentage of votes that party is forecast to have in that iteration of the simulation is also conducted. This is because if the Democrats win 45% of the vote in a district with two seats, such as in the Arizona House, they may win either zero or one seat, depending on how the votes are distributed.

SUPPLEMENTARY MATERIAL

To view supplementary material for this article, please visit https://doi.org/10.1017/S1049096518001555