DEVELOPING A STRUCTURE-X MODEL

To construct a Structure-X model, one initially forecasts with a strong structural model, then adjusts that forecast via application of expert judgement. This combination strategy should noticeably reduce the prediction error ensuing from the national structural model by itself, because it draws on local factors not captured in the usual, aggregated structural model (Graefe et al. Reference Graefe, Armstrong, Jones and Cuzán2014). In this way, it also addresses the question of omitted variables bias that the equation raises. Here we offer our structural theory of House elections, generating a forecast for the 2018 contest. Then, we correct that forecast according to the expert judgements available in Inside Elections (led by Nathan L. Gonzales, worthy successor to the Rothenberg Political Report).^{Footnote 1} If comparisons from past elections are any guide, this strategy—Structure + X, rather than Structure alone—should substantially reduce forecasting error.

As we have already observed, “the variables in the models must measure, at least by proxy, what we know for sure about voter decision-making” (Lewis-Beck and Tien Reference Lewis-Beck and Tien2008, 230). Fortunately, the electoral behavior literature commonly notes the conceptual centrality of the national economy and government popularity (Lewis-Beck and Stegmaier Reference Lewis-Beck and Stegmaier2013; Stegmaier and Lewis-Beck Reference Stegmaier, Lewis-Beck and Valelly2013). We utilize the following measures of such variables for the structural model of equation (1):

(1)$${\rm{House\ Seat\ Change\ = \ Presidential\ Approva}}{{\rm{l}}_{{\rm{t - 1}}}}{\rm{\ + \ Disposable\ Incom}}{{\rm{e}}_{{\rm{t - 1}}}}{\rm{\ + \ Mid - ter}}{{\rm{m}}_{\rm{t}}}$$

The ordinary least squares estimates for this equation, along with definitions and supporting statistics, appear below,

(2)$$\eqalign{ & {\rm{HS}} = \, - 44.83^* + \,0.82^* {\rm{P}}\,{\rm{ + \ 4}}{\rm{.91}}^{\rm{*}} {\rm{I}}\,{\rm{ - }}\,{\rm{28}}{\rm{.53}}^{\rm{*}} {\rm{M\ }} \cr & \,\,\left( {{\rm{ - 3}}{\rm{.44}}} \right) \left( {3.34} \right)\,\left( {2.92} \right) \left( { - 4.58} \right) \cr & R^2 = \,59,\,{\rm{adj}}.\,R^2 = \,.55,\,{\rm{RMSE}} = \,18.06,\,{\rm{D - W}} = \,1.87,\,{\rm{N}} = \,35 \cr} $$

where HS = presidential party seat change in the House of Representatives, I = change in real disposable income, for initial six months of the election year (from the Bureau of Economic Analysis’s National Income and Product Account Table 2.1: Personal Income and Its Disposition), P = June Gallup poll presidential popularity rating from Gallup’s Presidential Approval Center, M = midterm dummy (0 = presidential election, 1 = midterm election), figures in parentheses are t-scores, * = statistical significance beyond .05, R ² = coefficient of multiple determination, adj. R ² = adjusted coefficient of multiple determination, RMSE = root-mean-square error, D-W = Durbin Watson statistic, and N = the elections from 1948 to 2016.

To forecast ex ante the seat change for 2018, we plug in the appropriate independent variable values: I = 1.73 (December 2017 through June 2018), P = 42 (June 2018), and M = 1.

(3)$$\eqalign{ & {\rm{H}}{{\rm{S}}_{{\rm{2018}}}} = - 44.83 + \,0.82\,\left( {42} \right)\, + \,4.91\,\left( {1.73} \right)\, - 28.53\,\left( 1 \right) \cr & = \, - 30\,{\rm{Republican}}\,{\rm{seats}}. \cr}$$

Thus, the structural model, standing alone, forecasts a Republican seat loss great enough for the party to lose its majority in the chamber. How does the forecast change once expert judgements are considered?

THE REFERENDUM MODEL PLUS EXPERT JUDGMENT

Expert judges have long held a respectable place at the election forecasting table, even when they shun, as they usually do, the word, “forecasting” (Lewis-Beck and Stegmaier Reference Lewis-Beck and Stegmaier2014). The names of Charlie Cook, Stuart Rothenberg, and Larry Sabato, for example, regularly appear in election publications investigating conditions in congressional races. These analysts favor on-the-ground, intuitive methods rather than models or statistics. As Rothenberg (Reference Rothenberg2014) observed, “we use qualitative judgments and general rules of thumb to base our analysis. In other words, our process can’t really be replicated…. I use an ordinal scale of nine categories – Safe Democrat, Safe Republican, Democrat Favored, Republican Favored, Lean Democrat, Lean Republican, Toss-Up/Tilt Democrat, Toss-Up/Tilt Republican, Pure Toss-Up – to reflect my assessment of the relative vulnerability of seats.”

For 2014, we used Rothenberg’s (Reference Rothenberg2014) categories, what he labeled “seats in play” (including Favored, Lean, Toss-up/Tilt, Pure Toss-up), in order to separate safe seats from competitive seats. Then, to make the expert (X) prediction, we differenced Democratic seats in play from Republican seats in play. That is, we subtracted the presidential party number from the out-party number. To illustrate, in June 2014 Rothenberg identified 26 Republican seats in play, as compared to 24 Democratic seats in play. Therefore, X = (26-24) = + 2, our expert index of Democratic seat gains.

This expert index (+ 2) for the Democrats in 2014, taken alone, was clearly different from our 2014 of structural forecast (−31). How do we bridge the gap between these two divergent estimates? The Structure-X model simply joins the two numbers by averaging the two estimates. For 2014, that meant a June-based forecast of (−31 + 2)/2 = −15 net Democratic seat loss. As we now know, that Structure-X forecast did well, with an error of only two seats (i.e., actual seat loss = −13). Clearly, the combination of the two numbers worked better than the approaches individually.

The Structure-X strategy, which combines these two different approaches, appears empirically to work better than either approach taken alone. Why? It does so because of the underlying assumptions. First, it draws on statistical theory, which shows us the potential error reduction that can come from combining results (Graefe et al. 2014). In this case, each forecast contained errors that when combined, would tend to cancel out (i.e., as an analogy, imagine each forecast observes a different half of the “electoral elephant”; when joined the whole elephant is accurately observed). More specifically, the referendum model has the strength that comes from taking into account fundamental national political economic conditions and trends. However, it has the disadvantage of neglecting local, idiosyncratic forces, forces that might account for unexpected losses, more likely to be picked up by the expert judgments. Combining these two strategies, then, can be expected to reduce overall error.

Given that, the questions turn to the specific rules of combination. Our overarching assumption bases itself on the notion that seats in play serves as a proxy variable for seat races that are uncertain, with an incumbent party at risk. It is not to be taken literally, in the sense that one would not say all seats in play will be lost. But one could say, and we do, that the more seats in play, the more will be lost, as it is “a good proxy variable…strongly related to the unobserved variable of interest” (Clinton Reference Clinton, Lewis-Beck, Bryman and Liao2004, 878). Because we do not yet know how many of these designated 2018 seats in play will be lost, we must make an informed guess, one that we can apply across an election series. Here we draw on the forecasting evaluation criterion of parsimony, relying on the Principle of Ockham’s razor…. [and] variables based on strong theory” (Lewis-Beck Reference Lewis-Beck2005, 151).

For that purpose, recall that effectively we have a two-party system, where rule alternates between the Democratic and Republican parties. Further, historically the expected vote share—the “normal vote”—slightly favors a Democratic majority at the national level (Converse Reference Converse, Campbell, Converse, Miller and Stokes1966). However, that Democratic lead has been declining to the point where exit polls show “the difference in the proportions of actual voters who say they are Democrats and Republicans has been trivial since 1984” (Lewis-Beck et al. Reference Lewis-Beck, Jacoby, Norpoth and Weisberg2008, 132). This suggests a simple, working rule-of-thumb: suppose the outcomes of the uncertain seats are pretty much a coin toss, that is, they will break 50-50 at the ballot box, with about half going to the governing party. Of course, taken on its face, the assumption will not test out exactly right year in, year out. That is to say, there will be error—some years the incumbent will get somewhat more than half these seats, some years a bit less. Still, over the long-run, this rule-of-thumb, given its empirical validity as a proxy, permits us to track the changes (and eventually make adjustments as necessary).

Evidence favoring the empirical validity of this proxy comes from examination of how, in fact, such projected seats in play have actually turned out. Looking at the last three mid-term elections (2006, 2010, 2014), we find that 55% (101/182) of the unsafe seats in play flipped to the other party, a number quite close to our coin-toss rule. Could we make a still more accurate prediction of the number of seats flipped based on actual party turnover within each risk category? By definition of the categories, one can reasonably expect that there would be many more seats flipped from one party to the other in the pure toss-up category compared to the category where either the Democratic or Republican Party was favored, but this is not the case (see table 1). Indeed, there appears to be no clear pattern to seats flipped by category of the last three mid-term elections.^{Footnote 2} Thus, the 50-50 rule-of-thumb seems like a good rule to apply to the expert index of seats at play, as a surrogate for how many will actually be lost.

Table 1

Midterm Seats Flipped by Seats-in-Play Category

Indeed, adding this proxy variable to the compensating errors of the referendum model permits real forecasting gains, as our over time analysis shows, in table 2. First, we see that this 2014 accuracy was not a fluke. Looking at the six congressional elections from 2006 to 2016, we see the Structure-X forecasts cuts the absolute error a good deal, when compared to the structural model by itself. Take, as an example, the first contest in the series, the midterm of 2006. The structural model forecasted −33, while the expert index showed incumbent loss of −31. Averaging these gives a Structure-X forecast incumbent seat loss of −32, for an overall forecasting error of one seat. Making such a calculation for the subsequent elections, we observe (in the last column of table 1), the Structure-X absolute forecasting error averages only about seven seats, [i.e., (1 + 17 + 19 + 0 + 1.5 + 5.5)/6 = 7.3]. By way of contrast, the average absolute error from the structural model alone equals about 17 seats [i.e., (2 + 29 + 40 + 10 + 18 + 3)/6 = 17). Hence, we see that Structure-X lowers absolute forecasting error by well over one-half, when compared to the structural model acting alone. In sum, it would seem a promising method to consider for the upcoming 2018 contest.

Table 2

Structure-X Model House Forecast Performance

Where the Referendum model uses June Gallup approval, change in disposable income over the first six months of the election year, and a midterm dummy.

Rothenberg/Gonzales’s seats in play differential is calculated by subtracting the number of seats in play of the president’s party from the number of seats in play for the out-party (as reported in The Rothenberg Political Report: May 2006, July 2010, and June all other years until 2014, thereafter in June edition of Inside Elections with Nathan L. Gonzales).

THE STRUCTURE-X HOUSE FORECAST FOR 2018

In generating a Structure-X forecast for 2018 we employ the same procedure for the “expert” (X) calculation, using the total “seats in play” number drawn from Inside Elections with Nathan L. Gonzales (selecting the May, June, or July figure for the election year, depending on June availability). More precisely, we took the difference between the Democratic seats in play and the Republican seats in play (i.e., subtracted the president’s party number from the out-party number). For June 2018, Gonzales reports 68 Republican seats in play, versus 10 Democratic party seats in play. Thus, the X-number = (10 - 68) = −58. In other words, the expert index we use, taken literally, implies a pick-up of 58 House seats for the Democratic party this fall. Taken as is, this expert index (−58) clearly departs from our structural forecast (−30). To reconcile these differences, we propose a Structure-X model, combining the two methods into one simply by averaging the two estimates. In the case at hand, that means a June-based forecast of (−30 −58)/2 = −44 net Republican seat loss. If the forecast holds, that would represent a very large loss for the Republicans. Indeed, it would be close to twice the average seat loss (i.e., of 24 seats) over the electoral period 1950 to 2014.