Ninety-nine days before the elections I utilized national- and district- or state-level information to forecast the 2008 U.S. presidential, House, and Senate elections (Klarner Reference Klarner2008). Overall, the presidential model performed very well, while the House and Senate models understated the extent of the Democratic wave.

The presidential forecasts were extremely accurate. The model predicted that Barack Obama would attain 53.0% of the two-party popular vote, and, as of November 11, 2008, he had received 53.5% (see Table 1), although the vote tallies are as of this writing still being updated hourly. The model also predicted that he would obtain 346 electoral votes, while in reality he obtained 365. The state and district forecasts of the three models were not published in PS (Klarner Reference Klarner2008) due to space constraints, but were posted on the Pollyvote Web site well before the election.Footnote ¹ The presidential model also did well at calling individual states, forecasting 48 out of 51 correctly, missing only Arkansas, Indiana, and West Virginia. On average, the state-level forecasts were 0.05% off, indicating a lack of bias in the model. The median absolute value of the state-forecast error was 3.3%, indicating that half of the time the state forecast was within 3.3% of the actual vote (the mean error was 4.2%). More importantly, states that were won by 5% of the vote or less saw only an average error of 1.02% (1.49% on average for those states won by 10% of the vote or less), which in particular explains why the model called states so accurately.

Table 1.

Forecasts of Democratic Margin of Vote or Gains Versus Reality

The House and Senate models understated the number of seats that the Democrats picked up, as did Klarner and Buchanans' (2006a; 2006b) models of the 2006 elections. The House model forecast that the Democrats would have a total of 247 seats in the House of Representatives after the 2008 election, implying a gain of 11 seats. Although as of December 4, some seats had not been called, the Democrats had at least 255 seats in the House after the election, and may have as many as 259, meaning the model underestimated Democratic gains by at least eight seats, and perhaps as many as 12. At the district level, the model called 351 out of 375 (93.6%) races correctly (many were uncontested, bringing the total down from 435). The median absolute value of error between the forecasts and actual results was 2.92% (the mean was 3.87%), which appears to be a fairly accurate track record. What these data do not reveal, however, is that the model did poorly at calling races won by narrow margins. If races won by 10% or less are examined, the median absolute value of error goes up substantially to 6.34% (mean = 7.28).

This pattern of errors reveals something very interesting about how elections work. Figure 1 shows a scatterplot of the forecast Democratic percent of the two-party vote on the horizontal axis, and the actual vote on the vertical axis. It is evident that the model did a good job overall except in those districts in which it forecast a narrow Republican win (aside from a few errant elections in the southeast quadrant). Those are districts the Democrats often won. It can be argued that the forecasts are representations of how elections would go based on the normal causes of election outcomes unrelated to campaigns (incumbency, etc.). In an election year that favors one party, that party focuses its resources on the districts of the disadvantaged party that are closest to swinging, and these areas therefore appear more responsive to the national wave (in this case going toward the Democrats). Or perhaps it is because the voters there are more middle-of-the road and thus more subject to the national wave. Whatever the case, the scatterplot draws attention to a major shortcoming of district-level models. Aggregate models only forecast the net change in seats, and so if massive and consequential campaign activity occurs in these battleground districts it is implicitly modeled. District-level forecasting models will have to take this into account in the future to fully live up to their potential.

Figure 1.

Democratic Percent of the Two-Party Vote in House Districts: Forecasts versus Actual

The Senate model forecast that the Democrats would win 15 of this year's 35 races, a party gain of three to hold a total of 54 seats after the election (including Democratic-caucusing independents Bernie Sanders and Joseph Lieberman). The Democrats have picked up at least seven seats in the Senate and may pick up as many as eight total, depending on how the Minnesota election is resolved. State-level forecasts called 29 out of 34 elections correctly (85.3%), missing the Senate races in Alaska, New Hampshire, North Carolina, Oregon, and Virginia, a proportion that may worsen when the Minnesota race is resolved. The median of the absolute values of error in predictions for individual seats was 6.19% (mean = 6.55). Overall, the Senate model was not effective at forecasting the number of seats Democrats picked up or how the vote would go in individual states. As in 2006, the Senate model performed worse than the House model. ■