Election forecasting has become a hot ticket. Over the past 12 years or so, interest in election-related polls and forecasts has grown, and they now reach wide audiences. But whose interests are served by the popularization and commercialization of polling and forecasts? Before 2008, election forecasting was of interest primarily to political scientists, as a sort of biannual, wonky, pseudo-academic sideshow. Today, one might say forecasts are crumbling under the weight of their own success.
Today, one might say forecasts are crumbling under the weight of their own success.
What is the virtue of an election forecast? In this short commentary, I provide important historical context for the business of forecasting before detailing some downsides to the modern versions of it. I question the reliability of forecasting the behavior of an electorate inflicted with strong partisan polarization, affected by a pandemic, and evaluating a highly unconventional candidate. Even if forecasts are reliable, there is some reason to be concerned about their potential to affect voter turnout, provide outsized attention to the election horse race at the cost of more substantive matters, and contribute to a general anti-science sentiment that threatens democratic institutions. I throw hot water on some possible upsides and conclude with a short list of recommendations for anyone who publicly talks about election forecasts. There is not much virtue in forecasting, but since it is here to stay, we might as well figure out how to be as responsible about it as possible.
A Very Brief History of Election Forecasting
Today, forecasting is big business. Still, most of the academics engaged in forecasting are not monetizing their efforts. In the 1990s, election forecasting raised its game within political science by applying stronger theoretical frameworks—as was the broader trend across the entire discipline—to their models. Among the forecasters, one could sense a sort of healthy competition to produce a forecast that was both accurate and theory-driven, with bonus points given to those published or publicized well before the election. The forecasting panels at annual meetings, especially at the Midwest Political Science Association meetings at the Palmer House in Chicago every spring, and at the APSA national meetings, were some of the most highly anticipated and well attended of any political science panel presented in a year ending in an even number. And, of course, PS began publishing its forecasting issue with regularity more than 20 years ago. But in those early days, the audience for these matters was largely political scientists themselves. We were not particularly practiced, as a discipline, at communicating our findings to a general public audience, nor was there any consistent venue for doing so.
Ironically, it was not a political scientist who changed all this. When Nate Silver began Five Thirty Eight in 2008, the original audience was—again—mostly political scientists. Silver applied his Moneyball style of pragmatic statistical inference to the 2008 presidential election between Barack Obama and John McCain, and his approach and website became popularized. The commercial election forecast was born. His blog was picked up by the New York Times in 2010, and data journalism for politics boomed. Forecasting was not just for political science nerds anymore; now everyone could follow the high-stakes horse race of election forecasting. And it seems like everyone did.
But where does that leave political science? Are the original nerds who arguably started this industry relevant anymore? The popularization of election forecasting means more people want in on it. Back in 1992 PS published only four forecasts, and even in 2016 it published just nine. But at the time of writing it appears there may be well more than a dozen forecasts in PS for 2020, all with various theories and paradigms applied. Although the recidivism rate among forecasters is high, the expansion of voices publishing in this area is probably a good thing. But what is all this activity getting us?
One thing it has gotten us is a lot of flak—some deserved and some undeserved. While many public pollsters took heat for their 2016 forecasts, political science fared better than advertised because forecasting is more than just poll aggregating (usually). Looking at the 2016 forecasts in PS, only two of the nine predicted a Trump win, but nearly all of the forecasts were accurate. In other words, even the forecasts that predicted Hillary Clinton would win really predicted that she would win the majority of the popular vote, which she did. In this way, political science forecasting has an excellent track record in terms of predicting some election outcomes, but they may not be great at helping contextualize the political outcomes that occur as a result of the election.
The historical context is useful for assessing the virtue of forecasting. I begin by enumerating several downsides.
Downsides of Forecasting
Downside #1: Partisan Polarization Perverts the Fundamentals
If there is a conventional wisdom that has developed around forecasting the general election winner in presidential elections, in my view, it is the “Time for Change” model developed by Alan Abramowitz and its variants. That is, three fundamental features of a presidential election have correctly forecast every presidential election since 1988: party incumbency, status of the national economy, and presidential approval. These are the fundamental features of an electoral context that I teach undergraduate students. This symposium in PS includes many other valuable models, with a high degree of consistency among them.
In general, models rely on two categories of inputs: static features of the election cycle (e.g., “the fundamentals”), such as how long the incumbent party has been in the presidency, and variable features of the election cycle, such as incumbent approval ratings and economic statistics. When the predictors in the variable category have low variance, however, they operate more like fundamental features. In this way, most forecasters agree that day-to-day campaign events and news cycle highlights are less predictive of the outcome than are fundamentals. But this is where 2020 gets interesting and presents particular challenges to forecasting.
Unfortunately, two particularly salient features of the 2020 election season make forecasts both more appealing and less reliable: polarization and the pandemic. First, as politics has become nationalized (Hopkins Reference Hopkins2018) and partisanship is thought of as teamsmanship (Hetherington and Weiler Reference Hetherington and Weiler2018; Mason Reference Mason2018), predicting the outcome of a high-stakes election in a polarized political environment becomes of increasing interest to more people. Although a polarized electorate may have greater interest in forecasts, the polarization reduces the reliability of the measures in their models. For example, if we think of fundamentals like incumbent approval ratings or status of the economy, it turns out that Republicans and Democrats have very different evaluations of these factors (Pew Research Center Reference Center2020). Measuring the overall approval rating may provide an eagle’s-eye view of an incumbent’s popularity, but if partisans view the incumbent and their likelihood of supporting the incumbent party systematically differently, then it introduces a systematic bias for which the model does not account. Polarization therefore drives up interest in election forecasts and drives down their reliability.
Moreover, as a political scientist, I am uneasy about publicly recognizing the predictive nature of the state of the economy for understanding elections, because it tacitly suggests that voters justifiably assign credit or blame to presidents for economic conditions. Political science research shows that this sort of retrospective evaluation of candidates is a flawed means of holding elected officials accountable, largely because generic human cognitive biases (such as recency bias) cause us to overvalue recent or salient events over more consequential events (Achen and Bartels Reference Achen and Bartels2016). This suggests that the state of the economy is sometimes an irrational reason to credit or blame the president, but nevertheless, humans are flawed, and the factor remains predictive. A political scientist understands the difference between a predictive factor and a normative claim, but we should not expect everyone to.
In addition, the COVID-19 pandemic also drives up interest, and down reliability, of election forecasts. The pandemic has curtailed or eliminated professional and collegiate sports, and Americans may be extra hungry for a team to root for, a race to watch, or highlight reels to relive. The now immense and highly visible election forecasting industry fills that demand. Yet, even though demand may be up for election coverage, the pandemic itself messes with fundamental indicators included in forecasts. Statistics that capture the economic downturn, for example, may reflect the pandemic more than the president and therefore provide a tainted indicator for an election forecast. In this way, coronavirus infects the forecasts with measurement error, while stoking the public’s interest in forecasts’ findings. Not a great combination.
Finally, President Trump is an anomaly that creates challenges for forecasters using out-of-sample predictions. As a candidate in 2016 he was heavily discounted because of his lack of any experience in elected office and his scorched-earth campaign style that broke many norms of strategy and behavior in a national campaign. As the incumbent in 2020, Trump is less of an anomaly now than in 2016, because he is, well, the incumbent, but he is still an unconventional candidate in many respects. He defies democratic norms in nearly every public appearance by disrespecting political opponents, glorifying authoritarian practices, revering autocratic leaders, and undermining democratic institutions. His campaign style in 2020 so far is fairly similar to the antagonistic campaign he ran in 2016 that emphasized themes of misogyny and racism. For the first time in history, Americans have the opportunity to reelect an impeached president. Talk about your out-of-sample events.
In a world where the fundamental indicators are diseased, are they valid predictors of the election? If not, how can forecasters adjust their models without being arbitrary and introducing personal bias? The whole idea of fundamentals is that structural commonalities across time explain political outcomes that are governed by the Constitution and by fixed political contexts. If we discount the fundamentals because of anomalies in the cycle, then we are effectively admitting that fundamentals are a weak model. But that seems like an overstatement of reality.
In a world where the fundamental indicators are diseased, are they valid predictors of the election?
At the end of the day, forecasters who rely on variable factors of the election may overcorrect their models in arbitrary ways that reduce the value of their forecast; yet those who rely on fixed factors may insufficiently account for the odd features of this particular cycle. Damned if you do; damned if you don’t.
Downside #2: Forecasts May Affect Turnout
A study of the 2016 election showed that greater emphasis on forecasts in that cycle increased voters’ certainty about the election outcome and therefore depressed turnout (Westwood, Messing, and Lelkes Reference Westwood, Messing and Lelkes2020). Had 2016 seen fewer or paid less public attention to forecasts, the outcome of the election may have been different. In this way, forecasting itself is not unvirtuous; the business and aggrandized showiness of forecasting imposes costs on democratic institutions that may exceed their benefits to the general public, especially if an election is close.
Downside #3: Outsized Focus on the Election Horse Race
Although a polarized, partisan, and always-in-crisis political environment drives up interest in the election, forecasts contribute to a news environment that focuses on the horse race at the expense of more substantive election topics. To make matters worse, the pandemic likely increases this effect, as noted earlier. Political science research shows, however, that journalists rationally provide election horse race coverage, rather than covering candidate scandals or election issues, because the latter are simply more expensive to write about (Searles and Banda Reference Searles and Banda2019). In 2020, we should anticipate a louder, but lower-quality, election discourse. More forecasts contribute to that effect.
Downside #4: Forecasts Give a False Impression of Science and Certainty
When the public perceives that polling or forecasts are incorrect, it contributes to skepticism about science and media. When forecasts or reporting about forecasts overstates the level of confidence in an estimate or fail to communicate uncertainty associated with that estimate, it gives a false impression about what science is designed to do. Science is for explanation, not prediction. Increasingly, public distrust in science and media is contributing to democratic demise in the United States. I am not claiming that the forecasts are causing this, but how we talk about them matters.
The Upsides of Forecasting Are Endogenous
One might argue that the increased promotion of polling and forecasts generates interest in elections and voting. After all, it is not always bad to talk about the horse race. However, it is difficult to parse the relationship between electoral motivation and election reporting about things like forecasts. Presumably, they are strongly related. Forecasts might drive interest in voting, but it seems just as likely that interest in voting drives forecasters to do their thing. Interest in forecasting and voter mobilization may covary, but it is not at all clear that one can motivate more voters by doing more forecasting.
What Is a Responsible Political Scientist to Do?
The genie is out of the bottle. A biannual, nerdy political science sideshow is now a cottage industry. When forecasting was an exercise in statistical modeling intended to be interpreted by a trained audience, it had some virtue as a means of understanding political behavior and outcomes, in addition to entertainment value. But now that forecasting has become a monetized commodity, it imposes some real costs on democracy that undermine its value. Unfortunately, at this point, there appears to be little virtue in the exercise.
But, because there is no way to put the genie back in the bottle, the onus is on every political communicator to contextualize forecasts and attempt to limit their influence on election outcomes. We may not be able to stop people from seeing election forecasting as a salacious fortune-telling exercise, but we can improve the way we communicate about them. To that end, I urge our discipline to talk about forecasts in the following ways:
-
- Emphasize the precise parameters that a forecast estimates, and nothing more.
-
- Favor forecasts that predict vote share, rather than the probability of winning, because people are really bad at cognitively processing probabilities.
-
- Use data visualizations to emphasize uncertainty in any forecast.
-
- Whenever possible, provide transparency, clarity, and an honest context that allows readers to evaluate the limits of a forecast.
The public may still overinterpret some forecasts, but if we talk about them within these guidelines, at least political science will have remained loyal to its first principles as a social science.
