Serious forecasting of presidential (and other) elections has been going on in political science for 40 years (Lewis-Beck and Stegmaier Reference Lewis-Beck and Stegmaier2014) and now extends beyond academia, with many rival online and media forecasts. They are certainly interesting for the public and the candidates, but what is their value to political science? On the one hand, there ought to be epistemic gains to the profession in understanding of elections if we accurately forecast results (King et al. Reference King, Keohane and Verba1994; Schrodt Reference Schrodt2014). On the other, if commentators and the public misunderstand forecasts, or different methods provide rival forecasts, the forecasting business might throw our profession into disrepute (Blanchflower Reference Blanchflower2016).

In this comment I will, first, elucidate, in a fairly simple manner, the dangers of forecasts being misunderstood. Second, I will distinguish forecasting from scientific prediction.

The forecasting models in this symposium predict the results of the next presidential election. At least, that is one description. Some predict the popular vote of the candidates, some predict the electoral college make-up through predicting state results, some turn popular vote estimates into electoral college estimates, some predict the Congressional results. These specific predictions will be judged by the actual result in November.

One way of judging the 2016 symposium is to say only two of the nine forecasts predicted Trump’s victory, one with the unconfident caveat that “Hillary Clinton should probably be considered a strong candidate to win” (Abramowitz Reference Abramowitz2016, p. 660). However, six Clinton victory forecast models were on the basis of a popular vote whose range of 50.4–52.5% almost perfectly circle her actual vote of 51.12%. These six models might be judged as correct in terms of what they actually modeled. Of course, presidents are not elected by popular vote, but by electoral college. For that reason, eight of the current symposium articles also model, or at least take into consideration, the electoral college. The point is obvious: what we take to be the forecast affects our judgment of a model’s success. But the very fact that we can have different judgments based on different criteria may induce skepticism about the whole enterprise in the public and the media, bringing the discipline into disrepute.

Forecasters can be explicit about what they are actually forecasting even if commentators are not. However, there are also other, deeper problems of interpretation. The public do not understand probability estimates, nor appreciate that these come with a margin of error; hence people overestimate the certainty with which forecasts are made (Westwood et al. Reference Westwood, Messing and Leikes2020). Careful estimates derived from poll data can be wrong, since reported margins of error in polls only capture sampling and not total survey error (Shirani-Mehr et al. Reference Shirani-Mehr, Rothschild, Goel and Gelman2018). In other words, forecasts are not as certain as they are often purported to be, and they are often misinterpreted. We cannot expect every forecast to be correct, especially in elections as unusual as 2016. Many “wrong” forecasts in fact lie well within the bounds of forecasting uncertainty. But trying to explain that to the public might well trigger even greater public skepticism about “so-called experts.”

So the problem for political science is that the public might judge its scientific merits by the accuracy of forecasts of a single token event while misunderstanding what those forecasts actually claim. A better judgment of the merits of any set of forecasting models is how well they perform over a series of such events. Judged thus, presidential forecasts appear in better light (Cuzan Reference Cuzan2020). Even so, we should not confuse forecasting with scientific prediction (Dowding and Miller Reference Dowding and Miller2019). The merits of a good forecast are not quite the same as the merits of a good explanatory model.

We judge forecasting models primarily in terms of (1) predictive success, (2) narrow probability bounds, and (3) length of lead times.Footnote ¹ Scientific models are judged in terms of their explanatory value. To be sure, predictive fit with the world is a necessary feature of such explanation, but scientific prediction is not future forecasting. A scientific prediction is a logical implication drawn from some theory. It need have no future reference. Scientific predictions are existential and a necessary part of explanation. They are existential since they are conditional. They say “if condition C holds, then we expect Y”; or “if condition C holds, then we expect with some probability p that Y holds.” They are explanatory, since if C is supposed to explain Y, then it should do so in all analogous circumstances (Dowding Reference Dowding2016, ch. 3).

With enough data, of course, scientific models can forecast token events. Astronomers have sufficient data to forecast eclipses of the sun or moon, but not when a meteorite will land on the White House lawn. Their inability to predict the latter does not besmirch astronomy. Political science should only be able to reliably forecast events with relatively stable patterns, and only then with good data. Our ability to forecast election results depends on good data, and it depends on elections having relatively stable patterns. In 2016, forecasters knew that the characteristics of the two candidates and the abnormal features of the Trump campaign would make the election unusual—they had, as Erikson and Wlezien (Reference Erikson and Wlezien2016, p. 671), nicely put it, a “premonition of greater uncertainty.”

Scientific predictions concern the structural features of types for which specific data on contingencies is required when forecasting the detail of token examples. Furthermore, in scientific terms correctly predicting the winner each time is less important than evidential aspects of models. After all, the proximate cause of an election result in all its detail is how each voter casts their ballot, together with the counting rules. In science we want the ultimate causes, structural features that explain the reasons why voters cast their ballots as they do. Not for each voter, of course, since any given event contains different levels of detail—what is sometimes called “granularity” of description. Biden winning the presidency can be caused by many different events at lower levels. Forecasts can be incorrect, yet accurately track the evidence they utilize, or be correct without tracking explanatory factors at all.

Some forecasting models are simply aggregative—they extrapolate from polls. How people say they intend to vote is an indicator of how they will vote, but polls do not provide much by way of explanation. Intention to vote may track contingencies in the world, but theory and other evidence is required to connect the two. Contingent events are best tracked retrospectively. For example, Lewis-Beck and Quinlan (Reference Lewis-Beck and Quinlan2019) nicely test Hillary Clinton’s views on the structural and contingent factors that led to her loss, while Sances (Reference Sances2019) models the purportedly unusual flipping of counties to see if it is really unusual when viewed over time.

Structural models use objective economic and political indicators. They provide more by way of explanation and over greater lead times can be more accurate than aggregative forecasts (Lewis-Beck and Dassonneville Reference Lewis-Beck and Dassonneville2015). Combining the two can provide more accurate forecasts, though with less lead time, but do not in themselves provide further explanation because they still only map contingency. Thus improving forecasting does not necessarily improve scientific prediction. Poll changes a week prior to the 2016 election may tell us nothing about similar poll changes at the 2020 election, unless the same conditions C hold for both. Scientifically all we learn is that polls can switch in the last week.

Nevertheless, we might think the more reliable the forecasting model, the stronger its scientific basis. A good reason for believing that forecasting is the right route for political science is that models predicting unknown data are epistemically superior to ones only accommodating known data. Known data restricts potential explanatory theories and cannot increase our belief that any theory is true. Forecasting unknown data adds something (White Reference White2003). Highly predictive models accommodating known data can be achieved through overfitting. Forecasting unknown data suggests our models are not overfitted. However, there are ways of guarding against overfitting even when data is known. The epistemic advantage only accrues when new data tracks other differences between models that are evidentially significant (Hitchcock and Sober Reference Hitchcock and Sober2004). Furthermore, models can track evidential significance at the type level, yet fail to track evidential significance in a specific token example.

Scientific prediction and explanation is directed at types. To prove accurate of a given token event, scientific prediction needs to pick out what is evidentially important in that event. Where an election is close, where forecasts are made with low confidence, then some specific set of circumstances might be causally important. In presidential elections these could be something peculiar to a specific swing state, or something unusual—such as a pandemic! Accommodating after the event can lead to the charge of overfitting and not improve long-term model accuracy. Of course, the probabilities assigned to forecasts are designed to take contingent factors into account. Forecasting models that do best overall are those which model the most evidentially important factors for the type as a whole.

There are many stable patterns in social life, which tend to hold over long periods of time. Forecasts are made for specific events; at times, these will not conform to the overall pattern. Forecasting models are obviously important to political science for they give an epistemic check upon our accommodationist modelling. Yet there are dangers. Critics may deride the accuracy of the headline predictions and then use this to disparage the science in political studies. This is to misunderstand the nature of scientific prediction, and is liable to instill expectations of our ability to foretell future token events with complex causal determinants at a level of granularity which is never expected of natural scientists.