Expert Judgment

Although asking domain experts about what is going to happen is a popular method for making predictions, empirical evidence shows that the predictive value of expert judgment is limited. Especially when forecasting complex problems, experts’ forecasts often are no more accurate than those of laymen or simple statistical models (Tetlock Reference Tetlock2005).

Regarding the prediction of election outcomes, it nonetheless seems reasonable to assume that experts have value relative to alternative methods (e.g., polls). The reason is the availability of extensive domain knowledge, a condition under which Green, Graefe, and Armstrong (Reference Green, Graefe, Armstrong and Lovric2011) expected experts’ forecasts to be useful. For example, political scientists who study voting behavior and campaign effects should be able to assess how polling numbers may change relative to recent and future events. Similarly, pollsters have insight about response rates and know whether certain demographic groups are difficult to poll, which should help them to assess potential bias in poll numbers. In other words, we would expect experts to be able to improve on the accuracy of polls or, at least, to identify their directional error.

The limited evidence available supports the notion that expert judgment can add value for predicting election outcomes. Graefe (Reference Graefe2018) analyzed the accuracy of 452 individual expert forecasts across four US presidential elections from 2004 to 2016. Although a randomly selected expert forecast, on average, was less accurate than the RealClearPolitics (RCP) poll average from the same day, almost two thirds of the experts’ forecasts (62%) correctly predicted the RCP’s directional error. In other words, experts are somewhat better than guessing in identifying the directional error of polls. Therefore, expert judgment is likely to improve forecast accuracy when combined with polls.

Averaging forecasts from multiple experts also is beneficial. Jones and Cuzán (Reference Jones and Cuzán2013) found that the simple average of US presidential-election forecasts made by several experts outperformed polls for longer forecast horizons.

I emailed survey invitations to 11 experts (i.e., political scientists, journalists, and pollsters).Footnote ¹ In the online questionnaire, experts were asked to forecast vote shares of seven candidates (and all other candidates combined) in the first round of the 2022 French presidential election. Table 1 shows simple averages of forecasts from five experts who responded to the survey conducted January 18–21, 2022. In addition, experts were asked to predict vote shares for two hypothetical second-round runoffs (i.e., Macron versus Le Pen and Macron versus Pécresse). Table 2 shows these results.

Table 1

First-Round Vote-Share Forecasts (in %) of the PollyVote and Its Components

¹ Source: Datapolitics.fr

² I thank Éric Bélanger for calculating the forecast from the Iowa model.

³ Note that my calculation of the forecast for Pécresse differs from the forecast provided by Facchini (Reference Facchini2022), who predicted a vote share of 24.2%. I distributed the right-wing vote across Pécresse, Zemmour, and Dupont-Aignan, using latest poll numbers.

Data as of January 25, 2022, available at Harvard Dataverse (Graefe Reference Graefe2022).

Table 2

Macron’s Second-Round Vote-Share Forecasts (in %) of the PollyVote and Its Components, Conditional on the Challenger

¹ Source: politico.eu/europe-poll-of-polls/france.

² Expectations include only expert judgment. No second-round forecasts were available from citizen forecasts at the time of writing.

³ Retrospective models include only the forecasts from party ratings model, using the average of the two second-round forecasts provided by Facchini (Reference Facchini2022). The proxy and Iowa models did not provide second-round forecasts.

⁴ The naïve no-change forecast used the result of the 2017 runoff between Macron and Le Pen.

Data as of January 25, 2022, available at Harvard Dataverse (Graefe Reference Graefe2022).

Citizen Forecasts

Citizen forecasts are derived from survey respondents’ answers about who they think will win an election. Prior research found that citizen forecasts often outperform alternative forecasting methods. For example, analyzing 449 months of data for British general elections from 1950 to 2017, Murr, Stegmaier, and Lewis-Beck (Reference Murr, Stegmaier and Lewis-Beck2021) found that citizen forecasts were more accurate than polls in predicting both election winners and seat shares. Graefe (Reference Graefe2014) found that across the last 100 days prior to the seven presidential elections from 1988 to 2012, citizen forecasts were more accurate than four alternative methods: polls, betting markets, models, and expert forecasts.

Despite the predictive power of citizen forecasts, public-opinion surveys rarely include vote-expectation questions and only ask for people’s vote intention. One reason may be that there are no clear guidelines on how to best obtain survey participants’ electoral expectations and translate them into a forecast. As pointed out by Graefe (Reference Graefe2016), there are many methodological questions, especially in multiparty elections.

The good news is that researchers have started to collect citizen forecasts for different types of elections. For the 2022 French presidential election, Dufresne et al. (Reference Dufresne, Jérôme, Lewis-Beck, Murr and Savoie2022) provided citizen forecasts based on data from representative surveys of 1,000 respondents. Table 1 shows the citizen forecasts derived from their survey conducted in late November 2021.

Retrospective Models

The proxy model predicts a total vote share of 47.3% for all left-wing candidates, using the prime minister’s job-approval rate as a single independent variable. In the article that presents the proxy model in this symposium, Bélanger, Feitosa, and Turgeon (Reference Bélanger, Feitosa and Turgeon2022) also refer to the Iowa model, which has been used to forecast previous French elections (e.g., Lewis-Beck, Bélanger, and Fauvelle-Aymar Reference Lewis-Beck, Bélanger and Fauvelle-Aymar2008). The Iowa model is similar to the proxy model in that it also predicts the combined vote for all left-wing candidates using a measure of the incumbent’s popularity.Footnote ² The difference is that the Iowa model includes one additional variable: the national unemployment rate as a structured measure of the state of the economy. Because both models predict the total vote shares of all left-wing (and, therefore, all right-wing) candidates, the PollyVote translates these figures to vote-share forecasts for each candidate using daily updated poll numbers.Footnote ³ A third model, presented by Facchini (Reference Facchini2022), uses party ratings to predict party-vote shares. Again, the PollyVote uses daily updated poll numbers to convert vote shares of blocs (e.g., the right-wing) to vote shares for each candidate (e.g., Pécresse, Zemmour, and Dupont-Aignan). This also is the reason why some of the numbers in table 1 differ from those provided by Facchini (Reference Facchini2022).

Mixed Models

Mixed models include both retrospective and prospective aspects of voting. Two models of that type are available: those by Jérôme and Bélanger (Reference Jérôme and Bélanger2022) and by Nadeau, Mongrain, and Jérôme (Reference Nadeau, Mongrain and Jérôme2022). Essentially, these models consist of several equations that use different predictor variables that were estimated through multiple-regression analysis. Each equation then was used to predict the vote share(s) of a specific candidate or party (e.g., the incumbent) or a bloc of parties (e.g., the Diverse Right or the Total Left).

The models share several variables, including retrospective variables, such as the level of—and change in—the unemployment rate or the incumbent’s popularity, as well as prospective variables, such as vote-intention polls. The main difference between both models is that Jérôme and Bélanger (Reference Jérôme and Bélanger2022) predicted vote shares at the regional level and then aggregated the resulting forecasts to national vote shares (i.e., SUR regional), whereas Nadeau, Mongrain, and Jérôme (Reference Nadeau, Mongrain and Jérôme2022) directly predicted national vote shares.

As with the retrospective models described previously, the two mixed models do not predict vote shares for all candidates listed in table 1. In these cases, the PollyVote again uses daily poll numbers to translate the numbers to vote shares for each candidate.