1. Do campaigns matter more than in the past?

First, students of voting behavior have drawn attention to a shift in weight of different factors that influence voters' choices. It has been argued that “anchor factors” are becoming less important (Costa Lobo Reference Costa Lobo2006; Walczak et al. Reference Walczak, van der Brug and de Vries2012). The case for the decline of long-term factors has many entries; scholars have argued that partisanship is in decline (Dalton and Wattenberg Reference Dalton and Wattenberg2002)Footnote ¹, and there is strong evidence that cleavages such as religion (Knutsen Reference Knutsen2004) and social class (Evans and Tilley Reference Evans and Tilley2012) are shaping voters' choices less now than was the case some decades ago (Lipset and Rokkan Reference Lipset and Rokkan1967).

The decline of long-term factors is affecting voters' behavior.Footnote ² This is clear from the fact that electoral behavior has become more volatile, with increasingly large groups of voters switching their preferences during the election campaign and from one election to another (Dalton and Wattenberg Reference Dalton and Wattenberg2002; Dassonneville Reference Dassonneville2013). At the same time, voters appear to make their vote choice increasingly late (Box-Steffensmeier et al. Reference Box-Steffensmeier, Dillard, Kimball and Massengill2015; Schmitt-Beck and Partheymüller Reference Schmitt-Beck and Partheymüller2015). Late-deciding voters are known to be more influenced by what happens during the campaign (Fournier et al. Reference Fournier, Nadeau, Blais, Gidengil and Nevitte2004; Johann et al. Reference Johann, Kleinen-von, Kritzinger and Thomas2018). As a result, trends as these make it plausible that campaigns matter more now than they did in the past.

Second, changes in campaign practices and in the media environment have potentially strengthened the impact of election campaigns. Political parties nowadays make intensive use of sophisticated techniques to persuade voters. Marketing professionals are employed by political parties to brand and sell the parties' platforms (Lees-Marshment Reference Lees-Marshment2014). In addition, technological changes allow parties to contact voters in a more targeted and personalized way (Hersh Reference Hersh2015; Kreiss Reference Kreiss2016).

Both patterns of change in voters' behavior and changes in campaign practices give credence to the expectation that structural forecasting models, that predict the election outcome months out, and that rely on national-level indicators, are performing less well now than they did in the past. Our expectation is therefore that prediction errors of structural forecasting models have grown larger over time.

2. A parsimonious forecasting model

For testing our hypothesis, we develop a fundamentals model, apply it to electoral data, and trace its performance over time. Because the claims of changes in voters' behavior apply to most established democracies, and because we do not want our results to be driven by a single case, we apply the model to as many advanced democracies as possible.

We opt for a parsimonious forecasting model, that follows standard practice in the field and model the incumbent vote as a function of a political and an economic variable (Lewis-Beck Reference Lewis-Beck2005):

(1)$$ \hbox{Incumbent vote} = f\,(\hbox{politics, economics}) $$

We thus focus on predicting the vote of the incumbent, and model it as a function, first, of the economy. The theory guiding our “fundamentals model” is therefore the economic voting theory, which serves as the theoretical foundation for most structural forecasting models (Lewis-Beck Reference Lewis-Beck2005). Because our goal is to apply a similar parsimonious model to predicting the incumbent vote in different countries, we include GDP growth as the economic variable. After all, GDP growth has been referred to as the most “global” economic indicator (Norpoth et al. Reference Norpoth, Lewis-Beck and Lafay1991).

Our second indicator is a political one and captures citizens' general appreciation of the incumbent government before the start of the election campaign (Campbell Reference Campbell2005). Forecasters regularly rely on measures of public approval of the President's job (Campbell Reference Campbell2005) or government approval (Lewis-Beck et al. Reference Lewis-Beck, Nadeau and Bélanger2004) to capture this sentiment. However, only a handful of countries have long time-series of government or presidential job approval data. As an alternative, we rely on vote intention data, for the incumbent, measured three months before the election. Such a measure reflects “the public's general impressions of the in-party and its opposition on a wide range of issues (from war to the economy)” (Campbell Reference Campbell2005, p. 74).

Using vote intention data as the political variable and GDP growth as our economic indicator, our fundamentals model takes the following form:

(2)$$ {\rm V} = \beta_0 + \beta_1 {\rm VI} + \beta_2 {\rm GDP} + \epsilon $$

where V is the incumbent vote share, VI is the vote intention for the incumbent, GDP is the GDP growth rate, and ε is an error term.

3. Data, operationalization and methods

For six established democracies, we have vote intention data since at least the early 1950s, and these countries therefore constitute our sample. We apply the model to forecast 114 elections held between 1955 and 2017 in six countries: Australia (1958–2016), Canada (1957–2015), Denmark (1960–2015), Germany (1957–2017), the UK (1957–2017) and the USA (1956–2016). These countries are established democracies and also present some institutional variety. Our focus is on legislative elections, except for the USA, where we apply the model to presidential elections.

Our sampling period covers more than six decades, a time span during which changes in parties' and voters' behaviors may have increased the importance of campaigns and rendered electoral results less predictable. Since we are primarily interested in the overall trend, we merged the data for the six countries to form a balanced survey pool.Footnote ³

The entries in Table 1 highlight a number of interesting features of our dataset. First, the data include a sufficient number of elections (114 in total) to compare the performance of our forecasting model for meaningful sub-periods. These sub-periods, regardless of whether we define two or four sub-periods, are characterized by the good balance of their composition. That is, single countries never represent less than 13 percent and never more than 23 percent of the sample. Second, as the entries (columns by country) in Table 1 show, the distribution of elections among countries remains quite stable over time. This means that a change in performance of our forecasting model cannot be attributed to changes in the composition of our sample over time.

Table 1.

Distribution of national elections over time

The dependent variable measures the level of support garnered by all the members of the incumbent coalition. The coalition can consist of single party governments or administrations (USA, Canada and, with one exception, Great Britain), it can include quasi-permanent coalitions (like the Alliance and CDU/CSU partnerships in Australia and Germany) or larger and changing coalitions (like in Denmark and Germany). Details about the definition of the dependent variables are provided in Appendix A.

Vote intentions are measured three months before the elections (sources are reported in Appendix A). This implies that our model predicts the election outcome with information that is available before the start of formal electoral campaigns (or the post-Labor Day stretch for the USA) in virtually all cases. If campaigns indeed matter more now than in the past for voters' choices, models that rely on such a long lead time should perform less well now.

Finally, information on GDP growth was obtained from the OECD. Following common practice, we lag the GDP growth rate three months.Footnote ⁴ The three-month lead time is conventional (Nadeau et al. Reference Nadeau, Lewis-Beck and Bélanger2009).

We use OLS to estimate Equation 2 on the pooled dataset. To account for the nested structure of the data, with different elections by country, we include country fixed effects, and we cluster the standard errors by country. In addition, the data are weighted to ensure that the results are not overly driven by countries (like Australia) holding more frequent elections.Footnote ⁵

After estimating this model, we evaluate its predictive power and how this evolves over time. In the first step, we evaluate the MAE (mean average error) statistics of the model, and do so for different time periods. We then analyze the prediction errors in a more systematic way by saving the prediction errors that the model produces. We plot and visually inspect the absolute prediction errors and also use them as the dependent variable in a regression model in which the main independent variable is time.

5. Robustness of the results

These results can be challenged in a number of ways, which we address in a set of supplemental analyses.

First, we validated that our results are driven by outliers. The analyses that are reported in Appendix D suggest that even when excluding potential outlier elections, the time variable still fails to reach significance.

Second, some might question our reliance on vote intention data. We address this issue by replicating our analyses with a more standard government approval indicator in three of the six countries; Germany, the UK and the USA. These additional analyses nicely replicate our main results (see Appendix E).