Background

Economic voting is one of the most important accountability mechanisms at work in electoral democracies. The fact that voters reward or punish the incumbent government based on how the domestic economy is performingFootnote ¹ is traditionally viewed as normatively desirable, for it reflects popular control of representatives.Footnote ²

Recently, a number of scholars have challenged this optimistic view, by pointing out that domestic economic growth is often a weak proxy for government performance. When the local economy moves in sync with secular trends or global shocks, governments may be rewarded or punished for events beyond their control.Footnote ³ This is especially true in integrated economies, where domestic fortunes are tightly linked to events abroad, and responsibility is blurred.Footnote ⁴ Democratic accountability thus requires more from voters than a simple response to local economic conditions.Footnote ⁵

A new and influential strand of research argues that voters do, in fact, make rational judgements about government performance, because their evaluations are relative.Footnote ⁶ What matters to rational voters may not be how well the national economy is doing per se, but rather how it performs compared to the economies of other countries, or relative to some historical benchmark.

Benchmarking is a powerful idea, which can be traced back to the work of Powell and Whitten.Footnote ⁷ As these authors point out, voters are likely to ‘evaluate government relative to some expectations about how the economy should have performed’.Footnote ⁸ But since expectations are difficult to measure, ‘it seems reasonable to use the international average levels of growth, inflation, and unemployment to estimate a baseline against which each country’s citizens could judge the performance of their own economy’.Footnote ⁹ This approach is intuitive, since ‘abundant research in other domains of social science supports the proposition that individuals are sensitive to comparative assessments’.Footnote ¹⁰

Yet there are also good reasons to doubt that voters benchmark economic performance. First, the benchmarking hypothesis is at odds with a dominant view on the cognitive limitations of ordinary voters. Indeed, a long tradition of research in political science has depicted the citizenry as poorly informedFootnote ¹¹ and biased.Footnote ¹² It is difficult to imagine how such an unsophisticated electorate could systematically and accurately compare how well the national economy is performing relative to other countries or a historical benchmark. Secondly, even if some authors posit that the media could facilitate benchmarking by making implicit comparisons in their news coverage, evidence of the underlying mechanism is rather weak. For instance, Kayser and Peress (henceforth, KP) report that high-information voters – those most exposed to the media – do not engage in more benchmarking than low-information voters.Footnote ¹³ Finally, some empirical studies claim that voters act based on relative economic conditions, but others find that when it comes to evaluating government performance, ‘the effect of luck is larger than the effect of competence’.Footnote ¹⁴ In short, the theoretical case for benchmarking is implausible, and the empirical record is mixed.

In this article, we show that the empirical evidence of benchmarking is extremely weak. We argue that the way in which regression models are typically specified to test benchmarking is needlessly complicated, that it invites a fundamental misreading of the evidence and that it may lead researchers astray. We propose a simpler model specification that can be used to test benchmarking, which avoids common misconceptions and carries powerful intuitions about the theory. We also show how this simple model can be enhanced to test more complicated theories, such as when voters benchmark against multiple reference points, or when the strength of benchmarking depends on the context. We revisit a prominent empirical study of Benchmarking Across Borders,Footnote ¹⁵ and conduct a faithful replication of the models reported in that article. When correctly interpreted, the results do not support the contention that voters make rational comparative evaluations.

Our findings have important implications for the field of economic voting, and for our understanding of the mechanisms that underpin democratic accountability. More generally, our article makes useful contributions to political science methodology by highlighting the shortcomings of a widely used empirical strategy, and by proposing a better way to test theories of relative evaluation.

Benchmarking vs. Conventional Economic Voting

The core intuition of benchmarking is illustrated in Figure 1, which shows how domestic growth and a reference point can affect support for the incumbent party. The solid line represents the domestic growth rate during the election year (G _y ), and the dashed line represents the growth rate that voters use as a benchmark to evaluate the incumbent government’s performance. Depending on the analyst’s theory, the reference point could be the international growth rate (G _i ) or the historical level of growth in the country under study (G _h ).

Figure 1

Marginal effects of domestic economic growth and benchmark growth on votes for the incumbent.

The conventional view of economic voting is that votes for the incumbent (V) are tied to domestic growth. As we move from left to right, G _y increases in Figure 1a but stays constant in 1b. Thus the conventional prediction is that votes for the incumbent will increase in 1a but stay constant in 1b.Footnote ¹⁶

In contrast, proponents of benchmarking argue that what matters to voters is not domestic growth per se, but rather the difference between domestic growth and the benchmark (G _y − G _i ).Footnote ¹⁷ When the solid line is above the dashed line, domestic growth outperforms the benchmark, and voters should reward the incumbent. When the solid line is below the dashed line, domestic growth underperforms relative to the benchmark, and voters should punish the incumbent. As we move from left to right in Figure 1, the ‘performance gap’ or ‘competence signal’ increases in Figure 1a and decreases in 1b. Thus, benchmarking predicts that votes for the incumbent will increase in 1a and decrease in 1b.

These expectations can be restated using the language of multiple regression. When domestic growth increases and the reference point is held constant (Figure 1a), both theories predict that the incumbent’s vote share will increase. In other words, both theories predict that the marginal effect of domestic growth will be positive: ∂V/∂G _y >0. When the reference point increases and domestic growth is held constant (Figure 1b), benchmarking predicts that votes for the incumbent will decrease. In other words, benchmarking predicts that the marginal effect of the reference point will be negative: ∂V/∂G _i <0.

If we hope to discriminate between benchmarking and conventional economic voting, the main quantity of interest is the marginal effect of the reference point, since this is where benchmarking theory makes a distinctive prediction.

How do Scholars Test Benchmarking?

Conventional theories of economic voting are typically tested using models of this form:

(1) $$V{\equals}\beta _{y} G_{y} {\plus}{\rm \Psi \Omega }{\plus}\nu ,$$

where V is the incumbent’s vote share, G _y is the domestic economic growth rate during the election year, Ω is a vector of control variables, and ν is a disturbance term. Clearly, Model 1 cannot be used to test benchmarking, since it ignores relative evaluations altogether.

In their seminal article, Powell and WhittenFootnote ¹⁸ estimate a regression equation of this form:

(2) $$V{\equals}\lambda _{{y{\minus}i}} (G_{y} {\minus}G_{i} ){\plus}{\rm \Phi \Omega }{\plus}\upsilon ,$$

with G _i equal to a ‘reference point’, the international economic growth rate. Model 2 takes us very close to the benchmark story: When the gap between G _y and G _i is positive, the domestic economy outperforms the global economy, and voters should reward the incumbent government for its competence.

As KP note, however, Model 2 cannot be used to distinguish between benchmarking and conventional economic voting, because it suffers from omitted variable bias.Footnote ¹⁹ Indeed, the composite variable (G _y − G _i ) is highly correlated with the level of domestic economic growth (G _y ).Footnote ²⁰ As a result, we cannot parse out the effect of benchmarking from conventional economic voting, and λ _y−i captures both phenomena. Model 2 is thus useful if we want to estimate something akin to the ‘total effect’ of domestic growth and benchmarking on voting behavior, but not if we wish to compare and contrast the two theories.

To solve this problem, KP introduce an additional control for the reference point:

(3) $$V{\equals}\theta _{{y{\minus}i}} (G_{y} {\minus}G_{i} ){\plus}\theta _{i} G_{i} {\plus}{\rm \Gamma \Omega }{\plus}{\epsilon}.$$

In this model, G _y − G _i represents a ‘decomposed’ or ‘local’ component of growth, whereas G _i aims to control for changes in the reference point. Model 3 has had tremendous influence in the field. At the time of writing, KP’s article has been cited over 160 times, and several other researchers have adopted and adapted their empirical strategy.

A Widespread Misconception

An intuitive – but ultimately incorrect – way to interpret Model 3 would be to focus on the gap between G _y and G _i , and to treat the θ _y−i coefficient as the effect of relative economic performance on votes for the incumbent.

For example, Aytaç argues that a positive estimate of θ _y−i provides ‘evidence for the hypothesis that voters reward (punish) incumbents on whose watch the economic performs relatively better (worse) in domestic and international comparisons’.Footnote ²¹ KP define ‘local growth’ as the gap between G _y and G _i , and interpret θ _y−i as measuring the association between ‘an increase in local growth’ and an ‘increase in the leader party’s vote share’.Footnote ²² Goplerud and Schleiter follow in those footsteps, and discuss θ _y−i as the effect of some ‘benchmarked’ or ‘local’ component of growth on voting behavior.Footnote ²³ Using data on the American states, Ebeib and Rodden also interpret θ _y−i as the effect of ‘relative state conditions’ on votes.Footnote ²⁴

If the domestic economy outperforms a reference point, it may be reasonable for voters to infer that the government is doing good work. In that spirit, Leigh treats θ _y−i as the effect of ‘government competence’ on votes for the incumbent.Footnote ²⁵ In the American context, Wolfers considers the gap between state- and national-level economic growth, and calls θ _y−i the ‘effect of competence’.Footnote ²⁶

Interpreting θ _y−i as the effect of relative economic performance on votes for the incumbent appeals to common sense, but it is a mistake. The root of the problem lies in the fact that G _i appears twice on the right-hand side of Equation 3. This redundancy changes the substantive meaning of our regression coefficients.

To see how, take the partial derivative of Equation 3 with respect to G _y , and find the marginal effect of domestic growth:

(4) $${{\partial V} \over {\partial G_{y} }}{\equals}\theta _{{y{\minus}i}} .$$

This simple exercise demonstrates that the coefficient associated with G _y − G _i is exactly equivalent to the marginal effect of G _y . Against intuitive common sense, θ _y−i does not measure the effect of relative economic performance on votes for the incumbent. Since θ _y−i is the marginal effect of domestic growth, finding a positive coefficient for ‘benchmarked’ or ‘local’ growth is actually supportive of conventional economic voting.

Tests of benchmarking based on Equation 3 have been repeatedly misinterpreted in prestigious scientific journals, by leading scholars of economic voting. The inclusion of duplicate regressors on the right-hand side of Equation 3 has been a source of widespread confusion in the economic voting literature.Footnote ²⁷ To put this confusion to rest, we need a simpler, more direct test of benchmarking.

A Simpler Test of Benchmarking

From Figure 1, we learned that both benchmarking and conventional economic voting predict that the marginal effect of domestic growth should be positive. In contrast, only benchmarking predicts that the marginal effect of international growth should be negative. The simplest and most direct way to test those predictions is to estimate a model of this form:

(5) $$V{\equals}\delta _{y} G_{y} {\plus}\delta _{i} G_{i} {\plus}{\rm \Gamma \Omega }{\plus}{\epsilon}.$$

Since Model 3 includes redundant regressors, it carries no more information than the simpler Model 5. In fact, Models 3 and 5 are perfectly equivalent from a logical standpoint, and they produce identical numerical results: the marginal effect of domestic growth, the marginal effect of international growth,Footnote ²⁸ the intercept, the control variables’ coefficients, the residuals and all fit statistics are always the same in both models. In the online appendix, we present side-by-side estimates using Models 3 and 5 to illustrate this point numerically.

Yet even if the two models are formally equivalent, the simpler specification has major advantages in terms of transparency, presentation and interpretation.

First, the correct interpretation of KP’s Model 3 is highly counterintuitive: the coefficient that they call ‘Local Component of Growth’ in their regression tables (θ _y−i ) does not measure the effect of the local economy’s relative performance on votes for the incumbent. As we showed above, this has been a major source of confusion in the field, and benchmarking results have been repeatedly misinterpreted in print. Our simpler specification avoids this problem.Footnote ²⁹

Secondly, Equation 5 directly translates the theoretical intuitions conveyed by Figure 1, and it immediately reveals the relevant test statistics. Recall that the discriminating test of benchmarking is that international growth should have a negative marginal effect on votes for the incumbent. In Model 5, the marginal effect of international growth is the δ _i coefficient, and we can simply look at its p-value. In Model 3, the marginal effect of international growth is a linear combination of coefficients (∂V/∂G _i =θ _i − θ _y−i ), and we must conduct an extra Wald test to know if that combination is negative and statistically significant.

Finally, as we show below, our simpler specification offers a solid foundation on which we can build empirical tests for theories of benchmarking where voters compare multiple reference points, or where the strength of benchmarking is context dependent.

Replication: Benchmarking Across Borders

As we explained above, the key quantity of interest for tests of benchmarking is the marginal effect of international growth (holding domestic growth constant). Unfortunately, KP do not consistently report the statistics that are needed to test if that quantity is distinguishable from zero.Footnote ³⁰ As a result, readers cannot assess the strength of the evidence simply based on the findings printed in Benchmarking Across Borders.

To see if KP’s data support their theory, we re-estimated all of their models using the authors’ replication files, and we computed all the quantities of interest.Footnote ³¹ Table 1 shows the results for four models,Footnote ³² estimated using KP’s preferred measure of international growth (an index constructed via principal component analysis).

Table 1

OLS regression models with incumbent vote share as the dependent variable

Note: robust standard errors in parentheses. *p<0.1, **p<0.05, ***p<0.01

How to Test Benchmarking With Multiple Reference Points

The models considered above show little evidence of benchmarking. This surprising result could be an artifact of several factors. For instance, our models may be too simple to capture the complex processes at work, or KP’s dataset may be too small to conduct well-powered tests. In this section, we consider how to adapt our barebones empirical framework to the more complex case in which voters compare domestic economic performance to multiple reference points. Then, we illustrate by studying a larger dataset drawn from a more recent study of benchmarking.

AytaçFootnote ⁴³ develops a reference point theory that is highly reminiscent of KP’s, but which makes two important substantive changes. First, the author argues that voters use two reference points to assess their government’s performance: the level of international growth (G _i ) and their own country’s historical level of growth (G _h ).

Secondly, Aytaç points out that these reference points could be compared to two alternative measures of the incumbent’s performance: domestic growth during the election year (G _y ) or domestic growth during the incumbent’s full term in office (G _t ). The term-based measure is preferable if we adopt a rational voter model, since such voters can extract more information about the quality of government by observing performance over a longer period. The election year measure is preferable if we take the view – dominant in political psychology – that voters are cognitively limited, myopic and that they use end heuristics when engaging in retrospective evaluations.Footnote ⁴⁴ Here, we remain agnostic and estimate models using both measures.

In our framework, testing theories of benchmarking with multiple reference points is straightforward: we simply introduce the new reference point variable additively in Model 5. Again, there is evidence of benchmarking if the marginal effect of domestic growth is positive, and if the marginal effects of the benchmarks are negative.

In Table 2, we illustrate this by estimating six models using Aytaç’s replication data.Footnote ⁴⁵ In a first set of three models, we compare international and historical growth to domestic growth in the election year. In a second set of three models, we compare international and historical growth to the average domestic growth rate during the incumbent’s full term in office. We include the same control variables as Aytaç.

Table 2

Benchmarking with multiple reference points and alternative measures of domestic growth

Note: OLS regressions with country-clustered standard errors. Robust standard errors in parentheses. *p<0.1, **p<0.05, ***p<0.01

All six of the models in Table 2 show evidence of conventional economic voting: the coefficients for domestic economic growth (G _y or G _t ) are all positive and statistically significant. In contrast, none of the models allows us to reject the null hypothesis of ‘no international benchmarking’: the G _i coefficient is never statistically significant at the α=0.1 level.

The two right-most models in Table 2 show evidence of historical benchmarking: the G _h is negative and statistically significant. However, it is important to point out that those models rely on a highly unconventional assumption. Indeed, they assume that voters have long enough memories to accurately compare the average level of growth during the incumbent’s full term in office to the average level of growth during the previous government’s term. This assumption clashes with common wisdom in the field of economic voting, in which ‘virtually all macro-studies assume a short lag, generally of one year’.Footnote ⁴⁶ Most studies of benchmarking also use short-term measures of domestic growth.Footnote ⁴⁷

In sum, the results in Table 2 offer strong support for the conventional theory of economic voting, but evidence of benchmarking is mixed. None of the models allows us to confidently reject the absence of international benchmarking, and the only models that support historical benchmarking require us to jettison the widespread assumption that voters are myopic.

How to Test Conditional Theories of Benchmarking

The regression models that we have studied so far were relatively underspecified. Indeed, one of the major contributions of Powell and WhittenFootnote ⁴⁸ was to point out that the level of economic voting depends on the institutional context (for example, clarity of responsibility). Similarly, there are good reasons to think that benchmarking will vary across populations: some voters – such as those with high information – might engage in more relative economic evaluations than others.

If benchmarking is truly conditional, then the ‘pooled’ models that we estimated above would be inappropriate, and our null results would not be surprising. For this reason, it is extremely important to develop regression models capable of testing conditional benchmarking hypotheses. Again, this is very easy to do in our simple empirical framework.

We use the same starting point as before (Figure 1). Benchmarking predicts that the marginal effect of domestic growth should be positive, and that the marginal effect of the reference point should be negative. If a moderating variable M increases (decreases) the salience of the reference point, then the marginal effect of domestic growth should be more (less) positive, and the marginal effect of the reference point should be more (less) negative where M is high.

This idea can be captured by a simple extension of Model 5:

(6) $$V{\equals}\delta _{y} G_{y} {\plus}\delta _{i} G_{i} {\plus}\delta _{{ym}} G_{y} {\times}M{\plus}\delta _{{im}} G_{i} {\times}M{\plus}\delta _{m} M{\plus}\Gamma \Omega {\plus}{\epsilon},$$

where M stands for a variable that moderates comparative economic assessments.

As usual, a positive marginal effect of domestic growth (δ _y +δ _ym M>0) would be consistent with both conventional economic voting and benchmarking. A negative marginal effect of international growth (δ _i +δ _im M<0) would be consistent with benchmarking. The slopes of those marginal effects (δ _ym and δ _im ) measure the extent to which M moderates relative economic assessments.

To illustrate how one can apply Model 6, we revisit a secondary set of tests from Aytaç,Footnote ⁴⁹ where the author studies if benchmarking is more prevalent in countries with high trade intensity, GDP per capita or average level of schooling.Footnote ⁵⁰ We assess the moderating effect of all three variables,Footnote ⁵¹ include the same control variables as in Table 2, and use Aytaç’s two alternative measures of domestic growth (G _y and G _t ). The full regression results are reported in the online appendix.

Figure 2 shows the estimated marginal effect of international growth in six models. None of the marginal effects is clearly negative, and most lines are nearly flat. These results, estimated using a dataset that is over twice the size of KP’s, offer no evidence of international benchmarking, and no evidence that trade, income or education increase the salience of comparative economic assessments.

Figure 2

Marginal effect of international growth on votes for the incumbent .Note: the figure displays the results of six regression models with three different moderators and two alternative measures of domestic growth. 95 per cent confidence intervals in gray.