Stokes (Reference Stokes1963, Reference Stokes1994) famously argued that competition between parties involves assessing policy performance and party competence in the delivery of agreed-upon goals. There is assumed to be a general consensus on the desirability of goals such as economic growth, the effective delivery of public services and the reduction of crime. Vote choice is based on which party or candidate is perceived to be the most competent to deliver these sorts of outcomes. This theory contrasts markedly with approaches to voting that emphasize differences in issues and/or ideological positions between parties and voters. In recent years, perceptions of party competence have become important explanations of political choices and electoral outcomes. Voters evaluate parties and politicians in terms of their competence and performance, and vote for those they think have delivered, or will deliver, valued outcomes.Footnote 1 The significance of evaluating performance and competence for vote choice is now so embedded in our understanding of voting behavior that voter assessments of party competence on key issues are routinely included in election surveys. The American, British, Canadian, European and Irish election studies all now include questions on party issue competence (Wagner and Zeglovits forthcoming).
A particular model of valence voting advocated by the current British Election Study (BES) team has recently become highly influential. Clarke et al. (Reference Clarke, Sanders, Stewart and Whiteley2004, Reference Clarke, Sanders, Stewart and Whiteley2009, Reference Clarke, Sanders, Stewart and Whiteley2011) have instantiated an operationalization of valence voting that has reshaped understanding of British voting behavior in particular, but has also been assumed to have more general temporal and spatial applications.Footnote 2 The BES valence model has been influential because of its thorough and systematic empirical analyses and explicit specification and operationalization of what is involved in valence voting and how it should be measured. One recent reviewer concludes, “the valence model, and the sophisticated ways in which it is tested, provides a powerful, possibly unassailable, explanatory account of the overall pattern of party choice” (Johnston Reference Johnston2011, 213). Another offers that “Clarke et al. (Reference Clarke, Sanders, Stewart and Whiteley2004) make a compelling case for the predominance of valence voting in recent (and probably not so recent) elections” (Johns Reference Johns2010, 156). In this article we question whether the valence model advocated by Clarke et al. does actually present an unassailable explanatory account of party choice by examining whether the model of valence/performance voting they advance is empirically valid.
Our primary focus is on whether the key elements of the BES valence model—(1) which is the best performing party on the most important issue, (2) which party is evaluated most highly on salient issues and (3) which is the best party leader—can explain vote intention or reported vote. More specifically, we are interested in whether these associations between answers to questions, often measured sequentially in the same survey, can be treated as causal and unidirectional rather than simply correlational. If causation is not established, or if it is not unidirectional, estimates of ‘effects’ from recursive models will be at the very least biased upwards, or may even be found to be spurious. If so, inferences regarding the substantive impact of voters’ for evaluations of competence and performance will be invalidated, and the empirical basis of the valence model will therefore be discredited.
For this purpose we estimate cross-lagged model parameters using the multiwave BES 2005–10 panel study to examine the direction of causality that is assumed in the model: that is, that responses to questions about the best-performing party and party leader influence party preference and voting intention, rather than party preference and voting intention influencing evaluations of the competence of parties and party leaders. The idea that voters’ party preferences are powerful influences on competence and performance assessments has been influential since at least the American Voter (Campbell et al. Reference Campbell, Converse, Miller and Stokes1960; Green, Palmquist and Schickler Reference Green, Palmquist and Schickler2002; Bartels Reference Bartels2002a). In this view, voters’ party preferences provide an interpretative lens through which politics is perceived, which biases their judgments so that their preferred party is seen in a positive light. We compare the relative magnitudes of these reciprocal effects. We also use this approach to examine heterogeneity amongst voters and test claims concerning the generality of the valence model, which rest on the idea that performance evaluations are easily employed heuristics that enable even less-informed and less-motivated voters to make political choices.
The BES Valence Model
The BES valence model has three core elements: leadership assessments, party performance evaluations and party identification.Footnote 3 The theory proposes that the key processes informing party choice are “comparative assessments of parties’ managerial capabilities and their potential performance when attempting to deal with these problems. Voters maximize their utilities by choosing the party that they think is best able to deliver policy success in areas that concern them most” (Sanders et al. Reference Sanders, Clarke, Stewart and Whiteley2011, 289). The theory is operationalized and tested within a survey context by measuring performance evaluations and estimating their association with vote intention or reported vote. Thus “the valence indices for Labour and the Conservatives combine party leader evaluations, assessments of the party best able to deal with the most important problem facing the country, and evaluations of the prospective policy competence of a party with regard to the economy, the NHS, crime and terrorism” (Sanders et al. Reference Sanders, Clarke, Stewart and Whiteley2011, 304). Party choice is then regressed on these indices in either simple cross-section logit models, with all variables measured contemporaneously or near contemporaneously.Footnote 4 Alternatively, in their most recent (and arguably most sophisticated) analysis, Sanders et al. (Reference Sanders, Clarke, Stewart and Whiteley2011) use vote intention instead of reported vote and measure all covariates at the same time as the response, and include the lag of the dependent variable in their models.Footnote 5
In general, the BES valence model is found to have substantially higher levels of predictive power than models containing, for example, issue-proximity scales or demographics. As a result, earlier specifications of the valence model emphasized its predictive superiority to models containing only demographics, particularly class voting (Clarke et al. Reference Clarke, Sanders, Stewart and Whiteley2004), while more recent publications have also stressed its superiority to models containing issue proximities, as “…voters are always likely to give priority to valence issues over spatial issues” (Clarke et al. Reference Clarke, Sanders, Stewart and Whiteley2009, 51). In the most recent specifications of the model, it has been claimed to be predictively superior to, though partly conditioned by, issue proximity (Sanders et al. Reference Sanders, Clarke, Stewart and Whiteley2011).
Underlying the arguments of the valence thesis is the idea that “people typically are not particularly interested in politics and wish to minimize the cognitive burdens of selecting a party” (Sanders 2011, 290). Valence theory “merely requires a voter to judge the relative merits of a very limited number of prominent individuals (party leaders) and to assess the relative performance capabilities of the main alternative parties in an area that the voter prioritizes” (Sanders et al. Reference Sanders, Clarke, Stewart and Whiteley2011, 290–1). Valence attributes are thus cognitive heuristics that minimize the cost of acquiring detailed information about politics.Footnote 6 This further adds to the appeal of the model as an explanation of voting behavior of the large body of voters who are not especially informed about or motivated by politics. Valence considerations allow politically unsophisticated voters to choose a party.
Why Party Preference May Explain Performance Evaluation
Why might valence not be the major explanatory force that its advocates have purported? As noted above, the idea that party preference provides a basis for assessing competence and performance is well established: “The strength of relationship between party identification and the dimensions of partisan attitude suggests that responses to each element of national politics are deeply affected by the individual's enduring partisan attachments” (Campbell et al. Reference Campbell, Converse, Miller and Stokes1960, 128). Bartels (Reference Bartels2002a) likewise argues that the interpretation of party identification as a “running tally” of past satisfaction with a given political party's policies is untenable, as perceptions of past performance are themselves shaped by pre-existing partisan affiliation. Much ensuing research has established that partisanship can influence responsibility attributions so that favored parties are seen to be responsible for good outcomes but not for negative ones (Abramowitz, Lanoue and Ramesh Reference Abramowitz, Lanoue and Ramesh1988; Feldman Reference Feldman1982; Peffley Reference Peffley1984; Rudolph Reference Rudolph2003; Marsh and Tilley Reference Marsh and Tilley2010).
Following Gaines et al. (Reference Gaines, Kuklinski, Quirk, Peyton and Verkuilen2007), Tilley and Hobolt (Reference Tilley and Hobolt2011) outline the mechanisms through which partisanship can influence evaluations. Voters can refuse to update their beliefs in the face of new information by, for example, disregarding unfavorable information on performance if they support the incumbent. Bartels (Reference Bartels2002a) shows, for example, that many “strong Democrats” thought that inflation had increased during Reagan's presidency, while it had in fact declined precipitously. Voters can also reinterpret a given level of performance in a more positive light (selective evaluation) or attribute responsibility for the level of performance differently (selective attribution).Footnote 7 Inter-election panel analyses have likewise indicated that political preferences shape citizens’ perceptions of national economic performance (Evans and Andersen Reference Evans and Andersen2006; Evans and Pickup Reference Evans and Pickup2010), while other recent research by Ladner and Wlezien (Reference Ladner and Wlezien2007) finds that voters’ expectations about which parties are likely to win forthcoming elections influence their prospective economic assessments. That these biases are consequential beyond the voting booth has also been established by Gerber and Huber (Reference Gerber and Huber2009), who demonstrate that actual economic behavior is consistent with patterns of partisan differences in economic evaluations. Research into ‘motivated reasoning’ (Redlawsk Reference Redlawsk2002; Taber, Lodge, and Glather Reference Taber, Lodge and Glather2001; Taber and Lodge Reference Taber and Lodge2006) likewise implies that information gathering is unlikely to be accurate, as voters seek to bring their observations in line with prior opinions. The resolution of inconsistency in the case of performance evaluation therefore works to preserve longer-term beliefs associated with party preference rather than serve as a neutral information-gathering process, as implied in the valence model.
Evaluating the Empirical Validity of the Valence Model: The Limitations of Cross-Sectional, Within-survey Estimates of Valence Effects
As noted above, the BES valence model assumes that the association between performance measures and vote intention/reported vote, sometimes net of prior vote intention (as in Sanders et al. Reference Sanders, Clarke, Stewart and Whiteley2011), is causal. The literature on party effects on evaluations implies that the direction of effects is highly unlikely to be one way. We therefore need to estimate recursive models. One way of addressing such a need is to attempt to identify relative strengths of the direction of influence between performance evaluations and party choice using two-step procedures with panel data.Footnote 8 That such identification is inevitably arbitrary with cross-sectional data is well known, since it is difficult to find an instrument that is related to political attitudes at time t but is uncorrelated with the unobserved determinants of party preference at the same time (see Schickler and Green Reference Schickler and Green1993). There are also survey context effects that militate against attempting to estimate the strength and direction of effects within surveys.
An extensive literature addresses how the questionnaire context is a source of bias and error in survey research (Schuman and Presser Reference Schuman and Presser1981; Schaeffer and Presser Reference Schaeffer and Presser2003) and how question proximity affects respondents’ interpretation and retrieval of information, which is typically referred to as priming (Tourangeau and Rasinski Reference Tourangeau and Rasinski1988; Tourangeau, Rips and Rasinski Reference Tourangeau, Rips and Rasinski2000). Within the study of electoral behavior there are numerous examples of priming derived from question proximity.Footnote 9 Zaller's influential (Reference Zaller1992) monograph presents robust evidence of question-order effects and question-wording effects. Consistent with much research in cognitive psychology, Zaller and Feldman (Reference Zaller and Feldman1992) have proposed that most people respond to survey questions on the basis of whatever ideas happen to be salient and accessible to them at the time of answering. Such accessibility is conditioned by the content of preceding questions in a survey.
It is problematic, therefore, that the strongest evidential basis for the valence model has typically involved analyzing repeated batteries of questions that ask respondents (1) which party they think is most able to deal with the most important issue, (2) how well the party (first Labour and then the Conservatives) is performing, or would hypothetically perform, in various issue areas, followed by (3) the standard party identity questions and (4) vote intention or reported vote.Footnote 10Figure 1 presents the valence and party questions in a representative wave of the 2005–10 BES internet panel.
Question order and proximity in the BES internet panel questionnaire11
It can be seen that no less than 17 performance evaluation questions, plus three questions about party identification, precede the question on voting intention. There are almost no intervening questions. A battery on ‘leader images’ (affect towards party leaders) follows shortly after the vote intention questions.
The BES questionnaires thus contain an almost continuous stream of party-laden priming questions. These are questions that explicitly ask for an evaluation of a named party (or ‘the government’—the Labour Party at the time). It is thus unsurprising (and potentially uninformative about the motives underlying vote choice) that there should be a close link between answers to questions about party performance that explicitly label the party concerned, party identification and vote intention. The question batteries are so heavily primed with party cues that it is hard to imagine how there could be anything other than a high correlation between these various indicators of political orientation.
Such survey design effects represent a potential source of systematic measurement error, which is likely to seriously bias coefficient estimates. The implications of this bias have been demonstrated previously in research into economic performance evaluations, which are a key element of valence politics. Positioning questions on political attitudes immediately prior to economic evaluation questions influences respondents’ assessments of the national economy and their personal financial situation: it makes them more consistent with their partisan orientations (Sears and Lau Reference Sears and Lau1983). Wilcox and Wlezien (Reference Wilcox and Wlezien1996) find that both political responses and economic evaluations are subject to contamination when related questions are placed in close proximity to each other. Other scholars have found similar priming effects in areas such as foreign policy (Krosnick and Kinder Reference Krosnick and Kinder1990).
Footnote Our concern, then, is that the placement of explicitly party labeled performance evaluations adjacent to party preference questions will artificially inflate the correlation between responses. This hypothesized effect builds on a body of evidence indicating that people prefer to give consistent responses (Abelson Reference Abelson1968; Schwarz and Sudman Reference Schwarz and Sudman1992). Respondents condition their answers on their responses to previous questions. This preference for consistency produces a marked upward bias in the association between answers to proximal questions in social surveys. To the extent that this holds true, asking political questions in a context in which many other questions have cued the expected response is likely to produce an artifactual consistency between respondents’ performance evaluations, and between those and their party preferences. Similarly, partisan priming derived from the content of the question batteries, with a persistent emphasis on which party the respondent thinks is best, can be expected to have made partisanship salient. Notwithstanding the problem of directional indeterminacy implied by the party effects literature referred to above, the strength of association between performance evaluations and party preference cannot be estimated with any confidence from a within-survey, cross-sectional analysis.
Our solution to these two sources of bias in estimating valence effects—the indeterminacy of causal direction and the upward bias of priming and questionnaire proximity—is to use panel studies to simultaneously model the effects of past performance evaluations on current party preference and of past party preference on current performance evaluations, while controlling for prior party preference and performance evaluations, respectively. This approach builds on work that uses cross-lagged structural equation modeling estimation to examine the relative strength of partisanship and issue proximities (Evans and Andersen Reference Evans and Andersen2004), policy preferences (Milazzo, Adams and Green Reference Milazzo, Adams and Green2012; Highton and Kam Reference Highton and Kam2011), economic perceptions (Evans and Andersen Reference Evans and Andersen2006) and core political values (Goren Reference Goren2005; Evans and Neundorf Reference Evans and Neundorf2013). This method provides a more demanding and, in our view, less biased test of the impact of performance evaluations on party preference. It allows us to compare the relative importance of these competing ways of understanding survey responses and to examine whether some sorts of performance evaluations are of greater significance to party preference than others.
Hypotheses
If the BES valence model is valid, we would expect performance evaluations to have a significant lagged, net impact on party preference, but for party preference not to have such a lagged effect on performance evaluations.
Hypothesis 1: The dominant causal relationship is from performance evaluations to party preference, rather than vice versa.
If there are significant party effects, this would imply that the observed association between performance evaluations and party preference is at least in part attributable to reciprocal causal influence, and that the effect estimates in valence models, which assume unidirectional effects, are biased upwards. If partisan conditioning arguments are valid, we would expect party preference to have a significant lagged, net impact on performance evaluations.
Hypothesis 2: Party preference has a significant net effect on performance evaluations.
The use of performance evaluations as simplifying heuristics for people who are “not particularly interested in politics and wish to minimize the cognitive burdens of selecting a party” (Sanders 2011, 290) can be tested quite simply by estimating cross-lagged models separately for voters with different levels of attention to politics. If valence attributes are cognitive heuristics that minimize the cost of acquiring detailed information about politics, we would expect to find that valence effects are present among less politically interested and motivated voters as well as amongst the more attentive.
Hypothesis 3: The net effect of performance evaluations is not conditioned by voters’ levels of attention to politics.
It could conversely be argued that valence voting is not that simple—it requires a high degree of attentiveness to the media and the public debate in order to perceive salient problems (which vary over time), to link them to party competence at different time points, to perceive and evaluate differences among parties when their cabinets change, and to make an electoral choice anew in each election without relying on party identification as a standing predisposition (which is arguably the most cost-effective heuristic of all). If this is the case, we might expect to find that party preference effects are stronger (and valence effects weaker) among less politically attentive voters.
Hypothesis 4: The relative strength of effect of party preference on performance evaluations and vice versa is conditioned by voters’ levels of attention to politics: performance evaluations have weaker effects on party preference among less politically attentive voters, who instead use the heuristic of party preference to infer performance.
Data and Model
We use data from the 2005–10 BES internet panel, which was designed to produce representative samples of the adult population living in private households in Britain.Footnote 12 Three waves were carried out in the 2005 and 2010 election years: before the election campaign, during the campaign and soon after the general election. There were follow-up surveys in 2006, 2008 and 2009, with no survey in 2007. Due to the change in the Labour leadership in 2007, we focus on the last three years of the panel: 2008, 2009 and 2010 (pre-campaign wave).Footnote 13 We also use indicators from the 2006 wave to construct the lagged latent measures for Labour party performance and Labour preference.
To correct for measurement error associated with survey items, we use multiple indicators to estimate the latent constructs of Labour performance evaluation and Labour party preference at each time point.Footnote 14 Latent Labour performance is modeled with four valence indicators: perceptions of the Labour government's handling of the economy, Labour's handling of the NHS,Footnote 15 the Labour leader affect and whether the Labour party is best able to handle the most important issue facing the country. Latent Labour preference is estimated with the Labour party feeling thermometer, the Conservative party feeling thermometerFootnote 16 and Labour vote intention. All indicators are standardized. It is a flexible feature of cross-lagged structural equation models that the errors associated with each indicator are allowed to be correlated across all time points (see Goren Reference Goren2005; Highton and Kam Reference Highton and Kam2011; Milazzo et al. Reference Milazzo, Adams and Green2012).Footnote 17
The four-wave linear dynamic structural equation modelFootnote 18 is presented in Figure 2. We estimate the factor loadings (λk) of each indicatorFootnote 19 (xj or zm) on each latent construct (ξj - 'Labour reference’ or δm - 'Labour performance’, respectively) at each time point t. The estimated stability coefficients (βt for Labour preference and γt for Labour performance) measure the effect of each latent construct in the previous period (t − 1) on this same construct in the current period t. The higher the value of these estimates, the more stable the latent construct is over time. The cross-lagged effect parameters, μt and σt, are the effects of Labour preference in the previous period (t − 1) on Labour performance in the current period (t), and the reciprocal effects of Labour performance (t−1) on Labour preference (t), respectively, for respondent i in period t:
Cross-lagged four-wave structural-measurement model
Notes:
xj observed indicators for latent Labour preference.
zm observed indicators for latent Labour performance.
ξj Labour preference.
δm Labour performance.
βt stability coefficients for Labour preference.
γt stability coefficients for Labour performance.
μt net effect of Labour performance at (t−1) on Labour preference at (t).
σt net effect of Labour preference at (t−1) on Labour performance at (t).
λk unstandardised factor loadings of the observed indicators on the latent constructs.
$${ & \quad Labour\ preferenc{{e}_{it}}\,{\rm{ = }}\,{{{\rm{\ralpha }}}_{\rm{1}}}\,{\rm{ + }}\,{{{\rm{\rbeta }}}_{\rm{1}}}\,Labour\ preferenc{{e}_{i{\rm{(t - 1)}}}}\, \cr & \qquad \qquad \qquad \quad \qquad \quad{\rm{ + }}\,{{\rsigma }_{\rm{1}}}Labour\,\,performanc{{e}_{i{\rm{(t - 1)}}}}\,{\rm{ + }}\,{{{\rm{\rzeta }}}_{\rm{i}}}\, \cr & Labour\ performanc{{e}_{it}}\,{\rm{ = }}\,{{{\rm{r\alpha }}}_{\rm{2}}}\,{\rm{ + }}\,{{{\rm{\rgamma }}}_{\rm{1}}}Labour\,\,\:performance{{t}_{i{\rm{(t - 1)}}}} \qquad\cr & \qquad \qquad \qquad \qquad \qquad {\rm{ + }}\,{{{\rm{\rmu }}}_{\rm{1}}}Labour\,\,preferenc{{e}_{i{\rm{(t - 1)}}}}\,{\rm{ + }}\,{{{\rm{\rtheta }}}_{\rm{i}}} \cr}$$
Importantly, the cross-lagged parameters μt and σt represent the net effects of one construct on another, controlling for the values of each dependent variable in the previous period. Omitting the stability parameters βt and γt would bias the estimates of the cross-lagged effects upwards. Latent factors estimated using the 2006 indicators can have effects on their values at the next time point, but not the cross-lagged effects on the other construct. They are used merely as controls.
Although cross-lagged dynamic structural models offer a considerable improvement on cross-sectional analyses by using the natural time ordering of panel data, they do not solve all potential endogeneity problems. For instance, they do not allow for non-recursive simultaneous effects of Labour preference and government approval within the same wave. To the extent that the simultaneity bias occurs due to priming or context effects arising from questionnaire design, this is not necessarily a weakness of cross-lagged models. However, if the two constructs have purely exogenous simultaneous effects on each other, such reverse causality cannot be disentangled. We can establish ‘Granger causality’ only, by comparing the strengths of their lagged effects on each other, net of their respective stability effects.
Results
Before estimating the full model presented in Figure 2, we evaluate the fit of the four-wave stability-measurement models for Labour performance (Table 1) and Labour preference (Table 2). Observed indicators load very highly on the respective latent factors. Both models fit the data very well, confirming that the chosen observed indicators tap into the underlying latent constructs.Footnote 20 Both models show a high degree of factor stability over time, suggesting that voters’ assessments of government performance, and their preference for Labour, hardly fluctuate from wave to wave. The lowest stability coefficient is for 2006–08 in the performance model, as would be expected given the party leadership change and the onset of the global financial crisis in the intervening period.
Labour Performance Assessments Stability Model
Note: maximum likelihood estimates, with robust standard errors. All parameter estimates significant at p < 0.001, unless indicated otherwise. RMSEA = Root Mean Square Error of Approximation. Factor variances, error variances, error covariances, intercept estimates and disturbances omitted for clarity. All indicators are standardized.
Labour Preference Stability Model
Note: maximum likelihood estimates, with robust standard errors. All parameter estimates significant at p < 0.001, unless indicated otherwise. Factor variances, error variances, error covariances, intercept estimates and disturbances omitted for clarity. All indicators are standardized.
Table 3 presents the cross-lagged and stability estimates for the structural-measurement model in Figure 2, with the factor loadings omitted for clarity. Both latent constructs were remarkably enduring between 2006 and 2008, 2008 and 2009, and 2009 and 2010, although the stability estimates were significantly larger for Labour preference than for performance assessments. This finding suggests that the feelings about the majority party were, on the whole, more stable over time than the evaluations of government performance. The estimated cross-lagged structural coefficients of 0.33 (p < 0.001) and 0.31 (p < 0.001) indicate that performance evaluations and party preference, respectively, both had statistically significant positive effects on each other between 2008 and 2009, net of their stability effects. This is evidence of a reciprocal relationship between the two constructs during this period. Thus respondents who held more positive views of the incumbent government's performance in 2008 reported a substantially stronger preference for Labour in 2009, on average, controlling for their prior support for Labour. These reciprocal effects are not statistically significantly different from each other, suggesting that between 2008 and 2009, neither performance evaluations nor party preference dominated. In contrast, the lagged effect of performance evaluations on party preference is no longer significant in the 2009–10 period, while party preference continues to have a significant positive lagged effect on performance evaluations with the estimated coefficient of 0.37 (p < 0.001). The difference between the two cross-lagged estimates is significant at p < 0.05, suggesting a stronger party conditioning effect at the end of the electoral cycle.Footnote 21 Overall, the finding of significant net effects of party preference on performance evaluation in both periods (2008–09 and 2009–10), even after controlling for prior performance evaluations, gives support to Hypothesis 2 but not Hypothesis 1.
Cross-lagged Labour Preference and Performance Model
Note: maximum likelihood estimates, with robust standard errors. All parameter estimates significant at p < 0.001, unless indicated otherwise. Factor loadings, factor variances, error variances, error covariances, intercept estimates and disturbances omitted for clarity. All indicators are standardized.
To check whether the performance model is disproportionately driven by leader assessments, we excluded the leader affect indicators in the measurement part (Table 4). The estimated effect of party preference in 2008 on government performance in 2009 is 0.37 (p < 0.001), compared with the reciprocal effect of lagged government performance on party preference of 0.20 (p < 0.001). The difference between these two coefficients is statistically significant at p < 0.01, suggesting that if leader evaluations are excluded from the model, the effect of party preference dominated in this period. In the main model in Table 3, in which leader evaluations were included as indicators of government performance, the reciprocal effects of party preference and government performance are not significantly different from each other. During the 2009–10 period, party preference continues to have a significant positive lagged effect on performance evaluations, while the reciprocal effect is still statistically indistinguishable from zero. Thus, Labour preference now has significantly larger cross-lagged effects on performance evaluations than the reciprocal effects of performance assessments on Labour preference in both 2008–09 and 2009–10. These findings indicate that leader evaluations strengthen the overall effect of the valence indicators on party preference.Footnote 22 Thus it would appear that leader evaluations are less likely to be conditioned by party support than competence measures.Footnote 23
Cross-lagged Labour Preference and Performance Model (Without Leader Indicators)
Note: maximum likelihood estimates, with robust standard errors. All parameter estimates significant at p < 0.001, unless indicated otherwise. Factor loadings, factor variances, error variances, error covariances, intercept estimates and disturbances omitted for clarity. All indicators are standardized.
To evaluate Hypothesis 3, we re-estimate the original model (including leader assessments) separately for respondents who reported higher attention to politics in the 2005 pre-campaign wave (7–10 on the 0–10 scale)Footnote 24 and those who reported lower attention to politics. Table 5 shows the estimates for the sub-sample of the respondents with higher values of reported attention to politics. The cross-lagged (structural) coefficients are very similar to those based on the full sample: both performance assessments and Labour preference have significant positive reciprocal lagged effects of comparable size on each other in 2008–09. In 2009–10, only Labour preference has a significant lagged effect on the other construct. The effect of Labour preference is also significantly larger (p < 0.01) than the reciprocal effect of performance evaluations.
Cross-lagged Labour Preference and Performance Model (High Attention to Politics)
Note: maximum likelihood estimates, with robust standard errors. All parameter estimates significant at p < 0.001, unless indicated otherwise. Factor loadings, factor variances, error variances, error covariances, intercept estimates and disturbances omitted for clarity. All indicators are standardized.
In the model estimated on the sub-sample of respondents with lower self-reported attention to politics (Table 6), none of the cross-lagged effects is statistically significantly different from zero, net of the stability effects. The stability coefficients themselves are large and significant, suggesting that respondents with lower attention to politics are less likely to update their performance assessments and party feelings from year to year, and rely on their previous political attitudes. This does not support Hypothesis 4, in which less politically attentive respondents were predicted to use party preference as a heuristic for responding to questions about performance. Neither, however, is it consistent with Sanders et al.'s (2011) argument that valence considerations provide an effective, simple heuristic for updating party preference among those who are not strongly motivated by (or interested in) politics (Hypothesis 3).
Cross-lagged Labour Preference and Performance Model (Low Attention to Politics)
Note: maximum likelihood estimates, with robust standard errors. All parameter estimates significant at p < 0.001, unless indicated otherwise. Factor loadings, factor variances, error variances, error covariances, intercept estimates and disturbances omitted for clarity. All indicators are standardized.
It should be noted that none of the estimated models so far includes party identification. The inclusion of Labour party identity as an observed exogenous factor in the structural model would lead to multicollinearity and, therefore, inflated standard errors for all structural estimates. Alternatively, including party identity in the measurement part of the model as an indicator for both latent performance assessments and latent party preference precludes model identification. Since the valence model (Sanders et al. Reference Sanders, Clarke, Stewart and Whiteley2011) uses party identity alongside government competence and leadership assessments as a predictor of vote, it could be entered as an indicator of latent performance evaluations in our model. However, party identification can also be conceptualized as a long-standing commitment to a party (for example, Campbell et al. Reference Campbell, Converse, Miller and Stokes1960). Consistent with this latter view, we find that party identification taps effectively into the latent party preference,Footnote 25 along with the party feeling thermometers and vote intention.
The Labour preference measurement-stability model—identical to the model in Table 2 but including the strength of Labour party identificationFootnote 26 as the fourth indicator—fits the data well (see Table A1 in the Annex). Importantly, it fits better than the comparable Labour performance model with party identity as the fifth indicator (Table A2). Thus if we were to have included party identity in the analysis it would have been most appropriate to have included it as an indicator of Labour preference in the cross-lagged models. When operationalized in this way, Labour preference has a significant cross-lagged effect on performance assessment in both periods, but performance assessment has a reciprocal effect only in 2008 (see Table A3). This pattern replicates that observed in Table 3, in which party identity was not included as part of the party preference factor.
Conclusions
We have examined whether the direction of association postulated in Clarke et al.'s valence model of British voting from performance, competence and leader evaluations to party choice is empirically valid. Our findings indicate that in the earlier part of the electoral cycle, the observed association between competence measures and party preference is a consequence of their reciprocal influence, but the effect of latent party preference clearly dominates toward the end of the electoral cycle. Thus controlling for party preference in 2008, we find that performance evaluations in 2008 have a significant positive effect on party preference in 2009. Yet we also find a significant positive reciprocal effect of a similar size of past party preference on current performance evaluation in the same period. The lagged effect of performance evaluation in 2009 on party preference in 2010 is not significant, however, while past party preference continues to exert a substantial positive effect on current performance evaluation in 2010. As would be predicted from the literature reviewed in the introduction on the impact of party preference on performance evaluations, assessments of government performance are, to a significant degree, a product of party preference and cannot therefore be used to explain that preference. These estimates are shown to be robust to alternative specifications of observed indicators, such as omitting leader evaluations from the performance assessment model or including Labour party identification in the Labour preference model.
We additionally address the assumption in the BES valence model that government and leader evaluations are heuristics that voters use to minimize the cognitive burden of choosing a party. This implies that performance assessment effects should be observed among less attentive voters as well as the more attentive. We find, however, that respondents with lower self-reported attention to politics simply do not update their government competence assessments and party preference from one year to another. This suggests that, rather than acting as a cost-minimizing heuristic, competence and performance assessments require a higher degree of attentiveness to current affairs than is consistent with the general applicability of the valence model. The results for those who reported higher levels of attention to politics are in line with the estimates for the full sample, in that they are more likely to re-evaluate government and leader performance every year. However, as with the full sample, these assessments are strongly conditioned by prior party support.
These findings are inevitably subject to qualifications. First, we have only considered the 2005–10 British electoral cycle.Footnote 27 This was a period of sharp decline in popular support for the party in government as well as a time of significant economic turbulence, which may limit the generalizability of the results.Footnote 28 At the same time, analyses of this electoral cycle form the basis of claims concerning the predictive superiority of the valence model in the most recent publications on this topic (Clarke et al. Reference Clarke, Sanders, Stewart and Whiteley2011; Sanders et al. Reference Sanders, Clarke, Stewart and Whiteley2011). The BES valence model is believed to be no less valid in this electoral cycle than it is in other cycles. Our findings are therefore likely to be of general relevance with respect to the validity of the assumptions and procedures used to estimate valence effects.
Secondly, although the use of lagged models is an improvement on the strong assumptions of simultaneous, unidirectional causality embodied in cross-sectional analyses, they do not offer an irrefutable test of exogeneity of performance assessments and party preference. Cross-lagged models are becoming increasingly popular, but they come with an important limitation: only lagged effects can be estimated, with no allowance for simultaneous non-recursive effects. Thus, at most we can establish ‘Granger causality’ (Highton and Kam Reference Highton and Kam2011); party preference is said to cause performance assessments at a point in time because there is a significant association between prior party preference and current performance assessments, controlling for prior performance assessments. Given the evidence of endemic context effects in surveys, however, this limitation is not necessarily a weakness. As discussed in the introduction, there is strong evidence that simultaneous effects in surveys are powerfully influenced by the measurement context.
Our view is then that the use of lagged structural equation modeling, with multiple survey items measuring the latent constructs of interest, has provided more realistic estimates of the effects of performance evaluations than has hitherto been the case. In contrast, the measurement approach that has been adopted by Clarke et al. does not support the inferences drawn regarding its superiority vis-à-vis other approaches to explaining party choice. Cross-sectional models seriously over-estimate the magnitude of the impact of performance evaluations on party preference. This derives in part from misinterpretation of the direction of effects between performance evaluations and party preference, but it is also clear that the demands of the BES valence model—with respect to voter updating and use of performance as a heuristic—are also incorrect. The degree to which this is a measurement problem or a theoretical problem cannot be ascertained with certainty. It is possible that ‘purer’, less contaminated measures of performance evaluations have stronger influences on party preference than vice versa. However, to the degree that the BES valence theory is empirically disconfirmed, it cannot be taken as “a powerful, possibly unassailable, explanatory account of the overall pattern of party choice” (Johnston Reference Johnston2011, 213).
It should be made clear that the re-analysis and critique presented in this article relates most precisely to the operationalization of valence advanced with considerable influence within the discipline, particularly in Britain, by Clark et al. At the same time, we might want to consider what it suggests for empirical models of valence theories more generally. We would suggest, first, that if valence theory is to make progress, studies that implement its empirical implications need to take into account the effects of question proximity on estimates of valence effects. The best strategy for dealing with such context effects is to include unrelated questions, so called “buffer items”, between the questions for which associations between responses are being estimated. A further, particularly problematic feature of the operationalization of valence examined here is the extensive explicit cueing of parties in the content of questions. Voters may well be able to evaluate issue competence without using party as a heuristic, but unless questions are devised that remove partisan cueing, we cannot know if they do or not.Footnote 29
Valence theories might also benefit from refining their assumptions about how voters evaluate competence. An innovative though as yet unpublished study (Reference Wagner and ZeglovitsWagner and Zeglovits forthcoming) using cognitive interviewing procedures seems to indicate that the demands of the valence model on voter sophistication are such that refocusing on generalized party competence evaluations as advocated by Green and Jennings (Reference Green and Jennings2012a),Footnote 30 rather than issue-specific evaluations, might be a productive strategy. This would help to address, in particular, the failure of less attentive respondents to update their issue competence evaluations and link them with their party preference in ways the theory has assumed. Similarly, studies of issue salience (Johns Reference Johns2010) and leadership evaluations (Evans and Andersen Reference Evans and Andersen2005) suggest that the latter is a more robust source of influence on party preferences than the former, a finding that is also echoed in our analysis. Leadership effects are more robust to the tests we have presented in this article than are party performance evaluations. A related study (Evans and Chzhen Reference Evans and Chzhen2013) of defection from Labour during the 2005–10 electoral cycle likewise supports the argument that leadership effects are not spurious and do, at least under some conditions for some leaders, impact voter behavior. Rather than treating valence as a multiply defined construct, it could therefore be more effective in future research to examine the differential impact of its distinct components and discover which ones ‘do the work’.
As a final, more general point, we should note that the inferential problems produced for survey researchers by the survey context are not confined to valence models (though these models do appear to be particularly vulnerable to concerns about partisan labeling). Questions of endogeneity and causal direction bedevil all models that measure subjective independent variables (for example, issue positions and issue proximity, ideological positions and proximity, values, partisan identity) in the same survey as dependent variables (for example, vote intention, reported vote). Raising the hurdle for assessing the validity of models of political choice more generally should, in time, put our explanations of voting on a stronger empirical footing.
Annex
Labour Preference Stability Model (With Labour Party ID)
Labour Performance Assessments Stability Model (With Labour Party ID)
Cross-lagged Labour Preference and Performance Model (With Labour Party ID as an Indicator of Labour Preference)
Cross-lagged Labour Preference and Performance Model (2005–06)
