When do politicians seek out expert information on policy?Footnote 1 In theory, politicians should have at least two broad motivations for seeking out information. On the one hand, they may view themselves as policy-makers (Docherty Reference Docherty1997). The policy-making politician may make policy via a formal position – for example, as a cabinet minister. They may alternatively make policy as an entrepreneur, by persuading other elites or voters about the merits of some policy change (Kindgon 1997 [Reference Kingdon2003]). In either scenario, they should benefit from objective information about policy options. On the other hand, politicians may view themselves as persuaders, tasked with selling policy set by others (Grose et al. Reference Grose, Malhotra and Van Houweling2015; Malloy Reference Malloy1996; McGraw Reference McGraw1991). In this instance, knowing experts’ views on issues will make it easier for them to sell their own party’s position. Overlaying all of this is the reality that on some issues politicians will have an easy sell – voters will largely be in agreement with them and promoting their party’s position will be straightforward. On other issues, they will be far offside constituency opinion and will face a more difficult task in selling their own or their party’s position. We might expect politicians to be even more likely to seek out helpful information when this is the case.
In this paper, we explore whether politicians seek out more information about an issue when they are farther offside the average opinion in their constituency on that issue. To do so, we conducted a field experiment with 334 Canadian Members of Parliament (MPs). In the midst of a contentious national debate on federal government support for the oil industry, we invited MPs and their staff to watch a webinar or read a written summary of the webinar. The webinar was created by a school of public policy and contained a variety of viewpoints on the future prospects of oil extraction in Canada. For politicians on either side of this issue, the information could prove useful in future debate and conversation. Within our invitations, some politicians were randomly assigned to information about the distribution of opinion in their constituency on the issue of whether the government should be involved in actively helping the resource sector, including in the construction of pipelines.
Our analysis follows closely the logic of Butler and Nickerson (Reference Butler and Nickerson2011), in that we estimate the effect of constituency opinion on an MP taking action, conditional on the MP receiving information about that constituency opinion, though we focus more purposively on the degree to which opinion disagrees with a politician’s party position. We find that neither are politicians in the control group likely to be responsive to constituency disagreement in seeking information nor are those who receive a prime about constituency opinion more likely to seek out information conditional on disagreement. In short, politicians who are offside their constituency opinion do not appear more likely to seek out expert information on contentious policy issues.
In what follows, we provide some theoretical intuitions, followed by background information for our experiment. We describe our experimental design and present our results. We then note the limitations of our study and next steps.
Theoretical intuitions
Our experiment is guided by four simple theoretical intuitions. First, elected politicians have multiple roles to play – among them as policy-makers and as representatives of their parties (Docherty Reference Docherty2005; Fenno Reference Fenno1973; Franks Reference Franks1987). The degree to which they will play both roles varies both individually and systematically. A politician who sits on a policy-focused committee will have more opportunities to act as a policy-maker than one who does not. This is especially so if the politician is in the governing party. At the political system level, the extent to which politicians can influence actual policy development will depend on the openness of the policy-making system, the extent of party discipline, and the strength of the executive (e.g., Lijphart Reference Lijphart2012; Russell et al. Reference Russell, Gover and Wollter2016). As representatives of their parties, some individuals will be better persuaders than others and some will devote more resources to policy persuasion than others. This could vary based on their own skills or their marginality. At a systemic level, the degree to which politicians have to act as apologists for their party’s positions will depend on the accountability calculus in their own country, and on the degree to which their party is seen as responsible for policy-making. What matters most for our experiment is not the variation, however, but that each politician is likely to view their own role as at least somewhat about policy-making and somewhat about being a representative of their party’s positions.
Second, politicians at least sometimes want expert information on the policy issues before them. Expert information serves several functions, not least to help policy-makers understand the nature of a policy problem, the link between policy instrument and outcome, and the range of alternative policies that could have been chosen (Quirk and Bendix Reference Quirk, Bendix, Schickler and Lee2011). Moreover, understanding expert opinions on issues allows politicians to draw on arguments when they are in agreement with their position and develop responses against these arguments when they are counter to their position.
Third, politicians will sometimes know the policy preferences of their constituents, but these estimates are likely to be both biased and to have some error and/or uncertainty associated with them (Broockman and Skovron Reference Broockman and Skovron2018).
Fourth, politicians are responsive to constituency opinion (e.g., Soroka et al. Reference Soroka, Penner and Blidook2009), such that ceteris paribus, they are likely closer to their constituency’s mean opinion and they would like to know when they are likely to be perilously distant from constituency opinion.
Taken together, these four intuitions support the two principal hypotheses of our experiment. First, MPs whose political positions are farther away from their constituency mean will be more likely to seek out information on “offside” issues. Second, this will be more likely among MPs who are given accurate information about the distribution of opinion in their constituency. Below, we describe an experiment that tests both of these propositions.
Background on the case
Our experiment is centered around an ongoing issue of political contention in Canada: Federal government support for resource development, especially the extraction of oil and its export via pipelines. Canada is an oil-rich country, with reserves estimated in the top five globally. However, much of this oil is “landlocked,” in two senses. First, it exists not in free-flowing wells, but in “tar sands,” which require expensive and environmentally costly extraction technologies. Both of these increase the global barrel cost at which extracting oil is profitable. Second, the oil is mostly contained in the province of Alberta, which is a substantial distance from tidewater. To export this oil to refineries or end users requires either expensive and dangerous rail transportation or pipelines. The development of the oil sands and the pipelines necessary for their economic viability has been a major issue of political debate in Canada in the last decade.
Generally speaking, the leadership of each political party lines up on one side of the issue of energy extraction and pipeline construction. The Conservative Party and the Liberal Party of Canada both support the oil sands and the development of pipelines. The New Democratic Party (NDP), Bloc Québécois, and the Green Party all oppose this development.Footnote 2
There is heterogeneity within each party, however, at the level of their provincial counterparts and individual MPs. The Liberal Party is generally a broad party, with so-called business Liberals supporting the resources industry, while left or social Liberals support greater environmental restrictions. The NDP likewise houses heterogeneous opinion, as its labor wings are supportive of the jobs created through this industry, while its environmental and social justice wings oppose oil extraction. As importantly, there is substantial regional variation across the country, which is reflected in the distribution of opinions within each party (see Figure 1, below). The consequence of this is that it would be difficult to ascribe with certainty to any constituency the distribution of opinion or mean view based solely on its geographic location and the party of its MP. MPs face some uncertainty about their constituents’ opinions.
The distribution of constituency opinion on federal assistance for energy development, by party and overall.
Experimental design
In May 2017, the University of Toronto School of Public Policy and Governance hosted a webinar about energy policy in Canada. It featured presentations from three academic experts, who also took audience questions. In May 2018, we posted online – via a Qualtrics survey – a video recording and written summary of this webinar.Footnote 3 We then invited MPs and their staff to access these materials through the survey. Bilingual invitations were sent by email to the offices of 334 of the 338 MPs in the Canadian House of Commons.Footnote 4 The invitations reminded MPs about the ongoing uncertainty surrounding the proposed expansion of the Trans Mountain Pipeline and offered a clickable link to the webinar materials.Footnote 5
An experimental treatment was embedded in the text of the email invitation.Footnote 6 We randomly assigned MPs to either a treatment group or a control group. MPs in the treatment group received information on their constituents’ policy preferences with respect to federal assistance for energy development, including oil pipelines. We obtained and computed these constituency-level preferences from a publicly available online survey of Canadians conducted by the Local Parliament Project (LPP) during the 2015 general election (Loewen et al. Reference Loewen, Rubenson and Koop2018). The 2015 LPP survey was the largest ever election study of Canadians and its sample size allowed us to aggregate individual-level responses into unusually precise constituency-level estimates. MPs in the control group did not receive any constituency preference data as part of their invitations. The texts of the invitations – both treatment and control – are reported in the supplementary material. An important aspect of the experiment is the timing of the study. Emails were sent to MPs on May 28, 2018, a few days before the well-publicized “deadline” for a decision on the Trans Mountain Pipeline expansion issued by Kinder Morgan, Canada, after which the company said it would walk away from the project. In other words, the stakes were high and the issue was very much on the political agenda.
At the level of the individual survey respondent, our measure of support for federal energy development assistance was a five-point scale of disagreement/agreement with the statement: “The federal government should do more to help Canada’s energy sector, including building oil pipelines.” This scale was coded from −2 to 2 (with 0 being “neither agree nor disagree”) and a mean was computed for each constituency.Footnote 7 The mean number of survey respondents per constituency was 102 (SD = 37). As we describe below, we later use this constituency-level measure of support for energy development to construct a measure of the degree to which each constituency disagreed with the party position of their MP on the matter.
Figure 1 shows the distribution of constituency support for federal energy development assistance. We note two points here. First, there is an association between constituency-level opinion and the party of the MP. As one might expect, constituencies represented by Conservative MPs are, on average, more supportive of energy development than constituencies with MPs from other parties (p < 0.05). Second, with the exception of the one-MP Green Party, there is a range of constituency opinions within all of the parties. This implies there are likely to be cross-pressured MPs in each party and that MPs from all parties face some degree of uncertainty about opinion in their constituency.
Drawing on the theoretical intuitions outlined in the previous section, we set out two hypotheses:
H1: MPs/their staff are more likely to access the webinar materials if the average preference of their constituents differs from the policy of their party.
H2: The relationship described in H1 will be stronger among those MPs/staff that received the treatment information – that is, where the average constituency preference was provided in the email invitation.Footnote 8
Put differently, we expect that cross-pressured or “offside” MPs are more likely to access the webinar material (H1) and that this relationship will be accentuated among the randomly assigned subset of MPs who were provided with clear estimates of their constituency’s aggregate opinions (H2). We preregistered our hypotheses and analysis plan with the Evidence in Governance and Politics (EGAP) design registry.
The outcome of interest is whether – upon receiving an invitation – an MP or a member of their staff accessed the webinar materials. MPs/staff were identified as having done so using unique survey links generated from Qualtrics.Footnote 9 A binary indicator of whether the MP/staff accessed the material serves as our dependent variable. A total of 16 MPs’ offices accessed the webinar material.Footnote 10
To test our hypotheses, we use two explanatory variables. The first is a continuous measure of the degree to which an MP’s party and their constituency disagreed about federal assistance for energy development. To obtain this measure, we started with the constituency mean scores. Since higher scores on this scale indicate greater support for energy development, we use this as our measure of disagreement for MPs from parties that opposed the expansion of the Trans Mountain Pipeline: namely, for the NDP, the Bloc Québécois/Groupe parlementaire québécois, and the Green Party. For parties that supported the Trans Mountain expansion – that is, the Liberals and the Conservatives – we flipped the scale by multiplying by −1 so as to arrive at a measure where higher scores indicate greater disagreement between constituency and party.Footnote 11 The second explanatory variable is a simple binary indicator of whether the MP received a control invitation or a treatment invitation.
Results
Table 1 reports two sets of OLS regression estimates.Footnote 12 Model 1 regresses the binary indicator of whether the MP/staff accessed the webinar material on our measure of constituency party disagreement, as well as on the indicator of whether the MP/staff received the treatment of constituency opinion embedded in their invitation. The Disagreement coefficient is not significant in a one-tailed test at the p < 0.05 level. In other words, we find no evidence to support our expectation in H1 that cross-pressured MPs are more likely to access the webinar materials. Indeed, if anything, this relationship runs in the opposite direction: The greater the disagreement between an MP and their party, the less likely they were to seek out the expert information provided in our webinar materials. Although we did not have expectations about the direct effect of the treatment, Model 1 shows that it is also statistically nonsignificant.
OLS estimates of the impact of constituency party disagreement on the probability of accessing webinar material
NOTE: Standard errors in parentheses.
*p < 0.05, **p < 0.01, ***p < 0.001.
Model 2 shows the estimates of the impact of constituency party disagreement conditional on the treatment. Thus, here the Disagreement coefficient captures the impact of constituency party disagreement in the control condition. The Treatment × Disagreement coefficient estimates the difference between the impact of disagreement in the treatment condition and its impact in the control condition. While the estimated coefficient for this interaction term is positive – meaning that the impact of disagreement is less negative in the treatment condition than in the control condition – it is not significant at conventional levels. In short, we find no evidence to support H2.
In the supplementary material, we investigate three alternative operationalizations of constituency party disagreement.Footnote 13 We also produced two additional sets of model estimates: OLS models that include covariates for MPs’ party and province, and rare events models that account for the infrequency of ones in our binary-dependent variable (King and Zeng Reference King and Zeng2001).Footnote 14 These additional model estimates do not substantively change our conclusions.
Discussion and conclusions
We designed an experiment to test whether politicians seek out more information about an issue when they are more offside of constituency opinion. Our study returns null results. As we show in the supplementary material, we believe this null effect is real. MPs do not appear more likely to want or look for expert information when their party position is more distant from constituency opinion. This is so even when politicians are shown normally inaccessible local opinion data. We think the finding has important implications for understanding the conditions under which policy-makers will choose to become better informed.
Our results have three notable limitations and we anticipate further studies to address these limitations. First, questions could be raised about the attractiveness of the webinar and, perhaps its neutrality. Seeking out other instruments of information in a future experiment could be useful. Second, we may have conducted our experiment on an issue for which simply too much information already exists, making a low uptake more likely. An equally pressing but more novel or technical issue may be better for a replication. Finally, there is the possibility that effective divisions of labor and expertise mean that an experiment such as this will always lead to null findings, unless it is able to identify and test effects on the very small number of politicians tasked by their parties with being experts on any individual issue. That possibility is a serious challenge to our research design but could be addressed in a future experiment.
Each of these limitations underscore the challenge of elite experiments. The ever-present demands on politicians’ time mean that their participation in any one event or academic study is likely to be modest – a constraint also evident in other elite survey work and in more qualitative, interview-based research involving elite respondents. Scholars are well advised to take such limitations into account when designing future experiments that involve politicians as participants.
Despite these limitations, we believe our findings do present important new insights showing the lack of propensity among representatives to seek out nonpartisan expert information on a salient issue at a time when the issue was coming to a head. Moreover, this hesitance to take on information is present even when politicians are reminded their constituents may on average be far from their party on the issue – a situation that intuitions about representative democracy and electoral incentives imply ought to lead to the opposite effect. This has important implications for our understanding of the functioning of parliamentary representation.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/XPS.2020.42.
