Hostname: page-component-6898cfcb55-th4rj Total loading time: 0 Render date: 2026-11-11T12:43:59.790Z Has data issue: false hasContentIssue false
  • English
  • Français

The Minimal Persuasive Effects of Campaign Contact in General Elections: Evidence from 49 Field Experiments

Published online by Cambridge University Press:  28 September 2017

JOSHUA L. KALLA and
DAVID E. BROOCKMAN
JOSHUA L. KALLA*
Affiliation:
University of California, Berkeley
DAVID E. BROOCKMAN*
Affiliation:
Stanford Graduate School of Business
*
Joshua L. Kalla is a Graduate Student, Department of Political Science, University of California, Berkeley. kalla@berkeley.edu, http://polisci.berkeley.edu/people/person/joshua-kalla.
David E. Broockman is an Assistant Professor, Stanford Graduate School of Business. dbroockman@stanford.edu, https://people.stanford.edu/dbroock/.
Rights & Permissions [Opens in a new window]

Abstract

Significant theories of democratic accountability hinge on how political campaigns affect Americans’ candidate choices. We argue that the best estimate of the effects of campaign contact and advertising on Americans’ candidates choices in general elections is zero. First, a systematic meta-analysis of 40 field experiments estimates an average effect of zero in general elections. Second, we present nine original field experiments that increase the statistical evidence in the literature about the persuasive effects of personal contact tenfold. These experiments’ average effect is also zero. In both existing and our original experiments, persuasive effects only appear to emerge in two rare circumstances. First, when candidates take unusually unpopular positions and campaigns invest unusually heavily in identifying persuadable voters. Second, when campaigns contact voters long before election day and measure effects immediately—although this early persuasion decays. These findings contribute to ongoing debates about how political elites influence citizens’ judgments.

Information

Type
Research Article
Information
American Political Science Review , Volume 112 , Issue 1 , February 2018, pp. 148 - 166
Copyright
Copyright © American Political Science Association 2017 

Political elites can easily manipulate Americans’ political choices: this is the conclusion of a great deal of academic research and popular commentary (see Druckman Reference Druckman2004a; Issenberg Reference Issenberg2012; Jacobson Reference Jacobson2015; Lenz Reference Lenz2012). By its telling, Americans’ political judgments are susceptible to framing, priming, and other forms of influence political elites wield when they advertise to and contact voters. Understanding the effects of elite communication on Americans’ choices has important implications for theories of public opinion, polarization, democratic competence, and campaign finance. For example, in the case of framing effects, as Druckman (Reference Druckman2001, 226) reviews, many scholars conclude that “elites often use framing to manipulate citizens’ judgments.”

Nowhere would the implications of Americans’ susceptibility to such “elite manipulation” of their judgments be more theoretically and substantively significant than in their candidate choices in general elections. Americans voting in general elections determine the balance of power in Congress and state legislatures. They decide whether to grant incumbents an advantage. They decide whether to reward politicians who have focused on raising money for advertising instead of other activities. They pick which party controls the White House. And the legislators who cast deciding votes on major legislation are disproportionately accountable to general electorates.

How susceptible are American voters’ choices in general elections to influence from political elites in the form of campaign contact and advertising? It is surprisingly unclear on the basis of existing evidence. Reviews reach opposite conclusions, with some arguing that “the prevailing scholarly consensus on campaigns is that they have minimal effects,” (Brady, Johnston, and Sides Reference Brady, Johnston, Sides, Brady and Johnston2006, 4) and others indicating that many scholars believe “campaigns fundamentally shape voters’ decisions” (Druckman Reference Druckman2004b, 577).Footnote 1 If one consensus has been reached, it is that there is a dearth of studies in existing literature that credibly identifies causal effects (Brox and Shaw Reference Brox, Shaw, Katz and Crotty2009; DellaVigna and Gentzkow Reference DellaVigna and Gentzkow2010; Jacobson Reference Jacobson2015).

Speaking to enduring debates about the susceptibility of voters to elite persuasion, we analyze results from 49 field experiments on the persuasive effects of campaign contact and advertising. All these experiments rigorously estimate the effects of real campaigns’ choices about which voters to persuade and how to persuade them in the context of real elections. We find:

  • • The best estimate for the persuasive effects of campaign contact and advertising—such as mail, phone calls, and canvassing—on Americans’ candidate choices in general elections is zero. Our best guess for online and television advertising is also zero, but there is less evidence on these modes.

  • • When campaigns contact voters long before election day and measure effects immediately, campaigns often appear to persuade voters. However, this early persuasion decays before election day and the very same treatments usually cease working close to election day. This suggests political scientists and practitioners should consider whether an experiment was run close to an election when attempting to generalize its findings.

  • • Campaigns can sometimes identify pockets of persuadable voters, but even this only appears possible in some elections and when campaigns conduct within-cycle field experiments to identify responsive subgroups.

  • • We find campaigns are able to have meaningful persuasive effects in primary and ballot measure campaigns, when partisan cues are not present.

  • • Our evidence is silent on several questions. It does not speak to the effects of candidates’ qualities, positions, or overall campaign “message.” It does not indicate the optimal allocation of campaign spending across voter registration, get-out-the-vote, and persuasion efforts. It also remains possible campaigns could develop more effective persuasive messages. Future experimental research should consider these questions.

We contextualize these findings in a theoretical argument that draws on theories of partisanship and political communication to argue that when a partisan cue and competing frames are present, campaign contact and advertising are unlikely to influence voters’ choices. We present two forms of evidence that support this argument. First, we present the first meta-analysis of the emerging field experimental and quasi-experimentalFootnote 2 literature on campaign contact and advertising. Such evidence was once rare and the many studies that have now been done are often imprecise on their own. However, enough such evidence has been reported in recent years to conduct a relatively precise meta-analysis. This meta-analysis estimates that campaign contact and advertising can have persuasive effects in primaries and in ballot measure elections. However, their effects on election day in general elections are essentially zero. These results are robust across elections at every level of government and in both competitive and uncompetitive elections (terms we define below).

Our meta-analysis surfaced a surprising dearth of statistically precise studies that examine the effects of personal contact from campaigns, such as phone calls and face-to-face conversations, which could be expected to have the largest persuasive effects.Footnote 3 Therefore, our second empirical contribution is a series of original studies we conducted in partnership with a national door-to-door canvassing operation in 2015 and 2016. These studies all focused on measuring the effects of in-person, door-to-door persuasive canvassing in general elections, a common strategy (Enos and Hersh Reference Enos and Hersh2015). Exploiting recent advances in experimental design (Broockman, Kalla, and Sekhon Reference Broockman, Kalla and Sekhon2017), these studies are unusually precise: together, our original studies increase the amount of statistical evidence in the literature about the persuasive effects of personal contact in general elections by over tenfold. Nearly all these studies also found a zero effect on which candidates voters supported on election day.

Does campaign contact ever persuade voters in general elections? Both our meta-analysis and our original studies suggest two caveats to our otherwise consistent finding of null effects.

First, we find an intriguing pattern whereby campaign contact in general elections appears to have persuasive effects if it takes place many months before an election, but that these effects decay before election day. However, when these same tactics are deployed closer to election day, they do not even have immediate effects. We show this pattern both in aggregate and in the context of four studies where there is variation in the timing of both campaign contact and outcome measurement. In all these cases, we only see effects of campaign contact in general elections when voters receive contact far before election day and outcomes are measured immediately. But these effects are typically illusory: as election day approaches, the effects of early campaign contact and advertising decay and the immediate effects of subsequent contact and advertising almost always go to zero.

Can campaign contact in general elections ever have persuasive effects that matter on election day? In the existing literature and in our original studies, we also find that campaigns appear able to have persuasive effects in circumstances in which candidates take unusually unpopular positions and opposing campaigns invest unusually heavily in identifying persuadable, cross-pressured voters whom they can inform about these positions (Hersh and Schaffner Reference Hersh and Schaffner2013; Rogers and Nickerson Reference Rogers and Nickerson2013). In these cases, identifying cross-pressured persuadable voters requires much more effort than simply applying much-ballyhooed “big data” (Endres Reference Endres2016; Hersh Reference Hersh2015). For example, the organization we partnered with on our original studies conducted large-scale field experiments early in the electoral cycle in several states to identify subgroups of persuadable voters that were difficult to predict ex ante. They then shifted resources to focus on persuading these voters—a strategy that the data we present below suggests was successful. This strategy only appears able to find subgroups of persuadable voters in some elections, however, and can only be executed by campaigns with considerable resources and sophistication.

These findings are consistent with our theoretical argument that campaigns can provide new considerations or increase the salience of certain considerations before an election campaign is active, but that such effects nearly always diminish when competing frames and clear cues (such as partisanship and candidate attributes) are available. Voters in general elections appear to bring their vote choice into line with their predispositions close to election day and are difficult to budge from there (e.g., Gelman and King Reference Gelman and King1993). Supporting this interpretation, we also do not find clear evidence of generalizable subgroup effects, nor that persuasive campaigns have heterogeneous effects by “driving partisans home” to support their party’s candidate, nor that persuasive contact activates a candidate’s supporters to turn out.Footnote 4

To be clear, our argument is not that campaigns, broadly speaking, do not matter. For example, candidates can determine the content of voters’ choices by changing their positions, strategically revealing certain information, and affecting media narratives—dynamics which are outside the scope of our analysis but could be affected by advertising (Holbrook Reference Holbrook1996; Jacobson Reference Jacobson2015; Johnston, Hagen, and Jamieson Reference Johnston, Hagen and Jamieson2004; Sides and Vavreck Reference Sides and Vavreck2013). Campaigns can also effectively stimulate voter turnout (e.g., Gerber and Green Reference Gerber and Green2000; Green, McGrath, and Aronow Reference Green, McGrath and Aronow2013). Our argument is not that campaigns do not influence general elections in any way, but that the direct persuasive effects of their voter contact and advertising in general elections are essentially zero.

In concluding, we discuss the broader implications of our findings for theories of political communication and democratic accountability. Our results harken back to an oft-criticized literature on the “minimal effects” of campaign interventions (e.g., Berelson, Lazarsfeld, and McPhee Reference Berelson, Lazarsfeld and McPhee1954; Klapper Reference Klapper1960; Lazarsfeld, Berelson, and Gaudet Reference Lazarsfeld, Berelson and Gaudet1948). A common critique of the original literature on “minimal effects” was that campaigns may not appear to have aggregate effects because any advertising they engage in is immediately reciprocated with responses from their opponents that “cancel out” in aggregate. Importantly, because the studies we analyze and present are individually randomized, they are not susceptible to this critique: it is not possible for an opposing campaign to reciprocate advertising to the treatment group but not the control group in these experiments, unless it somehow had knowledge of the treatment and control group assignments.Footnote 5 As a result, our findings suggest that a relatively strong version of the minimal effects thesis may hold in general elections—not because campaign effects cancel each other out, but because they have no average effects at all. This finding may help explain why campaigns increasingly focus on rousing the enthusiasm of existing supporters instead of reaching across party lines to win over new supporters (Panagopoulos Reference Panagopoulos2016). Our findings also offer an important caveat to the widespread notion that political elites can easily manipulate citizens’ political choices. The circumstances in which citizens’ political choices appear manipulable appear to be exceedingly rare in the elections that matter most.

THEORETICAL PERSPECTIVES

Political behavior research generally depicts Americans’ political predispositions as highly durable and resistant to change (Campbell et al. Reference Campbell, Converse, Miller and Stokes1960; Green, Palmquist, and Schickler Reference Green, Palmquist and Schickler2002; Sears and Funk Reference Sears and Funk1999). Consistent with these findings, Page and Shapiro (Reference Page and Shapiro1992, 45) find “a remarkable degree of stability” in aggregate public opinion (see also Druckman and Leeper Reference Druckman and Leeper2012a). Research suggests two broad reasons why campaign advertising and contact might have effects on voters’ candidate choices nevertheless: providing voters new considerations and heightening the salience of existing considerations.Footnote 6 We argue that close to election day in a general election, it is difficult for campaigns to persuade voters with either mechanism.

First, when it comes to providing voters with new arguments, frames, and information, by the time election day arrives, voters are likely to have already absorbed all the arguments and information they care to retain from the media and other sources beyond the political campaigns themselves (Gelman and King Reference Gelman and King1993). This is not to say that voters will know all the relevant information campaigns could provide them, but that they are likely to have been exposed to all this information and that, of this information, they will have chosen to retain nearly all they care to (Petty and Cacioppo Reference Petty and Cacioppo1986). It is clearly the case that voters do not know everything about most candidates; but if voters still have not retained any of the information they lack after weeks of being exposed to that information in the media, it is unlikely that campaigns will prove any more effective in getting that information through to them.

We also expect that there is a shrinking amount of information that campaigns could give American voters in general elections that would produce meaningful persuasion. There are shrinking numbers of “cross-pressured” voters for campaigns to push to their side through such crossover appeals (Hersh and Schaffner Reference Hersh and Schaffner2013; Smidt Reference Smidt2017). Correlations between voters’ partisan predispositions and their racial and issue views have increased dramatically (Abramowitz Reference Abramowitz2010). This means that a dwindling number of voters have conflicting considerations that would lead them to abandon their party; by following partisan cues, most voters can make the same choices they would make had they decided using other attributes of the candidates. In such an environment, it may be difficult for campaigns to change voters’ minds by informing them about a candidate’s positions, as voters are likely to agree with their party on any issues on which they have opinions in the first place (Berinsky Reference Berinsky2009; Lauderdale Reference Lauderdale2016; Freeder, Lenz, and Turney Reference Freeder, Lenz and Turney2017).Footnote 7 This means that although campaigns may have some scope for persuasion in competitive primary elections, where there is no partisan cue, in general elections, there are few considerations they can provide today’s voters that would lead them to abandon their party; these considerations increasingly push voters to vote for their party anyway.Footnote 8

A second main mechanism for the persuasive effects of campaign contact and advertising is thought to be that they temporarily make certain considerations more salient as people decide what they think (Zaller Reference Zaller1992). However, conditions that sharply limit the effects of salience-raising frames are likely to be met in general elections. The salience-raising effects of communication diminish in the presence of clear cues (Druckman, Peterson, and Slothuus Reference Druckman, Peterson and Slothuus2013) and when individuals are exposed to competing arguments and information (Druckman Reference Druckman2004a). For example, being exposed to both sides of political debates “limit[s] and often eliminate[s]” these effects because all the considerations people believe are relevant have been made salient, especially the partisan cue that makes other frames irrelevant to many voters (Druckman Reference Druckman2004a, 683; see also Chong and Druckman Reference Chong and Druckman2007; Druckman and Leeper Reference Druckman and Leeper2012b; Sniderman and Theriault Reference Sniderman, Theriault, Saris and Sniderman2004). Consistent with this view, Leeper and Slothuus (Reference Leeper and Slothuus2015) find that providing voters with new information about the substance of their choices can change their attitudes, but that once they have this information, providing them additional frames or emphasizing certain considerations does little to affect their choices.

As a result, our empirical expectation is that contact from campaigns in general elections could have effects early in the electoral cycle before the media provides competing frames and relevant information, but that these effects would decay rapidly, consistent with campaigns being able to temporarily make certain considerations salient when competing messages are not yet present because the campaign has not yet started (Hill et al. Reference Hill, Lo, Vavreck and Zaller2013). But we argue that it will be difficult for campaigns to produce even these short-lived effects within a couple months of a general election, consistent with campaigns no longer persuading voters once the media environment naturally raises the salience of the considerations being provided by all sides.

These arguments yield the theoretical predictions shown in Table 1. As the Table notes, our argument does not pertain to effects candidates might have by actually changing their platforms and positions, by being of higher quality, by securing more favorable media coverage, and so on. However, we argue that what campaign contact and advertising typically does—providing information voters are already being exposed to and attempting to increase the salience of this information—is very unlikely to lead voters to cross partisan lines. For example, a typical Democratic candidate sending mailers to voters featuring some of her more popular positions the media has already told voters she has we expect to be unlikely to persuade many voters to vote differently. By contrast, it may well be the case that actually changing her positions on these issues would affect election outcomes; our argument does not pertain to that counterfactual.

TABLE 1.

Theoretical Predictions

Existing work does not clearly test these predictions. It is obvious that the effects of elite attempts to persuade voters will be smaller in real-world, competitive environments than in the artificial survey environments in which scholars typically study them (e.g., Barabas and Jerit Reference Barabas and Jerit2010). However, it is unclear whether such effects are merely smaller or if they indeed are so small they are essentially non-existent. We hypothesized that the dynamics we discussed—the shrinking numbers of cross-pressured voters and the presence of competing frames in environments with partisan cues—would mean that contact from political campaigns has minimal effects on American voters’ candidate choices in the run-up to a general election.

This argument is by no means obvious. Campaigns spend a great deal of money advertising to voters and the firms and consultants who profit from these activities argue that their effects are large. Consistent with this optimism, nearly every recent review of the literature on campaign effects argues that the consensus among a previous generation of scholarship that campaigns have “minimal effects” on voters can be decidedly rejected in the wake of new research (Druckman Reference Druckman2004a; Iyengar and Simon Reference Iyengar and Simon2000; Jacobson Reference Jacobson2015). However, the vast majority of the evidence that has been marshaled in favor of this claim comes from observational studies, studies of primary elections, and studies of campaign interventions that collect outcomes far before election day. We draw on the new wealth of carefully identified studies of campaign contact to shed new light on this question and test our theory.

META-ANALYSIS OF FIELD EXPERIMENTS AND QUASI-EXPERIMENTS

As a first test of our theoretical predictions, we see whether they fit patterns we observe in an original meta-analysis we conducted of the existing field experimental and quasi-experimental literature on the effects of campaign contact and advertising in US elections. In the wake of the “credibility revolution” in social science research, scholars have produced a wealth of rigorous research that credibly estimates the effects of campaign activity. A recent meta-analysis considers the average effect of campaign activity on turnout (Green, McGrath, and Aronow Reference Green, McGrath and Aronow2013), but we are aware of no similar meta-analysis on the effects of campaign activity on persuasion.Footnote 9

Data

Our meta-analysis began with an exhaustive processFootnote 10 to collect all public studies using plausible identification strategies to estimate the effect of campaign advertising and outreach through the mail, phone calls, canvassing, TV, online ads, or literature drops on voters’ candidate choices and evaluations: primarily randomized trials but also regression discontinuity designs, natural experiments, and difference-in-differences designs. We list all the studies we included in Online Appendix B. We also excluded a few studies, as discussed in Online Appendix B.27. For example, Arceneaux and Nickerson (Reference Arceneaux and Nickerson2010) did not include a control group and focused on differences between treatments only, so we could not include an estimate of the effect of the campaign they studied.

For each study, we carefully collected information on the following from the original write-ups, or, if necessary, from the authors:

  • • Treatment effect estimate and standard error in percentage points.

    • – Some studies code a vote for the opposing candidate as –1 and vote for the cooperating candidate as 1. In these cases, we recode the data as 0 for the opposing candidate and 1 for the cooperating candidate, so that the estimates always have the interpretation of “percentage point effect on vote share.”

    • – In some cases, vote choice was not measured, but rather favorability or approval. In these cases, we use whichever variable is closest to capturing vote choice.

    • – Some studies emphasized subgroup effects that were not pre-registered in advance; in these cases, we used the average effect estimates, not the effects among subgroups that were chosen post hoc. Given that the studies all examine the persuasive effect of campaign contact among voters that campaigns themselves decided to contact, the average treatment effect is arguably the estimand of greatest interest.

    • – Where possible, we used complier average causal effect (treatment-on-treated) estimates.

    • – When studies have multiple treatment arms that we are unable to combine into a pooled estimate given the information available in the articles or replication data, we enter each treatment arm’s estimates separately into our meta-analysis and cluster the standard errors at the study level, given the shared control group.

  • • Days after election the survey was taken. This is coded as a negative number if the survey is taken before the election.Footnote 11 For studies that measure outcomes at the aggregate (e.g., precinct) level rather than with surveys, this is 0 by definition.

  • • Days after treatment the survey was taken. The number of days between treatment delivery and outcome measurement. For studies that measure outcomes at the precinct level, this is the number of days before the election the treatment occurred.

  • • Mode of treatment. Examples include door-to-door canvassing, phone banks, and mail.

We also collected the following contextual information:

  • • Election stage. Primary or general election.

  • • Seat. US President, US House, mayor, and so on.

  • • Incumbency.

  • • Competitiveness. Our definition of competitiveness is whether a reasonable outside observer would expect the election outcome to be uncertain rather than a foregone conclusion. Recognizing this definition is somewhat subjective, we found that in most cases it was easy to categorize races as competitive or not. We provide details in Online Appendix B.Footnote 12

Results

The results of our meta-analysis are shown in Figure 1.Footnote 13

FIGURE 1.

Meta-analysis Forest Plots: General Elections

Panel 1(a) shows the average effect of campaign outreach in general elections when the treatments are delivered within 2 months of election day. Consistent with our theoretical expectations, the average effect is zero.Footnote 14 Indeed, only two studies have statistically significant point estimates, about what would be expected given mild publication bias and this number of public studies. We discuss these studies in more detail below; the campaign strategies in both are unusual and not easily scalable.Footnote 15 Figure 2 shows that t-statistics from these studies follow a normal distribution nearly exactly. The right panel shows a Q-Q plot consistent with nearly all studies finding zero effects with a slight underrepresentation of effects very near zero, as would be expected given publication bias.Footnote 16 Together, these studies suggest that the most optimistic estimate that could be warranted from the literature is that campaign contact persuades about 1 in 175 voters, but that our best guess is that it persuades about 1 in 800 voters, substantively zero.

FIGURE 2.

Distribution of t Statistics from General Elections with Estimates Close to Election Day

Panel 1(b) shows that in the subset of studies in which treatment is delivered long before election day and its effects are measured immediately, the effects are clearly positive on average.Footnote 17 However, Panel 1(c) shows that in the two existing studies that examined whether these initial positive effects persisted, they were found to decay. This is consistent with our theoretical argument that when scholars study persuasion far from election day, when competing messages are not present, it can appear that persuasion is possible, but that such effects evanesce rapidly and likely would not appear were the persuasion attempted close to election day. (We present more evidence consistent with this pattern later.)

Panels (a) and (b) of Figure 3 show the meta-analyses for primary and ballot measure elections, respectively. We see clear significant effects in both these election types. Nearly all of these elections were competitive, reinforcing our argument that competitive environments alone are not responsible for extinguishing campaign effects near general elections, but that partisan cues present only in general elections play a role in extinguishing persuasive effects.

FIGURE 3.

Meta-Analysis Forest Plots: Primary and Ballot Measure Elections

Our meta-analysis is consistent with our theory that persuasive effects can exist in primaries and far from election day in general elections, but decays rapidly and is nearly impossible close to election day.

ORIGINAL FIELD STUDIES IN 2015 AND 2016

Our meta-analysis of well-identified campaign research uncovered the relative imprecision of the existing studies of persuasive personal contact, such as door-to-door canvassing. The eight extant studies using personal contact and conducted within two months of election day have an average treatment effect of negative 2 percentage points with a pooled standard error of 1.7 percentage points. This uncertainty, coupled with the expectation from the voter turnout literature that in-person treatments tend to show larger effects (Enos and Fowler Reference Enos and Fowler2016; Gerber and Green Reference Gerber and Green2000; Green and Gerber Reference Green and Gerber2015), led us to collaborate with a nationwide door-to-door canvassing operation during two 2015 elections and the 2016 general election to conduct nine original studies on the effect of their canvassing on vote choice, with six of those conducted in the final two months of the 2016 general election. These studies improve on the statistical precision of the literature on the persuasive effect of personal contact close to election day more than tenfold.

We conducted these studies to rule out several alternative explanations for the null effects found in our meta-analysis:

  • • One reason voters in general elections are thought to be hard to persuade is because they do not “receive” political messages in the first place, being disinterested in political topics and not avid consumers of media bearing political news (Zaller Reference Zaller1992). But, as we discuss below, in our studies of personal contact, we can be confident a voter received a message because a campaign worker physically spoke with them about it face-to-face.

  • • We can show that our conclusions about null effects are not driven by low-quality campaign activity. First, we find that our partner organization had larger-than-typical effects in persuasion experiments conducted during a 2015 primary and a 2015 special election as well as in a 2016 voter turnout experiment. In addition, contact from them early on in the electoral cycle had effects consistent with our theory.

  • • Another possible alternative explanation for null effects is a simple “saturation” explanation; that is, an explanation whereby a marginal campaign contact has no effect because voters have already received so many contacts from other campaigns but that the average effect of these contacts is nonzero. For example, perhaps a marginal piece of persuasive mail is unlikely to have much effect if voters have already received 100 pieces of mail, even if the average effect of receiving mail is nonzero. In addition to our evidence that persuasion is possible in highly competitive primary and ballot measure elections, our focus on door-to-door canvassing also helps rule out this alternative. We show in Figure OA4 that the vast majority of voters in these competitive elections received no other door-to-door persuasive contact. We return to this question in the discussion.

  • • Research on voter turnout and other activities suggests that face-to-face conversations are the likeliest to have large effects on voters (e.g., Enos and Fowler Reference Enos and Fowler2016; Gerber and Green Reference Gerber and Green2000; Green and Gerber Reference Green and Gerber2015), meaning the door-to-door canvassing conversations we studied were, if anything, likely to overestimate the effects of other campaign activity.

  • • These studies estimate the effects of activities that represent the strategic choices of a real campaign about who to target and what to say, rather than the (potentially less generalizable) decisions of academics attempting to mimic what real campaigns do.

Design

We conducted three randomized experiments with this partner organization before the final two months of the 2016 election. In the final two months of that election, we conducted four additional randomized experiments and two difference-in-difference studies with them. Below, we discuss the common elements across the designs of these studies. In Online Appendix D, we discuss each experiment in detail, including the experimental universe, tests of covariate balance, tests of differential attrition, treatment scripts, the outcome measures, and the results.

These experiments were paid for and administered by the partner organization; no university funds were used, and the authors advised the organization on implementation in their personal capacity as unpaid consultants.

Persuasive Interventions

The scripts canvassers used across these studies generally followed the same approach, which this partner organization has developed across several election cycles. These scripts are similar to the scripts reported in the other canvassing experiments we found for the meta-analysis.

  • • Introduction: “Hi, my name is [X] with [PARTNER ORGANIZATION]. We’re out today talking with folks in the neighborhood about the future of [STATE]. Are you [NAME]? Great!”

  • • Identify important issue: “First, a quick survey. When you think about the upcoming election on November 8th, what is the most urgent issue to you and your family?”

  • • Identify current candidate preference: “In the upcoming election for [RACE], Republican [NAME] is running against Democrat [NAME]. If you were going to vote today, would you vote for [REP] or [DEM]?”

  • • Establish source credibility: “[PARTNER ORGANIZATION] is an independent organization that represents over [STATE NUMBER OF PEOPLE] who want an economy that works for working people. We are not part of any political party or campaign and support candidates based on their record.”

  • • Persuasion on important issue: “You said earlier that [ISSUE] was the most important issue to you. I understand. How you vote is a personal decision. [PARTNER ORGANIZATION] has done the research on the economic issues and the records of the candidates. [Explain relevant issue background and candidate record. This would typically include an explanation of candidates’ issue positions as well as valence qualities relevant to the issue, such as experience, competency, and integrity.]”

For all of our studies, the same staff at the partner organization researched and wrote all of the scripts, ensuring that they always conveyed similar information. The scripts for each campaign are given in Online Appendix D.

Field Experiment and Survey Designs

The design of the field experiments closely follows the four methodological practices for field experiments with survey outcomes outlined in Broockman, Kalla, and Sekhon (Reference Broockman, Kalla and Sekhon2017). In all the field experiments, the following steps were taken:

  1. (1) The partner organization identified voters it wanted to persuade and had their independent public opinion research division enroll them in online surveys by mail. These surveys included dozens of questions on political, social, and cultural issues. This was designed to separate the survey measurement from the treatment and to limit demand effects. Note that this activity was administered and paid for completely by the partner organization; university resources were not used in any way. This paid survey design with a variety of both political and nonpolitical questions has been shown to produce samples that are fairly representative, including in political knowledge and past levels of political participation (Broockman, Kalla, and Sekhon Reference Broockman, Kalla and Sekhon2017). Online Appendix D shows representative assessments of those who completed each survey relative to the sampling frames from which they were recruited.

  2. (2) The partner organization canvassed enrolled voters with either a treatment or placebo message. Both scripts started identically to identify compliers symmetrically, and only branched into different content after the voter at the door was identified. If multiple people in a household responded to the survey, every survey respondent living in that household received the same treatment assignment.

  3. (3) The partner organization invited all voters who had been reached at the door (compliers) in either the treatment or placebo condition to complete follow-up surveys. Voters received gift cards to encourage high response rates.

  4. (4) We analyzed data collected by the partner organization to measure the effect of their canvassing on vote choice and candidate evaluations. The surveys typically included a vote choice question, a favorability question for each candidate, and (sometimes) a “best qualified” question. We always constructed our outcome measure the same way: We take the first factor from the factor analysis of all of that race’s survey questions. Then, we standardize to mean 0 and standard deviation 1 in the placebo group, with higher numbers representing greater support for the candidate endorsed by the partner organization. When incorporating these estimates into our meta-analysis, we divide all estimates and standard errors by 2 to approximate a percentage point effect while maintaining the benefit of multiple outcome measures.Footnote 18 Online Appendix D gives all the question wordings. We then regressed the outcome measure on a binary indicator for treatment versus placebo and a series of pre-treatment and demographic covariates. We used cluster-robust standard errors at the household level.

In Online Appendix D we describe the design and identification strategy for our quasi-experimental difference-in-differences studies. The difference-in-difference studies included five waves of surveys conducted over the final weeks of the campaign, with the final wave on election day. Importantly, in these studies, we observe which voters the partner group actually contacted and have measures of voters’ opinions both before and after any contact.

Quality of Partner Organization: Evidence from Other Experiments

One potential concern with these 2016 experiments is that they were all conducted with the same partner organization. This raises the question of whether any null effects reflect that organization’s own low quality, rather than the voters’ unpersuadability. After all, not every campaign operation is of equal quality (Nickerson Reference Nickerson2007b). Fortunately, three experiments help establish that this partner organization is of unusually high quality.Footnote 19

The first experiment was conducted during the 2015 mayoral Democratic primary in Philadelphia. This was a competitive primary for an open seat. We found that the partner organization’s canvass six weeks before election day and measured a week later increased support for their endorsed candidate by approximately 11 percentage points (p = 0.01), which is nearly three times the average effect in our meta-analysis of other primary elections. In a follow-up survey conducted during the last week of the campaign, we continue to estimate effects of 9 percentage points (p = 0.19).

The second experiment was conducted during a 2015 special election for state legislator in Washington. This was a competitive election in which nearly $2 million was spent in total by the candidates and outside groups. The partner organization’s canvass had a substantively large 6-percentage-point effect on support for their endorsed candidate (p = 0.01), although in a post-election survey, consistent with our theory, the effect had decayed.

Our third experiment was a voter turnout experiment conducted during the 2016 general election in the battleground state of North Carolina. We found that the partner organization’s canvass increased turnout by nearly 2 percentage points (p = 0.04), which is 43% more effective than would be expected based on Green and Gerber’s (Reference Green and Gerber2015) meta-analysis of door-to-door voter turnout experiments. More details on all of these experiments are available in Online Appendix D.

Overall, these experiments suggest that the partner organization is capable of persuading and mobilizing voters to the extent this is possible, typically with effects greater than average based on the literature.

Results

Table 2 shows the results of the original canvassing persuasion studies, with all effects shown in terms of standard deviations (d) on the first factor of the candidate items in each survey. The subtables split the studies into categories. The first subtable shows the 2015 experiments we just described, conducted during the 2015 Philadelphia Democratic mayoral primary and a Washington state legislative special election. The second subtable shows the first experiment we conducted in the 2016 general election, over 2 months before election day. The third subtable shows experimental results when the measurement was conducted within 2 months of election day. The fourth shows two difference-in-differences quasi-experiments. The final subtable shows the results of a literature drop conducted at the end of the North Carolina canvasses.Footnote 20 When “Experiment” is the same across multiple rows in each subtable, it means the estimates are drawn from the same study. For example, in the Ohio experiment that began in August, canvassers attempted to persuade voters with respect to both the senate and presidential races and there were both immediate and election day outcome measurements, so this one study appears four times in the table. Online Appendix D gives the dates of the surveys, the scripts used, the balance checks for each experiment, and other details of interest.

TABLE 2.

Results of Original Canvass Experiments in 2015 and 2016: Effects in Standard Deviations (d)

Subtable (a) shows that the organization had effects in a 2015 primary and a 2015 special general election, as discussed, although in the case of the general election, their effects had decayed by election day, as predicted. Subtable (b) shows that the organization had effects in the 2016 Ohio Senate race when measured immediately, although we find in Subtable (c) that these effects decayed by election day. Subtable (c) reports our original field experiments estimating that the canvassing from late August to mid-October had no effects on vote choice as measured within two months of election day, with a pooled estimate of –0.005 standard deviations (SE = 0.020). Subtable (d) shows the results of our quasi-experimental difference-in-differences designs in Ohio and North Carolina. In each case, the organization found subgroups of voters it estimated as more likely to be persuadable, based on the experiments in Subtable (c) and focused their canvassers on targeting these voters. The evidence in Subtable (d) suggests this was likely successful and that they ultimately had some persuasive effects targeting these voters. However, an important caveat to these conclusions is that the difference-in-differences designs entail stronger assumptions than the field experiments from Subtable (c) does. We return to discussing the potential persuasion these quasi-experiments found in the next section. Subtable (e) reports the literature drop experiment and quasi-experiment. There, the only statistically significant estimates are the nonpartisan Supreme Court race, which is consistent with our theory that effects are more likely in the absence of partisan cues.Footnote 21

Placing these findings in the context of the existing literature underscores their contribution and the consistent support they provide for our theory. Statistically, these experiments increase the amount of evidence in the literature about the effects of personal contact in general elections by about a factor of 10.Footnote 22 We also increase the amount of evidence in the literature about the effects of personal contact on candidate preferences within two months of a general election by a factor of nine.

Underscoring the strong support for our argument these new studies provide, Figure 4(a) shows a meta-analysis of the effects of personal contact in general elections, now including our original studies that were conducted within 60 days of election day. From this, we conclude that, on average, personal contact—such as door-to-door canvassing or phone calls—conducted within two months of a general election has no substantive effect on vote choice. The average effect from our meta-analysis is 0.58 percentage points, with a 95% confidence interval ranging from –0.50 to 1.66 percentage points. The only statistically significant estimates that come from within two months of a general election with party cues are in the difference-in-differences estimates, which measured the effects of programs that had been carefully targeted based on the results of the prior experiments. We now turn to discussing our interpretation of these estimates.

FIGURE 4.

Effects Within 2 Months of Election Day With Original Studies, By Contact Type

WHEN PERSUASION IN GENERAL ELECTIONS APPEARS POSSIBLE

Across a large number of electoral settings, candidates, treatments, targets, organizations, and experimental designs, our best guess is that persuasion attempts near election day in general elections fail to persuade voters. Despite the wide variation in experimental settings in the studies we examined, we see treatment effect estimates of less than 1 percentage point more than half the time when measurement is conducted near election day. A formal test for heterogeneity across studies also finds none. These patterns suggest that null effects in general elections are the rule across most general elections; not only do we see zero persuasive effects on average, but we see the same in a wide variety of individual studies.

Here we discuss two potential exceptions to this pattern of null effects. Although both are in line with our theoretical argument, we caution that this discussion is more tentative. It is quite possible given the general pattern of null effects that the studies we discuss here are statistical flukes. However, in the interest of transparency and critically examining our theoretical argument, we discuss both patterns. In both cases, we believe these potential exceptions are consistent with our theory, proving the rule that campaign contact seldom has meaningful effects on general election outcomes.

In General Elections, Early Persuasion Rapidly Decays and Late Persuasion Rarely Appears

As we have shown, most field experiments on voter persuasion find null effects; but many survey experiments report significant effects. One potential reason for this discrepancy is the time at which each kind of study is typically done: most field experiments measure effects close to election day, whereas survey experiments tend to be conducted outside active electoral contexts or far from election day and measure effects immediately.Footnote 23 Our theory expects immediate persuasive effects will be commonplace outside an active electoral context, but for effects to be more difficult to achieve inside an active electoral context. Here we show that this potential explanation is supported by over-time variation in the effect sizes in field experiments.Footnote 24

In the field experimental literature, relatively few studies have been conducted more than two months before election day, but we need to observe the effects of this early persuasion to test our theory. Even fewer studies track whether early persuasion persists over time. Fewer studies still examine whether a treatment that had effects early in the cycle would have effects when deployed again closer to the election. However, we were able to locate two studies that test our predictions in the literature. Two of our own studies also do so.

Table 3 shows evidence from these four studies, with Subtable 3(a) restating our theoretical predictions.

TABLE 3.

Early Persuasion Decays; Late Persuasion Fails

Notes: * = p < 0.05.

Subtable 3(b) is a reanalysis we conducted of the data for Doherty and Adler (Reference Doherty and Adler2014), a rare study in the literature that introduced variation in the timing of campaign contact. Consistent with our predictions, the campaign mailers they studied had persuasive effects in state legislative general elections when they were mailed months before the election and their effects were measured immediately (first column). However, a subsequent survey of the same individuals closer to election day found that these persuasive effects had decayed (second column). Finally, a follow-up experiment found that the same mailers sent close to election day did not even have immediate effects (third column).

Subtable 3(c) shows Gerber et al.’s (Reference Gerber, Gimpel, Green and Shaw2011) field experiment with the Rick Perry campaign on its TV advertising. This experiment was conducted many months before the general election and found immediate effects. However, these effects decayed within a week. Moreover, additional data provided by the authors finds that this same advertising did not have effects closer to election day.Footnote 25

Subtable 3(d) is a study we conducted with our research partner in the 2016 Ohio Senate election. The first column shows that we found strong evidence in late August that their door-to-door canvassing program increased support for the Democratic candidate in the Ohio Senate election. However, the second column shows that when we resurveyed the same voters that had been persuaded in August closer to election day, this persuasion appears to have decayed. Moreover, as the third column shows, our subsequent measurement in the difference-in-differences analysis of the effects of the very same canvassing program conducted closer to election day found that it no longer had persuasive effects (the coefficients in the first and last column can be statistically distinguished).

Finally, Subtable 3(e) shows results from one of the other studies we conducted with our research partner. This study of door-to-door canvassing conducted in a 2015 special state legislative general election found that canvassing conducted early in the electoral cycle had immediate effects. A second measurement closer to election day found those effects decayed. Unfortunately, in this study we were not able to measure whether this canvassing would have had immediate effects closer to election day.

A majority of the field experiments in the literature that find persuasive effects in general elections are shown in Table 3. It appears we can account for all these effects by noting that they occurred early in the election cycle; in every case where data is available, these treatments did not have effects that lasted until election day, nor did they have immediate effects when repeated close to election day. This grants additional support to our theory and raises questions about the generalizability of treatment effects measured outside of active election campaigns.

Potential Exceptions Close to Election Day: Identifying Rare Cross-Pressure and Exploiting Unusual Candidates

There are three studies we are aware of in which an experiment or quasi-experiment found a statistically significant persuasive effect measured within two months of an election. With the renewed caveat that these estimates could be statistical flukes, we interpret the unusual features of all three of these studies as evidence consistent with our theory. Our theory expected that very few voters would typically be persuadable and that identifying them would be extremely difficult. In all these three cases, the campaigns invested unusually heavily in identifying persuadable voters (Hillygus and Shields Reference Hillygus and Shields2008) and were working to defeat an unusual candidate—circumstances our theory expected to be rare but, in the context of which, meaningful persuasive effects may be possible.

First, Rogers and Nickerson (Reference Rogers and Nickerson2013) worked with a pro-choice organization ahead of the 2008 US Senate election in Oregon to identify voters who identified as pro-choice in a pre-survey, many of whom did not realize that the Republican incumbent Senator was not consistently pro-choice. For the experiment, the group sent a mailer to only those voters who had identified as pro-choice in a pre-survey; the mailers attempted to correct these mistaken beliefs about the incumbent’s position on abortion. In follow-up surveys, Rogers and Nickerson (Reference Rogers and Nickerson2013) found that these mailers corrected these individuals’ beliefs and changed their vote in the Senate race. It is worth considering the multiple rare conditions this study met. An incumbent was demonstrably out of step on an issue—a Senator from Oregon who opposed abortion. Moreover, abortion is widely regarded as “easy” and important for many voters. Finally, the interest group had funded a pre-survey to identify supporters of its issue, purging individuals from the target universe for the mailers who were anti-abortion and who might have had a negative reaction to the mailers—a strategy that cannot be widely applied because most voters do not answer surveys and cannot be identified for individual targeting of this type without individual voter survey responses (Endres Reference Endres2016; Hersh Reference Hersh2015). We would expect persuasion to be possible in such conditions, but we expect such conditions to be extremely rare.

Second, in the experiment our cooperating group conducted in North Carolina, the door-to-door canvassers discussed the presidential and Senate candidates aloud with voters but left flyers at the door with endorsements of the Democratic North Carolina gubernatorial candidate. These flyers discussed how the Republican incumbent had cost the state billions of dollars as a result of supporting the unpopular HB 2 law that banned transgender people from using the bathrooms for the gender they identified with. The experiment found that black voters appeared to react positively to this material but white voters appeared to react negatively. In response to the experimental results, the group removed many white voters from their target lists going forward. We found in our follow-up measurement closer to election day that the overall program still had positive effects on vote for the Democratic gubernatorial candidate among blacks and negative effects on whites, and that the group’s targeting had changed enough that the overall average effect was more likely to be positive. It was not obvious to the partner organization that black voters would have positive effects and white voters negative effects; only by conducting an experiment was the group able to identify a responsive audience. Several aspects of this situation are quite unique. This same strategy would not have been possible in states where campaigns do not have access to voter race on voter rolls (Endres Reference Endres2016; Hersh Reference Hersh2015). In addition, only by conducting an expensive randomized experiment far in advance was the group able to identify the right audience for its message—and only by conducting door-to-door canvassing was it able to limit its message to only this audience (whereas with TV ads, this individual-level targeting would not have been possible). Finally, the persuasive material was able to exploit a unique situation in which the governor had supported a deeply unpopular piece of legislation on an easy issue that was salient to voters.Footnote 26

Third, our partner group found statistically significant effects in the difference-in-differences analysis of their canvassing on the presidential race in Ohio. This was a highly unusual race because (a) a Republican candidate (Donald Trump) taking many positions out of step with the party, and (b) the prevailing campaign messages from the Democratic candidate’s (Hillary Clinton) campaign did not focus on the economic messages one might expect to persuade Ohio voters, although the partner group canvassers emphasized these issues.Footnote 27 In addition, although the first experiment did not find significant effects, the partner organization, as it did in North Carolina, adjusted its targeting to focus on the voters it estimated to be most persuadable, a subset it was unable to predict in advance without conducting the experiment at significant expense.

In summary, although these three studies suggest that persuasion close to election day is possible sometimes, the broader context of these experiments and the substantively small effects they estimated underscore our broader pessimism that meaningful persuasion is possible for most campaigns close to most general elections.Footnote 28 These studies were all conducted in unusual electoral circumstances and among a subset of voters that most campaigns do not have the resources to identify, using data that most campaigns cannot even collect (Endres Reference Endres2016; Hersh Reference Hersh2015).Footnote 29

DISCUSSION

Both assembling and contributing to the theoretical and empirical research literature to date, we present unique evidence indicating campaign persuasion is extremely rare in general elections; the best estimate of the size of persuasive effects in general elections in light of our evidence is zero. Although exceptions are possible, the evidence we carefully assembled from the literature and from a series of unique original studies paints a consistent picture. When party cues are absent in ballot measures and primaries or when persuasion is conducted far in advance of a general election, it appears that campaign contact and advertising can influence voters’ choices. But when we focus on the choices voters actually make on election day in a general election, we find that any early persuasion has decayed and that any persuasion near election day fails reliably.

This pattern of findings is surprising in light of recent reviews of the literature on campaign effects that posit that the classic “minimal effects” view of campaign contact and advertising can be decidedly rejected. Our evidence suggests that minimal effects of campaign contact and advertising are the norm, with only rare exceptions. In this way, our findings are most consistent with a view stressing the “fundamentals” of an election in shaping the outcome rather than the role of campaigns; in other words, that “campaigns reinforce the political orientations of voters and mobilize them to vote rather than convert large segments of the population to new ways of thinking” (Ansolabehere Reference Ansolabehere, Brady and Johnston2006, see also Gelman and King Reference Gelman and King1993; Sides and Vavreck Reference Sides and Vavreck2013). More generally, our findings cast doubt on the view that political elites can easily manipulate citizens’ judgments.

With this said, we hasten to note several caveats to our argument. First, our argument is not that campaigns do not matter at all. Campaigns likely cannot directly change which candidates Americans support in general elections through the direct effects of contact and advertising. However, candidates can still decide to change their issue positions, attract media coverage, and engage in other activities that may change who voters support. For example, other research argues that the positions candidates take and the information they encourage the media to discuss, in part through their paid advertising, can influence elections (Sides and Vavreck Reference Sides and Vavreck2013; Vavreck Reference Vavreck2009). Our evidence does not speak to these forms of influence.Footnote 30

Campaigns clearly can also influence whether voters bother to vote at all. Indeed, another implication of our results is that campaigns may underinvest in voter turnout efforts relative to persuasive communication. Although the marginal effects of get out the vote interventions are smaller in competitive general elections, especially in presidential years, they are still clearly positive (Green, McGrath, and Aronow Reference Green, McGrath and Aronow2013). Indeed, we found that our partner canvassing organization had effects of nearly 2.5 percentage points on turnout in the 2016 Presidential election. If these canvassers had been working on persuading voters instead of mobilizing existing supporters, our best estimate is that they would have generated fewer net votes. In this way, our results speak to the puzzle of why campaigns have increasingly focused on rousing the enthusiasm of existing supporters rather than reaching out to and attempting to persuade moderates (Panagopoulos Reference Panagopoulos2016).Footnote 31 With this said, increasing turnout alone can only provide so many votes; if campaigns were able to have large persuasive effects, they would be able to change the outcome of many additional elections.Footnote 32

We also hasten to note several limitations to our evidence. First, the existing literature (and, by extension, our meta-analysis) provides only scarce evidence on the effects of television and digital advertising, which represent a great deal of campaign spending.Footnote 33 Although our theoretical argument would also apply to these forms of campaign communication and the little evidence we do have on them is consistent with null effects in general elections, more evidence about these mediums would clearly be welcome. In addition, our original evidence was largely from 2016. Although our meta-analysis drawing on experiments from other years is consistent with our findings, replicating these findings in future elections would clearly be of interest. Our new evidence also largely focused on general elections, although our meta-analysis and theory suggested that voter’s choices in primary elections—which are also politically significant, of course—are much easier for campaigns to influence.Footnote 34 Last, the field experiments we reviewed and presented by and large measured the marginal effect of one contact rather than the total effect of an entire campaign’s persuasive activity. It may well be that many very small marginal effects that field experiments do not have the statistical power to detect could add up to a large enough total effect to impact reasonable numbers of elections. This is a proposition that future research could test by randomly assigning entire campaign strategies, such as by having a party campaign committee randomly assign which legislative districts they target or what strategy they pursue in each district (e.g., Wantchekon Reference Wantchekon2003).

Another caveat to our findings is that it remains possible that existing persuasive tactics could be improved.Footnote 35 Despite the stability of both micro- and macro-level predispositions and opinion on issues (Druckman and Leeper Reference Druckman and Leeper2012a; Green, Palmquist, and Schickler Reference Green, Palmquist and Schickler2002; Page and Shapiro Reference Page and Shapiro1992; Sears and Funk Reference Sears and Funk1999), there are clearly exceptions where public opinion has changed, such as the American public’s embrace of LGBT equality and in the evolution of party coalitions (Schickler Reference Schickler2016). These changes tend to be gradual, but suggest that sustained political advocacy and generational replacement can together ultimately produce the large opinion changes political practitioners may like to affect. Nevertheless, recent counterexamples of more rapid changes do exist in the context of theory-driven interventions, and future work should continue to evaluate their promise (e.g., Broockman and Kalla Reference Broockman and Kalla2016; Paluck and Green Reference Paluck and Green2009).

Consistent with this cautious optimism, the history of research on voter turnout is replete with examples of scholars identifying how psychological theories could be productively applied to have sizable effects from forms of contact that were once thought to produce relatively small effects (Gerber, Green, and Larimer Reference Gerber, Green and Larimer2008; Nickerson Reference Nickerson2007a; Nickerson and Rogers Reference Nickerson and Rogers2010)—theoretical innovations which rapidly diffused to campaigns (Issenberg Reference Issenberg2012). It is conceivable that scholars and campaigns could work together to develop and test such advances in the realm of persuasion as well, advancing both campaign practice and scholarship. Indeed, recognizing that their existing approaches fall short might increase campaign practitioners’ interest in exploring new strategies. With this said, going forward, our consistent finding of null effects suggests that non-experimental studies of campaign communication or studies conducted outside of active campaign contexts that claim to find large campaign effects in general elections should be viewed with healthy skepticism.

SUPPLEMENTARY MATERIAL

To view supplementary material for this article, please visit https://doi.org/10.1017/S0003055417000363.

Replication material can be found on Dataverse at http://doi.org/10.7910/DVN/SMXWA9.

Footnotes

We acknowledge seminar participants at the Midwest Political Science Association conference, the Northeast Political Methodology Meeting at NYU, Berkeley, Stanford, and the University of British Columbia, as well as Adam Berinsky, Donald Green, Avi Feller, Shanto Iyengar, Jon Krosnick, Gabriel Lenz, Joel Middleton, Daron Shaw, Jas Sekhon, Eric Schickler, Laura Stoker, and Lynn Vavreck for helpful feedback. All remaining errors are our own. The original studies reported herein were conducted by Working America. The authors served as unpaid consultants to Working America in their personal capacity.

1 Likewise, Jacobson’s (Reference Jacobson2015) review argues that “the ‘minimal effects’ thesis. . .has not survived” (32) and Iyengar and Simon (Reference Iyengar and Simon2000, 150) summarize the “conventional academic wisdom” as that “the consequences of campaigns are far from minimal.” However, much of the evidence these reviews cover comes from a time when affective polarization was lower and voters might have been more persuadable than they are today (Iyengar, Sood, and Lelkes Reference Iyengar, Sood and Lelkes2012).

2 Our criteria for research is described in further detail below.

3 For example, they appear to have the largest effects on turnout (Enos and Fowler Reference Enos and Fowler2016; Gerber and Green Reference Gerber and Green2000; Green and Gerber Reference Green and Gerber2015).

4 Other research has found evidence for these phenomena in some cases, but this does not appear to be a reliable feature in our experiments.

5 That field experiments identify partial and not general equilibrium effects is often considered a key weakness (Deaton Reference Deaton2010), but in this case, it represents a strength: we are explicitly interested in identifying the partial equilibrium effects of campaign contact, as it can help us understand the nature of the general equilibria that may exist.

6 This conception is specific to memory-based models, but an analogous version of the argument that follows can be made for models of on-line processing: Voters already aware of a candidates’ attributes or positions are not likely to update their affective “running tally” toward the candidate when being informed of such attributes yet again; and the strength of affect toward partisan groups should typically overwhelm any candidate-specific affect (Iyengar, Sood, and Lelkes Reference Iyengar, Sood and Lelkes2012).

7 In addition, there are very few true independents who do not have a partisan cue to rely on (Klar and Krupnikov Reference Klar and Krupnikov2016; Smidt Reference Smidt2017).

8 The existence of split-ticket voters indicates there are clearly other candidate characteristics voters value, such as qualifications or ideology. Our argument similarly applies to information about these candidate attributes: if the media is already making these attributes clear to voters, it is unlikely that campaigns providing them again would change many voters’ minds.

9 Lau, Sigelman, and Rovner (Reference Lau, Sigelman and Rovner2007) conduct a meta-analysis on the effects of negative political campaigns. Their analysis largely focuses on laboratory studies and observational studies and is limited to negative political campaigns. As such, it may miss the effect of campaigns that are more positive or focus on the contrast with the other candidate, includes studies without identification strategies, and includes studies of hypothetical campaigns, which may raise external validity concerns. Nevertheless, these authors, too, conclude that “the research literature does not bear out the proposition that negative political campaigns ‘work’ in shifting votes toward those who wage them” (1183).

10 To ensure that we had the complete universe of public studies and unpublished working papers, we began with a list of studies identified in a recent methodological article (Broockman, Kalla, and Sekhon Reference Broockman, Kalla and Sekhon2017). We then e-mailed several listservs with our preliminary list of studies and contacted the authors of most of these studies to ask if our list was complete.

11 We always use election day and do not take early voting into account, as dates of early voting are not consistently available across studies.

12 For example, Nickerson (Reference Nickerson2005) studies the effects of a Michigan Democratic party organization’s outreach in targeted state legislative races; we assume the party organization selected races to target that were competitive.

13 Our meta-analysis uses random effects with standard errors clustered at the study level. Results are robust to using fixed effects or the permutation test described in Follman and Proschan (Reference Follman and Proschan1999). Follman and Proschan (Reference Follman and Proschan1999) demonstrate that random effects estimates in meta-analyses can inflate the type I error rate. Because we find a null, we are not concerned with the increased likelihood of a false positive. If anything, the random effects estimate is conservative in the case of the null findings we report below.

14 These results are not an artifact of survey measurement: in precinct randomized experiments that do not rely on self-reported survey data, we find an average treatment effect of –0.02 percentage points.

15 One study involved the candidates themselves knocking on doors. The other involved individually identifying persuadable voters with a pre-survey that most voters do not answer, limiting the reach of this strategy.

16 Our outreach to authors of previous experiments and listservs yielded at least five additional experiments with null effects that have not been written up (Franco, Malhotra, and Simonovits Reference Franco, Malhotra and Simonovits2014).

17 The one exception is the multiple treatments from Shaw, Blunt, and Seaborn (Reference Shaw, Blunt and Seaborn2017). While noisy, the 95% confidence intervals from these treatments all include positive values. Excluding Shaw, Blunt, and Seaborn (Reference Shaw, Blunt and Seaborn2017) results in an average effect of 3.16 percentage points with a 95% confidence interval from 1.34 to 4.99 percentage points. Excluding Shaw, Blunt, and Seaborn (Reference Shaw, Blunt and Seaborn2017) also increases the p-value from the test for heterogeneity to 0.36, reinforcing that Shaw, Blunt, and Seaborn (Reference Shaw, Blunt and Seaborn2017) might be an exception.

18 In a perfectly competitive election with voters split 50-50, the standard deviation of support for a candidate is 0.5. A 1-percentage-point shift would thus correspond to a 0.02 standard deviation increase.

19 We conducted a fourth voter turnout experiment in Missouri during the 2016 general election. This experiment followed the same design as the North Carolina voter turnout experiment reported in the Quality of Partner Organization section, but due to an implementation error there was covariate imbalance between the compliers in the treatment and placebo groups. Full results from this experiment are reported in Online Appendix D.

20 After completing the President and Senate persuasion scripts in North Carolina, canvassers would encourage voters to take literature on the gubernatorial and nonpartisan Supreme Court races. Beyond mentioning they were leaving this literature, canvassers did not engage in persuasion face-to-face on these races. For this reason, we do not include these in our later meta-analysis of personal contact.

21 Influential theories argue that “the campaign brings the fundamentals of the election to the voters” (e.g., Wlezien and Erikson Reference Wlezien and Erikson2002, 987; see also Gelman and King Reference Gelman and King1993). With this said, Figures OA1 and OA2 in the Online Appendix respectively find no evidence of consistent heterogeneous effects of the treatments in our original studies by “driving partisans home” to their parties and no evidence of effects on turnout of pre-existing supporters. However, these are likely underpowered tests as they reflect the impact of a single contact. It may well be that the campaign has cumulative effects that do not appear in these individual contacts. We return to this question in the discussion.

22 In particular, the precision of each study in the literature is $\frac{1}{SE_i^2}$ and the total precision of multiple studies is $\sum \frac{1}{\text{SE}_i^2}$ , where SE i is the standard error of study i. Using this metric, the total precision of the prior literature in competitive elections is 0.255. Across our studies, it is 3.05. This is in terms of Complier Average Causal Effect (CACE) (Treatment on Treated [TOT]) effects, but a similar ratio holds for Intent to Treat (ITT) effects.

23 Another difference between our studies and most survey experiments is that we focus on candidate choice, which is typically the choice voters are faced with, whereas survey experiments tend to focus on issue opinions, which appear to function differently than candidate choices (Berinsky Reference Berinsky2017; Lenz Reference Lenz2012).

24 Unfortunately these four studies were all conducted in general elections, so we are unable to test our prediction that effects would be larger but still decay somewhat if treatment were conducted early on in a primary or ballot measure campaign. All the experiments on primaries and ballot measures were conducted close to election day.

25 We thank Donald Green for providing these additional results. More details are provided in Online Appendix Section B.10.

26 In addition, one caveat to this experiment is that since the persuasive flyer was left with voters and they often answered the surveys in their homes, it is possible that they answered the survey with the persuasive flyer in view but that in the ballot box, they would not have remembered it.

27 Fowler, Ridout, and Franz (Reference Fowler, Ridout and Franz2016, see Figure 9) note that over 60% of Clinton’s TV ads were solely about candidate characteristics compared to those of Trump, over 70% of whose ads concerned policy, a figure much closer to what has typically been seen in presidential campaigns since 2000.

28 We also investigated whether individuals who identify as independents might on average be more persuadable. As we show in Figure OA3, we find no consistent persuasion effects among either pure independents or independents and party leaners, as measured during the pre-survey in our original studies.

29 Another exception may be cases in which the candidate herself does the outreach and persuasion. In a general election, Barton, Castillo, and Petrie (Reference Barton, Castillo and Petrie2014) find that candidate persuasion has a nearly 21 percentage point effect on vote choice, but their standard errors are large. Recent research from the United Kingdom (Foos Reference Foos2017) and Italy (Cantoni and Pons Reference Cantoni and Pons2017) suggests more muted effects of candidate canvassing. Regardless, this strategy is unlikely to persuade meaningful numbers of voters in most elections, as a candidate can only knock on so many doors.

30 One caveat to this view is that many apparent effects of campaign events may be due to differential nonresponse bias (Gelman et al. Reference Gelman, Goel, Rivers and Rothschild2016).

31 While most experiments in the literature have been conducted with Democratic or liberal-leaning organizations, the shift Panagopoulos (Reference Panagopoulos2016) identified appears in both parties.

32 However, campaigns may be able to use persuasive efforts early on in an election cycle to identify supporters in a manner that assists later mobilization efforts (Carpenter Reference Carpenter2016).

33 Our evidence that the “warm literature drop” in the North Carolina Gubernatorial and Supreme Court elections appear to have effects also suggest further research on this medium is warranted.

34 One intriguing implication of this result for American politics research more generally is that interest groups may find it more feasible to threaten officeholders through campaign spending in primaries than in general elections (e.g., Anzia Reference Anzia2011; Bawn et al. Reference Bawn, Cohen, Karol, Masket, Noel and Zaller2012).

35 Nevertheless, our findings are decidedly not vulnerable to a common critique of field experiments that the results of the interventions academics design and implement might not generalize to the effects of real campaigns. Since academic institutions are not legally allowed to intervene in candidate elections, the voters to target were selected and treated by a real campaign in every single study we have discussed. Since academic institutions are not legally allowed to intervene in candidate elections, in every single study we have discussed.

References

REFERENCES

Abramowitz, Alan I. 2010. The Disappearing Center: Engaged Citizens, Polarization, and American Democracy. New Haven, CT: Yale University Press.Google Scholar
Adams, William C., and Smith, Dennis J.. 1980. “Effects of Telephone Canvassing on Turnout and Preferences: A Field Experiment.” Public Opinion Quarterly 44 (3): 389–95.Google Scholar
Ansolabehere, Stephen. 2006. “The Paradox of Minimal Effects.” In Capturing Campaign Effects, ed. Brady, Henry E. and Johnston, Richard. Ann Arbor: University of Michigan Press, 29–44.Google Scholar
Anzia, Sarah F. 2011. “Election Timing and the Electoral Influence of Interest Groups.” The Journal of Politics 73 (2): 412–27.CrossRefGoogle Scholar
Arceneaux, Kevin. 2005. “Using Cluster Randomized Field Experiments to Study Voting Behavior.” Annals of the American Academy of Political and Social Science 601 (1): 169–79.CrossRefGoogle Scholar
Arceneaux, Kevin. 2007. “I’m Asking for Your Support: The Effects of Personally Delivered Campaign Messages on Voting Decisions and Opinion Formation.” Quarterly Journal of Political Science 2 (1): 43–65.Google Scholar
Arceneaux, Kevin, and Nickerson, David W.. 2010. “Comparing Negative and Positive Campaign Messages: Evidence From Two Field Experiments.” American Politics Research 38 (1): 54–83.Google Scholar
Arceneaux, Kevin, and Kolodny, Robin. 2009. “Educating the Least Informed: Group Endorsements in a Grassroots Campaign.” American Journal of Political Science 53 (4): 755–70.CrossRefGoogle Scholar
Bailey, Michael A., Hopkins, Daniel J., and Rogers, Todd. 2016. “Unresponsive, Unpersuaded: The Unintended Consequences of Voter Persuasion Efforts.” Political Behavior 38 (3): 713–46.Google Scholar
Barabas, Jason, and Jerit, Jennifer. 2010. “Are Survey Experiments Externally Valid? ”American Political Science Review 104 (2): 226–42.CrossRefGoogle Scholar
Barton, Jared, Castillo, Marco, and Petrie, Ragan. 2014. “What Persuades Voters? A Field Experiment on Political Campaigning.” The Economic Journal 124 (574): F293–F326.CrossRefGoogle Scholar
Bawn, Kathleen, Cohen, Martin, Karol, David, Masket, Seth, Noel, Hans, and Zaller, John. 2012. “A Theory of Political Parties: Groups, Policy Demands and Nominations in American Politics.” Perspectives on Politics 10 (3): 571–97.CrossRefGoogle Scholar
Berelson, Bernard R., Lazarsfeld, Paul F., and McPhee, William N.. 1954. Voting: A Study of Opinion Formation in a Presidential Campaign. Chicago, IL: University of Chicago Press.Google Scholar
Berinsky, Adam J. 2009. In Time of War: Understanding American Public Opinion From World War II to Iraq. Chicago, IL: University of Chicago Press.CrossRefGoogle Scholar
Berinsky, Adam J. 2017. “Measuring Public Opinion with Surveys.” Annual Review of Political Science 20: 309–29.CrossRefGoogle Scholar
Brady, Henry E., Johnston, Richard, and Sides, John. 2006. “The Study of Political Campaigns.” In Capturing Campaign Effects, ed. Brady, Henry E. and Johnston, Richard. Ann Arbor, MI: University of Michigan Press, 1–26.Google Scholar
Broockman, David, and Green, Donald. 2014. “Do Online Advertisements Increase Political Candidates’ Name Recognition or Favorability? Evidence from Randomized Field Experiments.” Political Behavior 36 (2): 263–89.Google Scholar
Broockman, David E., and Kalla, Joshua L.. 2016. “Durably Reducing Transphobia: A Field Experiment on Door-to-door Canvassing.” Science 352 (6282): 220–4.Google Scholar
Broockman, David, Kalla, Joshua L., and Sekhon, Jasjeet S.. 2017. “The Design of Field Experiments With Survey Outcomes: A Framework for Selecting More Efficient, Robust, and Ethical Designs.” Political Analysis.CrossRefGoogle Scholar
Brox, Brian J., and Shaw, Daron R.. 2009. “Political Parties, American Campaigns, and Effects on Outcomes.” In Handbook of Party Politics, ed Katz, Richard S. & Crotty, William. London: Sage, 146–59.Google Scholar
Campbell, Angus, Converse, Philip E., Miller, Warren E., and Stokes, Donald E.. 1960. The American Voter. Chicago: University of Chicago Press.Google Scholar
Cantoni, Enrico and Pons, Vincent. 2017. “Do Interactions with Candidates Increase Voter Support and Participation? Experimental Evidence from Italy.” MPSA Conference Paper.Google Scholar
Cardy, Emily Arthur. 2005. “An Experimental Field Study of the GOTV and Persuasion Effects of Partisan Direct Mail and Phone Calls.” The Annals of the American Academy of Political and Social Science 601 (1): 28–40.CrossRefGoogle Scholar
Carpenter, Daniel. 2016. “Recruitment by Petition: American Antislavery, French Protestantism, English Suppression.” Perspectives on Politics 14 (3): 700–23.Google Scholar
Chong, Dennis, and Druckman, James N. 2007. “A Theory of Framing and Opinion Formation in Competitive Elite Environments.” Journal of Communication 57 (1): 99–118.Google Scholar
Cubbison, William. 2015. “The Marginal Effects of Direct Mail on Vote Choice.” Paper presented at the Annual Meeting of the Midwest Political Science Association. URL: http://media.wix.com/ugd/3a8c0a_47330c730f56431f8f982a3d842f434a.pdf.Google Scholar
Cunow, Saul, and Schwenzfeier, Meg. 2015. Working America 2014 Persuasion Experiments Results Memo. Technical Report Analyst Institute.Google Scholar
Deaton, Angus. 2010. “Instruments, Randomization, and Learning about Development.” Journal of Economic Literature 48 (2): 424–55.Google Scholar
DellaVigna, Stefano, and Gentzkow, Matthew. 2010. “Persuasion: Empirical Evidence.” Annual Review of Economics 2: 643–69.CrossRefGoogle Scholar
Doherty, David, and Adler, E. Scott. 2014. “The Persuasive Effects of Partisan Campaign Mailers.” Political Research Quarterly 67 (3): 562–573.Google Scholar
Druckman, James N. 2001. “The Implications of Framing Effects for Citizen Competence.” Political Behavior 23 (3): 225–56.Google Scholar
Druckman, James N. 2004a. “Political Preference Formation: Competition, Deliberation, and the (Ir) Relevance of Framing Effects.” American Political Science Review 98 (4): 671–86.CrossRefGoogle Scholar
Druckman, James N. 2004b. “Priming the Vote: Campaign Effects in a U.S. Senate Election.” Political Psychology 25 (04): 577–94.Google Scholar
Druckman, James N., Peterson, Erik, and Slothuus, Rune. 2013. “How Elite Partisan Polarization Affects Public Opinion Formation.” American Political Science Review 107 (01): 57–79.Google Scholar
Druckman, James N., and Leeper, Thomas J.. 2012a. “Is Public Opinion Stable? Resolving the Micro/Macro Disconnect in Studies of Public Opinion.” Daedalus 141 (4): 50–68.Google Scholar
Druckman, James N., and Leeper, Thomas J.. 2012b. “Learning More from Political Communication Experiments: Pretreatment and Its Effects.” American Journal of Political Science 56 (4): 875–96.Google Scholar
Endres, Kyle. 2016. “The Accuracy of Microtargeted Policy Positions.” PS: Political Science & Politics 49 (4): 771–4.Google Scholar
Enos, Ryan D. and Fowler, Anthony. 2016. “Aggregate Effects of Large-Scale Campaigns on Voter Turnout.” Political Science Research and Methods: 1–19. https://doi.org/10.1017/psrm.2016.21 Google Scholar
Enos, Ryan D., and Hersh, Eitan D.. 2015. “Party Activists as Campaign Advertisers: The Ground Campaign as a Principal-Agent Problem.” American Political Science Review 109 (2): 252–78.CrossRefGoogle Scholar
Follman, Dean A., and Proschan, Michael A.. 1999. “Valid Inference in Random Effects Meta-analysis.” Biometrics 55 (3): 732–7.Google Scholar
Foos, Florian. 2017. “First Impressions - Lasting Impressions? The Short- and Long-term Effects of Candidate Contact on Voting Intentions During a High-salience Campaign.” MPSA Conference Paper.Google Scholar
Fowler, Erika Franklin, Ridout, Travis N., and Franz, Michael M.. 2016. “Political Advertising in 2016: The Presidential Election as Outlier?” The Forum: A Journal of Applied Research in Contemporary Politics 4 (14): 445–69.Google Scholar
Franco, Annie, Malhotra, Neil, and Simonovits, Gabor. 2014. “Publication Bias in the Social Sciences: Unlocking the File Drawer.” Science 345 (6203): 1502–05.Google Scholar
Freeder, Sean, Lenz, Gabriel S., and Turney, Shad. 2017. “The Importance of Knowing ‘What Goes With What’: Reinterpreting the Evidence on Policy Attitude Stability.” Working paper, available at https://www.ocf.berkeley.edu/~glenz/wgww/WGWW.pdf.Google Scholar
Gelman, Andrew, and King, Gary. 1993. “Why Are American Presidential Election Campaign Polls so Variable When Votes Are so Predictable?” British Journal of Political Science 23 (4): 409–51.CrossRefGoogle Scholar
Gelman, Andrew, Goel, Sharad, Rivers, Douglas, Rothschild, David et al. 2016. “The Mythical Swing Voter.” Quarterly Journal of Political Science 11 (1): 103–30.Google Scholar
Gerber, Alan S. 2004. “Does Campaign Spending Work? Field Experiments Provide Evidence and Suggest New Theory.” American Behavioral Scientist 47 (5): 541–74.Google Scholar
Gerber, Alan S., Kessler, Daniel P., and Meredith, Marc. 2011. “The Persuasive Effects of Direct Mail: A Regression Discontinuity Based Approach.” The Journal of Politics 73 (1): 140–55.CrossRefGoogle Scholar
Gerber, Alan S., Karlan, Dean, and Bergan, Daniel. 2009. “Does the Media Matter? A Field Experiment Measuring the Effect of Newspapers on Voting Behavior and Political Opinions.” American Economic Journal: Applied Economics 1 (2): 35–52.Google Scholar
Gerber, Alan S., and Green, Donald P.. 2000. “The Effects of Canvassing, Telephone Calls, and Direct Mail on Voter Turnout: A Field Experiment.” American Political Science Review 94 (3): 653–63.CrossRefGoogle Scholar
Gerber, Alan S., Green, Donald P., and Larimer, Christopher W.. 2008. “Social Pressure and Voter Turnout: Evidence from a Large-Scale Field Experiment.” American Political Science Review 102 (1): 33–48.Google Scholar
Gerber, Alan S., Gimpel, James, Green, Donald, and Shaw, Daron. 2011. “How Large and Long-lasting Are the Persuasive Effects of Televised Campaign Ads? Results from a Randomized Experiment.” American Political Science Review 105 (1): 135–50.Google Scholar
Green, Donald, Palmquist, Bradley, and Schickler, Eric. 2002. Partisan Hearts & Minds: Political Parties and the Social Identities of Voters. New Haven, CT: Yale University Press.Google Scholar
Green, Donald P., and Gerber, Alan S.. 2015. Get Out the Vote: How to Increase Voter Turnout. 3rd ed. Brookings.Google Scholar
Green, Donald P., Krasno, Jonathan S., Coppock, Alexander, Farrer, Benjamin D., Lenoir, Brandon, and Zingher, Joshua N.. 2016. “The Effects of Lawn Signs on Vote Outcomes: Results from Four Randomized Field Experiments.” Electoral Studies 41: 143–50.Google Scholar
Green, Donald P., McGrath, Mary C., and Aronow, Peter M.. 2013. “Field Experiments and the Study of Voter Turnout.” Journal of Elections, Public Opinion and Parties 23 (1): 27–48.Google Scholar
Hersh, Eitan D. 2015. Hacking the Electorate: How Campaigns Perceive Voters. New York: Cambridge University Press.Google Scholar
Hersh, Eitan D., and Schaffner, Brian F.. 2013. “Targeted Campaign Appeals and the Value of Ambiguity.” The Journal of Politics 75 (2): 520–34.Google Scholar
Hill, Seth J., Lo, James, Vavreck, Lynn, and Zaller, John. 2013. “How Quickly We Forget: The Duration of Persuasion Effects from Mass Communication.” Political Communication 30 (4): 521–47.Google Scholar
Hillygus, D. Sunshine, and Shields, Todd G.. 2008. The Persuadable Voter: Wedge Issues in Presidential Campaigns. Princeton, NJ: Princeton University Press.Google Scholar
Holbrook, Thomas M. 1996. Do Campaigns Matter? Thousand Oaks, CA: Sage.Google Scholar
Issenberg, Sasha. 2012. The Victory Lab: The Secret Science of Winning Campaigns. New York: Crown.Google Scholar
Iyengar, Shanto, and Simon, Adam F.. 2000. “New Perspectives and Evidence on Political Communication and Campaign Effects.” Annual Review of Psychology 51 (1): 149–69.CrossRefGoogle ScholarPubMed
Iyengar, Shanto, Sood, Gaurav, and Lelkes, Yphtach. 2012. “Affect, Not Ideology: A Social Identity Perspective on Polarization.” Public Opinion Quarterly 76 (3): 405–31.CrossRefGoogle Scholar
Jacobson, Gary C. 2015. “How do Campaigns Matter?” Annual Review of Political Science 18: 31–47.Google Scholar
Johnston, Richard, Hagen, Michael G., and Jamieson, Kathleen Hall. 2004. The 2000 Presidential Election and the Foundation of Party Politics. New York, NY: Cambridge University Press.Google Scholar
Kalla, Joshua L., and Sekhon, Jasjeet S.. 2017. “Do TV Ads Persuade? A Randomized Experiment in the 2016 Presidential Election.” Working Paper.Google Scholar
Keane, Laura, and Nickerson, David W.. 2013. “A Field Experiment on Nonpartisan Mobilization and Persuasion Down-Ballot.” Working Paper, URL: https://www.researchgate.net/publication/254065918_A_Field_Experiment_on_Nonpartisan_Mobilization_and_Persuasion_Down-Ballot.Google Scholar
Klapper, Joseph T. 1960. The Effects of Mass Communication. New York, NY: Free Press.Google Scholar
Klar, Samara, and Krupnikov, Yanna. 2016. Independent Politics. Cambridge University Press.Google Scholar
Lau, Richard R., Sigelman, Lee, and Rovner, Ivy Brown. 2007. “The Effects of Negative Political Campaigns: A Meta-Analytic Reassessment.” Journal of Politics 69 (4): 1176–209.Google Scholar
Lauderdale, Benjamin E. 2016. “Partisan Disagreements Arising from Rationalization of Common Information.” Political Science Research and Methods 4 (3): 477–92.Google Scholar
Lazarsfeld, Paul F., Berelson, Bernard R., and Gaudet, Hazel. 1948. The People’s Choice: How the Voter Makes Up His Mind in a Presidential Campaign. New York, NY: Columbia University Press.Google Scholar
Leeper, Thomas J., and Slothuus, Rune. 2015. “Can Citizens Be Framed? How Information, Not Emphasis, Changes Opinions.” Unpublished paper, Aarhus University.Google Scholar
Lenz, Gabriel S. 2012. Follow the Leader? How Voters Respond to Politicians’ Performance and Policies. Chicago, IL: University of Chicago Press.Google Scholar
Miller, Roy E., and Robyn, Dorothy L.. 1975. “A Field Experimental Study of Direct Mail in a Congressional Primary Campaign: What Effects Last Until Election Day.” Experimental Study of Politics 4 (3): 1–36.Google Scholar
Nickerson, David W. 2005. “Partisan Mobilization Using Volunteer Phone Banks and Door Hangers.” Annals of the American Academy of Political and Social Science 601 (1): 10–27.Google Scholar
Nickerson, David W. 2007 a. “Don’t Talk to Strangers: Experimental Evidence of the Need for Targeting.” Presented at the 2007 Annual Meeting of the Midwest Political Science Assocation. Available at https://www.scribd.com/document/98714549/Nickerson-independents.Google Scholar
Nickerson, David W. 2007 b. “Quality Is Job One: Professional and Volunteer Voter Mobilization Calls.” American Journal of Political Science 51 (2): 269–82.CrossRefGoogle Scholar
Nickerson, David W., and Rogers, Todd. 2010. “Do You Have a Voting Plan? Implementation Intentions, Voter Turnout, and Organic Plan Making.” Psychological Science 21 (2): 194–9.Google Scholar
Niven, David. 2013. “Hiding Racism from Ourselves: A Field Experiment on Race Effects in a Statewide Election.” Paper presented at the Annual Meeting of the Midwest Political Science Association, Chicago, April 11–14, 2013.Google Scholar
Page, Benjamin I., and Shapiro, Robert Y.. 1992. The Rational Public: Fifty Years of Trends in Americans’ Policy Preferences. Chicago, IL: University of Chicago Press.Google Scholar
Paluck, Elizabeth Levy, and Green, Donald P.. 2009. “Deference, Dissent, and Dispute Resolution: An Experimental Intervention using Mass Media to Change Norms and Behavior in Rwanda.” American Political Science Review 103 (4): 622–44.CrossRefGoogle Scholar
Panagopoulos, Costas. 2016. “All about that base: Changing campaign strategies in US presidential elections.” Party Politics 22 (2): 179–90.Google Scholar
Petty, Richard E. and Cacioppo, John T.. 1986. “The Elaboration Likelihood Model of Persuasion.” In Advances in Experimental Social Psychology Volume 19, ed Leonard Berkowitz. New York, NY: Academic Press, 123–205.Google Scholar
Potter, Philip B. K. and Gray, Julia. 2008. “Does Costly Signaling Matter? Preliminary Evidence from a Field Experiment.” Working paper, available at http://www.belfercenter.org/sites/default/files/files/publication/Potterr%202008%20FINAL%20DOC.pdfGoogle Scholar
Rogers, Todd, and Nickerson, David W.. 2013. “Can Inaccurate Beliefs about Incumbents Be Changed? And Can Reframing Change Votes?” Working Paper RWP13-018, Harvard Kennedy School. URL: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2271654.Google Scholar
Rogers, Todd, and Middleton, Joel. 2015. “Are Ballot Initiative Outcomes Influenced by the Campaigns of Independent Groups? A Precinct-Randomized Field Experiment Showing That They Are.” Political Behavior 37 (3): 567–93.Google Scholar
Sadin, Meredith L. 2014. “Campaigning with Class: Ambivalent Stereotypes and Candidate Wealth in U.S. Elections.” PhD thesis, Princeton University.Google Scholar
Schickler, Eric. 2016. Racial Realignment: The Transformation of American Liberalism, 1932–1965. Princeton, NJ: Princeton University Press.Google Scholar
Sears, David O. and Funk, Carolyn L.. 1999. “Evidence of the Long-term Persistence of Adults’ Political Predispositions.” The Journal of Politics 61 (1): 1–28.Google Scholar
Shaw, Daron, Blunt, Christopher, and Seaborn, Brent. 2017. “Testing Overall and Synergistic Campaign Effects in a Partisan Statewide Election.” Working paper.Google Scholar
Shaw, Daron R., Green, Donald P., Gimpel, James G., and Gerber, Alan S.. 2012. “Do Robotic Calls From Credible Sources Influence Voter Turnout or Vote Choice? Evidence From a Randomized Field Experiment.” Journal of Political Marketing 11 (4): 231–45.Google Scholar
Shaw, Daron R., and Gimpel, James G.. 2012. “What if We Randomize the Governor’s Schedule? Evidence on Campaign Appearance Effects From a Texas Field Experiment.” Political Communication 29 (2): 137–59.Google Scholar
Sides, John, and Vavreck, Lynn. 2013. The Gamble. Princeton, NJ: Princeton University Press.Google Scholar
Simon, Paul. 1994. P.S.: The Autobiography of Paul Simon. Boulder, CO: Taylor Trade Publishing.Google Scholar
Smidt, Corwin D. 2017. “Polarization and the Decline of the American Floating Voter.” American Journal of Political Science 61 (2): 365–81.Google Scholar
Sniderman, Paul M. and Theriault, Sean M.. 2004. “The Structure of Political Argument and the Logic of Issue Framing.” In Studies in Public Opinion: Attitudes, Nonattitudes, Measurement Error, and Change, ed. Saris, Willem E. and Sniderman, Paul M., Princeton, NJ: Princeton University Press: 133–65.CrossRefGoogle Scholar
Strauss, Aaron B. 2009. “Political Ground Truth: How Personal Issue Experience Counters Partisan Biases” PhD thesis, Princeton University.Google Scholar
Ternovski, John, Green, Jennifer, and Kalla, Joshua. 2012. Working America’s 2011 Persuasion Canvass Experiment on SB5 in Ohio. Technical report Analyst Institute.Google Scholar
Vavreck, Lynn. 2009. The Message Matters: The Economy and Presidential Campaigns. Princeton, NJ: Princeton University Press.Google Scholar
Wantchekon, Leonard. 2003. “Clientelism and Voting Behavior: Evidence from a Field Experiment in Benin.” World politics 55 (3): 399–422.Google Scholar
Wlezien, Christopher, and Erikson, Robert S.. 2002. “The Timeline of Presidential Election Campaigns.” Journal of Politics 64 (4): 969–93.CrossRefGoogle Scholar
Zaller, John R. 1992. The Nature and Origins of Mass Opinion. New York, NY: Cambridge University Press.Google Scholar
Figure 0

TABLE 1.Theoretical Predictions

Figure 1

FIGURE 1.Meta-analysis Forest Plots: General Elections

Figure 2

FIGURE 2.Distribution of t Statistics from General Elections with Estimates Close to Election Day

Figure 3

FIGURE 3.Meta-Analysis Forest Plots: Primary and Ballot Measure Elections

Figure 4

TABLE 2.Results of Original Canvass Experiments in 2015 and 2016: Effects in Standard Deviations (d)

Figure 5

FIGURE 4.Effects Within 2 Months of Election Day With Original Studies, By Contact Type

Figure 6

TABLE 3.Early Persuasion Decays; Late Persuasion Fails

Supplementary material: PDF

Kalla and Broockman supplementary material 1

Appendix

Download Kalla and Broockman supplementary material 1(PDF)
PDF 2.2 MB
Supplementary material: Link

Kalla and Broockman Dataset

Dataset

http://dx.doi.org/10.7910/DVN/SMXWA9
Link
Submit a response

Comments

No Comments have been published for this article.