When do people decide that negative rhetoric has gone too far? The stakes in the answer to this question are high if, as Stevens et al. (Reference Stevens, Sullivan, Allen and Alger2008) contend, perceptions of the fairness of campaign claims are at the heart of why some negative campaigns appear to stimulate turnout while others have the opposite effect. Previous research has focused on three, non-mutually exclusive, possibilities: that negativity is deemed unfair if the claims are (a) untrue (Stevens et al. Reference Stevens, Sullivan, Allen and Alger2008, Reference Stevens, Allen, Sullivan and Lawrence2015), (b) taken out of context (Cappella and Jamieson Reference Cappella and Jamieson1994), or (c) deemed irrelevant (Fridkin and Kenney Reference Fridkin and Kenney2011; Jamieson and Hardy Reference Jamieson and Hardy2012).Footnote 1 In this article, we present the results of an experiment designed to assess the importance of these three considerations in evaluations of the fairness of negative claims. In the process, we also shed light on the impact of “fact-checks.” We find little evidence that context or relevance are components of fairness but some evidence that the truthfulness of claims is a consideration.
PREVIOUS RESEARCH
While there is consensus about what negativity in campaign communication is (e.g., Geer's (2006, 29) description of a negative appeal as, “any criticism or reason to vote against the opposition”), research has consistently found that people do not see the various claims characterized in this minimal definition as equally acceptable (e.g., Kahn and Kenney Reference Kahn and Kenney1999). Scholars suggest that to understand what constitutes legitimate or illegitimate negativity, we must distinguish between negative claims that are perceived to contain fair rather than unfair criticisms (Bartels Reference Bartels1998; Jamieson Reference Jamieson2000; Kahn and Kenney Reference Kahn and Kenney1999). Fridkin and Kenney (Reference Fridkin and Kenney2011) combine the dimensions of civility and relevance in their research, arguing, “the relevance of the message is paramount” (309), while Jamieson and Hardy (Reference Jamieson and Hardy2012) add that accepted “rules of argument” require that relevant evidence be presented in its context.
Fairness surely matters, but the current research suffers from three limitations. First, such studies have generally ignored how partisanship is likely to moderate judgments. Our own research (Stevens et al. Reference Stevens, Allen, Sullivan and Lawrence2015) shows that while individuals hold principles about what is fair and unfair in advertising, partisans systematically regard criticism of their preferred party's candidate as unfair and criticism of her opponent as fair, regardless of their fairness standards—conforming to what we term a “strong partisan bias” model. Second, previous research findings tend to refer to unmediated negative messages, although contemporary exposure to political advertising is increasingly characterized by media deconstruction, contextualization, and examinations of claim accuracy (Geer Reference Geer2012; Ridout and Franz Reference Ridout and Franz2011). Third, none of this research has examined different dimensions of fairness simultaneously in a controlled experiment.
THEORY: FACT-CHECKING AND FAIRNESS
We begin with the assumption, consistent with previous research (e.g., Freedman and Lawton Reference Freedman and Lawton2000), that individuals hold principles about what constitute fair and unfair political ad claims. These principles interact with partisan predispositions, which can be either congruent or non-congruent with the target of the criticisms, such that partisans’ judgments of a claim's fairness tend to be consistent with partisanship (Stevens et al. Reference Stevens, Allen, Sullivan and Lawrence2015). This conjecture implies four combinations of principles and partisanship that may occur with exposure to a negative claim in an ad. For individuals in the Unfair-Non-congruent group, partisan motivations point in the same direction as principles and we expect, predominantly, criticisms will be considered unfair. Similarly, for individuals in the Fair-Congruent group, partisan motivations point in the same direction as principles and we expect criticisms to be judged fair. However, individuals in the Unfair-Congruent and Fair-Non-congruent groups are cross-pressured. In the Fair-Non-congruent group, the claim concerns a topic they consider to be fair grounds for criticism but it has been made of the candidate whose partisanship they share. In the Unfair-Congruent case, criticisms violate their principles but are congruent with their partisan sympathies. We have previously shown that cross-pressured individuals will exhibit strong bias, turning to partisanship rather than principle for a guide (Stevens et al., Reference Stevens, Allen, Sullivan and Lawrence2015), if for no other reason, we argue, than their ignorance of the truthfulness of the ad claim. When the veracity of a criticism is uncertain partisans tend to fall back on the cue of partisan identification.
Fact-checks may disturb these tendencies, however. Even among the putatively unpressured cases (Unfair-Non-congruent and Fair-Congruent), we may generally expect individuals to be less likely to judge a criticism as fair if they are told its accuracy has been challenged by an “independent organization” (Stevens et al. Reference Stevens, Allen, Sullivan and Lawrence2015). In fact, we have found the effects of fact-check disconfirmation are most pronounced on partisans for whom the criticism is congruent, particularly those in the Fair-Congruent group, although partisan bias is reduced rather than eliminated. Such findings suggest that the truthfulness of claims is one component of fairness judgments: even partisans for whom the subject matter is usually considered fair game struggle to conclude that an actual claim, leveled at the opposition, is fair, when its truthfulness is brought into question. If truthfulness were irrelevant, judgments of the fairness of the criticism would be unaffected.
We employ a similar logic to test three hypotheses about the meaning and importance of truthfulness in evaluations of fairness. H1 “Fairness is equated with bare factual accuracy”: partisan bias in judgments of the fairness of criticisms will therefore be reduced if the accuracy of those criticisms is questioned. H2 “Fairness is equated with providing the complete context for an accurate claim”: even if the bare facts of a criticism are accurate, taking acts or words of an opponent out of context may distort the truth to a degree that citizens judge as equally unfair—or even more unfair—than a bare-facts falsehood. If not being taken out of context is a component of fairness, we would expect that highlighting the content of a criticism as “not the whole story” would undermine perceptions of its fairness. H3 “Fairness is equated with relevance for an accurate and context-specified claim”: criticisms may be deemed unfair because they are irrelevant to the judgment at hand (Fridkin and Kenney Reference Fridkin and Kenney2008, Reference Fridkin and Kenney2011). The latter conjecture suggests one reason why pundits argue that voters object more to personal than to issue-based criticism of candidates: the facts may be accurate but the claim is objectionable because the voter finds that it has no bearing on a candidate's ability to serve in office. If relevance is a component of fairness, we would expect that undermining the relevance of a criticism would diminish perceptions of its fairness.
RESEARCH DESIGN
We conducted survey experiments on a random national telephone sample of 500 respondents during the 2008 US election campaign, along with 200 respondents we re-interviewed from earlier surveys (for more details, see online Appendix). We pool the two samples. Excluding the panel respondents makes no difference to our substantive conclusions.
Respondents were initially asked about the fairness of criticisms by one candidate of another (the Fair-Unfair dimension above) for their “voting record,” “sources of campaign contributions,” and for being “out of touch with ordinary Americans,” without reference to specific candidates or parties, i.e., in principle. We then provided some specific examples of criticisms on the same topics, made by the Democratic and Republican presidential candidates (the Congruent-Non-congruent dimension above) and their supporters in 2008 and randomly varied the text that followed.
-
• The control group was simply presented with the negative message (e.g., for “criticizing an opponent for taking campaign contributions from certain special interests” the text was “Barack Obama claims to get most of his money from middle class donors in small amounts, but Republicans say he receives the majority of his funds from trial lawyers and other special interests. According to the Republican National Committee, Barack Obama also has several lobbyists on his campaign staff and has accepted in kind contributions from many others. Is this a very fair, somewhat fair, somewhat unfair, or very unfair criticism of Obama?”).
-
• The treatment groups were presented with the negative message followed by a statement that an “independent organization” had checked the facts and found them to be: (a) Untrue (H1) (e.g., “But after this ad was run, an independent organization checked the facts. They found that claims about PAC funding for Obama are untrue today and applied only to his earlier campaigns. Obama has returned funds from lobbyists and PACs since declaring his candidacy for President”), (b) true but Not the Whole Story (H2) (e.g., “But after this ad was run, an independent organization checked the facts. They found that although the information presented by Republicans is generally correct, claims about PAC funding for Obama applied only to his earlier campaigns. Obama has returned funds from lobbyists and PACs since declaring his candidacy for President”), or (c) true but Irrelevant to the candidate's ability to govern (H3) (e.g., “But after this ad was run, an independent organization checked the facts. They found that although the information presented by the Republicans is generally correct, no modern presidential campaign has been possible without contributions from a variety of donors, including lobbyists.”).Footnote 2
We did not remind respondents that criticisms of Obama and McCain were on the same topics whose fairness in principle they had previously been asked to assess.Footnote 3
The experiments covered all three treatments for criticisms of Obama and McCain for the sources of their campaign contributions, the Untrue (H1) and Not Whole Story (H2) treatments on the topic of their voting records—that candidates’ voting records on energy policy were irrelevant to the election was not credible—and the Not Whole Story (H2) and Irrelevant (H3) conditions for criticisms of the candidates as out of touch.Footnote 4
Although the primary purpose of the experiments was to examine different components of perceptions of the fairness of criticisms, they also speak to the efficacy of fact-checking. While the language of exposé sometimes assesses the basic truth of a claim, it frequently fails to label a claim so straightforwardly as “false,” offering instead a judgment of “misleading” or “not the whole story.” Fact-checks less often characterize the claims in ads as “irrelevant,” but attributing such critique to them is in keeping with their style and allows us to examine relevance as a criterion in judgments of fairness.Footnote 5 Previous research has offered mixed findings on the effectiveness of fact-checks. However, it has tended to conflate different kinds of assessments, e.g., criticisms that are “false” and criticisms that are “not the whole story” (Ansolabehere and Iyengar Reference Ansolabehere and Iyengar1995; Cappella and Jamieson Reference Cappella and Jamieson1994; McKinnon and Kaid Reference McKinnon and Kaid1999; O’Sullivan and Geiger Reference O’Sullivan and Geiger1995; Pfau and Louden Reference Pfau and Louden1994).
RESULTS
We model perceptions of the fairness of specific criticisms of the presidential candidates as a function of three variables, partisanship (congruent/non-congruent)Footnote 6 , perceptions of the fairness of the criticism in principle (fair/unfair), and the treatment. Partisanship is congruent with the criticism if the respondent identifies as Republican and the criticism is of Barack Obama, or if the respondent is a Democratic identifier and the criticism is of John McCain. We also include interactions between each of these variables.Footnote 7
Our main interest is in the effects of the conditions and their interactions with congruent partisanship and viewing a criticism as fair in principle. Does having a criticism characterized as untrue, not the whole story, or irrelevant render partisans less likely to regard it as fair? Table 1 presents the estimates from logit models for the three kinds of criticisms—of campaign contributions, voting record, and being out of touch—of the two candidates, making six models in total.
Perceptions of the Fairness of Criticisms of Candidates Given Different Factchecking Judgments (Logits)
Notes: Estimates are from logit models. Congruent partisan, Fair in principle, Untrue, Not Whole Story, and Irrelevant condition, are dummy variables with a value of one indicating that a respondent was exposed to information undermining the accuracy of the criticism, and zero the control group.
*p < 0.05 **p < 0.10 (two-tailed test).
To ease interpretation, we present the average predicted effects from the models of the different combinations of partisan congruence, fairness principles, and experimental conditions on the probability of judging each of the criticisms to be fair in three figures. In each figure, the predicted probability that the criticism is fair defines the vertical axis. Each group is represented by a marker. When a marker is higher vertically, the predicted perceived fairness is higher for that group. The horizontal axis is defined by whether a criticism is congruent. Differences within lines moving from left to right are the partisan congruence effects; the distances between markers of the same shape—circle or diamond—represent the effects of fairness principles for non-congruent and congruent partisans; the distance between markers of the same fill—solid or hollow—represents the effects of the experimental conditions for non-congruent and congruent partisans.Footnote 8
Figure 1 begins by showing the probability of judging criticisms of Obama and McCain's voting records and sources of campaign contributions to be fair, in the control and Untrue conditions (H1) from the first four columns of results in Table 1. Figure 1 shows, as we would expect, that congruent partisans were more likely to judge the criticisms as fair, although we see variation due to fairness principles: if congruent partisans said a topic was fair in principle they were generally more likely to see it as fair in practice. We also see some signs of effects of the treatment among congruent partisans: the circles that represent the control conditions show congruent partisans tending to perceive the criticisms as fairer than when the accuracy of the criticisms is brought into question. If the experimental treatment actually reduced partisan bias in perceptions of fairness we should see flatter lines between non-congruent and congruent partisans than in the control condition. We see some of that in Figure 1—in Table 1 all of the three-way interactions Congruent partisan x Untrue condition x Fair in principle are negative, i.e., in the expected direction, though none of them are statistically significant (the p-value for McCain's voting record is 0.27 and for Obama's campaign contributions it is 0.17). This is in keeping with our previous finding that undermining the factual accuracy of claims may attenuate but does not eliminate partisan bias (Stevens et al. Reference Stevens, Allen, Sullivan and Lawrence2015). It provides at most weak evidence in support of H1, however, although we note that while the interactions with partisanship are not statistically significant being told that the criticism of McCain for the sources of his campaign contributions was factually inaccurate diminished perceptions of the fairness of this claim for congruent and non-congruent partisans alike, which supports the notion that the veracity of claims is a component of fairness judgments.Footnote 9
Untrue Manipulation
Turning briefly to the results for non-congruent partisans in the other three examples—Obama's campaign contributions, Obama's voting record, McCain's voting record—Table 1 and Figure 1 show that neither fairness principles nor the experimental conditions made much difference: all the probabilities in Figure 1 are below the 0.5 threshold and there is relatively little variation. Thus, even if a topic of criticism had been deemed fair in principle by non-congruent partisans, they were no more likely to judge actual examples of that criticism as fair in practice when it was of their favored candidate. Hearing that an independent organization had said the facts behind the criticism were untrue made little difference.
All six models in Table 1 include estimates of the impact of the Not Whole Story condition, while Figure 2 shows the average marginal effects on perceptions of fairness that are implied by these estimates (H2). They illustrate modest effects of the Not Whole Story condition on perceptions of the fairness of criticism of the candidates’ voting records both on congruent and non-congruent partisans: the lines throughout Figure 2 tend to be of similar slope and the distance between markers of the same fill modest. Thus, while the effects on perceptions of the fairness of criticism of the candidates’ campaign contributions, or for being out of touch, present the familiar pattern of partisan bias, there is little evidence of effects of the Not whole story condition on judgments of fairness, even of the size of the impact of the Untrue condition. Neither do we see any main effects of the treatment. These results provide no support for H2.
Not Whole Story Manipulation
The estimates in columns 3–6 of Table 1 and the average marginal effects displayed in Figure 3 show the effects of the Irrelevant conditions on perceptions of the fairness of charges pertaining to sources of campaign contributions and being out of touch with ordinary Americans. The results for campaign contributions are underwhelming, indicating that contextualizing the criticism as irrelevant did little to alter partisan bias, but also that the manipulation was as likely to make congruent partisans view the criticism of Obama as fairer than in the control condition. For the more personal charge of being out of touch with ordinary Americans, the results for Obama also show that the condition left partisan bias in judgments of fairness largely unaffected: regardless of whether they said such criticism was fair in principle, Republicans were inclined to see the specific charge as fair criticism of Obama and Democrats as unfair criticism; indeed Republicans who viewed the criticism as fair in principle were somewhat more likely than other groups to view the specific charge as fair given the experimental manipulation questioning its relevance. For McCain being out of touch, contextualizing the claim had little impact on non-congruent partisans, who were inclined to think the charge was unfair, but the condition reduced perceptions of fairness among Democrats who had said such criticism was fair in principle. On the other hand, it did not do so among those who had said it was unfair. In sum, the evidence in Figure 3 indicates that relevance does not appear to be at the core of perceptions of the fairness of criticisms (H3).
Irrelevance Manipulation
CONCLUSION
We have explored three possible components of perceptions of the fairness of campaign criticisms by examining the effects of characterizing criticisms of candidates as untrue, not the whole story or irrelevant in the guise of real world fact-checking. We have previously shown that one kind of fact-checking judgment—that a claim is inaccurate and in that sense untrue—may attenuate but does not eliminate what we refer to as “strong partisan bias” (Stevens et al. Reference Stevens, Allen, Sullivan and Lawrence2015). In this paper, we examined a broader range of fact-checking judgments over more claims and in a different election and found further evidence of strong partisan bias in the widespread absence of effects. Fact-checking judgments in our experiments sometimes had an effect on perceptions of the fairness of criticisms but more broadly their impact was minimal or even opposite to expectations. These findings yield two implications. First, while evaluations of the truth of claims appear to be a component of fairness (although even characterizing a claim as untrue also often has no impact on perceptions of its fairness) we found little evidence that fairness necessitates that a claim be placed in its context or be deemed relevant. Second, presenting the context for an ad claim provided with fact-checks will have very little impact on the perceptions of partisans. In less partisan times, perhaps, fact-checks alerting voters that they are getting less than the “whole story” may hold more sway, but in the contemporary period covered by our experiments, partisan considerations dominate.
SUPPLEMENTARY MATERIALS
To view supplementary material for this paper, please visit http://dx.doi.org/10.1017/XPS.2015.6.
