This paper experimentally demonstrates one way that opposing partisans can come to state different beliefs about political facts and processes through rationalization. These kinds of partisan disagreements about the state of the political world have come under increasing scrutiny as partisan polarization in the United States has reached historically high levels (McCarty, Poole and Rosenthal Reference McCarty, Poole and Rosenthal2006). Both academic and non-academic observers have become increasingly concerned that Americans see different political realities, rather than simply disagreeing about political choices. In the context of US politics, scholars have documented cases in which Democrats and Republicans express strikingly different beliefs about the performance of the economy (e.g., Bartels Reference Bartels2002; Evans and Pickup Reference Evans and Pickup2010), about trends in economic inequality (Bartels Reference Bartels2007), about the existence of weapons of mass destruction in Iraq (Kull, Ramsey and Lewis Reference Kull, Ramsey and Lewis2003), about the involvement of Saddam Hussein in the September 11th terrorist attacks (Kull, Ramsey and Lewis Reference Kull, Ramsey and Lewis2003; Prasad et al. Reference Prasad, Perrin, Bezila, Hoffman, Kindleberger, Manturuk and Powers2009), about the political positions of congressional representatives (Wilson and Gronke Reference Wilson and Gronke2000), and about the evidence for climate change (Hamilton Reference Hamilton2010). However, the origin of these disagreements is far less clear than their existence.
Partisan disagreements could plausibly arise several possible mechanisms. One explanation is that they are a survey artifact: “partisan cheerleading.” Respondents give answers that support their party, but they actually know that these answers are inaccurate. Two studies have found that giving monetary incentives for accurate survey responses somewhat diminishes, but does not eliminate, measured disagreements (Prior Reference Prior2007; Bullock et al. Reference Bullock, Gerber, Hill and Huber2013). Another potential explanation is that respondents have not received correct information about these facts from a partisan source that they trust, perhaps due to variations in media exposure and public disagreement about certain political facts. To test this hypothesis, researchers have attempted to correct existing partisan disagreements, demonstrating that these disagreements are barely reduced (and sometimes increased) in magnitude by subsequent provision of accurate information, even when that information comes from co-partisans (Nyhan and Reifler Reference Nyhan and Reifler2010; Nyhan, Reifler and Ubel Reference Nyhan, Reifler and Ubel2013). Thus, while it seems that partisan information sources may play some role in creating these disagreements, they do not seem to provide a sufficient explanation for the persistence of factual disagreement. A third explanation is that citizens are reasoning about political information in partisan ways: rationalizing backwards from their partisan preferences to beliefs about political facts and processes that would justify those preferences (Kuklinski et al. Reference Kuklinski, Quirk, Jerit, Schwider and Rich2000; Taber and Lodge Reference Taber and Lodge2006; Gaines et al. Reference Gaines, Kuklinski, Quirk, Peyton and Verkuilen2007; Jerit and Barabas Reference Jerit and Barabas2012; Lenz Reference Lenz2012). Whether consciously or unconsciously, citizens reasoning this way find their way to factual beliefs that will not call their political commitments into question. While there is growing evidence that rationalization is a major component of the explanation for observed partisan factual disagreements, there is a missing link in this research. There are no experimental demonstrations that these rationalization mechanisms can create the kinds of partisan factual disagreements that are observed at the population level. This paper provides such a demonstration.
Rationalizations can take on many forms, so in the next section, I describe two specific and related types of rationalization, “projection”, and “inferred justification” that can arise as a citizen reasons about a political issue. It makes sense to study these types together because they involve overlapping sets of political attitudes. Projection is a form of rationalization about the attitudes and behaviors of other people; typically in political psychology these are the attitudes and behaviors of political elites. The desire for cognitive consistency encourages individuals to believe that the political actors whom they like must have positions that match their own positions (Krosnick Reference Krosnick1990; Wilson and Gronke Reference Wilson and Gronke2000). As opposing political partisans have different “liked” political actors, this kind of rationalization can yield a pattern of beliefs where Republicans and Democrats who share the same issue position themselves, nonetheless disagree about which partisan elites share that position.Footnote 1 Jon Krosnick has criticized the evidence for the claim that this psychological process of projection meaningfully shapes political attitudes by noting significant flaws with the cross-sectional, observational designs used in past research (see Krosnick Reference Krosnick1990; Krosnick Reference Krosnick2002). One of the contributions of this paper is to improve the evidence that projection shapes measured public opinion.
Inferred justification is a form of rationalization that involves substantive facts about the political world that are related to some political issue or choice. For example, in early 2003, Republican citizens were more likely than Democratic citizens to state a belief that Iraq was involved in the September 11 attacks (Kull, Ramsey and Lewis Reference Kull, Ramsey and Lewis2003, 588). These survey results motivated Prasad et al. (Reference Prasad, Perrin, Bezila, Hoffman, Kindleberger, Manturuk and Powers2009) to do extensive follow-up interviews with Republicans who held these beliefs, in order to try to understand their origin. While the authors of that study find indications of a number of mechanisms, one of the most striking and prevalent is the idea that such Republicans were trying to infer factual justifications for their partisan positions. When pressed, these Republicans often stated that there “must be a reason” for the Iraq invasion, assuming that there must be a justification that they would agree with for the position on the Iraq War taken by their co-partisans.Footnote 2 Inferred justification might involve beliefs about political facts (was the Iraqi government involved in the September 11 attacks?) or beliefs about political processes (does regime change in hostile states tend to reduce the threat of terrorism?). The unifying feature is that beliefs about political facts and processes are both components of a substantive justification for a policy position.
After developing these theoretical arguments about rationalization, I present the design of a survey experiment that puts citizens in a position to engage in these kinds of rationalizations. One treatment condition provides elite positions on a political issue, and assesses rationalization effects on beliefs that might provide substantive justifications for positions on that issue (inferred justification). A different treatment condition provides facts that might provide substantive justifications, and assesses rationalization effects on beliefs about elite positions on that issue (projection). The fact that Democrats and Republicans hold different prior beliefs about how the elite position in question (President Obama’s) relates to their own positions makes different rationalizations more appealing as ways to resolve potential inconsistencies. This in turn creates the potential to generate new partisan disagreements about the positions of elites (via projection) and about the factual justifications for issue positions (via inferred justification). Based on this design, I report evidence from two projection and two inferred justification experiments: increased partisan factual disagreements in the theoretically expected directions, of substantively significant magnitude (4–8 percentage points), across the treatment versus control group comparisons implicated by the projection and inferred justification mechanisms. I conclude with a discussion of the external validity of the experiment and some implications for our understanding of how citizens reason about the empirical features of the political world.
Rationalization and Partisan Disagreement
A wide variety of psychological models predict that in at least some circumstances, changes to an attitude can induce individuals to change their related attitudes in order to maintain internal consistency (Osgood and Tannenbaum Reference Osgood and Tannenbaum1955; Festinger Reference Festinger1957; Heider Reference Heider1958; Kunda Reference Kunda1987).Footnote 3 This is relevant here, because if a citizen changes her attitude about a political issue, all the other attitudes she holds that are related to that issue might also “need” to change in order to maintain cognitive consistency. While we would not expect strongly held attitudes like partisanship to change in response to a changing position on a single issue, less firmly held beliefs about elite positions and political facts or processes might change relatively easily.
This kind of rationalization is intrinsically an indirect phenomenon. If an individual is to be observed changing her beliefs about some political fact or process due to rationalization, something has to first cause her to change the issue position that she is rationalizing. Without something to induce a change in one’s own position on an issue, there is no reason to change one’s beliefs about the positions of (dis)liked elites (projection) or facts and processes substantively relevant to that issue (inferred justification). Thus, to have any prospect of an experimental test for the presence of rationalization, we need to consider experimental stimuli that might cause a citizen to change her stated position on an issue.
Thus, the experimental design presented in this paper begins with two situations in which citizens change their stated positions on political issues in response to new information. The first of these situations are those where citizens revise their stated political preferences in response to learning substantively relevant factual information. There are many experiments in the political science literature demonstrating this kind of “issue learning” can occur (e.g., Gilens Reference Gilens2001; Kam Reference Kam2005; Bullock Reference Bullock2011), although the results of any particular experiment will depend on the particular substantive fact, the implicated political issue, and broader aspects of the experimental design. The second of these situations are those where citizens revise their stated preferences toward those of trusted, co-partisan elites and/or away from those of distrusted out-partisan elites, usually called “cue-taking” (e.g., Cohen Reference Cohen2003; Kam Reference Kam2005; Bullock Reference Bullock2011; Nicholson Reference Nicholson2012). Again, the magnitude of these effects vary substantially by context. The key feature of both of these phenomena is that they are mechanisms by which citizens are exposed to facts about the political world—substantive facts relevant to an issue or elite positions on an issue—and revise their own attitudes toward the issue. These processes (issue learning and cue-taking) create the potential for the two kinds of rationalization (projection and inferred justification, respectively) that are the subject of this study.
To see why issue learning and cue-taking are useful starting points for an experiment trying to demonstrate the presence of projection and inferred justification, consider Figure 1. The figure shows how these attitude change processes are linked together and motivates the structure of the survey experiment. On the left of the figure, we see the processes arising from learning an elite position (a cue treatment). When a citizen learns that a trusted elite takes a particular position, she may revise her own position (cue-taking). If she believes that a particular substantive fact being true would help to justify that position, she may also revise her beliefs about that fact to match her changed position (inferred justification). Thus, if we set up a cue-taking experiment where the treatment group learns an elite partisan’s position, while the control group does not, the inferred justification hypothesis implies that we should observe differences between treatment and control not only for respondents’ own issue positions, but also in their beliefs about the facts that are substantively relevant to justifying their positions. Because the cue of a particular elite partisan’s position will have different effects on respondents as a function of their own partisanship, the inferred justifications we observe will also be different for opposing partisans.
A graphical representation of the attitude changes arising from learning either a substantive fact relevant to a political issue or an elite position on that political issue Note: the solid arrows show the immediate effects of new information about an elite position (top left) or a substantively relevant fact (bottom right), while the dashed and dotted lines show the intermediate effects on citizens’ issue positions and the indirect effects on other attitudes via rationalization, respectively
On the right of the figure, we see opinion change processes running in the opposite direction through the same set of attitudes. When a citizen learns about a fact or process that is substantively related to a political issue (a fact treatment), she may revise her position with respect to that issue (issue learning). If the citizen is confident that a particular elite actor usually shares her issue positions, the citizen may also revise her beliefs about the elite actor’s position (projection). Here, again, there is an experimentally verifiable implication. If we provide a treatment group a fact that is relevant to some issue, we should observe differences between treatment and control not only for the respondents’ own issue positions, but potentially also in their beliefs about the issue positions of elite partisan actors. Because opposing partisans have different expectations about which elites they will agree with, if they change their positions similarly in response to learning the substantively relevant fact, their views about a particular partisan elite’s position will change in different ways.
Thus, the prediction of both of these rationalization mechanisms is a differential treatment effect by partisanship. When a Democratic citizen and a Republican citizen learn an issue position of the Democratic President Barack Obama (cue-taking), they will tend to diverge in their beliefs about facts that they believe are substantively relevant to that political issue (inferred justification). In particular, supporters of the president will become more likely than opponents to believe that the facts that would justify the president’s issue position are true. Similarly, when a Democrat and a Republican observe a fact that they believe is substantively relevant to a political issue, and agree about that fact’s implications for that political issue, they will tend to diverge in their expectations about the likely position of President Obama with respect to that political issue (projection). The Democrat will have a greater tendency than the Republican to infer that the president holds the issue position indicated by the newly learned substantive, issue-relevant fact. Thus, the projection and inferred justification mechanisms describe situations where providing the same factual information (a elite position or substantive-relevant fact/process) can lead to different changes in beliefs among Democrats and Republicans about an indirectly related political fact (a substantively relevant fact/process or elite position).
This kind of differential causal effect by partisanship at the individual level is precisely what is needed to create population-level partisan factual disagreements where they do not previously exist, or to exacerbate existing disagreements. In the next section, I describe a survey experiment that provides the relevant kind of information and then tests for heterogeneous treatment effects as a function of pre-treatment partisanship.
Survey Design
There are three important goals in choosing issues to explore projection and inferred justifications in a survey experiment. First, the issue can neither be so obscure that citizens have no relevant prior beliefs, nor so familiar that citizens have immutable positions. Second, the provided cues and substantively relevant facts must not be known by all citizens already, or we have no reason to expect them to have any effects. Third, the substantive linkages between facts and policies must be ones that citizens have intuitions for, even if the actual relationships are very complex. The two issues that I adopted were the question of whether to support the Egyptian revolution and whether to support increasing international trade. The exact policy realization of that support was left vague in order to keep question length down and avoid overwhelming respondents with detail.
The substantively relevant facts were polling data on Egyptian citizens’ negative attitudes toward the United States and data on US manufacturing job losses during a period of increasing international trade from 2000 to 2008. The assumption in using these facts was that they would suggest to most US citizens that Egyptian democracy and international trade might be bad for the United States. On substantive grounds, the relevance of these facts can easily be disputed: Egyptian citizens might have negative views toward the United States because of the historical failure to support democracy in Egypt, and the correlation between US manufacturing job loss and increasing international trade might not be causal or might be outweighed by benefits of increased trade.Footnote 4 However, the fact that citizens’ beliefs about the relevance of these facts might not be correct is immaterial to testing whether rationalization is occurring: all that matters is whether and how most citizens believe that the facts are relevant.
For both the Egypt and trade cases, the cue provided was a statement by President Obama, which introduces a partisan asymmetry in the experiment that is important for interpreting the results. In both the Egypt and the trade experiments, the presented substantively relevant facts seem to argue against Obama’s positions, creating more dissonance for Democrats to resolve through rationalization. The cue treatments are similarly asymmetric with respect to party: we can expect that the cues from Obama would have larger magnitude effects for Democrats. The expectations for Republicans are less clear. Some previous research has found that partisans respond negatively to other-party elite positions (Kam Reference Kam2005; Nicholson Reference Nicholson2012), however, Republicans could also treat Obama’s position as uninformative, especially if there is no indication given of partisan disagreement on the issue. In either case, however, the sign of the difference between Democrats and Republicans is clear: among those who learn Obama’s position, Democrats will move toward Obama’s position to a greater extent than Republicans.
Subjects were independently randomized into four treatment groups, with each subject receiving the same treatment condition for a set of questions on Egypt and a set on trade. The four treatment conditions were: no treatment (control), a presidential cue treatment (cue), a substantive fact treatment (fact), and both treatments. Subjects answered three questions on Egypt and three on trade. The three questions queried their beliefs about a substantively relevant fact or process, about President Obama’s position, and about their own positions on the issue. The core tests of the predictions are comparisons of the groups receiving a single treatment with the control group. The response distributions for the group receiving both treatments are reported in the supplemental appendix, but are not used in the hypothesis testing. The exact text of the questions is provided in Table 1, with response options in bold, and text visible to the respondent italicized. The survey was administered by computer, with the questions asked in the order given in Table 1, keeping the intro and treatment information visible on the screen throughout each set of questions. The two question sets were provided in random order, with the three questions within each set appearing in fixed order.
Full Question and Treatment Wording for the Survey Experiment
Note: text seen by survey respondents is italicized; response options are bolded.
Because the survey instrument allows a “don’t know” response, there are several possible ways to aggregate the observed counts over three categories into a single statistic. When analyzing the results of the experiment, I report averages based on treating “don’t know” responses as intermediate to yes/no and support/oppose (i.e., as y=0.5). In the appendix, I report fractional support/opposition and agreement/disagreement among those who offer a response other than “don’t know,” the results are very similar.
Tests of Projection
How does this experiment test the hypothesis that projection can lead to an increase in partisan disagreement about the position of a political elite? Here, we consider the comparison versus the control group for the treatment group that received the issue-relevant facts about Egyptian citizens’ negative attitudes toward the United States and about US manufacturing job losses during 2000 to 2008. In Figure 1, this corresponds to a citizen receiving information about a substantive fact at the bottom of the figure, and then engaging in the reasoning processes ascending along the right side of the figure.
The most direct effects of learning this information are those on the factual questions “Do you believe that the new government of Egypt is going to be less favorable towards the US than the old government?” and “Do you believe that increasing international trade leads to job losses in US manufacturing?” One can think of these questions as a strong form of a manipulation check to ensure that respondents processed the information that constituted the treatment (Mutz Reference Mutz2011, 84); many manipulation are more minimal checks that respondents have read or listened to the prompt. These questions do not ask directly about the fact provided, but about a very closely related belief. The information provided is intended to be as politically neutral as possible so that respondents of both parties might be influenced by it, but there may be partisan variation in other beliefs that affects responsiveness to the information. Where F
i
is an individual response to the factual questions, and
$\overline{{F\!\mid\!S}} $
the average response F over some subset of respondents S, I estimate the treatment effect on the directly implicated fact, for Democrats and Republicans, as follows:
$$\eqalignno{ & \hat{\delta }_{{Fact\,manipulation,\,Dem}} =\left( {\overline{{F\!\mid\!Fact,\,Dem}} {\minus}\overline{{F\!\mid\!Control,\,Dem}} } \right), \cr & \hat{\delta }_{{Fact\,manipulation,\,Rep}} =\left( {\overline{{F\!\mid\!Fact,\,Rep}} {\minus}\overline{{F\!\mid\!Control,\,Rep}} } \right). $$
Following the logic of Figure 1, reading the right side from bottom to top, if this manipulation is successful, it may also lead to a change in respondents’ beliefs about the implicated political issue, which I have referred to as issue learning. In the Egypt case, this is measured by the survey question asking about whether the respondent supported the Egyptian revolution; in the trade case, this is measured by the survey question asking about whether the respondent supports increasing international trade. We can think of any such changes as an intermediate effect of the treatment. If we define P
i
be the stated position of a respondent and
$\overline{{P\!\mid\!S}} $
the average position over some set of respondents S, than these issue-learning effects can be estimated as follows:
$$\eqalignno{ & \hat{\delta }_{{Issue\,learning,\,Dem}} =\left( {\overline{{P\!\mid\!Fact{\rm ,}\,Dem}} {\minus}\overline{{P\!\mid\!Control{\rm ,}\,Dem}} } \right), \cr & \hat{\delta }_{{Issue\,learning,\,Rep}} =\left( {\overline{{P\!\mid\!Fact{\rm ,}\,Rep}} {\minus}\overline{{P\!\mid\!Control{\rm ,}\,Rep}} } \right). $$
Finally, again following the logic of Figure 1, if the treatment has influenced the issue positions of Democrats and Republicans, it could, via projection, influence their beliefs about President Obama’s position on the issue. For Democrats, this influence will be in the direction of their own change in position; for Republicans, we do not expect to see a change in the direction that their own position changed, and there may be a change in the opposite direction. If we define O
i
as the respondent’s stated belief about Obama’s position and
$\overline{{O\!\mid\!S}} $
the average position over some set of respondents S, than these projection effects can be estimated as follows:
$$\eqalignno{ & \hat{\delta }_{{Projection,\,Dem}} =\left( {\overline{{O\!\mid\!Fact{\rm ,}\,Dem}} {\minus}\overline{{O\!\mid\!Control{\rm ,}\,Dem}} } \right), \cr & \hat{\delta }_{{Projection,\,Rep}} =\left( {\overline{{O\!\mid\!Fact{\rm ,}\,Rep}} {\minus}\overline{{O\!\mid\!Control{\rm ,}\,Rep}} } \right). $$
The clearest expectation of the theory is with respect to the difference in these differences: that Democrats will change their assessment of Obama’s position more in the direction that their own positions change than will Republicans, leading to an increase in partisan disagreement about Obama’s position. To test this, we define the projection effect on partisan disagreement as:
$$\eqalignno{ \hat{\Delta }_{{Projection}} & =\delta _{{Projection{\rm ,}\,Dem}} {\minus}\delta _{{Projection{\rm ,}\,Rep}} \cr & =\left( {\overline{{O\!\mid\!Fact{\rm ,}\,Dem}} {\minus}\overline{{O\!\mid\!Control{\rm ,}\,Dem}} } \right){\minus}\left( {\overline{{O\!\mid\!Fact{\rm ,}\,Rep}} {\minus}\overline{{O\!\mid\!Control{\rm ,}\,Rep}} } \right). $$
The null hypothesis to be tested is whether this difference in differences is zero, the alternative hypothesis is that
$\hat{\Delta }_{{}} Projection$
is negative (because the provided facts are evidence against the policy in both the Egypt and trade cases). These difference in differences can be estimated either by taking the difference in differences in sample means as specified in the equation above, or using the estimated coefficient on an interaction of party (Democrat=1, Republican=0) and treatment status (fact treatment=1, control=0), these are mathematically identical.
Tests of Inferred Justification
For the tests of inferred justification, we instead evaluate the cue treatment versus control comparison, and we consider the same survey questions in the reverse order (reading the left side of Figure 1 from top to bottom). As the provided information is an Obama quote, the manipulation check is whether this changed respondents’ assessment of Obama’s position on the issue. Using the same notation as above, this estimate for the effect of the treatment is:
$$\eqalignno{ & \hat{\delta }_{{Cue\,manipulation,\,Dem}} =\left( {\overline{{O\!\mid\!Cue{\rm ,}\,Dem}} {\minus}\overline{{O\!\mid\!Control{\rm ,}\,Dem}} } \right), \cr & \hat{\delta }_{{Cue\,manipulation,\,Rep}} =\left( {\overline{{O\!\mid\!Cue{\rm ,}\,Rep}} {\minus}\overline{{O\!\mid\!Control{\rm ,}\,Rep}} } \right). $$
The intermediate outcome is the effect of the treatment on respondents’ own position on the issue:
$$\eqalignno{ & \hat{\delta }_{{Cue\,taking,\,Dem}} =\left( {\overline{{P\!\mid\!Cue{\rm ,}\,Dem}} {\minus}\overline{{P\!\mid\!Control{\rm ,}\,Dem}} } \right), \cr & \hat{\delta }_{{Cue\,taking,\,Rep}} =\left( {\overline{{P\!\mid\!Cue{\rm ,}\,Rep}} {\minus}\overline{{P\!\mid\!Control{\rm ,}\,Rep}} } \right). $$
The primary outcome of interest for testing whether inferred justification has occurred is then whether there is a causal effect of the Obama quote about the issue on respondents’ stated beliefs about the substantively relevant facts: the likely favorability of a democratic Egyptian government and the effect of international trade on jobs. The effects for each party are:
$$\eqalignno{ & \hat{\delta }_{{Inferred\,justification,\,Dem}} =\left( {\overline{{F\!\mid\!Cue{\rm ,}\,Dem}} {\minus}\overline{{F\!\mid\!Control{\rm ,}\,Dem}} } \right), \cr & \hat{\delta }_{{Inferred\,justification,\,Rep}} =\left( {\overline{{F\!\mid\!Cue{\rm ,}\,Rep}} {\minus}\overline{{F\!\mid\!Control{\rm ,}\,Rep}} } \right). $$
The overall inferred justification effect on the partisan disagreement between the parties is:
$$\eqalignno{ \hat{\Delta }_{{Inferred\,justification}} & =\delta _{{Inferred\,justification{\rm ,}\,Dem}} {\minus}\delta _{{Inferred\,justification{\rm ,}\,Rep}} \cr & =\left( {\overline{{F\!\mid\!Cue;\,Dem}} {\minus}\overline{{F\!\mid\!Control;\,Dem}} } \right){\minus}\left( {\overline{{F\!\mid\!Cue;\,Rep}} {\minus}\overline{{F\!\mid\!Control;\,Rep}} } \right). $$
The null hypothesis to be tested is again whether this difference in differences is zero, the alternative hypothesis is that
$\hat{\Delta }_{{Inferred\,justification}} $
is negative (because Obama’s statement in favor of the policies will tend to make Democrats less likely to believe the facts, which themselves seem to argue against Obama’s position). As in the test of projection,
$$\hat{\Delta }_{{Inferred\,justification}} $$
can be estimated as the difference in differences described above or as the interaction between partisanship and treatment status in a regression.
Implementation
The survey was fielded via the Time-Sharing Experiment for the Social Sciences (TESS) to a Knowledge Networks (KN) panel. TESS involves a peer-review process for proposals, and so I have included the initial proposal in the supplementary appendix to serve as pre-registration of intent to assess the differences between Democrats and Republicans specified above. The survey was available to 5313 potential respondents from June 23 until July 7, 2011, from whom 3600 responses were received (67.8 percent). Because the respondents were part of an existing KN panel, their (seven category) party identification was measured in previous surveys, rather than having been asked immediately before these questions, which mitigates concern about priming partisan attitudes. Of the sampled respondents, 50 percent self-reported as strong, not strong, or leaning Democrats, 47 percent as strong, not strong, or leaning Republicans, and 3 percent as pure independents. The pure independents are omitted from all of the following analyses, as they are so few in relative number. For the purposes of mapping the survey experiment onto the theoretical expectations described in the preceding section, I take party identification to be indicative of the degree to which citizens expect to share Obama’s issue positions. In order to preserve statistical power, the presented results do not use the probability weights for the KN panel when estimating treatment effects, so they should be understood as sample average treatment effects (Mutz Reference Mutz2011, 123). The corresponding population average treatment effects, using the probability weights, are provided in a supplemental appendix.
The reported p-values for the experimental treatments are based on permutation tests that simulate the distribution of differences between treatment and control samples under the null hypothesis of no population-level effect. Each iteration of the simulation randomly reassigns the labels for which individuals received treatment, and then recalculates the treatment effects as if those were the treatment groups. As under the null hypothesis, there is no difference between the individuals receiving different treatments, repeating this procedure simulates the distribution of treatment effects that would be measured if the null hypothesis were true. Reported p-values are based on 100,000 permutations. In the appendix, I show that these p-values are nearly identical to those for the interaction term in the regression approach described above.
Results
The supplemental appendix has tabulations of the response distributions for all questions by treatment condition and partisanship, as well as the precise subsample sizes by treatment condition and partisanship (varying from 408 to 475 respondents). Here, in the main text, I focus on the contrasts relevant to the processes of attitude change and rationalization discussed earlier. The treatment effects by partisanship defined in the preceding section are provided in Table 2. Now, for the projection and inferred justification experiments, I step in turn through the results for the manipulation checks, the intermediate outcomes (respondents’ positions) and the primary outcomes of interest (the indirectly implicated facts that might be the subject of rationalization).
Treatment Effects on the Response Distributions of Republicans and Democrats, Comparing a Single Treatment Group to Control, for Each of the Six Survey Items
Tests of Projection
First, did the fact treatment have any effect on responses to the immediately implicated factual questions? Respondents receiving polling data about Egyptian citizens’ negative attitudes toward the US government were more likely to say that the revolution would make Egypt less friendly by 6.4 percent among Democrats and 10.2 percent among Republicans. Respondents receiving the jobs and trade data were more likely to say that trade reduces US manufacturing jobs by 6.4 percent among Democrats and 2.7 percent among Republicans. Thus, while there is some variation by party and by experiment, we see evidence that the manipulation generally succeeded in changing responses to the facts directly implicated by the treatment information.
Second, looking at the intermediate outcome for the projection experiments, we see that on average treated Democrats expressed different issue positions than control Democrats, but this was not true for Republicans. Respondents receiving polling data about Egyptian citizens’ negative attitudes toward the US government were less likely to say they supported the revolution, by 4.6 percent for Democrats and 0.3 percent for Republicans. Respondents receiving jobs and trade data were less likely to express support for increased trade by 5.5 percent among Democrats and 0.2 percent among Republicans. One possible explanation for this partisan difference, which is not of the kind predicted by the rationalization hypotheses, is that Republicans simply had more firmly held preferences on these issues than Democrats.Footnote 5 Regardless of the reason for the lack of evidence for Republican changes in position, this has implications for the primary outcome of interest: the indirectly relevant fact of Obama’s position on these issues. If rationalization is at work, we would expect to see that if there is evidence of an increase in factual disagreement in the projection experiments, it will be primarily due to changes in Democrats’ perceptions of Obama’s position, not those of Republicans.
Third, is there evidence that beliefs about Obama’s position are polarized along partisan lines by the treatments, in the directions predicted by the projection hypotheses? The relevant treatment effects are provided numerically for Democrats and Republicans in Table 2 and graphically in Figure 2. Hypothesis tests related to whether there is a significant divergence between Democrats and Republicans are provided in Table 3.
Divergent treatment effects on factual beliefs of Democrats (D) and Republicans (R) when exposed to indirectly relevant factual information
Treatment Effects on the Magnitude of the Difference Between the Fractions of Republicans and Democrats Stating That They Believe in a Given Fact
Note: the reported p-values are based on permutation tests.
In the projection experiment on Egypt, revealing that Egyptian citizens had very negative attitudes about the US government increased the partisan difference in beliefs about Obama’s position by 4.5 percentage points (p=0.072). In the projection experiment on trade, revealing the decline in US manufacturing employment from 2000 to 2008 increased the partisan difference in beliefs about whether Obama supports increased international trade by 4.2 percentage points (p=0.086). As the two panels in the top row of Figure 2 show, in the control group there was no difference in beliefs about Obama’s positions, and so these differential treatment effects had the consequence of creating partisan disagreement about Obama’s position in the treatment group that received substantively relevant facts. In both of these experiments, it was primarily the Democrats whose factual beliefs were different in the treatment group versus the control group, which is consistent with the fact that it was mostly the Democrats whose own issue positions changed in response to treatment.
Tests of Inferred Justification
First, did respondents exposed to Obama’s statements on these issues change their beliefs about Obama’s position? Respondents learning Obama’s statement about the revolution were more likely to give responses that Obama supported the Egyptian revolution by 4.7 percent among Democrats and 3.4 percent among Republicans. Respondents receiving Obama’s statement about trade were more likely to give responses that Obama supports increased trade by 13.7 percent among Democrats and 9.0 percent among Republicans. There is some variation by party and by experiment, but we do see evidence that the manipulation generally succeeded in changing respondents’ beliefs about Obama’s position.
Second, having changed their beliefs about Obama’s position on these issues, was there a cue-taking effect on respondents’ own positions? Looking at the intermediate outcome, we see that cue-taking was much stronger in the trade experiment than in the Egypt experiment. Respondents receiving Obama’s statement on trade were more likely to indicate support for trade by 9.8 percent among Democrats and 1.7 percent among Republicans. In contrast, the corresponding differences associated with receiving Obama’s statement about Egypt were only 1.2 percent among Democrats and 1.7 percent among Republicans, both indicating little success at manipulating respondents’ issue positions on this issue. In both of these cases, we had no reason to expect large effects for Republicans due to the asymmetry of cues’ credibility; however, the small treatment effects for Democrats and Republicans on Egypt suggest that any increase in factual disagreement when we turn to examining the inferred justification hypothesis may be resulting from some other mechanism.
Third, is there evidence that citizens rationalized their changed positions by changing their responses about substantively relevant facts? In the inferred justification experiment on trade, revealing that Obama expressed support for increased international trade increased the partisan difference in beliefs about whether international trade leads to US job losses by 5.1 percentage points (p=0.095). In the potentially problematic inferred justification experiment on Egypt, revealing that Obama expressed support for the Egyptian Revolution increased the partisan difference in beliefs about whether the revolution would make Egypt less friendly by 8.6 percentage points (p=0.003). Here, again, the effects were mostly concentrated among Democrats (Table 2), which is expected given that under treatment versus control, Democrats had much larger changes in issue position in need of rationalization. In the trade inferred justification experiment, as in the two projection experiments, there was no difference between partisans in the control group (Figure 2), and so the treatment effect created partisan disagreement where it was not previously present. In the Egypt inferred justification experiment, there was some pre-existing disagreement which, combined with the weak results on the intermediate outcome, raise the possibility that other mechanisms might have contributed to the large increase in partisan disagreement about whether the revolution would make Egypt less friendly toward the United States.
Summary
While the increase in disagreement is largest for the treatment involving Obama’s statement on Egypt, we have reason to suspect that this is not due to the hypothesized causal mechanism, because the cue-taking effect is weak for that treatment and there was pre-existing disagreement on the outcome of interest. Thus, being conservative by setting that experiment aside, we are left with three treatment effects, of 4.5, 4.2, and 5.1 percentage points, all in the predicted directions, with one-sided p-values of 0.072, 0.086, and 0.095, respectively. To test the joint evidence across these three experiments, I simulate the distribution of the mean of these three treatment effects under the null hypothesis that all effects are zero by a permutation test over the cue treatment group, the fact treatment group, and control. Under this simulation of the joint null, the probability of observing a mean treatment effect across these three experiments greater than or equal to the actual average effect is p=0.011 (including the omitted fourth case, the joint p-value falls to <0.001).Footnote 6
Discussion and Conclusion
Together, the results of these experiments provide evidence in favor of the proposition that the rationalization mechanisms of projection and inferred justification contribute to partisan disagreement about the state of the political world. The provision of factual information that is relevant to a political issue can increase partisan disagreement about other facts that are relevant to that issue. In these experiments, providing information about historical changes in international trade and in US manufacturing jobs from 2000 to 2008 and about surveys of Egyptians attitudes toward the United States, increased expressed partisan disagreements about President Obama’s likely position on increasing international trade and the Egyptian revolution, respectively. Providing statements by President Obama about Egypt and about international trade increased expressed partisan disagreement about whether post-revolutionary Egypt would be less friendly toward the United States and about the cost in jobs to the United States of increased trade, respectively.
If we interpret these findings in terms of the motivation presented earlier in this paper, we come to the following specific conclusions with respect to the Egyptian revolution and international trade issues. At least some respondents were sufficiently unsure about the facts surrounding these issues such that they were responsive to the experimental treatments. Citizens’ partisanship shaped their interpretations when they were presented with facts, not just in terms of the direct implications of these informational treatments, but also in how they reconciled them with their other beliefs. In particularly, Democrats took cues from President Obama about their issue positions and responded to substantive facts about the policies in question, following cue-taking and issue-learning mechanisms that have been explored in previous research. But these same Democrats also seem to have rationalized their changed positions by modifying their responses to related factual questions. Democrats were more inclined to believe the best about Obama. Democrats tended toward profiles of responses in which their own position was supported by the substantive fact and that position matched Obama’s position. The consequence was increased partisan disagreement about the factual issues that were not the direct subject of the treatment. The concentration of these effects among Democrats may reflect a greater propensity to revise their beliefs, but it is at least in part because the experimental design put Democrats’ beliefs in greater tension with one another, giving them the stronger motivation to rationalize. Given previous research, there is no reason to believe that these effects are generally confined to Democrats as opposed to Republicans.
The reported experimental effects are non-trivial in magnitude, ranging from 4 to 8 percentage points. These are large enough effects to be important components of an explanation for observed partisan disagreements previously found by political scientists, particularly given the limits on what can be done in a short text-based survey experiment. While some of the pre-existing partisan disagreements discussed in the literature are substantially larger than this—Bartels (Reference Bartels2002) shows disagreements of 15–40 percentage points for different response thresholds of beliefs about inflation and unemployment—such disagreements could have arisen from years of political information.
Combined with the demonstration by Bullock et al. (Reference Bullock, Gerber, Hill and Huber2013) that a significant fraction of existing partisan disagreements can be eliminated by incentives to answer correctly and by the provision of a don’t know response option, the results presented here provide additional caution to scholars designing surveys to explore such disagreements. In the context of a survey, it is always appropriate to question whether one has recovered pre-existing views as opposed to causing individuals to generate views at the time of interview. Particularly when respondents have had no reason to express a view about the question previously, they are likely to be generating a view from the relevant considerations that they can recall as they answer the survey (Zaller Reference Zaller1992; Tourangeau, Rips and Rasinski Reference Tourangeau, Rips and Rasinski2000). Thus, responses will reflect contemporaneous rationalizations about the issues and facts at stake, as well as the incentives and response options provided by the survey. If providing factual information in a question can induce partisan disagreement in respondents’ responses about other facts, studies of partisan factual disagreement need to be conducted with great care in order to ensure that the partisan disagreements are not unintentionally created by the survey itself. As Figure 2 shows, for most of the facts manipulated in the experiment, there was little evidence of partisan disagreement in the control condition: it was the provision of factual information that created disagreement. In some survey instruments, similar pieces of factual information are provided in pre-question vignettes to help respondents contextualize their response. By making the political context of factual questions more salient, such vignettes could inadvertently increase measured partisan factual disagreement.
Daniel Patrick Moynihan is often credited with having observed that “everyone is entitled to his own opinion, but not his own facts.” While this sounds like a sensible basis for political contestation in a democracy, it neglects the fundamental difficulty of substantive engagement with political issues. Citizens cannot possibly know all the facts necessary to form views about the substantive merits of all the political issues in a modern democracy. The facts relevant to making decisions about governance are almost universally obscure. As a result, a citizen inevitably has his own opinions about facts. The experiment in this paper shows how rationalization—the desire to hold consistent beliefs—can quickly turn ignorance about facts into partisan opinions about facts. If facts are actually relevant to policy decisions—and we hope that citizens believe that they are—citizens will try to infer what facts are most likely to be true given their other beliefs about the political world. Of course rationalization is not the only way to get partisans to express disagree about political facts. Differential media consumption by party may create disagreements and the way we elicit citizens’ attitudes can shape how we measure them. Nonetheless, it is important to recognize that citizens’ desire to hold and express internally consistent attitudes may play an important role in creating the partisan disagreements that have been observed across a range of political facts.
