Racial Priming Revived” is a response to a paper we published in the American Journal of Political Science about the relative effectiveness of implicit and explicit racial communication in altering public opinion.1
Our research tests the thought-provoking hypothesis advanced in Professor Mendelberg's award-winning book, The Race Card, and in her earlier work, that implicitly racial messages will be more effective than explicitly racial ones in priming racial resentment in opinion formation.2 In our analysis of data from two randomized experiments with over 6,300 respondents, however, we found little support for this claim. Instead, “implicit [racial] appeals are no more effective than explicit [racial] ones in priming racial resentment in opinion formation.”3Huber and Lapinski 2006, 439.
In this note, we rebut these criticisms of our article. We emphasize three points. First, Professor Mendelberg's essay provides a valuable review of existing research documenting the possibility of racial priming. However, the existence of racial priming is not in dispute. Indeed, our findings confirm that many white Americans hold negative views of African Americans, and that the importance of those views in opinion formation can be magnified through political communication. Rather, there is only one point at issue. It is Professor Mendelberg's specific claim that implicit appeals are more effective than explicit ones in priming racial predispositions in opinion formation.4
The general literature on racial priming is therefore irrelevant for deciding whether existing data support the hypothesis that explicit racial messages are less effective than implicit ones in priming racial resentment.Second, we demonstrate that our experiments were properly designed and implemented and do not consist entirely of “Null” results. Professor Mendelberg provides no data to support the claim of treatment failure and erroneously compares our experiment to one fielded by the same survey firm three years earlier. Furthermore, we show that participants in our experiments reported receiving their assigned treatments, that their responses to our post-treatment manipulation check confirm this self-reported treatment, and that the other problems of research design identified by Professor Mendelberg are inconsequential in explaining our findings.
Third, we argue that the prior research supporting the implicit-explicit model, all of which is authored by Mendelberg alone, suffers from important limitations of experimental design, including randomization failures across treatments and an inability to compare directly the effects of implicit and explicit messages in priming racial resentment. While some of the differences between our findings and prior work may be explained by the way in which data are analyzed, we cannot reconstruct Professor Mendelberg's analysis because the proprietary experimental data underlying her work have been destroyed in a flood.5
We provided Professor Mendelberg with the data (and our Stata code) from our experiments in April of 2006. At that time, we requested the data underlying her experiments. It was not provided to us. We again requested those data in June of 2007. In July of 2007, Professor Mendelberg reported to us that the data were unavailable because they were stored in a cardboard box that was damaged due to flooding in her home in April of 2007.
In further developing our argument, the remainder of this essay is divided into three parts. First, we review the theory suggesting greater efficacy of implicit than explicit appeals in priming racial resentment and provide an overview of the findings from Huber and Lapinski that relate to these hypotheses.6
Huber and Lapinski 2006.
The Implicit-Explicit Model and Our Earlier Findings
Mendelberg's model of the different effects of implicit and explicit appeals in priming racial attitudes (hereafter the IE model) has three components: (1) political communication containing references to race primes underlying anti-black predispositions; (2) an explicitly racial message, however, also causes citizens to become aware of the racial nature of the appeal; (3) therefore, citizens resist explicit appeals because even those who hold negative views of African Americans also embrace the widely held egalitarian antiracist ideal.7
Consequently, implicit appeals—those containing visual images of African-Americans—prime underlying anti-black predispositions more effectively than explicit appeals, which evoke an egalitarian counter reaction.In order to test these claims, we designed and implemented a pair of randomized and controlled experiments with over 6,300 respondents. Complete results from our analysis of those experiments appear in Huber and Lapinski and are summarized here.8
Huber and Lapinski 2006.
Second, we found that implicitly racial messages were no more effective than explicitly racial ones in priming racial resentment in opinion formation. Thus, we were unable to replicate the IE model's central claim that implicit appeals increase the importance of racial predispositions in opinion formation relative to explicit ones.
Third, we demonstrated why implicit messages are no more effective than explicit messages in priming racial resentment. Low-education respondents do not differentiate between implicit and explicit appeals. For these citizens, explicit appeals therefore do not generate the egalitarian counter-reaction that inhibits racial priming. High education respondents, by contrast, do differentiate between the two forms of appeals, but are relatively impervious to priming in the first place. This is because they already bring their racial resentment to bear in expressing policy opinions on important issues that might otherwise be vulnerable to “racialization.” Together, these two facts produce a pattern in which those less-educated but resentful citizens most susceptible to racial priming are also least likely to reject explicit appeals.9
On the greater susceptibility of less sophisticated individuals to image priming, see Druckman and Holmes 2004, 770. In responding to our argument that low education respondents are more amenable to priming, Mendelberg cites a paper by Federico 2004 in which the correlation between racial resentment and opinions about welfare is attenuated for low education respondents. This is not a study of priming, however, but of the “unprimed” correlation between racial resentment and racial attitudes. It is precisely because low education respondents do not naturally bring their racial predispositions to bear in forming opinions about welfare that they are amenable to priming, which makes those considerations more accessible. In a second study reported in that article, education is shown to strengthen the correlation between evaluations of particular welfare recipients and beliefs about welfare policy, but that too is not evidence of greater priming of general racial resentment in opinion formation.
Mendelberg 2001, ch. 6.
See Gerber, Green, and Nickerson 2001.
Criticisms of Our Work and Our Response
Professor Mendelberg raises two distinct criticisms of our experimental research. The first is built around the claim that our data do not demonstrate any evidence of racial priming. We show, however, that for those groups most amendable to priming, racial appeals are effective in priming racial resentment. The second focuses on arguments about the design and implementation of our experiments, raising questions about the validity of our manipulation check, whether respondents viewed their assigned political advertisements, and the effects of question order on our results. After undertaking additional analysis of our data, we demonstrate that these concerns are unfounded. We address each criticism in greater detail below.
First Claim: We fail to find much evidence of racial priming
The first criticism of our work is that we find little evidence that any form of racialized appeal primes racial resentment in a pooled analysis of all experimental participants. In particular, table 4 in our original work displays analysis of data from our “Experiment A,” in which respondents were randomly assigned to view a generic Get Out the Vote appeal (the control group), an implicitly racial appeal, or an explicitly racial appeal. We find that only the “baseline” effect of racial predispositions (across treatments and the control group) is statistically distinguishable from zero at conventional levels, a finding that Professor Mendelberg states is at odds with prior research on racial priming (115).
However, as we suggested above and articulated more fully in our original article, there is little theoretical reason to believe that there will be large racial priming effects for all citizens.12
Indeed, the coefficients in table 4 show that, on average, both implicit (in three of four policy areas) and explicit appeals (for all four policy areas) prime racial resentment (increase the effect of racial predispositions) relative to the control group, but these effects are too small to be statistically significant at the .05 level. We do find that the effect of predispositions in the implicit treatment is larger and statistically distinguishable from the effect of predispositions in the control group for the strengthen welfare work requirements item (p-value of .10 in a one-tailed test), but we can never differentiate the effect of the implicit appeal from the effect of the explicit one.
Confirming this hypothesis, we found that among experimental subjects assigned to the control group (no racial appeal), racial predispositions are more powerful predictors of opinions for higher sophistication respondents (See table 5 and figure 3 in our original article). Furthermore, as we demonstrate in table A in this article, for low education respondents, implicit appeals do prime racial resentment. Comparing the effect of anti-black predispositions for respondents in the control group with the effect for those viewing the implicitly racial appeal, we find that the implicitly racial appeal increases the effect of racial predispositions on opinions for all four policy areas queried in our survey, and that this increase is statistically significant for three of the four policy areas.13
This table does not appear in our original article, but this analysis is the basis for Figure 5 there. In that original figure, we show that there are no differences between implicit and explicit appeals in priming racial resentment, but do not discuss effects relative to the control group. We report results only from Experiment A because Experiment B did not have a control group.
Table A: The effects of non-racial and implicitly racial appeals on low education respondents
Second Claim: Our Experiment Suffers from Defects of Design and Implementation
The second set of criticisms leveled against our experiments was that they were flawed in their implementation and design. First, Mendelberg posits that many experimental subjects did not receive the treatments to which they were assigned. To support this claim, she first asserts that we find only small differences in how respondents evaluated the implicitly and explicitly racial appeals. Neither claim is correct.
In our experiments, prior to viewing their randomly assigned advertising treatment, participants were prompted with the following text:
Occasionally, advocacy groups run political advertisements asking voters to contact elected officials in Washington to express their opinion about important issues. We would like to know your opinion about these types of ads.
Immediately after viewing their assigned treatment and being asked whether they had viewed it, respondents were asked a series of questions about the advertisement they had just viewed. That battery began with the following item:
Some people argue that issue ads distort the political process, while others argue that they help voters learn about important issues. Do you think that it is good for democracy that groups run these types of ads?
We created an Issue Advertisements Bad for Democracy scale from responses to that question and demonstrated that scale scores were more negative (by .07 units in experiment A and .06 units in experiment B) for respondents assigned to view the explicitly racial appeal rather than the implicit one. (These differences are statistically significant. See table 3 in our original article). Professor Mendelberg states that these are small differences, but it is largely meaningless to compare average differences in scale responses across treatments without considering the natural distribution of those responses. Indeed, these raw differences represent, respectively, 19 percent and 16 percent proportional increases in scale scores.14
(.44 − .37)/.37 = .19, (.43 − .37)/.37 = .16.
Moreover, as the data shown in table B demonstrate, the preponderance of negative evaluations of issue advertisements is much larger after viewing the explicit treatment rather than the implicit one. Proportionally, viewing the explicit appeal increases the likelihood of offering the “Bad for Democracy” response by 60 percent in experiment A and 95 percent in experiment B. If we consider the “Bad” and “Somewhat Bad” responses together, proportionally the explicit appeal increases the likelihood of offering either response by between 37 percent and 57 percent. These are dramatic, not small, differences in perceptions.
Table B: Evaluations of non-racial, implicitly racial, and explicit racial appeals
Professor Mendelberg also states that the “issue advertisements bad for democracy” question we use to measure these differences is inappropriate because it does not directly ask about the racial content of the particular advertisement a respondent had just viewed. We believe that this is the correct way to verify treatment for two reasons, however. First, because this is a randomized experiment, any ambiguity about the measure should be balanced across treatments, so that if respondents relied on divergent evaluative criteria in answering the question (e.g., they recalled other issue advertisements they had viewed outside of the experimental context), the only feasible source of systematic variation is the treatment to which a respondent was assigned.
Second, while we could have asked respondents directly whether they thought the advertisement they had viewed was a racial appeal, doing so would risk creating perceptions of the advertisements as racial among those susceptible to concerns about racial appeals. By asking whether respondents believed the advertisement was “good for democracy” we sought instead to measure whether they perceived some norm violation directly, such as might arise from invoking race visually or verbally. Measuring whether respondents perceived a norm violation is particularly important because a critical assumption in the IE model is that “when the norm is egalitarian, explicit messages backfire, and only implicit messages, which appear to adhere to the norm, can succeed.”15
Mendelberg 2001, 8.
Professor Mendelberg next compares our work to an early Knowledge Network study authored by Clinton and Lapinski and suggests that errors associated with the video delivery capabilities of Knowledge Networks may have contributed to high levels of non-treatment in those experiments.16
Clinton and Lapinski 2004.
Our survey also included a prompt, displayed immediately after a respondent viewed (or tried to view) their assigned treatment, which asked respondents whether they had been able to view the advertisement to which they were assigned. Just 8 percent of respondents in experiment A and 11 percent in experiment B reported that they did not view their assigned treatment. These numbers are likely overestimates of treatment failure, because respondents may have thought that reporting an inability to view the video would end the survey (it did not).17
Restricting our data analysis to the sample of individuals who reported success in viewing their advertisement produces findings substantively identical to those reported in our earlier paper. Our reason for not restricting the analysis in this way was that reported failures may have been affected by whether respondents agreed with the treatment to which they were assigned. Insofar as we are interested in how individuals react to perceived norm violations, it is a conservative approach to include respondents who did not receive any treatment.
Professor Mendelberg's final substantive criticism of our experiment is that by measuring anti-black predispositions prior to exposing respondents to a treatment we “likely primed racial predispositions for all subjects” (116). This is irrelevant and false for two reasons.18
Additionally, our survey was designed to minimize the linkage in respondents' minds between the questions about predispositions and the post-treatment questions of policy opinions. After respondents were asked the questions which we used to measure racial resentment, they were then shown an advertisement and asked three questions about it (the aforementioned question about the effect of advertisements on democracy, a question about the quality of issue advertising, and a final question about whether advertising has an effect on the political process). Only then, after these intervening distracter tasks were respondents asked their policy opinions.
Second, even if asking respondents the queries that form the basis for the racial resentment scale primes anti-black predispositions, this fact is irrelevant for testing the IE model. To highlight the reason for this claim, suppose Mendelberg is correct and that our pre-treatment questions prime racial resentment. If the IE model is correct, implicitly racial appeals (or non-racial appeals, as in the control group) should not upset this initial priming. But the explicit appeal, by consciously evoking race and the negative portrayal of African Americans, should cause an egalitarian counter-reaction that causes respondents to suppress their negative feelings toward African Americans. If the IE model is correct, we would then find that both the control group and those viewing the implicit appeal would have similar levels of racial priming, while for those viewing the explicit appeal this effect would be suppressed. But the data do not support this prediction. To reiterate, we do not find that the explicit appeal is less effective than the implicit one in priming racial resentment and neither do the other studies Mendelberg cites.
Considering Prior Work Supporting the IE Model
In our original article, we argued that our experiments, with a control group and a large national sample (including many high-resentment and low-education respondents), provide a superior vehicle for testing the IE model. Professor Mendelberg's response highlights the strength of her prior experiments and the robustness of her results across samples. However, the studies supporting the IE model are small in number and suffer from crucial limitations of design and implementation.
To make clearer the basis for our original argument, we summarize in table C the features of the existing research that directly tests the IE model.19
Notably, this list excludes Valentino, Hutchings, and White 2002; Terkildsen 1993; and White 2007, whose work we do not rebut. Valentino, Hutchings, and White do not compare implicit to explicit racial appeals. Terkildsen's experiment compares the effects of visual images of White, dark-skinned Black, and light-skinned Black candidates on evaluations of those candidates and finds that either image of a Black candidate increases the importance of prejudice in candidate evaluations. However, the effect of prejudice on evaluations of the candidate in the light-skinned case is not statistically distinguishable from its effect in the dark-skinned case. Additionally, dark-skinned versus light-skinned candidates are not directly comparable to the explicit versus implicit racial messages posited to have differential effects on racial priming in the IE model. (Although it might be the case that survey participants might have been more concerned about expressing negative evaluations of the more clearly Black candidate on a survey.) White 2007 reports the results from two experiments. In the first, focusing on different types of statements opposing the Iraq war, treatments vary in both their racial content and substantive arguments (e.g., in one treatment a politicians opposes the war for hurting racial minorities, while in the other the politician suggests that there are other more important domestic priorities like housing and healthcare). In the second experiment, focusing on arguments about Food Stamp reform, treatments vary only in their racial content (e.g., references to inner city families, poor families, and African Americans), but the results are mixed as they relate to the IE model. Both the direct reference to African-Americans and the inner city families reference increase the likelihood that Whites name crime as an important problem, but the inner-city reference appears to make resentment of African-Americans a more powerful predictor of opinions about food stamps reform. The latter finding is not found for the other implicit racial message (poor families), however, and is based on the comparisons of only 35 respondents in the explicit (African-Americans) condition to 34 in the implicit (Inner-city families) condition.
Table C: Summary of prior work testing the IE model
In (1), with 77 subjects, there is no explicit message with which to compare the implicit Horton appeal. Instead, participants in this experiment were exposed either to a message focusing on Dukakis' care of Boston Harbor or the Horton appeal.20
We note that this comparison is also a weak test of the generic claim of racial priming because the Horton appeal both contained implicitly racial cues and substantive criticisms of Dukakis' record on crime. A more persuasive design would compare a Horton-like appeal with an identical one in which the individual portrayed in the advertisement was White.
Study (2) has a large sample of low-education and high resentment respondents. It is an observational study, however, correlating the changing effects of racial resentment on evaluations of Dukakis and Bush over the course of the 1988 presidential race with over time rollout of the Horton advertisement (implicit period) and then criticism of that advertisement (explicit period). Because it is not a randomized experiment, however, any other events occurring during the campaign that affected the candidates' fortunes in a way correlated with beliefs about race would be conflated with the effects of the Horton appeal. As others have noted, many other features of the 1988 campaign were also dynamic over this period in a pattern that followed the fortunes of the Horton appeal (e.g., the Dukakis campaign's failure to address the Horton issue when it was initially raised, the resurgence of the Dukakis campaign and its focus on non-crime issues at about the same time the Horton appeal was criticized, and the fact that criticisms of the Horton advertisement included substantive defenses of Dukakis' record on crime in Massachusetts, ignited elite condemnation of the Bush campaign, and sought to rally Democratic partisans, etc.).21
See, e.g., Valentino 2001 and Hutchings 2002 for previous discussion of the relationship between other dynamics of the 1988 campaign and the Horton appeal. The specific campaign events identified are discussed in Marelius 1988, Haas 1988, Reid 1988, and McGrory 1988.
Mendelberg does consider whether the Horton appeal primed fear of crime, but fear is measured after the fall campaign when reported fear was likely affected by both that contest and an individual's susceptibility to it. A similar problem arises with respect to racial resentment, which was also measured after the election. A more general concern is that anything else correlated with racial resentment—including, for example, fear of bad leadership, fear of social disorder, fear of leniency in the criminal justice system—which might also have been activated by the Horton appeal and mitigated by its subsequent criticism could also cause there to be an apparent correlation between when a respondent was interviewed during the fall campaign and the effects of the racial resentment measure on opinions. In the experimental context one can isolate the effects of racial images and language from the effects of both the other content of political communication and the larger campaign environment.
Study (3) provides the strongest support for the IE model with a direct comparison of the effects of an implicit and explicit appeal, but Professor Mendelberg discards members of the control group in her analysis of this experiment because “a group that viewed a message about the environment had significantly different characteristics and thus could not be included.”23
Mendelberg 2001, 197.
See, e.g., Gerber, Green, and Nickerson 2001.
Additionally, by discarding the control group, it is impossible to conduct a full test of the IE model's prediction that implicit appeals are more effective than both explicit and non-racial appeals in priming racial resentment. Finally, the demographics of this sample of 251 are unrepresentative of the national as whole (extremely high education levels and low levels of racial resentment).25
Forty-six percent of respondents in that sample had “some post-graduate education” (Mendelberg 2001: 196). Given the strong correlation between racial animosity and education, small differences in treatment populations, measurement error, or model misspecification may contribute to incorrect conclusions from aggregate (pooled) analysis.
Available analyses of Study (4) do not allow testing the IE model. In particular, the relative success in priming racial resentment of implicit, explicit, and non-racial appeals is not reported. These results are therefore irrelevant for testing the claim that implicit appeals are more effective than explicit ones in priming racial resentment. (Furthermore, these data have been destroyed, so further analysis is not possible.)
Finally, studies (5) and (6) are from Huber and Lapinski.26
Huber and Lapinski 2006.
Cumulatively, then, there is only one randomized experiment directly testing the IE model that finds explicit appeals are less effective than implicit ones in priming racial resentment (Study 3 in table C). That experiment is small, and analysis of its data does not compare effects relative to the control group which was discarded prior to the analysis. Furthermore, those findings are not supported in two experiments with much larger samples, one of which also includes a comparison to priming effects in a non-racial control treatment. Professor Mendelberg is therefore incorrect when writing that “the particulars of Mendelberg's study are now virtually irrelevant given the large number of studies that replicate her basic results” (117). Rather, her work stands alone in supporting the IE model, and that work suffers from intrinsic problems that limit its value in theory testing.
Conclusion
The IE model predicts a greater effect of implicitly racial appeals than explicitly racial ones in priming racial predispositions. Is it an accurate characterization of how individuals form opinions? Can candidates really harness racial animosity through subtly racialized appeals, and can these moves be counteracted by “calling” the race card to make racial appeals visible as such to all citizens? Answering these questions is vitally important for understanding the role of race in contemporary U.S. elections and American politics more generally. In Huber and Lapinski (2006) we employ two large-sample controlled experiments to address these questions, and find little support for the IE model of opinion formation.27
Huber and Lapinski 2006.
What are the implications of our findings for the role of race in American politics? As we wrote in our earlier article: “Nonetheless, our results should not be held as evidence that race is unimportant in policy and political campaigns. Rather, our findings suggest that it is. Racial predispositions are a powerful predictor of opinions on a host of issues.”28
Huber and Lapinski 2006, 438.
Of course, our own research is not without its limitations. It is, after all, only a single study composed of two experiments. Nonetheless, its size and the consistency of our results call into question the work that precedes it. Relative to extent data, the balance of the evidence rebuts the predictions of the IE model of opinion formation. Moving forward, researchers may wish to conduct new experiments on the relative effects of implicitly racial and explicitly racial messages. Without such new data gathering, however, the IE model should be seen as unsupported conjecture rather then enjoying privileged status as a confirmed theory.
