Introduction
The political polarization of the COVID-19 pandemic poses a crucial challenge for policymakers. American conservatives are less willing to recognize the threat of COVID-19, reject COVID-19 conspiracy theories, engage in risk-mitigation strategies such as hand-washing and mask-wearing, support policies meant to reduce the public health impacts of COVID-19, and to vaccinate (Callaghan et al. Reference Callaghan, Moghtaderi, Lueck, Hotez, Strych, Dor, Franklin Fowler and Motta.2020; Calvillo et al. Reference Calvillo, Ross, Garcia, Smelter and Rutchick2020; Kushner Gadarian et al. Reference Kushner Gadarian, Goodman and Pepinsky2020; Miller, Reference Miller2020a; Miller, Reference Miller2020b). It is, therefore, important to identify messaging strategies that encourage conservatives to combat the pandemic.
While it might be appealing to simply present people with the facts about COVID-19, past research suggests that people often ignore facts in favor of previously held values (Druckman and Bolsen, Reference Druckman and Bolsen2011; Slothuus and de Vreese, Reference Slothuus and De Vreese2010). It is possible to create more persuasive messages by framing an issue in complementary ways, i.e. by combining messages that appeal to multiple groups and share a common persuasion goal (Wise and Brewer, Reference Wise and Brewer2010). However, crafting complementary frames is difficult for polarized issues where most frames will already be associated with one side of the political spectrum (Detenber et al. Reference Detenber, Ho, Ong and Lim2018). It is thus unclear whether complementary frames will be successful for the polarized issue of COVID-19.
In this paper, we first present preliminary evidence from an original survey experiment conducted in March 2021. We find that it may be possible to create an effective COVID-19 messaging campaign across ideological barriers using complementary frames. When participants are exposed to a factual message about the negative impact of COVID-19 on both the economy and public health, conservatives increase their willingness to engage in risk-mitigation practices including vaccination, as well as their support for policies meant to limit COVID-19 spread. We tested the robustness of these effects in a larger-n replication in August 2021. There, we fail to replicate our initial findings. Overall, we conclude that while complementary frames show some promise in public persuasion, they may struggle to overcome partisan polarization on especially salient issues.
Existing research
COVID-19 messaging
While it may be that conservatives were inherently likely to oppose (and liberals to support) ‘big government’ risk mitigation strategies such as masking mandates and lockdowns of non-essential travel, COVID-19 attitudes have also been affected by the widespread polarization of the American public (Druckman et al. Reference Druckman, Klar, Krupnikov, Levendusky and Ryan.2021). High levels of misinformation about the pandemic in right-leaning media, coupled with former President Trump’s inconsistent messaging about the virus’ severity, appear to have led many conservatives to view COVID-19 as overblown (Graso et al. Reference Graso, Chen and Reynolds2021; Moniz, Reference Moniz2020; Motta et al. Reference Motta, Stecula and Farhart2020). Liberals are more likely to believe in the virus’ threat – perhaps in part due to their low approval of Trump (Calvillo et al. Reference Calvillo, Ross, Garcia, Smelter and Rutchick2020).
Effective containment of the pandemic, however, necessitates a widespread public response. Unfortunately, attempts to reach the public with up-to-date COVID-19 information has also been hampered by polarization. While medical organizations and liberal elites have largely tried to focus messaging on the public health crisis, many conservative elites have focused more on the economic toll of restrictions, with the implication that COVID-19 risk-mitigation is not worth the cost (Green et al. Reference Green, Edgerton, Naftel, Shoub and Cranmer2020; Rutledge, Reference Rutledge2020). Conservatives in the general public are accordingly more likely to be concerned about the economy than the COVID-19 pandemic (Pew Research Center, 2021).
While some studies find that messages emphasizing COVID-19’s public health threat is broadly effective in increasing concern about the pandemic and support for risk mitigation in the US context (Everett et al. Reference Everett, Colombatto, Chituc, Brady and Crockett.2020; Jordan et al. Reference Jordan, Yoeli and Rand.2020), other studies find that health messages fail to surmount the existing ideological divides, and may even backfire and decrease support for COVID-19 risk mitigation among conservatives (Case et al. Reference Case, Eddy, Hemrajani, Howell, Lyons, Sung and Connors2021; Utych, Reference Utych2021). It may be unsurprising that conservatives are inclined to disregard or counterargue COVID-19 messages which focus on public health (a ‘liberal’ message) rather than the economy (a more ‘conservative’ concern). Encouraging conservatives to mask, wash hands, and vaccinate to hasten the reopening of the economy, therefore, could be a good way to appeal to their priorities. Researchers who directly compared public health and economic message frames found, however, that while frames about personal risk and social responsibility are effective in encouraging vaccination, messages about COVID-19 economic loss are not (Motta et al. Reference Motta, Sylvester, Callaghan and Lunz-Trujillo2021). We are thus left uncertain how to reach conservatives with vital messages about COVID-19 health risks.
Complementary frames
Message framing research shows that people interpret frames according to their preexisting political attitudes via motivated reasoning (Druckman and Bolsen, Reference Druckman and Bolsen2011; Slothuus and de Vreese, Reference Slothuus and De Vreese2010). This is especially true for competing frames – when people have the opportunity to choose between multiple frames of the same issue, they will gravitate to the one that best accords with their political worldview (Chong and Druckman, Reference Chong and Druckman2007). Given the existing framing competition over COVID-19, individuals may perceive economic and public health frames to be conflictual. This may lead liberals to reject economic frames (even those promoting public health), while conservatives reject health risk-mitigation procedures such as vaccination (even when framed as economically beneficial). If true, our ability to communicate about the pandemic across ideological boundaries is sharply limited.
Different frames are not always in conflict, however. When multiple frames promote the same goal, these frames can have a stronger effect than any frame alone (Wise and Brewer, Reference Wise and Brewer2010). It may be that combining economic and public health frames will be perceived as complementary – i.e. sharing the goal of encouraging COVID-19 risk mitigation to reduce both negative health and economic outcomes, even though the frames are associated with different sides of the US political spectrum. Yet complementary frames have thus far only been tested in non-polarized contexts, where participants typically have weak issue preferences (Detenber et al., Reference Detenber, Ho, Ong and Lim2018; Ho, Reference Ho2021). In a polarized context, it may be that complementary frames will nonetheless be perceived as competing. That is, conservatives will use the economic portion of the frame to justify their existing opposition to COVID-19 risk mitigation, while liberals will use the public health portion to justify their existing support. We must empirically test the effectiveness of complementary frames in the polarized context of COVID-19.
Hypotheses
We propose two competing hypotheses for the combination of economic and health frames of COVID-19. First, we theorize that combining public health and economic framing of COVID-19 will be complementary and increase support for COVID-19 risk mitigation by giving people multiple reasons to support risk mitigation regardless of which individual frame they find more persuasive. We thus expect the combined frame to be effective among both liberals and conservatives.
H1: Combined economic and public health frames will increase support for COVID-19 risk mitigation for both liberals and conservatives.
The alternative, however, is that individuals will view the combined frame as two polarized competing frames. Due to liberal and conservative elites’ differing emphasis on public health and economic effects of COVID-19, viewing these messages together may lead individuals to bolster the parts they already agree with, while denigrating the parts they don’t. In this case, we would expect greater polarization in reaction to the combined frame, with liberals increasing their support for risk mitigation while conservatives decrease in support.
H2: Combined economic and public health frames will increase support for COVID-19 risk mitigation among liberals, but decrease support for risk mitigation among conservatives.
Study 1
Data
Study 1 was conducted on March 8th 2021 through the online survey platform Prolific. Prolific recruits a large and diverse pool of potential participants through social media, physical flyers, and referrals, and researchers have successfully replicated established experiments through the platform (Palan and Schitter, Reference Palan and Schitter2018; Peer et al. Reference Peer, Brandimarte, Samat and Acquisti2017). Because our hypotheses involve an interaction with ideology, we used Prolific quota sampling to contact 662 American resident adults (of 149,111 eligible participants), evenly divided between self-identified liberals, moderates, and conservatives based on Prolific pre-screening categories. Full demographics are given in Appendix Table A1. While it is unknown how many participants viewed the study invitation, 60 declined to participate after viewing the study. This leaves 602 participants potentially analyzed below, 558 of whom answered all necessary measures.
Measures
COVID-19 messaging
Participants were randomly assigned one of four messages. The combined frame message contained factual information about both the public health toll of COVID-19 and its economic toll, as well as the then-anticipated delay for widespread vaccination. Participants could also receive a message including only public health information, only economic information, or a control including no reference to COVID-19 at all. Footnote 1 All four messages are in the appendix, along with all other analyzed survey items.
Engagement was increased by the employment of a brief writing task. After reading the COVID-19 messages, participants were asked to write a few sentences about ‘the negative effects and impacts of the coronavirus pandemic.’
Ideology
Ideology was measured using a standard seven-point self-identification question rescaled to run from 0 (extreme liberal) to 1 (extreme conservative). Although we recruited equal proportions of liberals, moderates, and conservatives using Prolific’s pre-screening data, slight response instability and the inclusion of ‘slightly’ liberal/conservative options resulted in 278 liberals, 105 moderates, and 219 conservatives. Their distribution between conditions is shown in Table A2.
Risk-mitigation attitudes
Participants were asked whether they planned to wash their hands longer or more often, socially distance, reduce trips outside the home, wear a mask when in public, shop online rather than in person, or get the COVID-19 vaccine when one was made available. Each question was asked on a six-point scale, rescaled to run from 0 (extremely unlikely) to 1 (extremely likely). Responses to these six items were averaged to form an index of individual behavior intention (α = 0.81).Footnote 2
Participants were also asked about their support for mask mandates, limits on in-person worship or church services, limits on indoor dining, lockdown on all non-essential travel, and school closures. Again, each question was asked on a six-point scale, rescaled to run from 0 (strongly disapprove) to 1 (strongly approve), and averaged to form an index of policy attitudes (α = 0.93).
Controls
All models control for respondents’ age, gender, race/ethnicity, education, income, census region, whether they previously tested positive for COVID-19, whether friends or family tested positive for COVID-19, and finally whether they live in a more urban or rural area. All variables are coded to run from 0 to 1, except age which is coded in years. Due to relatively low levels of nonresponse, observations with missing data were simply excluded from the relevant analysis.
Results
We first conducted OLS regressions examining the main effect of the COVID-19 conditions (compared to the control) on each risk-mitigation attitude index. Second, we tested for the interaction between ideology and condition.
Columns 1 and 3 in Table 1 show no consistent evidence of a condition main effect. Each message has a somewhat positive effect on individual risk-mitigation intentions and policy support (b = 0.02 – 0.04), but these effects are not statistically significant (all p > 0.10).
Study 1
Standard errors in parentheses
Comparison condition: Control
+ p < 0.10, * p < 0.05, ** p < 0.01
The impact of condition changes dramatically, however, when the interaction with ideology is taken into account. In columns 2 and 4, we detect a positive and statistically significant interaction between the combined message and ideology on both indices. Substantively, the combined message (compared to the control) increases extreme conservatives’ individual risk-mitigation behavior intention from 0.60 to 0.75 (p < 0.001), and support of risk-mitigation policy from 0.33 to 0.48 (p = 0.02). Extreme liberals, meanwhile, are largely unaffected by the combined message; individual risk-mitigation behavior intention drops slightly from 0.96 to 0.90 (p = 0.09), and policy attitude support goes from 0.98 to 0.92 (p = 0.25).Footnote 3
At first, this result might appear to contradict both H1 and H2 – the combined message is only effective in changing conservatives’ attitudes, and has little effect on liberals. A closer examination, however, shows that liberals may be subject to a ceiling effect. The average risk-mitigation support of extreme liberals in the control condition is so high that it is difficult to imagine any message increasing it. Extreme conservatives, however, have more room to be persuaded. Overall, the combined frame broadly increases conservatives’ support for a range of risk-mitigation practices and policies, bringing them closer to liberals, rather than increasing polarization and pushing conservatives further down in support. The balance of evidence is thus stronger for H1 (complementary frames) rather than H2 (polarizing frames).
Study 2
While Study 1 provides preliminary support for H1, with only 558 analyzed participants, it is underpowered to detect whether the combined condition is more effective than the public health or economic conditions alone. We thus conducted a larger-n replication, retaining the same hypotheses, with some changes to the manipulations and outcome variables in order to create a better empirical test. Study 2 was registered through the Open Science Foundation before any data collection or analysis was conducted.Footnote 4
Data
Study 2 was conducted on September 7–8 2021 through the online survey platform Lucid (Coppock and McClellan, Reference Coppock and McClellan2019). Full demographics are given in Table A4. While it is unknown how many participants viewed the study invitation, 2,426 American-resident adults were initially recruited. A total of 467 participants did not complete the study, and 167 failed a basic attention check described below. This leaves 1,792 participants analyzed below, 1,550 of which answered all necessary measures.Footnote 5 Table A5 shows the distribution of participant ideology by condition.
Measures
The procedure and design of Study 2 are identical to Study 1 with a few key exceptions listed below.
COVID-19 messaging
Participants were again randomly assigned to read one of four messages – a COVID-19 public health message, a COVID-19 economic message, a combined frame with key elements of both messages, or a control. These messages differ from Study 1 only in that they were updated to reflect then-current statistics, and the length of the manipulations was equalized to eliminate it as a potential confound. Additionally, the Study 1 writing task may have made the treatment effect more difficult to isolate by increasing the salience of participants’ individual concerns. Therefore, we eliminated the writing task and instead used a brief multiple-choice attention check to ensure participants read the paragraph. Participants who skipped the attention check or were unable to identify the topic of the paragraph were removed from the analysis.
Ideology
Ideology was measured identically to Study 1. We also measured party identification, using a seven-point scale derived from standard branching questions, recoded to run from 0 (strong Democrat) to 1 (strong Republican). This allows for a robustness check of the main hypotheses, to identify whether the effects are replicated when focusing on partisanship.
Risk-mitigation attitudes
Since conducting Study 1 in March 2021, the US pandemic response evolved rapidly. In summer 2021, the Biden administration advised that vaccinated individuals may go maskless and much of life could return to normal (Miller and Balsamo, Reference Miller and Balsamo.2021). At the same time, many epidemiologists advised caution, even for those who are vaccinated (Sanger-Katz et al. Reference Sanger-Katz, Cain Miller and Quealy2021). As Study 2 was fielded, the Delta variant was surging, and many states were resuming lapsed restrictions (O’Donnell and Lambert, Reference O’Donnell and Lambert2021).
Rather than attempting to predict the pandemic’s progress, in Study 2, we asked participants about their prospective behavior or policy support in anticipation of future COVID-19 waves or variants. This approach allowed us to avoid making judgments about what kinds of behavior vaccinated or unvaccinated people should endorse in the brief window that the study was conducted, and instead focus on the more generalized concept of risk-mitigation. It also allows us to provide guidance to those looking ahead to the future of COVID-19 and the polarization of future pandemics.
Participants were asked about their prospective intention to wash their hands longer or more often, socially distance, reduce trips outside the home, wear a mask when in public, shop online rather than in person, get a vaccine and/or booster shot, wear a mask when indoors with others, avoid crowded places, and avoid socializing with unvaccinated people. Each question was asked on a six-point scale, rescaled to run from 0 (extremely unlikely) to 1 (extremely likely). Responses were then averaged to form an individual behavior intention index (α = 0.91).Footnote 6
Participants were then asked about their prospective support for mask mandates, limits on in-person worship or church services, limits on indoor dining, lockdown on all non-essential travel, school closures, limits on international travel, limits on interstate travel, vaccine ‘passports,’ and vaccine mandates. Again, each question was asked on a six-point scale, rescaled to run from 0 (strongly disapprove) to 1 (strongly approve), and averaged to form an index of policy attitudes (α = 0.95).
Controls
We controlled for all factors included in Study 1, employing the same measurement and recoding procedures.Footnote 7
Results
We conducted an analogous set of analyses to those employed in Study 1. The results are shown in Table 2. Overall, Study 2 fails to replicate Study 1 or support either hypothesis. Columns 1 and 3 find negative (and statistically significant) main effects of the health and economic conditions on risk-mitigation attitudes, suggesting that our manipulations may have backfired when compared to the control. An F-test reveals that the combined condition does have less of a negative effect on policy support than the other experimental conditions (compared to health p = 0.08, economic p = 0.02), but the effects are not significantly different when predicting individual behavior (health p = 0.26, economic p = 0.29). Moreover, as shown in columns 2 and 4, the interactions between condition and ideology fail to achieve significance, and are almost entirely in the negative direction, indicating that they tend to exacerbate, rather than reduce, the existing conservative opposition to risk-mitigation. There are no significant differences between the interactions of the combined condition with ideology when compared to the interactions with the other conditions, either when predicting individual behavior (compared to health p = 0.63, economic p = 0.69), or policy support (health p = 0.70, economic p = 0.59).
Study 2
Standard errors in parentheses
Comparison condition: Control
+ p < 0.10, * p < 0.05, ** p < 0.01
Post-hoc tests
Overall, we conclude that despite the initial findings of Study 1, the failure to replicate in the larger-n Study 2 shows that COVID-19 attitudes may be resistant to complementary frames. In this section, we briefly describe post-hoc tests to rule out any obvious alternative explanations for replication failure.
First, the results are substantively similar when condition is interacted with partisanship, rather than ideology (Table A6). We also tested whether the expected interaction was nonlinear by examining ideology as a categorical variable; we find no significant effect of any condition at any level of ideology. The results are also unaffected when controlling for or interacting with respondent attention, vaccination status, or pretreatment concern about the pandemic (see Appendix pages 5–6).
Ultimately, the failure to replicate the findings of Study 1 may be due to an unreliable effect or the changing context of the pandemic. It may also be that the Study 1 writing task, eliminated to preserve a ‘pure’ manipulation in Study 2, was more crucial than we believed.
Conclusion
The polarization of COVID-19 attitudes and rhetoric is a major challenge for public health. Finding messages that appeal broadly is key to slowing and eventually halting the pandemic. Unfortunately, despite promising preliminary results, we fail to find consistent evidence that complementary frames can be successful in changing COVID-19 attitudes.
Our results may indicate a limitation of complementary frames. Past research has focused on non-polarized contexts, in which opinion change is easier to achieve. Complementary frames may be less effective when attitudes are highly polarized, or may require greater participant engagement to overcome salient, polarized attitudes.
Regardless, this null result may help guide policymakers and researchers alike. Both COVID-19 and future health crises require carefully researched and effective health messaging in order to reach the public. As we continue to seek messages that will be effective across partisan lines, complementary frames appear to have less promise than initially anticipated.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/XPS.2022.6
Data Availability
The data and code required to replicate all analyses in this article are available at the Journal of Experimental Political Science Dataverse within the Harvard Dataverse Network, at: doi: 10.7910/DVN/RLJLWT.
Acknowledgments
We thank Howard Ernst for his advice and assistance at the outset of this project, as well as anonymous reviewers of TESS for their invaluable suggestions. We also thank the editor and anonymous reviewers of the Journal of Experimental Political Science.
Conflicts of Interest
The authors declare no conflicts of interest. Support for this research was provided by Oklahoma State University and the United States Naval Academy Volgenau fellowship.
Ethics Statement
This project was approved as exempt by the United States Naval Academy Human Research Protection Program (#2020.0017), and adheres to APSA’s Principles and Guidance for Human Subjects Research. Further information is provided on page six of the supplemental Appendix.
