1. Introduction
A large number of studies have shown that authoritarian governments use government-controlled media to effectively change attitudes, behaviors, and outcomes (Geddes and Zaller, Reference Geddes and Zaller1989; Stockmann and Gallagher, Reference Stockmann and Gallagher2011; Yanagizawa-Drott, Reference Yanagizawa-Drott2014; Adena et al., Reference Adena, Enikolopov, Petrova, Santarosa and Zhuravskaya2015; Huang, Reference Huang2015; Bleck and Michelitch, Reference Bleck and Michelitch2017; Szostek, Reference Szostek2017). Existing studies of government media, however, tend to examine the content produced by government-controlled outlets in a memoryless manner, focusing, for example, on how effective government-controlled media is in influencing people to adopt a particular position on an issue with limited regard to prior efforts to shape public views about the same issue. As a result, we know little about what happens to the effectiveness of government-controlled media when authoritarian regimes change or flip positions on issues and alter media output accordingly to try to shift public attitudes.
Authoritarian governments, like all governments, inevitably need to shift positions on some issues to adapt to changing domestic and international conditions. Sometimes, they are required to flip their position and move to the opposite side of an issue. When this need arises, we often observe authoritarian governments using government-controlled media to frame issues in new ways to support policy changes (Field et al., Reference Field, Kliger, Wintner, Pan, Jurafsky and Tsvetkov2018; Rozenas and Stukal, Reference Rozenas and Stukal2019). Although public opinion is generally considered less consequential in authoritarian regimes than in democracies because the public cannot vote or easily sanction politicians for their policy decisions, research in autocracies shows that public attitudes can constrain domestic and foreign policy (FP)-making and implementation (Weeks, Reference Weeks2008; Weiss, Reference Weiss2013). This means public opposition to policies may require authoritarian regimes to give ground. Shaping public attitudes on policies can forestall opposition and the chance that the regime will need to give way.
Can authoritarian governments move the public to opposing policy positions by changing how an issue is framed? To what extent can attitudes be changed in this way? When we apply existing theories, there are reasons to believe that such efforts would have limited effectiveness as well as reasons to believe that new frames would be effective. On the one hand, the effectiveness of new frames may be limited by predisposition or political knowledge. On the other hand, authoritarian regimes have the ability, through their control of the media, censorship, and other information control measures, to create elite and/or mainstream consensus that prevents those who are predisposed against policy changes from linking their predisposition to the state's new policy position.
In this paper, we conduct an experiment to assess whether framing policy issues differently allows government-controlled media to move public attitudes toward opposing sides of the same policy issue. We assess the effect of different communication frames to measure whether government-controlled media can use framing to move policy positions in divergent directions. Then, we look at whether the effects of frames are limited by predisposition and knowledge. If different frames of the same issue are effective in moving individuals to adopt divergent policy positions and framing effects are not, or are only weakly, moderated by predisposition, this provides support for the notion that government-controlled media can be used to effectively change public attitudes in different directions on the same issue. If only some but not all frames have an effect or if frames are strongly moderated by predisposition, this would suggest that efforts to use government-controlled media to shift public attitudes on the same issue may have limited effectiveness.
The treatments for our experiment deal with issues where the Chinese Communist Party (CCP) has substantially changed its policy position with accompanying changes in framing through state media. We created short segments of news videos in conjunction with media professionals in China that draw upon original state broadcasts and realistically resemble Chinese government-controlled media. We embed these videos in an online survey conducted among 830 respondents in China.
We find that, for both domestic and foreign policies, framing policy issues differently is an effective way to move individuals to adopt divergent policy positions espoused in government-controlled media. We find no evidence that frames affect broad outcomes such as regime trust and patriotism or self-reported behaviors such as complaint-making and discussing government policies. We find that most respondents had prior knowledge of the policy issues contained in the experiment, which means it is less likely that their attitudes are changing because they are learning new information. The questions we asked in the experiment are not politically sensitive, which decreases the potential for social desirability bias. In addition, the method we use—an experiment embedded in an online survey—minimizes incentives of respondents to provide the “right” answer, also known as demand effects (Mummolo and Peterson, Reference Mummolo and Peterson2019). While we cannot completely rule out learning or social desirability bias as motivators for some respondents, these results are more consistent with framing effects.
Our results show that frames are effective regardless of individual predispositions. Our treatment approximates the mainstream consensus found in some authoritarian regimes because respondents are exposed to only one frame and the treatment does not provide contextual information linking predisposition to the state's espoused policy position. This finding may also be influenced by the type of content—narrow and issue-specific—and the video medium of our government-controlled media treatment that activates multiple sensory channels and elicits strong emotional and physiological responses (Houts et al., Reference Houts, Doak, Doak and Loscalzo2006; Kensinger and Schacter, Reference Kensinger and Schacter2006; Lang et al., Reference Lang, Bailey and Connolly2015).
Finally, we find that after controlling for predisposition, framing effects are stronger among individuals who have higher levels of political knowledge, consistent with the view that those who are knowledgeable tend to have stronger predisposition, and if predisposition is controlled for, knowledge should magnify framing effects because knowledge increases the availability and comprehensibility of frames (Druckman and Nelson, Reference Druckman and Nelson2003).
We proceed in four sections. The next section provides a theoretical framework for the use of framing in government-controlled media. We then describe the experimental design and survey, followed by our main results and heterogeneous effects by predisposition and knowledge. We conclude by discussing the implications of the findings.
2. Theoretical framework
We draw upon a conventional expectancy value model to characterize attitudes toward an object (Azjen and Fishbein, Reference Azjen and Fishbein1980; Nelson et al., Reference Nelson, Clawson and Oxley1997; Chong and Druckman, Reference Chong and Druckman2007). In such a model, every object, such as an issue, event, or policy, has i attributes. An attitude consists of the sum of the product of an individual's evaluation of each attribute i and the salience weight of each attribute i. For example, let's say the object is state-owned enterprise (SOE) reform. If the attributes of SOE reform are efficiency and public interest, a person's attitude toward SOE reform is the salience of efficiency times evaluation of efficiency plus the salience of public interest times evaluation of public interest. In this model, framing works by changing the salience of attributes.Footnote 1 In the example of SOE reform, framing would be at work if government-controlled media made efficiency or public interest more salient, lending a particular attribute greater weight in the public's evaluation of a particular approach to SOE reform.Footnote 2
A large number of studies have sought to capture the processes underlying framing effects (Zaller, Reference Zaller1992; Iyengar, Reference Iyengar1994; Cappella and Jamieson, Reference Cappella and Jamieson1997; Price and Tewksbury, Reference Price, Tewksbury, Barnett and Boster1997; Nelson and Oxley, Reference Nelson and Oxley1999; Chong and Druckman, Reference Chong and Druckman2007). Typically, these studies find that conditional on exposure, an individual's attitude about an object may change if a given attribute of the object is (1) stored in a person's memory, (2) accessible, and (3) judged to be applicable or relevant when encountering new information. This means that framing can work to change attitudes by making new attributes about an issue available and by making certain attributes more accessible and/or applicable.
Why would new frames be ineffective?
Suppose government-controlled media has traditionally framed an issue to promote a certain policy position, but later adopts a new frame for the same issue to promote a different, and perhaps opposing, policy position. Some existing theories would predict that new frames would have limited effectiveness. One reason is source credibility, and a second reason relates to moderators of framing effects—specifically, predisposition and political knowledge. If a state media outlet frequently flips positions, audiences may discount it as a credible source and be less amenable to influence.Footnote 3 In addition, an individual's predisposition, including prior beliefs and values, shapes their reactions to and evaluations of alternative frames. Predisposition has consistently appeared as an important moderator of government propaganda and framing effects.Footnote 4 Strong predispositions can reduce framing effects because they increase individuals’ resistance to disconfirming information such that attributes emphasized in the frame are less accessible and applicable.
Political knowledge is another moderator of propaganda and framing effects that has received extensive attention. While studies of government-controlled media have found that messages have stronger effects among less politically knowledgeable individuals (Geddes and Zaller, Reference Geddes and Zaller1989; Stockmann and Gallagher, Reference Stockmann and Gallagher2011; Bleck and Michelitch, Reference Bleck and Michelitch2017), research on framing is more mixed, with some studies finding that less knowledgeable individuals are more susceptible (Kinder and Sanders, Reference Kinder and Sanders1990; Haider-Markel and Joslyn, Reference Haider-Markel and Joslyn2001), and others showing the opposite (Nelson et al., Reference Nelson, Clawson and Oxley1997; Miller and Krosnick, Reference Miller and Krosnick2000; Slothuus, Reference Slothuus2008). Druckman and Nelson (Reference Druckman and Nelson2003) argue that these contradictory results are related to predisposition—specifically, that those who are knowledgeable tend to have stronger predispositions, and these stronger predispositions reduce susceptibility to framing. They argue that after controlling for predisposition, knowledge should magnify framing effects because knowledge increases the availability and comprehensibility of frames to individuals. If government-controlled media was initially effective in moving the public's preferences toward the regime's position, then the public should be predisposed to support the status quo. This means when the authoritarian regime moves to change its policy, it will face individuals who are predisposed against policy change and resistant to new frames. In addition, regardless of predisposition, individuals with different levels of political knowledge may be differentially influenced by new frames, such that some will remain unconvinced by new frames.
New frames, if ineffective, risk damaging public perceptions of the regime when the regime is forced to change positions on issues. When officials and politicians change their position on policies, the bulk of research shows that it makes officials appear less trustworthy and less competent unless the new policy position is supported by a supermajority of the public (McCaul et al., Reference McCaul, Ployhart, Hinsz and McCaul1995; Kartik and McAfee, Reference Kartik and McAfee2007; Tomz and Van Houweling, Reference Tomz and Van Houweling2012; Doherty et al., Reference Doherty, Dowling and Miller2016). In modern autocracies that do not rely on intensive repression or mass indoctrination, convincing the public that leaders are competent is crucial to regime durability (Guriev and Treisman, Reference Guriev and Treisman2019).
Why would new frames be effective?
If government efforts to change public attitudes can overcome challenges of source credibility and the moderating effects of predisposition and knowledge, they can be effective. Usually, authoritarian governments do not continuously flip-flop on an issue but rather enact changes only when the need arises. This may lessen damage to source credibility. In addition, by providing new frames that highlight different attributes of an issue rather than trying to persuade individuals to change their evaluations of a particular attribute, authoritarian governments may make the position change less obvious and lessen the risks associated with changing policy positions.
On overcoming moderators of framing effects, existing research on public opinion suggests that this may be possible if individuals lack information and the ability to link their predisposition with new messages. When an authoritarian government seeks to shift public attitudes on a policy issue, it can do so by framing the issue in a way such that predisposition is not judged to be applicable or relevant. For example, suppose some individuals have attitudes in support of market competition, preferring policies that allow the market to allocate resources rather than the state to do so. If the state's policy on reforming SOEs shifts so that the state begins to insulate SOEs from market pressure and competition, these individuals should in theory resist such a change because it is inconsistent with their pro-market predisposition. However, for them to actually resist such a change would require that their predisposition is accessible and judged to be applicable when encountering new information about SOE reform. If the state provides a new frame for SOE reform unrelated to efficiency—for example, framing SOE reform as being about public welfare and avoiding all references to efficiency—the predisposition of individuals who would have opposed such a policy change on the grounds of efficiency may not be accessible or judged to be applicable. This reflects the “resistance axiom” of Zaller's Receive-Accept-Sample model of public opinion, which states that although people tend to resist arguments that are inconsistent with their political predisposition, this is true only to the extent people possess contextual information necessary to perceive a relationship between the message and their predisposition (Zaller, Reference Zaller1992).
Preventing individuals from making the link between their predisposition and new frames is possible in authoritarian contexts where the state controls the media and other elite messages such that there is “mainstream consensus” (Geddes and Zaller, Reference Geddes and Zaller1989). Through control of media outlets, co-optation and intimidation of journalists and other elites, censorship, and other information control strategies such as flooding (McMillan and Zoido, Reference McMillan and Zoido2004; Roberts, Reference Roberts2018; Pan and Siegel, Reference Pan and Siegel2020), some authoritarian regimes may prevent the public from being able to choose and ultimately wanting to choose different sources of information (Chen and Yang, Reference Chen and Yang2019). If there is mainstream consensus, the authoritarian regime can prevent alternative information from providing such contextual information and thus succeed in overriding the barriers presented by predisposition to attitude change. Examples of alternative information that would provide such contextual information include alternative frames that link the proposed SOE policy to lower efficiency or arguments that make a connection between efficiency and public interest (i.e., that public interest is obtained at the cost of reduced efficiency). Such information can emerge if there is policy disagreement among elites and the government cannot suppress dissenting voices.
Testable implications
If government-controlled media can effectively frame policy issues in new ways, we should find that exposure to government-controlled media containing different frames moves respondents to the policy position espoused in government media in which policy positions are substantially different. In contrast, if government-controlled media cannot effectively offer new frames on the same issue, we should find that exposure to some frames moves respondents to the position espoused by government media, while other ways of framing do not.
In addition, for government-controlled media to be effective in offering new frames, we should observe that framing effects are not, or are only weakly, moderated by predisposition or political knowledge. In contrast, if framing effects are strongly moderated by predisposition—if those who are predisposed against the policy do not resist any change to their attributes—or constrained by political knowledge—if those with higher knowledge are less affected by framing effects—then offering new frames is unlikely to generate the supermajority of supporters needed for officials to change their policy position without incurring costs.
3. Methods
We worked closely with media professionals in China to create six short news video segments that closely resemble the content and style of China's main television broadcast, Xinwen Lianbo. We embedded these news segments in an online survey experiment to measure the effects of government-controlled media on policy preferences. We first describe the experimental context, then the video segments, and finally the survey in which the videos were embedded. All aspects of this research were approved by our university IRB (see Appendix for additional discussion of ethics).
Background
Television—in traditional and online streaming forms—remains one of the most powerful forms of government-controlled media available to autocrats despite the increasing complexity of media ecosystems and fragmentation of audiences (Webster and Ksiazek, Reference Webster and Ksiazek2012; Chadwick, Reference Chadwick2017). Audiences in countries ranging from Azerbajian to Russia to Iran continue to regard television as the most authoritative source of information.Footnote 5
Our experimental context is China, where the regime describes government-controlled media as foundational to the strength of the ruling CCP,Footnote 6 and where, perhaps more than any other country in the world, state control over information is deep and far-reaching, encompassing media, education, culture, sports, and even non-governmental organizations and research (King et al., Reference King, Pan and Roberts2013; Cantoni et al., Reference Cantoni, Chen, Yang, Yuchtman and Zhang2017).
The National Radio and Television Administration, which operates the country's predominant public broadcaster, China Central Television (CCTV), reports directly to the CCP Propaganda Department. A nationally representative survey from 2015 showed the primary way Chinese people consume news is through television (CUGS, 2015). China's most-watched television news program is Xinwen Lianbo (CCTV Evening News), a nightly broadcast at 7pm the CCP uses to try to shape public opinions on domestic policy and foreign issues (Zhao and Guo, Reference Zhao, Guo and Wasko2005). Xiwen Lianbo captures 55–60 million viewers each evening, which represents 4.6 percent of the approximately 1.2 billion people in China with access to television,Footnote 7 and additional viewers watch Xinwen Lianbo news segments online.Footnote 8
Experimental treatments
We create six 2-minute-long news segments that mimic the style of Xinwen Lianbo. Three of the segments are focused on the domestic policy issue of SOE reform. Three of the video segments are focused on the foreign policy issue of disputes in the South China Sea between China and the Philippines. We choose SOE reform and China-Philippines relations because we want to include both a domestic and a foreign policy issue and because they are areas where the Chinese government has adopted different policy positions over time. For external validity, policy positions in the videos are drawn from original Xinwen Lianbo transcripts, edited together for coherence while ensuring similarity in length and types of content.Footnote 9
SOEs in China account for approximately 30 percent of GDP and slightly less than 20 percent of total employment.Footnote 10 The three videos on SOE reform capture the two policy positions the Chinese government has taken—letting SOE reform be driven by the market (Market) and state-led SOE reform (State)—as well as a control condition where background and facts related to SOE reform are provided, but the position of the Chinese state is not stated (Control). The Market condition frames the pro-market policy position in terms of improvements in efficiency, focusing on how market competition can create incentives for competition and improve performance. The State condition frames the pro-state policy position in terms of public interest, describing how SOEs serve the people's interest and provide for public welfare.
Since the early 2000s, China and the Philippines have clashed over sovereignty of islands and shoals in the South China Sea. The three videos about China and the Philippines also capture the two policy stances of the Chinese government—cooperation (Dove) and aggressive measures to stake China's claim (Hawk)—as well as a control condition where facts about China-Philippines relations are provided but no state position is given (Control). The Dove condition frames the dovish policy position in terms of common prosperity, describing how mutual economic gain and trade can sustain regional stability. The Hawk condition frames the hawkish policy position in terms of national sovereignty, emphasizing that China's territorial integrity is sacred and inviolable. The treatment conditions operationalize framing, rather than persuasion or other media effects, by not introducing any information that would change viewers’ evaluation of attributes and by only emphasizing different attributes.
Each video also contains interviews we filmed and edited with two Chinese academics with expertise on the two policy issues. We include such interviews because they are frequently used in Xinwen Lianbo broadcasts. The same academic provides the explanations and rationales for both frames of SOE reform, as well as background facts on SOE reform for the control condition. The second academic provides the same for China–Philippine relations. Using the same expert in all treatment and control conditions of the same topic and by filming and editing the interviews ourselves ensures that the characteristics of the academics do not influence any outcomes we observe.
The videos include voiceovers by a broadcaster trained at the Communication University of China, which produces CCTV anchors and reporters. The videos are fully captioned in Chinese. Video images come from a free Chinese image library. See Appendix for English translations of all video transcripts.
Survey sample
We recruit an online sample of 830 respondents from 26 provinces in China.Footnote 11 Half (51.8 percent) of the sample came from richer provinces (based on 2017 per capita income), and the remainder from poorer provinces.Footnote 12 This sample is not representative of China's overall population, but we adopt a quota sampling strategy to ensure that the sample is diverse in gender, age, and educational background. Table A3 reports the summary statistics of the sample.
Survey design
The overall flow of our survey is illustrated in Figure 1.
Survey and experimental flow.
After screening, we ask the respondents to answer two sets of questions to measure their predispositions on economic and foreign policies. One set contains seven questions related to preferences on market economy—for example, whether private ownership and sale of land should be allowed—where responses range from preferences for free market and private capital to preferences for state control of the economy and limits on the influence of private capital. The other set contains seven questions related to preference on national sovereignty, such as whether it is appropriate to use military force to unify Taiwan with China. Here, responses range from highly nationalistic to not nationalistic (see Appendix Table A2 for list of questions). We construct two simple additive indices based on these questions to measure predisposition.Footnote 13
After we measure predisposition, the respondents watch two videos, one on each policy issue. The order in which the videos are watched is randomized, and randomization for each video is independent. Respondents are well balanced in terms of demographics (see Appendix Table A4 for the balance table). As shown in Figure 1, immediately following each video, we conduct manipulation checks by asking respondents two multiple-choice questions about the content of the video they have just viewed. We also assess the construct validity of our treatments by asking respondents whether they thought the source of the video was CCTV.
We measure policy preferences after the manipulation checks in two ways. First, we ask the respondents to explicitly select the statement that best reflects their preferences on each issue. For SOE reform, respondents can select from three options: SOE reform should be led by the state (1 State); SOE reform does not matter as long as economic outcomes are good (2 Neutral); or SOE reform should be market driven (3 Market).Footnote 14 Larger values denote a more pro-market position. For China–Philippines relations, respondents can select from three options: China must defend its national sovereignty and territorial integrity (1 Hawk), China's FP should change with changing circumstances (2 Neutral); or China should cooperate with the Philippines and set aside territorial disputes for regional peace (3 Dove). Larger values denote a more dovish, cooperative position.Footnote 15
Second, we create a composite measure by asking respondents several questions in closely related policy domains. We ask nine questions such as whether SOEs should receive preferential treatment from the government and whether SOEs should receive more preferential treatment than private enterprises (see Table A13 in the Appendix for the list of questions). Responses to these questions vary from supporting free market to supporting state control. We construct a simple additive index with these questions. Similarly, after the China–Philippines relations video, we ask nine questions about China's FP in Southeast Asia and China's diplomatic strategy, which vary along the dove versus hawk dimension. We use these closely related questions to construct an additive index of dovish FP.Footnote 16
To measure political knowledge, we ask the respondents four factual questions (see Table A13 in the Appendix for details). We also ask the respondents for demographic information, including their age, ethnicity, marital status, and income, as well as a number of other outcomes such as political knowledge and political trust. Finally, because research in media effects suggests that effects can quickly decay (Hill et al., Reference Hill, Lo, Vavreck and Zaller2013), we conduct a follow-up survey 48 hours after the first survey to ask the respondents about their understanding of the policy positions espoused in the video, their position on SOE reform and China–Philippines relations, and their policy preferences on the role of SOEs in the economy and FP more generally.Footnote 17
To ensure the quality of responses, we embed an attention filter in the predisposition questions prior to treatment assignment,Footnote 18 and screen out those who fail from the survey. We also see whether the birth year reported by the same respondent is consistent at the beginning and at the end of the survey. Finally, we measure the amount of time respondents spend watching the video and completing the entire survey to identify speeders.Footnote 19 Reducing the sample to respondents who pass the age-consistency test and who are not speeders reduces the sample size from 830 to 762.Footnote 20
4. Results
In line with the expectation that government-controlled media can provide new frames to shift public attitudes on an existing issue, we find that exposure to our video treatments moves respondents toward the espoused policy position of the state, regardless of what that position is and regardless of the predisposition of the respondent. These effects are observed in direct and composite measures of policy attitudes. Many of these effects persist up to 48 hours after exposure. We begin by showing the main results, and then show heterogeneous effects for predisposition and political knowledge, ending with results from the 48-hour follow-up. For details of the manipulation checks, see Appendix.
Framing effects
We find that the respondents respond to different frames and are more likely to adopt policy positions closer to the policy position espoused in the video. Figure 2 presents the estimated average treatment effects on preferences toward SOE reform.Footnote 21 The outcome in the left panel is a three-value indicator of direct policy preference (a bigger value means a more pro-market approach), and the outcome in the right panel is a standardized index of composite policy preference (with 0 mean and 1 standard deviation) based on nine closely related policy questions. The thick and thin bars in both plots represent 95 and 99 percent confidence intervals for the point estimates, respectively (same for subsequent Figures 3 and A1). We estimate the effects using regressions, controlling for a full set of individual characteristics and provincial fixed effects (see Table A7 in the Appendix for the full regression table).Footnote 22
Treatment effects on policy preferences: SOE reform. (a) Agree to market reform. (b) Index of market-oriented reform.
Treatment effects on policy preferences: South China Sea. (a) Agree to more cooperation. (b) Index of dovish foreign policy.
We see that framing SOE reform in terms of efficiency (Market condition) on average shifts the respondents’ stated preference on SOE reform toward a more market-oriented approach. On the direct measure (left panel of Figure 2), there is a shift of approximately 0.3 points on a 1-3 point scale. On the composite measure (right panel of Figure 2), the pro-market video content also shifts respondents approximately 0.3 standard deviations toward a more market-oriented position on related policy issues. Similarly, framing SOE reform in terms of public interest (State condition) reduces the respondents’ preference for market-oriented reform by roughly 0.3 points based on the direct question (left panel of Figure 2), and 0.2 standard deviation based on the composite measure (right panel of Figure 2). It is worth noting that both the Dove and Hawk treatments shift respondents’ preference on SOE reform toward a more state-led approach (see Dove and Hawk estimates in Figure 2), compared with the Control FP condition. This is likely because a video about FP may make the central role of the state in national interests more salient.
Figure 3 shows the estimated average treatment effects on policy preferences regarding China–Philippines relations in the South China Sea. The left panel shows direct policy preferences (a bigger value means a more dovish approach), and the right panel shows a composite measure (with 0 mean and 1 standard deviation) based on nine closely related policy questions. The full regression results are presented in Table A8 in the Appendix. From Figure 3, we see that framing China–Philippines relations in terms of mutual prosperity (Dove condition) on average shifts the respondents’ stated preference on the issue of China–Philippines relations toward a more dovish, cooperative approach by around 0.15 points on a 1–3 point scale based on the direct question (left panel of Figure 3) and around 0.19 standard deviation based on the composite measure (right panel of Figure 3). In contrast, framing China–Philippines in terms of national sovereignty (Hawk condition) reduces the respondents’ support for the dovish approach also by roughly 0.3 points based on the direct question (left), and 0.26 standard deviation based on the composite measure (right). Policy preferences on FP do not appear to be affected by the content of the video on SOE reform.
Predisposition
Predisposition does not appear to moderate the effects of government-controlled media. We measure predisposition toward economic policies and national sovereignty before exposing respondents to the video treatments. We find strong correlation between predisposition and post-treatment policy preferences in both the economic and the sovereignty dimensions (Figure A2 in the Appendix). This suggests, first, that many respondents have relatively coherent policy preferences. Second, respondents are not simply answering questions to satisfy the researchers. Strikingly, regardless of predisposition, exposure to the different frames presented in our video treatments changes policy preferences.
The left panel (a) of Figure 4 shows the marginal effect of State (light gray) and Market (dark gray) conditions on the probability of supporting market-oriented SOE reform moderated by predisposition toward a market economy. The right panel (b) of Figure 4 shows the marginal effect of Hawk (light gray) and Dove (dark gray) conditions on the probability of supporting more cooperative FP moderated by predisposition toward national sovereignty. Both are produced using a kernel estimation method proposed by Hainmueller et al. (Reference Hainmueller, Mummolo and Xu2019). Estimates above zero on the y-axis in panel (a) represent a marginal effect in support of market-oriented SOE reform while estimates below zero on the y-axis in panel (a) represent a marginal effect supporting state-led SOE reform. Estimates above zero on the y-axis in panel (b) represent a marginal effect in support of more dovish FP while estimates below zero on the y-axis in panel (b) represent a marginal effect in support of more hawkish FP. The (stacked) histograms at the bottom of the plot show the number of respondents under each treatment condition given different values of predisposition.
Marginal effects of the treatments by predisposition. (a) Treatment 1 on pro-market. (b) Treatment 2 on pro-cooperation.
Note: Panel (a) shows the marginal effect of videos on SOE reform on the probability of agreeing to market-oriented SOE reform moderated by predisposition toward market economy in percentiles. Panel (b) shows the marginal effect of videos on China–Philippines relations on the probability of agreeing to more cooperation in China's dealing with the Philippines moderated by predisposition on sovereignty in percentiles. Political knowledge is controlled for in both kernel estimations. Stacked histograms at the bottom of each plot show the number of respondents under each treatment condition given different values of the moderator. The estimates are based on a kernel regression method for multiplicative interaction models (Hainmueller et al. Reference Hainmueller, Mummolo and Xu2019).
The left panel (a) of Figure 4 shows that respondents with different predisposition toward how resources should be allocated, both those who support market allocation of resources (toward the right of this figure) and those who support state allocation of resources (toward the left of this figure) are influenced by differing policy frames on SOE reform. When exposed to frames about efficiency (Market video in darker gray), support for market-oriented SOE reform goes up across the board. When exposed to frames about public interest (State video in lighter gray), support for market-oriented SOE reform goes down across the board.
Likewise, the right panel (b) of Figure 4 shows that respondents with different predispositions toward national sovereignty, both those who are less nationalistic (toward the right of this figure) and those who are more nationalistic (toward the left of this figure) are influenced by differing policy frames on China–Philippines relations. When exposed to frames about mutual prosperity (Dove video in darker gray), support for dovish FP goes up across the board. When exposed to frames about the inviolability of territorial integrity (Hawk video in lighter gray), support for dovish FP goes down across the board.
Political knowledge
After controlling for predisposition, we find that those with higher levels of political knowledge in some cases experience stronger framing effects. This falls in line with the theoretical expectations of Druckman and Nelson (Reference Druckman and Nelson2003).
The left panel (a) of Figure 5 shows the marginal effect of State (lighter gray) and Market (darker gray) videos on the probability of supporting market-oriented SOE reform moderated by political knowledge, and the right panel (b) of Figure 5 shows the marginal effect of Hawk (lighter gray) and Dove (darker gray) videos on the probability of supporting more cooperative FP moderated by knowledge. Estimates above zero on the y-axis in panel (a) represent a marginal effect in support of market-oriented SOE reform while estimates below zero on the y-axis in panel (a) represent a marginal effect supporting state-led SOE reform. Estimates above zero on the y-axis in panel (b) represent a marginal effect in support of more dovish FP while estimates below zero on the y-axis in panel (b) represent a marginal effect in support of more hawkish FP. The (stacked) histograms at the bottom of the plot show the number of respondents under each treatment condition given values of political knowledge ranging from 0 to 4.
Marginal effects of the treatments by political knowledge. (a) Treatment 1 on pro-market. (b) Treatment 2 on pro-cooperation.
Note: Panel (a) shows the marginal effect of videos on SOE reform on the probability of agreeing to market-oriented SOE reform. Panel (b) shows the marginal effect of videos on China–Philippines relations on the probability of agreeing to more cooperation in China's dealing with the Philippines. In both plots, the moderator is a measure of political knowledge ranging from 0 to 4. We control for predisposition in both kernel estimations. Stacked histograms at the bottom of each plot show the number of respondents under each treatment condition given different values of the moderator. The estimates are based on a kernel regression method for multiplicative interaction models (Hainmueller et al. Reference Hainmueller, Mummolo and Xu2019).
The left panel (a) of Figure 5 shows that respondents with higher levels of political knowledge (toward the right of this figure) appear slightly more influenced by differing policy frames on SOE reform, this is especially true for the public welfare frame (State video in lighter gray) in support for state-led reform. Similarly, the right panel (b) of Figure 5 shows that respondents with higher levels of political knowledge (toward the right of this figure) are more influenced by frames about mutual prosperity (Dove video in darker gray).
Other respondent characteristics
We also explore where the effects of government-controlled media vary in statistically significant ways for individuals with different socioeconomic and demographic characteristics. Using regression controlling for individual characteristics and provincial fixed effects, we conduct subgroup analyses and explore heterogeneous treatment effects. The results are reported in Figure A1 in the appendix. In general, we find that the two sets of treatments change respondents’ policy preferences across almost all subgroups. However, we do not find differences in the amount of attention respondents of various subgroups paid to the content of the videos, so this pattern is not likely driven by differences in attention.
Moreover, we explore treatment effects on other, more diffuse and complex outcomes—including measures of patriotism and nationalism, efficacy, trust in the central and local governments, and self-reported political behavior such as complaint-making, voting in local National People's Congress elections, and discussing government policies. We find no evidence that the video treatments affected any of these outcomes (see Table A10 in the Appendix for details).
Robustness checks
We conduct a number of robustness checks by pruning the data based on different criteria. We present the results in Table A9 in the appendix. The outcome variables in panels A and B are the two indices of policy preferences. Column (1) shows the benchmark result using the full sample. In columns (2) through (6), we limit the sample to respondents who satisfy various conditions, including those who identified the source of the videos as CCTV (86 percent of all respondents), those who passed the age consistency filter and were not speeders (92 percent of all respondents), those who answered both factual questions correctly (83 percent of all respondents), those who agreed the policy toward SOE reform is correct (94 percent of all respondents), and those who agreed the policy toward the South China Sea is correct (96 percent of all respondents). The results remain substantively the same, and in some cases, the effects became even stronger.
Results from the 48-hour follow-up survey
We are able to reach 353 of out of 556 respondents 48 hours after the first survey (recontact rate of 63 percent).Footnote 23 We find that respondents exposed to the Market condition remain strongly supportive of market-oriented reform of SOEs, while those exposed to the State condition remain less supportive of market-oriented SOE reform. Similarly, those exposed to the Dove condition remain more supportive of cooperation and those exposed to the Hawk condition less supportive of cooperation. Due to power limitations, we conduct permutation tests on the effects of Treatment 1 and Treatment 2 against the sharp null that these treatments have zero effects on outcomes measured in the follow-up survey. We find that we can reject the sharp null hypotheses of zero effects in comparisons between the State and Control conditions with Treatment 1, between the Market and State conditions with Treatment 1, and between the Dove and Hawk conditions with Treatment 2, with a high level of confidence (with p-values equal to 0.001, 0.000, and 0.012, respectively). These results indicate that, at least for some respondents, the treatments had a lasting effect up to 48 hours. For details of these analysis, as well as manipulation checks for the follow-up survey, see Appendix.
5. Conclusion
Through this experiment, we study whether authoritarian regimes can use government-controlled media to re-frame policy issue and change public opinion toward policies. From existing theory, we identify reasons why this may or may not be possible. Our experimental results suggest that when authoritarian regimes need to change their position on an issue, they can use government-controlled media to provide new frames for the issue and effectively shift public attitudes if they can achieve mainstream consensus, preventing alternative frames and contextual information from being voiced. Specifically, we find that government-controlled media, which frames domestic and foreign policy issues in different ways, moves respondents to adopt policy positions closer to the position, including opposing positions, espoused in government media. We find this to be the case whether we use direct or composite measures of attitudes, and we find that framing effects persist up to 48 hours after exposure.
Importantly, we find that different frames are effective regardless of individual predispositions. Both those who have pro-market and those who have pro-state preferences with regard to economic policy become more likely to support market-led reform of SOEs when they are exposed to a frame about the importance of efficiency, and both those who hold pro-market and those who hold pro-state economic preferences become more likely to support state-led reform of SOEs when they are exposed to a frame about the role of SOEs in serving public interests. Both those who are nationalist and those who are anti-nationalist become more likely to support dovish foreign policy when such policies are framed in terms of mutual prosperity, and respondents with differing predispositions toward nationalism all become more likely to support hawkish foreign policy when such policies are framed in terms of sovereignty and territorial integrity.
Finally, after controlling for predisposition, we find that those with higher levels of political knowledge are sometimes more influenced by framing effects than those with lower levels of political knowledge. This aligns with existing theories that knowledge increases the availability and comprehensibility of frames to individuals after accounting for predisposition.
While our results on predisposition diverge from the bulk of research on the role of predisposition as a moderator of government media and framing effects, these results align with previous theories that predict such a result when propagandists can dominate and control the media environment, suppressing competing frames and contextual information that could link predisposition to new messages from government media. Such a circumstance is extremely unlikely in any competitive media system, and hence unlikely in most democracies, but mainstream consensus is likely and indeed the reality in many authoritarian regimes where those in power effectively stifle alternative voices. Although not all authoritarian regimes can achieve such an outcome, there are numerous contemporaneous (e.g., China, Cuba, Iran, Russia, Saudi Arabia, Turkmenistan) and historical (Peru under Fujimori, U.S.S.R) examples of such regimes. This finding suggests that media market structure and context matter a great deal to understanding the boundaries of framing effects.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/psrm.2021.35.
Acknowledgements
Our thanks to Zhijun Bi, Jiajun Li, Qixing Mo, Zhongfu Su, Jiaru Yu, Liangyuan Zeng, Zhaotian Zu, and others for excellent research support; to the interviewees featured in the videos; to Andrew Chadwick, Lance Holbert, Rasmus Nielsen, and participants at the Washington University in St. Louis Political Science Seminar and APSA Political Communication Pre-Conference for many helpful comments and suggestions.
), 
