Partisans hold positive perceptions of in-party media outlets and negative perceptions of out-party media outlets (Levendusky and Malhotra, Reference Levendusky and Malhotra2016). However, the online news environment may facilitate cross-cutting exposure over time, changing partisans' views toward partisan news outlets. It is difficult to know whether people's views of partisan news remain polarized because it is difficult to design a treatment that mimics daily exposure to a diverse online news environment. Do perceptions of partisan media change upon repeated exposure to an online news media environment?
Existing studies are typically constrained to one-shot, forced exposure designs to one source, and are unable to answer this question. An advantage of one-shot designs is the experimental control and assurance of a primary delivery of treatment. A disadvantage of one-shot designs is limited external generalizability (Arceneaux and Johnson, Reference Arceneaux and Johnson2013). As the media environment is increasingly characterized by choice, gaging news behaviors in a strictly controlled experimental setting is unlikely to provide the full picture. We use a novel experimental design in which consumers use an online news portal, manipulated to vary in its balance of partisan news outlets, for a week. Our method approximates real-world media use while varying partisan news exposure.
We find Democrats who repeatedly encounter Fox News in their feeds consider the network to be fairer than those who do not see Fox News in their feeds. Republicans that encounter Fox News find the outlet to be less fair. These results suggest that repeated exposure to out-party and in-party partisan news can diminish the gap in perceptions of outlet fairness among partisans. We conclude with some discussion of what drives these asymmetric effects. This study broadens our knowledge of people's partisan news perceptions in an online news environment beyond one-shot exposure.
1. Partisan news effects online
Partisan news consumers see bias in media accounts that diverge from their worldview (Vallone et al., Reference Vallone, Ross and Lepper1985), but not in in-party news (Coe et al., Reference Coe, Tewksbury, Bond, Drogos, Porter, Yahn and Zhang2008). This research was conducted when channels were limited (Gunther et al., Reference Gunther, McLaughlin, Gotlieb and Wise2017), using one-shot designs. Today, people encounter diverse sources online, making it important for researchers to incorporate characteristics of the information environment into research designs (Reid, Reference Reid2012).
Fox News and MSNBC are known to be biased (Stroud, Reference Stroud2011) and rely on negative news about the out-party (Smith and Searles, Reference Smith and Searles2014). These characteristics enable partisans to sort outlets as prototypically preferred by in- or out-group members (Stroud et al., Reference Stroud, Muddiman and Lee2014). People draw on media exemplars and inaccurate forecasts when evaluating opposing content, resulting in biased perceptions (Druckman et al., Reference Druckman, Klar, Krupnikov, Levendusky and Ryan2019). We can generate expectations regarding how attitudes may change toward in-party and out-party news organizations after sustained exposure to a non-partisan news feed, or news feeds that feature MSNBC or Fox News.
Same as it ever was (hypothesis 1). Upon daily consumption of a non-partisan news feed, partisans likely maintain animus toward out-party outlets, while elevating in-party outlets. People make assumptions about out-group opinions based on social expectations (Zhang and Reid, Reference Zhang and Reid2013), and such social accounts of prototypical out-group characteristics mean that people can distinguish themselves from others without first-hand knowledge of the traits that make others different (Hogg and Reid, Reference Hogg and Reid2006). If supported, we would expect partisans assigned to a feed without partisan news to maintain their negative perceptions of the out-party outlet, regardless of exposure.
Shifting to partisans in the two treatment conditions, a news feed featuring MSNBC or a news feed featuring Fox News, we can pose two possible effects that may change given repeated exposure.
Learning from the enemy (hypothesis 2). Daily consumption of a news feed featuring out-party outlets likely decreases negative perceptions of opposing news, relative to co-partisans assigned to a non-partisan news feed. Sustained exposure to out-party outlets in a portal offers partisans more opportunity to learn from inadvertent exposure (Kobayashi et al., Reference Kobayashi, Hoshino and Suzuki2017). When given choices, partisans do not avoid counter-attitudinal news (Garrett and Stroud, Reference Garrett and Stroud2014), particularly in a portal that can increase overall news exposure (Beam and Kosicki, Reference Beam and Kosicki2014). Extant studies uncovered evidence that people are exposed to considerable cross-cutting news in an online environment (Kim, Reference Kim2011). Thus, online users are more likely to be exposed to cross-cutting news in their feed and may find that out-party news is not as opinionated as they assumed. Although partisan news outlets feature more ideologically extreme voices on opinion shows, partisan news programs represent more centrist perspectives (Smith and Searles, Reference Smith and Searles2013). Online partisan news content is more like traditional news content than assumed, given similarities in coverage (Maier, Reference Maier2010). If hypothesis 2 is supported, then we would expect partisans assigned to a feed including an out-party outlet to become more positive in their perceptions of the outlet.
Familiarity breeds contempt (hypothesis 3). Another possibility is that daily exposure to in-party news in a portal increases negative outlet perceptions, relative to co-partisans assigned to a non-partisan news feed. Inasmuch as people make assumptions about the dominant traits of the out-group, they also categorize themselves and media into smaller segments that distinguish “us” from “them” (Tajfel, Reference Tajfel1982). Giner-Sorolla and Chaiken's (Reference Giner-Sorolla and Chaiken1994) study underscores this: among the factors that account for partisans' judgments of news, prior beliefs in bias play a bigger role than actual content. Having previously drawn upon biased samples of media exemplars, daily exposure to a feed featuring in-party news is likely to change perceptions of the outlet. If hypothesis 3 is supported, then we would expect that partisans assigned to a feed including in-party news will become less positive in their perception of the outlet.
2. Methods: sustained online exposure
Treatments in this study consisted of timely news stories delivered daily over a week. We accomplished this by creating a custom online news portal for participants to use that resembled a simplified version of Google News (Figure 1, also online Appendix A). Google News is a market leader, accumulating the most (58.9 percent) unique online visitors each month (Webster and Ksiazek, Reference Webster and Ksiazek2012). News stories were automatically added to our portal by a PHP script that checked RSS feeds of Google News every 30 min. The news portal ran from 29 May to 5 June 2016, and included 10,774 stories. Headlines were displayed chronologically alongside publication date, news outlet, and a description.
Participant view of the news portal.
When a new story was detected in the RSS feeds of Google News it was automatically assigned using Amazon's Mechanical Turk (MTurk) API to human coders who checked that the stories were news. After this check, stories were pushed to the portal with an average delay of 27 min. Six trained researchers monitored workers' coding decisions.
A convenience sample of US adults was recruited from MTurk, which was 53 percent female, 72 percent white, and 53 percent college educated, with an average age of 35 (SD = 11.6 years). The sample included more Democrats (44 percent) and Independents (32 percent) than Republicans (19 percent). For the purpose of an experimental factor that introduced state-specific news stories, participants were required to be residents of one of the ten most populous states (California, Florida, Georgia, Illinois, Michigan, New York, North Carolina, Ohio, Pennsylvania, or Texas). Users who had completed any of our news story coding tasks were not allowed to sign up as study participants, and vice versa.
The study proceeded in four stages. First, participants took a pre-test survey (N = 1395) for $1. Second, they were asked to regularly use the news portal for 1 week. Third was a post-test survey (N = 1051), available 1–36 h after the portal closed. For the post-test survey participants were paid $2 instead of $1. A bonus of $2 was awarded to users in the top 1/3 of portal usage. Finally, subjects were offered $1 to complete a follow-up survey (N = 897) 4–5 days after the news portal closed. This study utilizes data from the pre- and post-tests.
3. Design and treatments
Our primary factor of interest was partisan news, which randomized exposure to a news feed with no partisan news outlets,Footnote 1 a feed with Fox News added, or a feed with MSNBC added. To create the partisan news dose, stories were added from Fox News or MSNBC feeds using a Google News RSS with a source query. The broader design is fully factorial 2 (state factor: no state news added, state news added) × 3 (partisan news factor: no partisan news added, Fox News added, MSNBC added) × 3 (problem news factor: all problems, all problems except crime, no problems). These factors were all delivered continually within the portal. Although we include these factors in our estimates, they are not relevant to this study (see online Appendix A). Participants saw the same streams, but those in the partisan news conditions saw feeds featuring Fox News or MSNBC stories. Additional robustness checks in online Appendix B.
3.1 Measures
3.1.1 Fox/MSNBC fairness
Following Arceneaux and Johnson (Reference Arceneaux and Johnson2015), we use a single-item scale to capture participants' perceptions of Fox News/MSNBC fairness.Footnote 2 In the post-test survey, respondents rated their agreement on a 7-point scale (1 = strongly disagree, 7 = strongly agree) with two statements: “In general, Fox [or MSNBC] news is fair.” Two outcomes are computed, “Fox is fair” (Wave 2/Post-test: M = 2.67, SD = 1.66), and “MSNBC is fair” (Wave 2/Post-test: M = 3.51, SD = 1.49).
3.1.2 Partisanship
Respondents were asked which party (Democratic, Republican) they felt closest to; collapsed into a binary variable. Democrats comprised of 68.1 percent of this partisan sample (N = 680 post-test); Republicans comprised of 31.9 percent (N = 319 post-test).Footnote 3
3.1.3 Portal usage
As is typical with self-selected exposure to news, participants varied in their use of the news portal. Incentives encouraged use by participants in terms of distinct headlines encountered (M = 253.5, SD = 272.39, median = 160, min = 0, max = 1827), and stories clicked on (M = 18.27, SD = 22.10, median = 12, min = 0, max = 193).
Portal usage patterns were consistent across the three conditions. Although participants in “no partisan news added” condition (M = 255.38) encountered slightly fewer headlines than those in “Fox News added” (M = 268.45) and slightly more than people in “MSNBC added” (M = 237.32) conditions, these differences are not statistically significant (F [2, 2515] = 2.76, p = 0.063). There was little variance in stories clicked for “no partisan news added” (M = 18.78), “Fox News added” (M = 18.47), and “MSNBC added” (M = 17.59) conditions, F [2, 2515] = 0.65, p = 0.52.Footnote 4
3.1.4 Covariates
Although random assignment mitigates concern, we control for confounds that may explain respondent perceptions of Fox News/MSNBC. Our models include these controls: “Portal Usage,” the natural logarithm of a continuous variable that captures the extent to which a participant used our news portal (untransformed statistics: M = 922.51, SD = 1007.51, median = 654, min = 0, max = 8266), and two other factors that were not analyzed in this study and their interactions (Table A5, online Appendix A).
4. Results
4.1 Manipulation check
This study utilizes a news portal that randomly assigns participants to receive news feeds with partisan news doses (no partisan news outlets added, MSNBC news added, and Fox News added). We first check that there is variance in participants' exposure to the partisan news feed, captured first using the number of partisan headlines encountered (M = 1.37, SD = 5.60, median = 0, min = 0, max = 89) and then the number of partisan stories clicked on (M = 0.06, SD = 0.34, median = 0, min = 0, max = 5); previous study shows headlines to be a cue on similar platforms (Sundar et al., Reference Sundar, Knobloch-Westerwick and Hastall2007). First, participants in each condition encountered distinct partisan news headlines (F [2, 1258] = 60.29, p = 0.000). Relative to “MSNBC news added” (M = 0.41) and “no partisan news added” (M = 0) conditions, participants in “FOX News added” (M��= 3.70) were exposed to more partisan news headlines. Second, we observe a similar trend for number of partisan stories clicked (F [2, 1258] = 33.56, p = 0.000): participants in “Fox News added” (M = 0.17) condition clicked on significantly more partisan stories than those in “MSNBC news added” (M = 0.01) and “no partisan news added” (M = 0) conditions.
4.2 Panel regression models
We have panel data which requires controls for time-invariant differences correlated with our moderating variable “partisan media exposure” and the dependent variable “perceived media fairness.” Hypothesis 1 posits tests within one condition, so we restrict the sample to only those in the “no partisan news added” condition and use a lagged dependent variable mode:
Here, Fairness i,t+1 indicates the fairness rating of Fox News/MSNBC in the post-test, β 0 is an intercept, Party i,t represents a dummy variable indicating respondent identification with the Democratic Party (Fox News model) or Republican Party (MSNBC model) in the pre-test,Footnote 5 Fairness t indicates the assessment of the same network's fairness in the pre-test, ΓCovariates i,t represents a set of variables capturing participant age (M = 34.98, SD = 11.56), gender (53.56 percent are female), ethnicity (73.80 percent are White), education (53.09 percent college-educated and above), political ideology (measured as the natural logarithm of the seven-point scale; 1 = very liberal, 7 = very conservative; M = 3.34, SD = 1.64), and their interest in politics (measured as the natural logarithm of the 7-point scale; 1 = not interested at all, 7 = very interested; M = 5.25, SD = 1.50), Usage i,t+1 represents portal usage (described above), Γconditions i represents variables indicating inclusion in the other conditions, and ɛi a stochastic disturbance term.
We analyze hypotheses 2 and 3 using panel regression random-effects models, modeling the fairness of Fox News and MSNBC according to respondent party and condition. Given the relationship between partisanship and media choice, we control for individual characteristics using fixed-effect models. A Hausman test shows we cannot reject the null hypothesis that the within-entity error term is uncorrelated with the predictors (Greene, Reference Greene2007). Thus, it is appropriate to use a random-effects model which accounts for within-subject variance while modeling between-subject differences. This model is given in Equation 2:
In Equation 2, Δfairnessi is change in fairness rating from pre- to post-test, β 0 is an intercept, Dem a partisanship dummy with 1 representing Democrat and 0 representing Republican from the pre-test, Partisan news condition represents either Fox News or MSNBC exposure with the “no partisan news added” condition excluded as the baseline, Demi × Partisan news conditioni represents the interaction between Democrat and each partisan news condition (i.e. “Democrat × Fox News added” and “Democrat × MSNBC added”), Γ(Demi × Conditionsi) represents a set of interactions between the Democrat dummy variable and the other conditions, Usage i,t+1 represents portal usage as above, and ɛ i is a stochastic disturbance term.
Same as it ever was. Results from Equation 1 (hypothesis 1) are shown in Figure 2 as coefficient plots: we compare the interaction terms of each relevant condition with 95 percent confidence intervals to show which coefficients are statistically distinguishable from zero. Fox News fairness ratings appear on the left, MSNBC ratings appear on the right (alternative specifications: online Appendix D).
Coefficient plots for partisan ratings of outlet fairness, respondents in “no partisan news added” condition. Corresponds to Equation 1. Point estimates of coefficients with 95 percent confidence intervals (Table A3, online Appendix A).
We expect that partisan evaluations of out-party news remain unchanged in the control condition, where participants encountered a non-partisan feed. The results visualized in Figure 2a support this hypothesis. The point estimate of the coefficient for Republicans' MSNBC evaluations was nearly zero, whereas Democrats' Fox News evaluations was negative but insignificant. Absent exposure, impressions of media fairness did not change.
Results from our tests of hypotheses 2 and 3, examining out-party and in-party assessments of Fox News/MSNBC fairness using random effects models (Equation 2), are shown in Figure 3 as coefficient plots.
Coefficient plots for partisan ratings of outlet fairness across conditions. Corresponds to Equation 2. Point estimates of coefficients with 95 percent confidence intervals (Table A4, online Appendix).
Learning from the enemy. Out-party news exposure increased fairness evaluations, but only for Democrats' evaluations of Fox News fairness (p < 0.05). There was no change in Republicans' MSNBC evaluations following this increased exposure. Over the course of a week, when shown more Fox News headlines, Democrats rated the network as more fair than those who were not exposed to a feed featuring the network. Seeing the actual news content improved Democratic evaluations of Fox News.
Familiarity breeds contempt. Republicans rated Fox News as less fair upon repeated exposure to a feed featuring the outlet (p < 0.05), whereas Democrats' MSNBC evaluations were unchanged. Republicans who saw Fox News in their feeds rated it as less fair, suggesting that repeated exposure also has consequences for in-party evaluations of partisan media—at least on the right.
5. Conclusion
News consumers may have ideas about out-party or in-party news that do not endure upon repeated exposure to those outlets. One-shot experimental designs do not sufficiently replicate real-world exposure to news. Our design addresses that deficiency by supplying users with experimentally manipulated news feeds over the course of a week, allowing us to assess perceptions given exposure to a diversity of online new.
Although our hypotheses are largely supported, the results are asymmetric and largely driven by Fox News: Republicans think less of Fox News after repeated exposure, whereas Democrats think Fox News is fairer. Why would this be the case? Online content published by these news brands differs from on-air content. Partisans may base their perceptions of Fox News and MSNBC on their opinion shows, which differ substantially from news coverage (Smith and Searles, Reference Smith and Searles2013). Partisans started with very different perceptions of out-party news fairness: in the pre-test, Democrats average Fox News rating is 2.17 on our 1–7 scale, compared to an initial MSNBC rating of 3.0 by Republicans. Attitudes toward Fox News are simply stronger than views of other networks (Iyengar and Hahn, Reference Iyengar and Hahn2009). Fox News exerts a unique influence on policymakers (Arceneaux et al., Reference Arceneaux, Johnson, Lindstädt and Vander Wielen2016), on candidates (Arceneaux et al., Reference Arceneaux, Dunaway, Johnson and Vander Wielen2020), and on voter choice (Martin and Yurukoglu, Reference Martin and Yurukoglu2017). Asymmetric effects deserve additional scrutiny, but in the case of Fox News, there is reason to believe its partisan reputation is unique.
We believe that enrolling respondents in experimentally manipulated news feeds represents a promising path forward for media effects research. This approach is unobtrusive and offers more external validity. We show how repeated exposure to partisan news networks—which partisans have strong preconceived notions about—can change those attitudes.
The online news environment can lead to cross-cutting exposure that may change consumers' perceptions of partisan news outlets. In particular, Fox News evaluations by both Democrats and Republicans change upon repeated exposure to the outlet. Our knowledge of partisan media effects and the designs used to test those effects need to be updated for the online news environment, and this study represents a step in that direction.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/psrm.2021.21.
