Candidate biographies. Voting records. Issue analysis. Economic trends. Editorial endorsements. Campaign ads and speeches. Polling results. Registration requirements. Voting locations. The arrival of the so-called information age means that a vast and diverse array of political information is seldom more than a few clicks away. Hurdles making it difficult to obtain information were once held to be a chief impediment to an informed citizenry (Katz and Lazarsfeld Reference Kinder1955; Converse Reference Converse1962). Today, these impediments have all but vanished, making available to citizens a smorgasbord of information that has no practical limit.
A great irony of this sea change is that many citizens may acquire less political information—or at least a narrower sampling of information—than they did in the past. There is now considerable agreement that the exploding number of sources makes citizen choice a far more central factor in determining the dispersion of information. Previously, much of what citizens knew about politics was thought to come as a by-product of incidental exposure to information (e.g., LoSciuto Reference Mahapatra1972; Epstein Reference Festinger1973; Robinson Reference Ryan1976; Graber Reference Green, Palmquist and Schickler1988; Fiorina Reference Gadarian and Albertson1990; Neuman, Just and Crigler Reference Parker-Stephen1992). Now, newly empowered to choose, many citizens opt for narrowly tailored sources, and many others opt out of political coverage altogether (Chaffee and Metzger Reference Chaffee and Metzger2001; Prior Reference Prior2005; Bennett and Iyengar Reference Bennett and Iyengar2008; Lawrence, Sides and Farrell Reference Lippmann2010; Arceneaux and Johnson Reference Arceneaux and Johnson2013; though see Barberá Reference BaumN.d. on the diversifying potential of social media).
An age of abundant options makes understanding the basis of information choice all the more important, inviting a renaissance of efforts to explain how citizens choose what to read and watch. After all, the question of who gets what information is relevant to so many topics in the study of politics. Informed citizens are more likely to participate and vote (Verba, Schlozman and Brady Reference Verba, Schlozman and Brady1995), so patterns of choice could empower some constituencies while disempowering others. If citizens have a preference for some styles of coverage (e.g., confrontational) over others, it could explain why some ways of covering the news would succeed, even if they have damaging effects (Mutz and Reeves Reference Nyhan2005). The possibility that citizens have a preference for attitude-consistent information could explain a rise in hostile feelings toward political opponents and other indicia of mass polarization (Prior Reference Prior2007; Iyengar, Sood and Lelkes Reference Kiesler2012; Mason Reference Mercier and Sperber2014) and render those same citizens less responsive to new ideas and evidence that might change their minds (Zaller Reference Zaller1992).
This last concern calls attention to the long, tangled history of research on selective exposure in political science and social psychology (Lazarsfeld, Berelson and Gaudet Reference Lin1948; Festinger Reference Fiorina1957). Despite a great deal of research and widespread belief in the phenomenon both within and beyond scholarly circles, the proposition has always faced, as a recent review laments, daunting methodological challenges (Prior Reference Rahn, Aldrich, Borgida and Sullivan2013). Among these are, as we expound below, (1) a particular vulnerability to error-prone self-reports, (2) the difficulty of distinguishing motivated selective exposure from selective exposure that is a by-product of social milieu (Sears and Freedman Reference Stroud1967), and (3) the possibility that citizens’ behavior could be affected by studies that interrupt the naturalism of everyday exposure decisions. Moreover, substantial variance in the size of effects that have been identified suggest that selective exposure will never boil down to a simple “yes or no” account. Rather, exposure choices seem to depend on a rich set of circumstantial factors (Hart et al. Reference Hetherington2009, for a review).
We present results that contribute to the discussion in several ways. First, following other recent research in this area, our study eschews a simple “yes or no” perspective on selective exposure. We deliberately set out to take account of conditional preferences for attitude-consistent information. Second, although our study is interested in when a preference for attitude-consistent information arises, we begin to examine dimensions other than attitude-consistency that guide exposure decisions on the idea (which we develop below) that past work has focused too narrowly on whether or not information is congenial. Third, we employ a unique design—an online field experiment using advertisements administered on the social networking website Facebook—that allows exposure decisions (gauged by choices about which ads to click) to be examined with uncommon naturalism while nevertheless retaining the benefits of random assignment.
To preview our results, we identify attributes of electoral message content that make a preference for attitude-consistent information more and less likely. When it comes to a policy-focused message, attitude-consistent information is only slightly more likely to attract citizens than attitude-inconsistent news. On the other hand, when it comes to an embarrassing personal failing (a gaffe by the opposing party’s candidate), attitude-consistent news attracts much more attention than attitude-inconsistent news, a result we interpret as consistent with work highlighting the political significance of schadenfreude (Combs et al. Reference Combs, Powell, Schurtz and Smith2009). Moreover, we find that partisans and nonpartisans alike are more likely to click to learn about a candidate’s personal failing than damaging economic news, evidence that highlights a psychological basis for a lamented media emphasis on candidate gaffes (Patterson Reference Patterson and McClure1994).
We begin with a brief review of the selective exposure literature keyed to highlight psychological motivations that guide exposure decisions in a political context. From there, we describe how the design we employ complements several weaknesses of past studies before presenting our results.
A Preference for Attitude-Consistent Information?
There are several reasons why citizens might exhibit a preference for attitude-consistent information. Most prominently, cognitive dissonance theories suggest that there is a motivation for consistency in mental representations (Festinger Reference Fiorina1957; Harmon-Jones and Mills Reference Healy and Lenz1999). Perhaps because inconsistent representations generate a threat to one’s self-concept (Steele Reference Stroud and Muddiman1988), or because it is unpalatable to realize oneself is incorrect (Kiesler Reference Kinder1971), people may be averse to information that they expect to challenge an existing attitude. In a somewhat different vein, many individuals seem to have a need for cognitive closure, part of which manifests as a desire to maintain beliefs as long as possible (e.g., Kruglanski Reference Kurzban and Athena Aktipis1996). This motivation may cause them to avoid information that challenges their current views. Recent work grounds such findings in an evolutionary framework by positing the existence of mental programs that function as a sort of “press secretary,” seeking information that might be useful for persuasion (Kurzban and Aktipis Reference Lau and Redlawsk2007; Mercier and Sperber Reference Mutz and Reeves2011).
In the study of politics, this attitude-consistency hypothesis has a rugged intellectual history. After the initial bout of enthusiasm bolstered by dissonance theory, it spent years in scholarly limbo following an unnerving review that critiqued the evidentiary basis of existing studies (Sears and Freedman Reference Stroud1967; also see Stroud Reference Taber and Lodge2011, 22–5, for a discussion). Research on the topic dwindled and evidence that citizens prefer congenial information was taken to be so weak that John Zaller’s model of opinion formation simply assumed it away: “To the extent selective exposure occurs at all, it appears to do so under special conditions that do not typically arise in situations of mass persuasion” (Zaller Reference Zaller1992, 139).Footnote 1 Until quite recently, weaknesses in the data marshaled in favor of selective exposure have warranted considerable circumspection, leading researchers to deem the evidence “uncertain” (Taber and Lodge Reference van Dijk, Ouwerkerk, Goslinga and Nieweg2006, 756), “equivocal at best” (Iyengar et al. Reference Katz and Lazarsfeld2008, 187), or “thin” (Kinder Reference Kruglanski and Webster2003, 369).
Recently, innovative approaches have addressed many of weaknesses that hindered earlier studies and, we think, leave little doubt that an attitude-consistency bias arises, at least when the information is not crucial for completing a task (see Hart et al. Reference Hetherington2009, for a review). To highlight just a few compelling studies, Redlawsk (Reference Reinemann, Stanyer, Scherr and Legnante2002) presents subjects with an information board containing details about fictional candidates and finds that subjects access information about liked candidates about twice as often as they access information about disliked candidates. Also using an information board, Taber and Lodge (Reference van Dijk, Ouwerkerk, Goslinga and Nieweg2006) find that each of eight groups examined sought more attitude-confirming pieces of information than attitude-disconfirming ones. Valentino et al. invited subjects to view George W. Bush’s and John Kerry’s campaign websites and found subjects were unlikely to view the website of the candidate they disliked (Valentino et al. Reference van Dijk, Ouwerkerk and Goslinga2009, figure 3). Iyengar et al. (Reference Bennett and Iyengar2008) mailed a representative sample of Americans CDs that contained campaign information and tracked recipients’ usage. Both Republicans and Democrats exhibited preference for congenial information, and for Republicans, the difference was statistically significant. Similarly, Iyengar and Hahn (Reference Iyengar, Sood and Lelkes2009) presented laboratory subjects with a menu of news headlines and randomly assigned what news agency they came from. Democrats and liberals exhibited a preference for CNN and NPR, while Republicans and conservatives exhibited a preference for Fox News. Laboratory findings are consistent with recent observational evidence: for example, most Americans maintain social networks with highly homogenous political views (Mutz Reference Neuman, Just and Crigler2006), the vast majority of those who turn to blogs for political information regularly consult only those blogs that share their ideological and partisan affinities (Lawrence, Sides and Farrell Reference Lippmann2010), and broadly accessible events such as the party nominating conventions exhibit marked partisan biases in their viewership (Stroud Reference Taber and Lodge2011).
On the other hand, attitude-consistency motivations must have limits. In the extreme, they would almost negate the possibility of persuasion, as any attitude-challenging information will be avoided—but persuasion sometimes occurs. In the terminology of social psychology, attitude-consistency motivations compete with accuracy motivations. People have an interest in making correct decisions, and sometimes uncongenial information is crucial to doing so (Kunda Reference Krupnikov1990, 481–2). Thus, in keeping with the idea that individuals reflexively weigh the costs and benefits of acquiring information, the bias in favor of congenial information seems to be lower when the information is instrumental to a goal (Hart et al. Reference Hetherington2009).
A political context provides little guidance with respect to whether accuracy or congeniality motivations should prevail. On one hand, citizens have a stake in good (however they define this word) political outcomes, and political information can be diagnostic with respect to, for instance, which candidate to support or how to vote on a particular ballot initiative. On the other hand, many citizens possess strong emotional affinities for particular parties and are psychologically invested in seeing “their side” prevail; thus elections are opportunities to reap expressive benefits from partisan “cheerleading” (e.g., Campbell et al. Reference Campbell, Converse, Miller and Stokes1960; Green, Palmquist and Schickler Reference Greene2002; Gerber and Huber Reference Graber2010; Bullock et al. Reference Bullock, Gerber, Hill and Huber2013). It is clear that both motives shape political behavior, but there is much to learn about how they interact and when each applies.Footnote 2
A Preference for Personalities Over Policies?
The attitude-consistency hypothesis is focal in selective exposure research. It is easy to see why: the notion that citizens systematically avoid challenging information clearly has many implications for democratic governance, tolerance, and participation (Mutz Reference Neuman, Just and Crigler2006). But citizens’ exposure choices could be biased along dimensions other than attitude-consistency. Prior (Reference Prior2007), for instance, accumulates impressive evidence that the taste for political information in general matters: the growth of high-choice media leads politically uninterested individuals to select out of political coverage, with concomitant decreases in participation (Arceneaux and Johnson Reference Arceneaux and Johnson2013). One can imagine many different properties that could influence exposure decisions: geographic proximity or relevance (are Americans more likely to read about foreign news when links to the US are made clear?), satirical versus serious coverage (Stroud and Muddiman Reference Valentino, Banks, Hutchings and Davis2013), familiarity of the source, novelty of the news, or threatening content (Gadarian and Albertson Reference Gerber and Huber2014).
Here, we consider the relative attractiveness of two types of content in media coverage of elections: policy-focused versus person-focused messages. By policy-focused messages, we mean facts, statistics, and revelations about current issues that might be diagnostic of the past or future effectiveness of a candidate’s policies. By person-focused messages, we mean information on candidates’ personal attributes, especially as reflected in one-off events such as rhetorical triumphs or embarrassing gaffes.
We acknowledge that our distinction between policy- and person-focused messages will not neatly categorize all media coverage.Footnote 3 In particular, some aspects of candidates’ personalities are relevant to performance in office, such as if voters perceive a candidate to have a temperament that makes him or her ill-suited for the job. Still, much contemporary analysis and criticism of how the media covers politics has contrasted sober and substantive issue content with news focused on game aspects of politics—personalities, polls, and campaign hoopla (Patterson Reference Patterson and McClure1994; Cappella and Jamieson Reference Cappella and Jamieson1997). From early on, observers worried about the impact of television on politics have fretted that “personal image” takes precedence over policy substance (Patterson and McClure Reference Popkin1976). This is the basic relationship we seek to test.
There is good reason to believe that citizens, on average, prefer information about personalities to information about policies. Campaigns and elections inundate voters with information—the volume and sophistication (e.g., budgetary implications of policies, economic figures) of which is more than most people are equipped or motivated to assimilate. One way voters can cope with the overload is by focusing on candidate characteristics: what sort of people—intelligent, compassionate, competent—are the candidates? Who would be more reliable in a time of crisis? Who is more charismatic? Voters often formulate opinions on such questions, even if their policy understanding is lacking, as they offer a familiar and manageable way to distill the deluge of information (Kinder Reference Kinder and Roderick Kiewiet1986; Rahn et al. Reference Redlawsk1990; Aldrich, Gronke and Grynaviski Reference Aldrich, Gronke and Grynaviski1999). For example, when candidates commit “gaffes”—mangling or misspelling words, slipping and falling, seeming out of touch or unfamiliar with an everyday practice, referring to people in an offensive manner—voters may draw inferences about their intellect, motivations, trustworthiness, or ability to understand the problems of others (Popkin Reference Portmann1994).
Nonetheless, a preference for personality over policy content at election time is hardly a foregone conclusion. Although political scientists have long debated the extent and sophistication of policy voting in the American electorate, there has been much stronger consensus that (at the very least) voters evaluate the performance of politicians in office, relying especially on judgments of economic performance (Fiorina Reference Freedman1981; Kinder and Kiewiet Reference Kunda1981; Hibbs Reference Iyengar and Hahn2000; Healy and Lenz Reference Hibbs2013). Especially in times of economic woe, figures on unemployment and job creation from the Bureau of Labor Statistics are a recurring object of media scrutiny. Moreover, compared with learning personal details about the candidates, citizens may find it much easier to develop an impression of the state of the economy directly from their own experiences and those of their peers, unmediated by news organizations.Footnote 4
The relative preference of voters for personalities over policy holds substantive importance. Past work documents—and laments—the media’s hyper-focus on candidate missteps, as this focus might generate animosity and detract from constructive discourse about the issues (Patterson Reference Patterson and McClure1994). It is not known, however, whether the media’s focus on candidate gaffes and the like arises from audience preferences, or some other market pressure, such as pressures to appear objective.Footnote 5 We shall have more to say on this matter in the conclusion.
Schadenfreude
Hearkening back to our earlier discussion of attitude-consistency biases, taking account of person versus policy focus can in turn illuminate conditions that make a preference for attitude-consistent information more or less likely. Specifically, we conjecture that a person-focused message may be especially likely to activate partisan selective exposure. Our thinking draws from research into the phenomenon popularly known as schadenfreude: the idea that, when misfortune befalls people we dislike or members of rival groups, it generates a positive emotional payoff (e.g., Portmann Reference Prior2000; Combs et al. Reference Combs, Powell, Schurtz and Smith2009).
Why do the findings concerning schadenfreude suggest that attitude-consistency biases would be more prominent for a (negative) person-focused message than a (negative) policy-focused message? We see two potential reasons. First, there is considerable evidence that pleasurable feelings are amplified when the disliked individual is responsible for the misfortune (Feather and Sherman Reference Fiorina2002; van Dijk et al. Reference Epstein2005; van Dijk, Ouwerkerk and Goslinga Reference Feather and Sherman2009), and this characteristic is more prominent for person-focused messages (such as surround a campaign gaffe) than policy-focused messages. Second, person-focused messages, such as a campaign gaffe, are localized, typically having ramifications for the person in question, but not society at large. In contrast, policy-focused messages (e.g., discouraging news about the economy) might generate positive feelings insofar as a disliked person’s policies are failing, but these could be countervailed by negative feelings about national prosperity. Consistent with these conjectures, Democrats reported considerably more pleasure upon reading about George W. Bush falling off his bicycle than about economic news that reflects poorly on his policies (Combs et al. Reference Combs, Powell, Schurtz and Smith2009). It is not known whether the anticipation of such a payoff would guide prior exposure decisions, but the idea seems plausible and is consistent with our sense that moments of political embarrassment are met by political opponents with particular relish.
Hypotheses
The preceding discussion leads us to several hypotheses, some of which are offered as competitors. First, if people find information on candidates as people to be a more appealing way to distill political news than information about policy differences and details, then they should find an advertisement that focuses on a candidate gaffe more interesting and enticing than news linking the candidate’s policies to economic performance. In terms of behavior, citizens will be more likely to pursue additional information (i.e., click) when an internet ad focuses on a gaffe, than when it focuses on economic performance (H1a, the Gaffe Hypothesis). Alternatively, citizens might perceive gaffes as haphazard events that provide little useful information for reaching a political decision. This might especially be true compared with news about the success or failure of economic policies, a factor which seems to weigh heavily on voters’ decisions. If this inclination predominates, we would expect the promise of policy-focused news to generate greater interest, and therefore a higher probability of clicking (H1b, the Policy Hypothesis).
Next, we consider how citizens’ partisan attitudes might bear on their exposure decisions. As we note above, people often seek to avoid attitude-inconsistent thoughts (because these create dissonance) and to avoid revisiting a decision once it has been made (because doing so removes cognitive closure). When these motivations prevail, citizens might avoid exposing themselves to information that implies their preferred candidate made an error—they will be relatively more likely to click on an ad that promises to reveal bad news for the opposing candidate (H2a, the Congeniality Hypothesis). But as we also note, politics might evoke accuracy motivations. If accuracy motivations prevail, then citizens might be even-handed in their information search, finding information equally appealing irrespective of which candidate is harmed—or they might prefer uncongenial information.Footnote 6 We label this latter possibility the Uncongeniality Hypothesis (H2b).
Finally, the discussion of schadenfreude suggests that citizens might anticipate a particularly large payoff of positive feelings when they think they have encountered news that is damaging to a disliked individual (or member of a rival group) and the individual is personally responsible for his or her own misfortune. This suggests advertisements will entice the most clicking when they are simultaneously congenial and gaffe-focused. Thus, the Schadenfreude Hypothesis (H3) predicts that citizens are particularly likely to click on ads that promise to reveal a gaffe committed by a candidate from the opposing party.
Methods
As we noted at the outset, selective exposure research has grappled with measurement difficulties. Sears and Freedman’s (Reference Stroud1967) landmark review chilled the research enterprise for decades by pointing out just how daunting the challenges surrounding self-reports of information exposure are. There are the familiar problems of subjects misremembering what they read or exaggerating their sophistication and balance. But on top of these, there is the problem that so much of what citizens see and read is a by-product of their environment and other unrelated choices (Fiorina Reference Gadarian and Albertson1990). A liberal who works amidst conservatives often benefits from hearing perspectives that would otherwise have been missed, and vice versa. Likewise, the prominence of particular topical interests and other facets of one’s social milieu will cause many people to stumble onto certain kinds of information in spite of themselves. Where there is a strong correspondence between the partisanship of the audience and the source, it could be due to selective exposure, persuasion, or it may not reflect a direct causal relationship at all (e.g., a conservative community is likely to give rise to conservative local media and a conservative readership, even if neither influences the other). As Prior states the difficulty, “empirical analysis is severely hampered by a seemingly simple problem: we do not know how many and what kind of people are exposed to which messages” (Reference Rahn, Aldrich, Borgida and Sullivan2013, 102).
One remedy to these difficulties is for researchers to exert control over what information is available and monitor exposure decisions in the context of this constructed environment. Controlling the “menu” of information available addresses the problem of de facto selective exposure (Iyengar et al. Reference Katz and Lazarsfeld2008; Valentino et al. Reference van Dijk, Ouwerkerk and Goslinga2009), but the approach has a liability of its own. In these approaches, subjects typically have reason to believe that their behavior is being monitored. As Stroud (Reference Taber and Lodge2011, 25) notes, this awareness could increase accuracy motivations and thus dampen selective exposure, leading to a systematic underestimation of biases.Footnote 7 In fact, some studies intentionally boost this motivation. Taber and Lodge (Reference van Dijk, Ouwerkerk, Goslinga and Nieweg2006, 759) explicitly tell subjects to view information “in an even-handed way.” This approach is commendable in the spirit of epistemological conservatism—it handicaps the researchers’ hypothesis that a bias exists—but overall it could lead us systematically to underestimate the real degree of selective exposure.
We employ an approach that provides an unusual degree of naturalism while leveraging the benefits of random assignment and avoiding the pitfalls of self-reports. Following Ryan (Reference Ryan and Broockman2012; see also Broockman and Green Reference Broockman and Green2014) we conduct a field experiment in which internet users located in the United States are presented with randomly assigned political advertisements in the course of their routine web browsing. We do so by placing the ads on the social media website Facebook. Facebook has several benefits as a site for the research. It is a large and heterogeneous user community (148 million US adults as we write, or about 62 percent of the US adult population).Footnote 8 It allows the easy and cost-efficient purchase of a large number of advertisements. It allows advertisements to be targeted based on information users provide in their profiles, including information that conveys their political leanings. And key for the present study, it allows us to examine advertisement click-through rates—the proportion of advertisements displayed that were actually clicked—as a dependent measure. As such, we are able in a sense to extend the granular information search measures developed by Lau and Redlawsk (Reference Lazarsfeld, Berelson and Gaudet2001) for a laboratory context to a fully naturalistic setting in which subjects do not even know their behavior is being monitored.
Our study was conducted among users in the United States from October 18 through November 7, 2012, as the US presidential election was at peak intensity. The study presented Facebook users with ads that insinuated the revelation of information damaging to one of the presidential candidates, and invited them to click to find out more information.Footnote 9 As shown in Figure 1, there was a 2×2 manipulation of which candidate was supposedly damaged (Mitt Romney or Barack Obama) and whether there was a policy focus (“Learn how the latest economic numbers hurt [candidate]”) or a person focus (“Learn about [candidate’s] latest embarrassing gaffe on the economy”).
Experiment stimuli
We consider the Romney/Obama manipulation to be face valid. As we foreshadow above, the second manipulation maps onto a distinction between the facets of campaigning that have a clear link to public policy, versus those that focus on personal image while being a step removed from policy (Patterson and McClure Reference Popkin1976). We examine this distinction by creating exemplars of these broader categories. The specific instrumentation was inspired by events we observed in 2012. Throughout the campaign, various happenings were presented—not without reason—as being diagnostic of the quality of the candidates’ policy positions. Examples include fluctuations in the price of gas, changing indices of economic inequality, improvements or declines in students’ standardized test scores, statistics on the prominence of crime, and perhaps most prominent, monthly reports from the Bureau of Labor Statistics on the rising or falling unemployment rate. Candidates routinely reference facts like these in advertisements, debates, and interviews, and many perspectives would consider them among the most estimable criteria for citizens to use in forming their opinions (Lippmann 1955). Juxtaposed against this sort of information are one-off events that reflect well or poorly on a candidate, but in a shallower way. Recent politics offers no shortage of these: Mitt Romney’s awkward “binders full of women” moment during the second presidential debate; Barack Obama’s assurance (amidst a languishing economy) that the “private sector is doing fine;” Marco Rubio’s ill-timed need for a drink of water during his response to President Obama’s State of the Union address. Citizens’ appetite for learning about these different types of revelations is what we seek to test. We conducted a manipulation check on an external sample to verify that the policy-relevant advertisement increased subjects’ focus on “public policy,” “politically relevant facts and figures,” and “the state of the national economy” without appearing more novel (Supporting Information, Table A1).Footnote 10
All four ads were presented to both liberals and conservatives in equal degree. To do so, we took advantage of Facebook’s ability to target advertisements based on users’ interests. We targeted ads at US users whom Facebook classifies as having either a liberal or a conservative affiliation.Footnote 11 Because some individuals in each ideological group saw ads attacking each candidate, the resulting configurations can be coded as attitude-consistent (e.g., a liberal user seeing an ad attacking Mitt Romney) or inconsistent (e.g., a liberal user seeing an ad attacking Barack Obama).
We are also interested in the responses of non-ideologues, as they present a useful point of comparison: What does exposure look like when information pushes neither for nor against existing partisan attitudes? Non-ideologues are also useful in assessing the extent to which our findings pertain only to individuals who have exhibited some interest in politics. The capabilities of Facebook’s advertising engine limited how completely we could isolate non-ideologues. We could not target non-ideologues directly, as there was no targeting category for them.Footnote 12 We could not exclude ideologues from being targeted, as the software does not allow exclusion of specific demographics. In lieu of these preferable options, we administered our advertisements to users who registered an interest in sports or music. We chose these demographics to represent a non-ideological category after investigating several alternatives, based on three criteria. First, the sports and music categories are nonpolitical in nature. Second, they contain a relatively low proportion of ideologues.Footnote 13 Third, to the extent they do overlap with the ideological categories, they overlap with liberals and conservatives in roughly equal degree. But we wish to emphasize that the “non-ideological” grouping we analyze below is best thought of as considerably less ideological than the comparison points, but one that still contains some internal heterogeneity.
Although an individual-level randomization would of course be desirable, it is not possible in this sort of study. The reason is that the advertisements that appear on a user’s screen when he or she loads a page are determined by an instant auction in which advertisers automatically bid (up to a maximum) for the opportunity to appear. The outcome of such auctions likely have some stochastic properties, but are not truly random. Instead, following Ryan (Reference Ryan and Broockman2012), our approach is to conduct a large number (hundreds) of miniature campaigns, randomly assign one ad version to each of them, and examine campaign performance (as measured by the click-through rate in each). Thus, campaigns, rather than individual people or ad displays, are the unit of analysis in the results we report below.
How does one conduct a valid random assignment when campaigns are the unit of analysis? Similar to Ryan (Reference Ryan and Broockman2012), we use Facebook’s targeting capabilities to partition Facebook’s American user base into clusters defined by arbitrary demographic characteristics, such as age, gender, and where they live. Then, we randomly assign each cluster to be the target of a campaign featuring just one of our four advertisements. By using clusters that are mutually exclusive, we can be certain that any given person will see only one type of ad. Because the partitioning occurs before the random assignment, any particular cluster (e.g., 34-year-old married females living in California) has an equal ex ante probability of being assigned to each of the four conditions. Because the demographic information used to partition the clusters is retained, it can be used to conduct subgroup analyses or as controls in multivariate regression. Ryan and Broockman (Reference Sabato2012) provide additional practical details on how to implement this randomization procedure, as well as an illustrative example.
In the present study, we partition each ideological group into clusters based on discrete ages, gender, geography (where users live), and relationship status.Footnote 14 We create 282 clusters within each of the three ideological groups, for 846 clusters overall. We block our randomization (cf. Gerber and Green Reference Graber2013, 71–9) to ensure that treatment conditions will be balanced in terms of age, gender, geography, and relationship status.
Clicking the ad caused a subject’s computer to load a webpage (designed by the authors) with an array of links congruent with the advertisement (e.g., a link to GOP.com for ads attacking Barack Obama, a link to Democrats.org for ads attacking Mitt Romney).
Results
Overall, our ads were displayed 14,017,732 times and received 1444 clicks, a click-through rate of 0.0103 percent. This low click-through rate is typical for online advertisements, which succeed by force of number and low cost. Table 1 gives a sense of the data, reporting the total number of impressions (i.e., displays of the ad) and clicks, pooling across all clusters randomly assigned to each experiment condition.
Impressions and Clicks, by Condition
Some details require explanation before we can move to the main tests of our hypotheses, namely a campaign- (or cluster-) level analysis. In a study of this sort, the number of impressions is different for each cluster. This occurs for two reasons. First, the population in some clusters is smaller than in others; consider that 64-year olds are about four times more scarce on Facebook than 18-year olds. Second, some targeting combinations are more coveted by competing advertisers than others, making it more difficult for the ads we purchased to win the automated auctions. As a result, the number of ad displays varies widely, from just 384 in one extreme to 362,102 in the other.Footnote 15 To make the observations comparable with each other, we use each cluster’s click-through rate—simply clicks divided by impressions—as our primary dependent measure. This choice leaves a concern about heteroskedasticity: the clicking rate is measured more precisely to the extent the number of impressions is large. Happily, we can incorporate our knowledge of the differing variance by using weighted least squares (WLS) to correct standard errors (Greene Reference Groenendyk2008, 167–9). In the multivariate analyses below, we weight each observation by the number of impressions that cluster received. In addition, we use heteroskedasticity-consistent standard errors throughout.
Further, we face a choice concerning how to estimate out statistical models. Many of our campaigns received no clicks at all, although we are confident that if we had the resources to run them for longer, they eventually would. We could approach this as a censoring problem, specifying 0 as the mass point and estimating Tobit models. In such an approach, parameter estimates would be consistent, but they would only be interpretable as linear relationships to an underlying latent variable. Alternatively, we can estimate WLS models. WLS parameter estimates are biased in the face of a censored dependent variable, but the bias is typically toward 0, which would make our hypothesis tests more conservative (Greene Reference Groeling1981; Chay and Powell Reference Chay and Powell2001). WLS results also have the virtue of being easier to interpret. We opt to report WLS models here, but estimate all models by Tobit and report the results in the online Supporting Information. Results are consistent across the two approaches.
Finally, to make results easier to interpret, we multiply the dependent variable (i.e., the click-through rate in each cluster) by 10,000, reducing the clutter of leading zeroes. Thus, our first model—assessing the effect of the gaffe manipulation—is
$$\left( {{\rm Clicks}/{\rm Impressions}} \right){\times}10,000\,{=}\,\beta _{0} {\plus}\beta _{1} \,{\rm Gaffe}\,{\rm dummy}{\plus}{\epsilon}.$$
The first column of Table 2 shows the results. They suggest that, where clusters that were exposed to policy-focused ads had an average clicking rate of 0.00909 percent (the constant term), clusters exposed to gaffe ads had a clicking rate of 0.01144 percent. This might seem like a minor difference, given the low baseline. But it is statistically significant and, more important, big enough to matter for the success of a campaign. For instance, the effect of 0.235 implies that the gaffe advertisement would increase the number of clicks garnered over a $1000 campaign from 1272 to 1601, lowering the cost per click from $0.78 to $0.62.Footnote 16
Responses to Gaffe Manipulation
Weighted least squares models. Dependent variable=cluster click-through rate×10,000. Robust standard errors in parentheses.
***p<0.01, **p<0.05, *p<0.1.
The results in Table 2 are relevant to our hypotheses. Specifically, the significant increase in clicking caused by the gaffe-focused advertisement is inconsistent with the Policy Hypothesis (H1b), but corroborates the Gaffe Hypothesis (H1a). We probe this result in several ways. First, we present a model that uses our available covariates.Footnote 17 We find that older Facebook users are more likely to click advertisements, and that the estimated treatment effect increases slightly. Second, we repeat the analysis for our three ideological subgroups: liberals, conservatives, and non-ideologues, looking for texture in whether one ideological group responds more strongly to the treatment than others. The treatment effects are of a comparable magnitude in each subgroup, suggesting that the gaffe manipulation resonates with each category of user in roughly equal degree.
To assess the hypotheses related to attitude-consistency, we conduct a similar analysis focused on the congeniality manipulation. In Table 3, the same dependent variable is regressed on an indicator for assignment to a congenial ad. The ads are neither congenial nor uncongenial as applied to non-ideologues, so they are not included in this analysis. Working toward a substantive point, we look at the results in two stages. First, we consider the Congeniality Hypothesis within the context of subjects who saw an ad regarding the latest economic news. Second, we examine the results in the other group—people who saw a gaffe-focused ad. Of course, the same data could be pooled and analyzed in an interactive model, which we provide in the Supporting Information (Table A4).
A Conditional Preference for Attitude-Consistent Information
Weighted least squares models. Dependent variable=cluster click-through rate×10,000. Robust standard errors in parentheses.
***p<0.01, **p<0.05, *p<0.1.
The left side of Table 3 supports neither the H2a nor H2b, as the coefficients are not distinguishable from 0. As is always the case, we cannot rule out the possibility of an effect that is genuine, but too small to detect. Still, a power analysis suggests that even within this reduced sample, we are well equipped to detect effect sizes of the magnitude seen in Table 2. We thus interpret this as evidence that the effect of the congeniality manipulation is near 0; where messages emphasizing the performance of the economy are concerned, neither liberals nor conservatives exhibit a systematic preference either for ads that attack their disliked candidate nor for ads that attack their preferred candidate.
The story on the right side of Table 3 is rather different. We see that, among subjects assigned to the gaffe condition, the congeniality treatment has a pronounced positive effect, significant in each subgroup. The effect here is about three times as large as the main effect seen in Table 2; it implies a 71 percent increase in clicking when exposed to an attitude-consistent ad. As before, there is little evidence of a differential response by liberals and conservatives: ideologues of both stripes are more likely to click ads attacking the opposition candidate for an embarrassing remark. This leaves us with strong evidence against the idea that citizens are receptive to adjusting their pre-existing views by pursuing attitude-inconsistent information (H2b).
The results in Table 3 belie a simple understanding of congeniality and selective exposure, as effects are small or 0 among subjects in the Policy condition, but convincingly positive for subjects in the gaffe condition. What of the Schadenfreude Hypothesis, which predicted an interaction between the candidate manipulation and the gaffe manipulation? To assess it, we estimate
$$\matrix{ {\left( {{\rm Clicks}/{\rm Impressions}} \right){\times}10,000\,{=}\,\beta _{0} {\plus}\beta _{1} \,{\rm Gaffe}{\plus}\beta _{2} \,{\rm Congeniality}{\plus}} \hfill \cr \qquad \qquad \qquad \qquad \qquad \quad \qquad {\beta _{3} \,{\rm Gaffe}{\times}{\rm Congeniality}{\plus}\beta _{4} \,{\rm Female}{\plus}} \hfill \cr \qquad \qquad \qquad \qquad \qquad \qquad \quad {\beta _{5} \,{\rm Geography}{\plus}\beta _{6} \,{\rm Relationship}{\plus}\beta _{7} \,{\rm Age}{\plus}{\epsilon},} \hfill \cr } $$
among the partisans in our sample. As reported in the Supporting Information, the interaction term is marginally significant (β 3=0.434, SE=0.254, p<0.09). As H3 is directional—it implies that β 3 should be positive—we can apply a one-tailed test and reject the null with reasonable confidence (p<0.05). Figure 2 conveys the relevant results graphically by plotting the predicted click-through percentages, by condition, with control variables held at their means. Here again we see evidence that congeniality elicits clicking, but that the combination of gaffe-focus and congeniality is especially effective.
Predicted click-through rates, based on interactive model. Note: whiskers are ±1 SEE.
We sum up by reviewing how the evidence we present relates to our hypotheses. Citizens do not seem to seek out policy-related information that might make vote choices more responsive to policy successes and failures. Nor do they exhibit much appetite for information that might challenge their existing attitudes. Instead, person-focused revelations (gaffes) and attitude-consistent information pique the most interest. According to the Schadenfreude Hypothesis, partisans seem especially likely to seek out personally embarrassing information about the opposing side.
Can we conclude that the effects predicted by the Gaffe Hypothesis (H1a) are general, rather than limited to the special case of an interaction with the congeniality manipulation? We think so. As Figure 2 reflects, the estimated effect of the gaffe manipulation among individuals who saw an uncongenial ad is positive (but not significant: p<0.22, two-tailed). More important is the significant positive effect of the gaffe manipulation among non-ideologues (see Model 5 in Table 2). These results suggest that, even among a group weak in ideological leanings, and across ads equally likely to attack Mitt Romney or Barack Obama (the candidate manipulation is orthogonal here), the gaffe-focused ad evokes significantly more clicks. (Though it is also worth noting that the magnitude of the effect is considerably smaller than among partisans.)
Conclusion
The choices citizens make about what to read, watch, and listen to bear on how they interact with each other and the broader political system. We present data that move the discussion forward in several productive ways.
First, we bring to this area of study a new research design with distinct advantages. We combine randomized control over what subjects see with the ability passively and naturalistically to observe their behavior in a powerful combination. Where self-reports have difficulty separating motivations from context and laboratory studies could activate motivations for balance and accuracy, we think our design results in an especially pure view of exposure motivations in the real world. The online setting of our study also enhances the contemporary relevance of our results. Especially for younger generations, the internet is the new default for learning about nearly every topic, including politics.
We find strong evidence that belies a simple understanding of partisan selective exposure. The preference for congenial information is sometimes stark, but at other times fades to a whisper—a result that may help explain why conclusive results with respect to the Congeniality Hypothesis have proven so elusive. Moreover, we find that exposure decisions are markedly influenced by factors that have nothing to do with congeniality—a result that bolsters Prior’s (Reference Prior2007) entreaty to expand the scope of selective exposure research.
Substantively, our results contribute to the study of media institutions and how they shape citizens’ views of government. In Out of Order, Patterson (Reference Patterson and McClure1994) documents an increase in the media’s focus on candidate gaffes and campaign controversies.Footnote 18 The attention lavished on significant missteps, such as Mitt Romney’s “binders full of women” remark, or Barack Obama’s lackluster performance in the first 2012 presidential debate, remind us that these episodes remain focal in campaigns. This evolution is regrettable, as viewers learn little of substance (about the choice they face) from such coverage and it leads to frustration and disaffection with the political process.
Blame for the focus on gaffes, by Patterson’s account, lies at the feet of media: citizens desire policy-focused coverage, but the media focuses on candidate events because they are event-driven rather than analytical and more easy to cover in a detached, neutral style (Patterson Reference Patterson and McClure1994, 149). In contrast, our results suggest that at least some of the fault is not in our media, but in ourselves. Perhaps the media focuses on missteps not because doing so is easy, nor because it allows the media to avoid the perception of bias, but rather because over time, they have learned that when citizens can choose from a range of topics, the gaffe is what sells.
Finally, we note some weaknesses in the present research as well as opportunities for future studies. First, the results above are limited to the Facebook social media network. Facebook is attractive because its targeting capabilities facilitate a randomized study such as ours and because of its size and heterogeneity. The style of the website, however, could bear on the results we see. Facebook has a reputation as a hub in which users fill idle moments and procrastinate. As such, its users could be in a mindset more friendly to distraction, especially from an advertisement that is novel. On the other hand, this conforms to the way a great number of citizens are thought to come across political information—as an incidental by-product of other, more primary pursuits (Fiorina Reference Gadarian and Albertson1990), and we might expect political ads of all stripes would attract greater attention on sites where people are actively seeking political content (e.g., news websites, political blogs). Future studies could test the extent to which context bears on our results by conducting studies through other online forums.
Second, as we note above, our study took place just before a presidential election, when candidate coverage and the general salience of political topics were at peak intensity. It is possible that the relationships we identify would be more subtle if examined at the midpoint of an election cycle. This said, we can think of reasons that both congeniality and accuracy motivations would be less prominent. Partisan identities might retreat into the background when candidates are not clashing on a daily basis. Likewise, information might be seen as less useful when a decision point is distant. Thus, we do not think the timing of our study stacks the deck too much in favor of one possibility or the other.Footnote 19
Third, learning is a multi-step process that begins with exposure, but probably also requires some degree of concentration, elaboration, and solidification (Petty and Cacioppo Reference Sears and Freedman2011). We have presented evidence that bears only on the first step, and in a fairly sparse way at that. Future studies should consider more elaborate ways to measure exposure, such as by presenting advertisement clickers with a menu of information and observing exposure behavior after the first click.Footnote 20 There is also the possibility of combining the experimental design we use here with survey methods to gauge how much advertisements affect opinion change and candidate recognition (cf. Broockman and Green Reference Broockman and Green2014). Nonetheless, our focus here has been on that crucial first step, asking what shapes citizen choices to expose themselves to election messages.
Fourth, we have focused on one dimension (other than congeniality) that could affect exposure decisions: the emphasis on candidates as people (and specifically, gaffes) versus policy-related (in this case, economic implications) information about the candidates. As we noted earlier, there are many additional dimensions worthy of exploration. Even with respect to the Congeniality Hypothesis, our focus here was entirely on attacks, as opposed to information that might discuss positive attributes of one’s own candidate or the opponent. Given evidence that positive and negative information are processed differently (Krupnikov Reference Lawrence, Sides and Farrell2011, 798), the dynamics of positive congenial information could easily merit separate tests.
Finally, we note that our study examines the spontaneous aspect of selective exposure—decisions that occur as citizens go about their routine social and entertainment activities. These decisions might be different from those that occur when scanning the news for politics deliberately, or decided which websites to bookmark for habitual reading. This design attribute limits how grandly we can draw conclusions from our results, but also comes with a distinct advantage: the information age is making citizens accustomed to accessing any kind of information they want with a few keystrokes, and a substantial proportion of learning might occur through precisely such spontaneous exposure decisions.
Research on media effects has always suggested that “who learns what?” results from a complex interaction between what information is available, and choices about whether and how to consume it. Having an abundance of information at our fingertips certainly has the potential to empower, as almost anyone who is motivated can accumulate a degree of expertise that once would have been considered exceptional. At the same time, the results we report here suggest a troubling by-product comes with the ability to affirm one’s preconceptions “on demand.”
Supplementary Material
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/psrm.2015.62
