Over the last decade, scholars of political opinion formation and voting have debated whether events irrelevant to politics influence attitudes and behaviors. Two noteworthy studies, that use observational data, report that college sport victories and shark attacks can influence incumbents’ vote shares (Achen and Bartels, Reference Achen and Bartels2016; Healy et al., Reference Healy, Neil and Mo2010). Others contest these two results, claiming that they are false positives (Fowler and Montagnes, Reference Fowler and Montagnes2015; Fowler and Hall, n.d.; although see Healy et al., Reference Healy, Malhotra and Mo2015). Numerous experimental studies demonstrate that irrelevant events, such as the weather, sporting events, and random lotteries, can influence political opinions and/or behaviors (e.g., Healy et al., Reference Healy, Neil and Mo2010, study 2; Bagues and Esteve-Volart, Reference Bagues and Esteve-Volart2016; Bassi, Reference Bassi2017; Busby et al., Reference Busby, Druckman and Fredendall2017; Huber et al., Reference Huber, Hill and Lenz2012).
Yet, these experiments have not been replicated on different samples or with different outcomes. We seek to replicate one of these studies (i.e., Busby et al., Reference Busby, Druckman and Fredendall2017). Such replication attempts can indicate if the aggregate literature over-states the presence of irrelevant event effects. This could occur from happenstance if particular conditions likely to generate irrelevant event effects happen to be met in extant studies.Footnote 1 To be clear, this is not a critique of existing papers, which faithfully report careful studies that establish the existence of irrelevant event effects (i.e., researchers did not actively set up studies most likely to produce effects). Rather, replication with a different event, sample, and time is a way to move the literature forward to assess robustness and the conditions under which irrelevant event effects occur.
THE ORIGINAL STUDY
A “politically irrelevant” event is one that occurs outside the control of elected officials (Healy and Malhotra, Reference Healy and Malhotra2013). The event can affect individuals’ moods. Affected individuals then may unknowingly use their moods as information when they evaluate a politician or office holder: a positive (negative) mood leads to a favorable (unfavorable) assessment (Bassi, Reference Bassi2017; Huber et al., Reference Huber, Hill and Lenz2012, 731).
Busby et al. (Reference Busby, Druckman and Fredendall2017) offer evidence of such a process in their study of the 2015 College Football Playoff National Championship game, where The Ohio State University beat the University of Oregon. The authors drew a random sample of students from each school and then randomly assigned them to receive a survey, advertised as a study of the “social, economic, and political attitudes of college students,” either two days before or after the game.
They find that, relative to comparable students surveyed before the game, students from the winning school (OSU) who were surveyed after the game reported a more positive mood, more satisfaction with their school, and more approval of President Obama. Oregon students displayed the opposite dynamics. Thus, the football game, which was beyond political control, ostensibly affected moods and then presidential approval.Footnote 2 The effect on approval, however, dissipated one week later. Busby et al. (Reference Busby, Druckman and Fredendall2017) provide causal evidence of a short-term irrelevant event effect but are far from definitive. The authors conclude (2017, 349) that “caution should be taken in generalizing our results. In some sense, our sample size was two – two schools around one event. . .”
OUR REPLICATION
We utilized the same design and procedure as Busby et al. (Reference Busby, Druckman and Fredendall2017), focusing this time on the 2016 College Football National Championship game. The game took place at 8:30 PM Eastern time on January 11, 2016, and pitted the University of Alabama against Clemson University. Alabama won the game 45–40.Footnote 3 We randomly selected Alabama and Clemson students to participate; we then randomly assigned them to receive a before- or after-game invitation to participate in the survey. There thus, in essence, are four cross-sections of data: the before- and after-groups for Alabama and Clemson. We launched the before-game survey on January 9th, closing it at 5:00 PM Eastern time on January 11th; we launched the after-game survey on January 12th, closing it at 5:00 PM Eastern Time on January 14th. This was the time 1 (T1) survey. We sent one reminder, and launched a time 2 (T2) follow-up eight days later. The T2 follow-up asked those who participated at T1 to complete a survey with near identical items to the initial survey.
Like Busby et al. (Reference Busby, Druckman and Fredendall2017), we measured: presidential approval, satisfaction with one's university, positive and negative mood, evaluation of the economy, and willingness to post on social media.Footnote 4 To explore the boundaries of irrelevant effects, we also asked how favorably respondents viewed Pope Francis and how satisfied they were with their lives. The Pope Francis item tests whether effects extend to notable figures beyond the President, while the life satisfaction item comes from classic mood-as-information studies (e.g., Schwarz and Clore, Reference Schwarz and Clore1983, Reference Schwarz and Clore2003). Finally, we asked respondents how important their university was to their identity since college sport events can influence identity (Cialdini et al., Reference Cialdini, Borden, Thorne, Walker, Freeman and Sloan1976). We provide additional sample, procedural, and question wording details in the Supplementary Appendix.
RESULTS
We assume that, for each school, the before- and after-game survey groups are equivalent, on average (since we randomly assigned participation to the before- or after-game survey). Therefore, any average differences between the groups for each school (we do not compare across school) would indicate an effect of the game. In the Supplementary Appendix, we present demographics and balance checks that suggest comparability of the before- and after- groups.Footnote 5 As mentioned, we invited those who responded at T1 to participate in an (identical) survey eight days later. We use these data to assess whether any effects found at T1 endure (e.g., do other factors/events eliminate the immediate irrelevant event effect?).Footnote 6
We present the results in Table 1, which shows the before- and after-game mean scores (and the differences between the means) on each outcome variable for each school. The results replicate Busby et al.’s (Reference Busby, Druckman and Fredendall2017) findings on presidential approval, university satisfaction, and mood only for the losing school Clemson. We find no effects for Alabama respondents, in contrast to Busby et al.’s (Reference Busby, Druckman and Fredendall2017) finding that the game affected respondents from both schools. Moreover, even for Clemson, we find no significant differences on any other outcome measure, suggesting that the impact is confined to a limited set of attitudes.Footnote 7
Effects on Clemson (Losing Team) and Alabama (Winning Team) Respondents
***p ≤ 0.01, **p ≤ 0.05, *p ≤ 0.10 for one-tailed tests. Slight differences between the values in the difference column and the subtraction of the before and after-game groups are due to rounding.
We also explored whether the effects sustained one week later. We focus exclusively on the two significant Clemson effects (we did not measure mood in the T2 survey), since those are the effects that might last. In Table 2, we present the T1 and T2 means (and the differences between them) for the Clemson before- and after-game respondents. Note the T1 means in Table 2 do not match the T1 means in Table 1 since in the former we look only at those who responded at T2.Footnote 8 We find, as did Busby et al. (Reference Busby, Druckman and Fredendall2017), that the impact of the game was fleeting. Both after-game T2 scores significantly increased to resemble those of the before-game group, suggesting that people reverted back to attitudes unaffected by the game (also see Pierce et al., Reference Pierce, Rogers and Snyder2016).Footnote 9 In the Supplementary Appendix, we provide a host of robustness checks for our T1 and our over-time results (including accounting for non-response at T2).
Clemson Over-Time Effects
***p ≤ 0.01, **p ≤ 0.05, *p ≤ 0.10 for one-tailed tests.
CONCLUSION
The first generation of experimental studies of irrelevant event effects have clearly established that such effects can occur. Our results echo this reality as we replicated one such effect. On the other hand, we failed to replicate the effect for the winning team and found that the effects appear to be transitory and confined to selected outcome variables. This should not be taken as definitive evidence that the extant literature over-states the extent of irrelevant events; yet, it serves as a (cautionary) prompt to the next generation of work.
The obvious question is “under what conditions to irrelevant event effects occur?” Our read of the research thus far suggests they occur more often when people are surprised by the event (e.g., Eldar et al. Reference Eldar, Rutledge, Dolan and Niv2016, 15–16; Healy et al. Reference Healy, Neil and Mo2010, 12806),Footnote 10 are able to (unconsciously) attribute their mood to the object under evaluation (e.g., the president, their university, but not the economy or the Pope) (Schwarz and Clore, Reference Schwarz and Clore2003, 299), and are not motivated to systematically assess information beyond mood. It may be that existing evidence inadvertently happens to meet these and other unspecified conditions. Replications under different circumstances is one way to move the literature forward to isolate conditions and ultimately assess whether politically irrelevant events occur regularly and widely matter for politics.
SUPPLEMENTARY MATERIAL
To view supplementary material for this article, please visit https://doi.org/10.1017/XPS.2017.22
