Public confidence in the US Supreme Court is at near-record lows. Many scholars and pundits contend that increasing partisan polarization, along with the growing perception that the Court behaves in a partisan manner, threatens to further undermine the Court’s legitimacy.Footnote 1 Recent controversial decisions, including Bush v. Gore, Citizens United v. FEC, Shelby County v. Holder, and Obergefell v. Hodges have given partisans of all stripes reason to complain about judicial partiality.Footnote 2 In this light, it is easy to understand the tremendous partisan rancor over Supreme Court vacancies, seen most recently in the especially confrontational confirmation of Brett Kavanaugh. The threat of the appearance of partisan bias by the Supreme Court is so serious that Supreme Court justices themselves have worked to reinforce the idea that the Court is an apolitical arbiter of disputes—that is, a court, in the highest sense of the word—and not a “political” institution, in the low sense of the term. For example, in a speech in October 2018, Justice Elena Kagan warned: “Every single one of us needs to realize how precious the court’s legitimacy is…. The only way we can get people to do what we think they should do is because people respect us” (quoted in Associated Press 2018, n.p.). In a speech just days after Kavanaugh’s confirmation, Chief Justice John Roberts tried hard to distinguish the Court from the “political branches.” “Our role is very clear,” he said: “We are to interpret the Constitution and laws of the United States, and to ensure that the political branches act within them…. That job obviously requires independence from the political branches” (quoted in Shapiro Reference Shapiro2018, n.p.; see generally Johnson and Strother, forthcoming). In other words, some justices have spent time during their public appearances, interviews, speeches, and the like, working to explicitly reinforce the Supreme Court’s legitimacy. In fact, recent work has shown that this behavior is quite common for Supreme Court justices (Schmidt Reference Schmidt2013; Glennon and Strother Reference Glennon and Strother2019; Krewson Reference Krewson2019). We contend that when justices take to the airwaves to reinforce Court legitimacy, we should understand this behavior as strategic institutional maintenance (Hall Reference Hall2014).Footnote 3
While we know that justices regularly engage in these activities, it remains unclear whether judicial rhetoric can meaningfully influence public perceptions of judicial legitimacy. Though the public is known to be quite receptive to elite signals in general (for example, Bullock Reference Bullock2011), we simply do not know at this point whether justice rhetoric meaningfully shifts public perceptions of Court legitimacy. That is, we do not know whether justices’ efforts to protect their institution are effective. In this article, we ask whether Supreme Court justices can actually shore up the Court’s institutional legitimacy by speaking this way—in other words, whether justices can succeed in reinforcing Court legitimacy through their off-bench rhetoric. While there is an existent literature debating the marginal effects of judicial outputs on diffuse support, we still know very little about the effects of justices’ off-bench behavior on perceptions of high court legitimacy.
The above quotations indicate that Supreme Court justices consciously prioritize working to maintain diffuse support for the Court in their off-bench behavior, however, it is unclear if they are successful in doing so. In an effort to address this question, we field three original surveys with embedded experiments, one of which is based on a nationally representative sample, and find that Supreme Court justices can in fact measurably increase legitimacy with their off-bench rhetoric. Furthermore, we contribute to ongoing debates in the legitimacy literature in two key ways. First, we find that the legitimating effect of rhetoric is moderated by policy agreement with Court outputs, consistent with a growing body of research showing that legitimacy is directly affected by judicial decisions (Bartels and Johnston Reference Bartels and Johnston2013; Nicholson and Hansford Reference Nicholson and Hansford2014; Christenson and Glick Reference Christenson and Glick2015; Gibson and Nelson Reference Gibson and Nelson2016; Strother and Gadarian Reference Strother and Gadarian2019). Second, we find that perceptions of the Court’s nature as a political institution are highly relevant to individuals’ assessments of legitimacy, which is also consistent with recent work in this area (Bybee Reference Bybee2010; Strother and Gadarian Reference Strother and Gadarian2019).
SUPREME COURT LEGITIMACY
The Supreme Court relies on public support to be effective. Public support is typically conceptualized as the Court’s institutional legitimacy.Footnote 4 And while the literature suggests that Supreme Court legitimacy has long been deep and robust (Gibson Reference Gibson2007; Gibson and Nelson Reference Gibson and Nelson2015), recent works suggest that legitimacy may erode in the wake of performance dissatisfaction, perhaps especially in the context of highly politicized rulings (Bartels and Johnston Reference Bartels and Johnston2013; Christenson and Glick Reference Christenson and Glick2015; Zilis Reference Zilis2015; Strother Reference Strother2016; Hitt and Searles Reference Hitt and Searles2018; Strother and Gadarian Reference Strother and Gadarian2019). This means that the Court may be uniquely susceptible to institutional decay during the present moment of intense partisan polarization.
The Supreme Court’s typically high levels of diffuse supportFootnote 5 are generally attributed to one of two sources. Many adhere to the view that legitimacy is reinforced by ideological or partisan agreement with the Court’s outputs (Bartels and Johnston Reference Bartels and Johnston2013; Christensen and Glick Reference Christenson and Glick2015). Others contend that the Court’s legitimacy is owed to public beliefs that courts are fundamentally unique institutions that reach decisions through fair and neutral processes (Tyler Reference Tyler1990; Tyler and Rasinski Reference Tyler and Rasinski1991; Gibson, Caldeira, and Spence Reference Gibson, Caldeira and Kenyatta Spence2003). The latter perspective centers on the idea that legitimacy is largely immune to partisan- or policy-based erosion because “courts are different.” This perspective has been advanced most forcefully by James Gibson and various colleagues, under the auspices of “positivity theory” (Gibson Reference Gibson2007; Gibson and Caldeira Reference Gibson and Caldeira2009; Gibson and Nelson Reference Gibson and Nelson2016). Many believe that the widespread public perception that court decisions are reached by careful and fundamentally fair processes is a key source of judicial legitimacy (for example, Tyler Reference Tyler1990; Scheb and Lyons Reference Scheb, John and Lyons2000; Zink, Spriggs II, and Scott Reference Zink, Spriggs and Scott2009; Braman and Easter Reference Braman and Easter2014). Based on this view, the public is thought to believe that “courts are different” (Christenson and Glick Reference Christenson and Glick2015). Legitimacy, as a result, concerns the perceived “rightfulness” of governmental uses of authority (Easton Reference Easton1965; Tyler Reference Tyler1990; Braman Reference Braman2016). From this perspective, legitimacy is rooted in an understanding of the Supreme Court as an impartial arbiter of controversies, as opposed to being just another “political” institution containing election-minded actors subject to the pressures of interest groups or popular opinion. Positivity theory, then, holds that exposure to information about the Supreme Court inherently evokes ideas about neutrality and the like and thus supports the idea that courts are apolitical and, in turn, reinforces its legitimacy.
However, according to the former perspective, public assessments of the Supreme Court are not much different than assessments of the “political” branches (Nicholson and Howard Reference Nicholson and Howard2003; Nicholson and Hansford Reference Nicholson and Hansford2014; Christenson and Glick Reference Christenson and Glick2015). For example, Dino Christenson and David Glick (Reference Christenson and Glick2015) and Brandon Bartels and Christopher Johnston (2013) show that ideological or policy disagreement with even a single Supreme Court decision can significantly reduce Court legitimacy. Dan Simon and Nicholas Scurich (Reference Simon and Scurich2011, Reference Simon and Scurich2013) reach similar conclusions using an alternative approach to measuring legitimacy. Taken together, these studies strongly suggest that politics and policy preferences powerfully influence the legitimacy of the Court. In this light, the threat to judicial legitimacy posed by polarization and negative partisanship is clear.
An important dimension of the relationship between judicial legitimacy and Supreme Court outputs is media coverage. To the (very limited) extent that the public monitors the Supreme Court, it does so via media coverage (Solberg and Waltenburg Reference Solberg and Waltenburg2015; Linos and Twist Reference Linos and Twist2016; Strother Reference Strother2017). In the present moment of intense polarization, media coverage of Supreme Court decisions has itself become increasingly partisan in tenor (Bass Reference Bass2018; Hitt and Searles Reference Hitt and Searles2018). Matthew Hitt and Kathleen Searles, for example, show that media increasingly presents Court outputs in a “game frame,” which emphasizes which partisan side is “winning” and which is “losing,” rather than according to policy or principle (Hitt and Searles Reference Hitt and Searles2018; see also Bass Reference Bass2018). And given this tendency toward casting decisions in terms of winners and losers, it should come as no surprise that partisan and political considerations play an increasingly prominent role in an individual’s thinking about Court decisions and the institution itself (Hitt and Searles Reference Hitt and Searles2018; Strother and Gadarian Reference Strother and Gadarian2019; see also Linos and Twist Reference Linos and Twist2016).
THEORETICAL PERSPECTIVE: JUSTICES’ OFF-BENCH RHETORIC
Against this backdrop, we ask whether justices’ off-bench rhetoric can meaningfully increase public perceptions of Supreme Court legitimacy. Justices regularly give speeches, interviews, and the like, aimed at both elite and mass audiences. This behavior is certainly nothing new, but the incidence of extrajudicial appearances and interviews has increased markedly in recent years (Hasen Reference Hasen2015). Recently, the justices have collectively averaged more than 150 such public talks and appearances per year (Krewson Reference Krewson2019). In previous work, we analyze the substance of every televised mass media interview given by a Supreme Court justice from 1998 to 2014, and argue that in these interviews, the justices’ primary focus is on legitimacy reinforcement (Glennon and Strother Reference Glennon and Strother2019). Such legitimacy-supporting rhetoric includes arguments about how the Court is fundamentally apolitical and how its decisions are achieved through fundamentally fair and neutral processes.
When justices make these types of public statements, they seek to shore up the Supreme Court’s public legitimacy (Hasen Reference Hasen2015; Glennon and Strother Reference Glennon and Strother2019) because they recognize that the Court relies at least in part on public support for its decisions to be effective (Hall Reference Hall2014; Keck and Strother Reference Keck and Strother2016; Strother Reference Strother2019). Put differently, justices engage in what is sometimes referred to as strategic institutional maintenance. Many scholars have argued that justices respond to public and elite preferences in order to avoid sanctions that would harm the institution (for example, Clark Reference Clark2009; Engel Reference Engel2011). We contend that when justices address the public, they do so in the interest of maintaining and enhancing the Court’s institutional standing before the public.
In a recent study, Christopher Krewson (Reference Krewson2019) argues that justices’ speeches can significantly increase law students’ favorability ratings for the speaker. Additionally, he argues that faux newspaper stories describing judicial rhetoric significantly increase favorability ratings of the justice whose speech the participants read about and decreases perceptions that the Court is “politicized” in a sample of workers from Amazon.com’s Mechanical Turk (MTurk). These findings square with earlier research indicating that elite cues can powerfully shape public opinion above and beyond policy outputs (for example, Bullock Reference Bullock2011). However, at this time, it remains unclear as to whether justice rhetoric can meaningfully increase Supreme Court legitimacy. In Krewson’s (Reference Krewson2019) study, legitimacy ratings were statistically indistinguishable across those treated with judicial rhetoric and those not treated. This is perhaps the case because law students’ views about Court legitimacy are sufficiently high that they cannot be meaningfully increased. Alternatively, it is of course possible that this rhetoric is simply ineffective at reinforcing diffuse support for the Court. At least one piece of evidence suggests that this is not likely the case, however. Krewson finds that, among both law students and MTurk participants, justice rhetoric significantly reduces the perception that the Court is “political.” Logan Strother and Shana Gadarian (Reference Strother and Gadarian2019) show that the perception of how “political” the Court is seen to be is the key factor in shaping attitudes about judicial legitimacy in the present political moment: the more political the Court is perceived to be, the less legitimate it is. Drawing on these recent works, then, we theorize that justice off-bench rhetoric will significantly increase Supreme Court legitimacy among the general public.
Beyond this empirical expectation, we designed our studies to shed new light on a long-running debate in the legitimacy literature: does policy agreement with the Supreme Court matter when individuals reassess legitimacy and how much and why? Prior works are largely in agreement that policy preferences among the public do matter, but there is considerable debate as to how much (Bartels and Johnston Reference Bartels and Johnston2013; Nicholson and Hansford Reference Nicholson and Hansford2014; Christenson and Glick Reference Christenson and Glick2015; Gibson and Nelson Reference Gibson and Nelson2015). Furthermore, the pathways and mechanisms by which policy dis/agreement with the Court moderates or mediates legitimacy are not well understood. Thus, Studies 2 and 3 assess the effects of judicial rhetoric in the context of real-world politics; Study 2 manipulates the respondents’ agreement with a Supreme Court decision, while Study 3 presents a political event that maps directly onto a major partisan cleavage. Moreover, all three studies are designed to test a key mechanism that is theorized to link policy preferences to legitimacy: perceptions of how “political” the Supreme Court is.
RESEARCH DESIGN AND DATA
To test this theory, we designed and fielded three original surveys with embedded experiments, including one on a nationally representative sample. The experimental design allowed us to isolate the effects of judicial rhetoric on public perceptions of the Supreme Court’s legitimacy. As Table 1 illustrates, we also manipulated a number of other factors, such as the framing of the story and the political context in which the rhetoric is presented in order to assess whether any effects of judicial rhetoric are conditional on, or mediated by, other factors. The full text of the survey instruments, including vignettes and all question wording, can be found in Appendix 1, and sample demographics are presented in Appendix 2. All of the estimates presented in the following sections come from linear probability models. The dependent variables are continuous and are scaled to vary between zero and one to facilitate interpretation. The key independent variables are dummy variables that indicate treatment (versus not). Together, the studies demonstrate that justices’ off-bench rhetoric does in fact meaningfully influence public views of the Supreme Court’s legitimacy.Footnote 6 Put simply, we found considerable evidence that justices can “go public” to protect and preserve their institution across a range of political contexts and treatment frames.
Summary of Research Design
ANALYSIS
Study 1: Survey Experiment (MTurk)
Study 1 is a survey experiment fielded at the MTurk workplace. Study 1 was designed to provide leverage on the influence of judicial rhetoric on public perceptions of Supreme Court legitimacy. We restricted participation to MTurk participants who were US residents, at least eighteen years of age, and who had at least a 97 percent MTurk approval rating. This study was fielded on June 6, 2018. Respondents spent approximately five minutes completing the survey and were paid one dollar for participation. Six hundred individuals participated in this study. The respondents were randomly assigned into one of three treatment conditions or a true control condition (that is, no information). Each treatment condition consisted of a faux newspaper article featuring quotations attributed to Chief Justice John Roberts. In an effort to maximize the external validity of the treatments, the quotations were drawn from real-life justice interviews. In other words, in designing the treatments, we drew on justices’ actual public-facing rhetoric. The full text of the treatment vignettes is presented in Appendix 1, and a randomization check is presented in Appendix 2.
In one treatment, which is labeled “legitimizing,” the vignette features Chief Justice Roberts making remarks about the Supreme Court being fundamentally apolitical. These types of remarks are typical of the rhetoric that justices use when they attempt to shore up diffuse support for the Court in their rhetoric. Glennon and Strother (Reference Glennon and Strother2019) show that this legitimizing rhetoric is the most common type of off-bench rhetoric that justices employ in their public-facing interviews. As such, the “legitimizing” treatment will help us ascertain the effect of justices’ standard legitimizing rhetoric on public views about the Court’s legitimacy (that is, by comparing the treatment to the control condition). In doing so, it will shed light on whether justices’ legitimizing rhetoric can be effective in achieving its apparent aims.
The next treatment, which is labeled “mixed,” embeds some legitimizing judicial rhetoric in a larger narrative about the Supreme Court being “more mixed up in politics” than justices might care to admit. In this treatment, even some of the justices’ rhetoric mentions the importance of politics. This language is drawn from actual statements made by Justice Antonin Scalia (though, in the vignette, we attribute the quotation to Chief Justice Roberts; see Barnes Reference Barnes2016). In a conversation with a law student, Justice Scalia recommended that students who want to get into federal judgeships would be wise to get involved in politics—his point being that lawyers who become federal judges are often those with political connections. We adapt this “get involved in politics” line and construct a vignette depicting justice rhetoric to highlight both the legal and political dimensions of the Court. This vignette is intended to provide leverage on the effects of legitimizing rhetoric in the context of real-world media coverage, which often emphasizes the Court as a political actor (Hitt and Searles Reference Hitt and Searles2018).
The final treatment, which is labeled “political,” provides no legitimizing rhetoric but does feature the same quotation about getting “involved in politics” as the treatment described above. Rather than legitimizing rhetoric, this vignette features an extended discussion of the Supreme Court being a thoroughly political institution. This language is reflective of language used in media coverage that is critical of the Court’s work, especially in the context of decisions that the writer is unhappy with (see Bass Reference Bass2018; Hitt and Searles Reference Hitt and Searles2018). This vignette is designed to shed light on whether the simple fact of exposure to a quotation from a Supreme Court justice is sufficient to increase legitimacy (as positivity theory suggests it should), even when the content of the message suggests that the Court is “political” in nature.
After this treatment, the respondents were asked a series of questions about the Supreme Court. They were first asked the standard battery of legitimacy questions developed by Gibson and various colleagues (Gibson, Caldeira, and Spence Reference Gibson, Caldeira and Kenyatta Spence2003; Gibson and Nelson Reference Gibson and Nelson2015).Footnote 7 The respondents were also asked (in random order): “how well does the word ‘political’ [‘legal’] describe the Supreme Court?” A series of questions on social and demographic characteristics followed. We expected that the respondents in the legitimizing rhetoric treatment would exhibit higher levels of diffuse support for the Court and view the Court as being less “political” compared to those in the control condition. We expected respondents in the “mixed” treatment (that is, political and legitimizing rhetoric) to respond to the legitimizing rhetoric in the same way as those in the “legitimizing” treatment but to a lesser degree (due to the presence of the “political” messaging). Alternatively, we expected the respondents in the “political rhetoric” treatment to view the Court as more political and, contrary to positivity theory, less legitimate.
Figure 1 depicts the findings graphically (point estimates and 95 percent confidence intervals [CIs]). First, as expected, we found that those in the legitimizing rhetoric treatment (“legitimizing” in the figure) view the Supreme Court as a significantly less “political” institution than those in the control condition. Those who received the “mixed” treatment also view the Court as less political. However, those in the “political” rhetoric treatment view the Court as significantly more political than those in the control condition.
Effect of Justice Rhetoric on view the Court is “Political” in Nature, MTurk.
Next, we turn to views about Supreme Court legitimacy. Consistent with our expectations, Figure 2 shows that those in the legitimizing treatment view the Court as significantly more legitimate than those in the control condition. This suggests that justices’ off-bench rhetoric can in fact serve to reinforce the Court’s legitimacy. Somewhat surprisingly, we do not find a statistically discernible effect of the “mixed” treatment on views about legitimacy. The relationship falls in the expected direction but cannot be statistically distinguished from zero. Also as expected, respondents in the “political treatment” view the Court as less legitimate compared to the control condition, though this effect is only marginally significant (p = 0.059).
Effect of Justice Rhetoric on Supreme Court Legitimacy, MTurk.
This study is the first to demonstrate empirically that the mass public can be influenced by justices’ legitimating rhetoric. In other words, it does appear that justices can effectively “go public” to build support for their institution. In summary, we find, as expected, that the legitimizing rhetoric that justices frequently employ when speaking before the public can significantly enhance public assessments of Court legitimacy (full models and robustness checks are presented in Appendix 3). Beyond establishing this empirical regularity, this study also helps to develop our understanding of how and why respondents incorporate new information about the Court into their belief systems. Consistent with our expectations, those who are exposed to the legitimizing rhetoric view the Court as significantly less “political” in nature compared to those in the control condition. Finally, we find that those who are exposed to judicial rhetoric emphasizing the Court’s relationship to politics view the Court as more political and less legitimate than those in the control condition. This finding is consistent with our expectations, and, notably, suggests that the relationship between the public and information about the Court may be more complicated than positivity theory suggests.
Study 2: Survey Experiment (Nationally Representative Sample)
Study 2 is designed to extend Study 1 and to further test the efficacy of justices’ rhetoric at reinforcing Supreme Court legitimacy. Study 2 is a survey experiment, fielded by Survey Sampling International/ResearchNow (SSI), on a representative sample of voting-age Americans. SSI maintains a large online (opt-in) panel from which respondents are recruited; recruitment is balanced on a range of social and demographic characteristics to ensure the sample is demographically similar to the general population, according to census records. The survey was distributed in the field between June 12 and June 22, 2018; 1,013 individuals participated. We designed this study to be a more challenging test of the theory, in that we introduced new contextual factors that were reflective of the politicized nature of most media coverage of the Supreme Court (Bass Reference Bass2018; Hitt and Searles Reference Hitt and Searles2018).
This experiment was a two x two x two design (full factorial, plus an additional control condition) where the dimensions were (1) the presence (versus the absence) of a justice’s legitimizing rhetoric; (2) whether the Supreme Court decision depicted involved striking down versus upholding a statute; and (3) whether the vignette featured a “legal” versus “political” framing, plus a true control providing no information (the randomization check is presented in Appendix 2). The rhetoric versus no rhetoric dimension provided leverage on the specific effect of justices’ legitimizing rhetoric on the outcomes of interest, which is the main focus of this study. The legal versus political framing dimension allowed us to test whether media coverage that emphasizes the “political” nature of the Court conditions, amplifies, or otherwise moderates the relationship between justice rhetoric and public views of the Court. Finally, the strike versus uphold condition allowed us to directly manipulate respondents’ policy agreement with a specific, real-world Supreme Court decision: we measured policy preferences before treatment and then randomly assigned respondents to read a vignette depicting a decision that either comported with, or ran counter to, their expressed preference. In all treatment conditions, respondents read a story about a case that was similar, in broad strokes, to Masterpiece Cakeshop v. Colorado (the “gay wedding cake case”).Footnote 8 In the “strike” condition, the baker was depicted as the winner; in the “uphold” condition, the couple was depicted as the winner. Having distinct outcomes helped to assure that we would observe a large number of respondents who disagreed with the decision (since, as the common refrain in the literature goes, “legitimacy is for losers,” as people who like a decision surely accept it as authoritative).
Immediately after treatment, the respondents were asked (1) whether they agreed with the Supreme Court’s decision; (2) whether the decision was based on law, politics, or both; and (3) whether, in general, the Court bases its decisions more on law or more on politics. The respondents were then asked the legitimacy battery. The full text of the vignettes and question wording are presented in Appendix 1.Footnote 9 Analyses are restricted to those who passed an attention check.Footnote 10 To reiterate, Study 2 was designed to validate and extend Study 1 by running a similar design and set of analyses in a distinct context (a Supreme Court decision as opposed to an interview) and on a different sample frame. Moreover, this study allowed us to directly test a potential key mediator in a real-world context: policy agreement with a Court decision. Building on prior works, we expected that policy disagreement with the Court decision would muzzle the legitimating potential of the justice’s rhetoric (Bartels and Johnston Reference Bartels and Johnston2013; Christenson and Glick Reference Christenson and Glick2015; Strother and Gadarian Reference Strother and Gadarian2019).
We began by estimating the effects of treatment on views of how political the Supreme Court is. Figure 3 depicts the findings graphically (point estimates and 95 percent CIs). First, as expected, we saw that those treated with legitimizing justice rhetoric (“rhetoric” in the figure) view the Supreme Court as a significantly less “political” institution than those in the control; those who were in the “no rhetoric” treatment were indistinguishable from the control condition in terms of views about how political the Court is. Second, and again consistent with our expectations, we found that those randomly assigned to agree with the decision viewed the Court as much less political compared with those in the control condition. Finally, and in line with our expectations, those in the “legal” frame treatment viewed the Court as significantly less political than those in the control condition. Somewhat surprisingly, though, those in the “political” frame treatment did not view the Court as more political than those in the control—indeed, if anything, those in the political treatment viewed the Court as somewhat less political than those in the control (p = 0.055). This finding, though somewhat unexpected, is perhaps consistent with positivity theory, which holds that any information about the Supreme Court inherently reinforces legitimacy.
Effect of Justice Rhetoric on view Court is “Political” in Nature, SSI.
Next, we turned to the effects of treatment on the key outcome of interest: Supreme Court legitimacy. These findings are depicted graphically in Figure 4. As expected, we found that treatment with justices’ legitimizing rhetoric led respondents to view the Court as significantly more legitimate than those in the control or no rhetoric conditions. In other words, we again found that justices can in fact reinforce diffuse support for the Court through their off-bench rhetoric (full models and robustness checks are presented in Appendix 3). Taken together, the findings from Study 2 strongly reinforce those from Study 1. Justices’ off-bench rhetoric reduces individuals’ perceptions that the Court behaves “politically” and, in turn, increases perceived legitimacy, even in the context of a controversial and potentially divisive Supreme Court decision such as Masterpiece Cakeshop v. Colorado.
Effect of Justice Rhetoric on Supreme Court Legitimacy, SSI.
Additionally, we found that those who were treated to agree with the case outcome (that is, those who were assigned to the outcome they had previously stated they preferred) also viewed the Court as significantly more legitimate. Moreover, the direct effect of rhetoric on legitimacy was concentrated among those who agreed with the Court’s decision (p = 0.024). This finding strongly suggests that, in this real-world scenario, the legitimating promise of justice rhetoric may be limited to those who are happy with the Court’s policy outputs. This finding runs contrary to positivity theory in that it indicates that policy agreement powerfully mediates legitimacy.
Study 3: Survey Experiment (MTurk)
Study 3, which was the final study presented, is a survey experiment fielded at MTurk. Study 3 was designed to extend and validate the findings of the first two studies in further contexts. We restricted participation to MTurk participants who were US residents, at least eighteen years of age, and who had at least a 97 percent MTurk approval rating. This study was fielded on July 3, 2019. Respondents spent approximately eight minutes completing the survey and were paid one dollar for participating. In total, 682 individuals participated in this study.
Study 3 was a two x two + two design, where the relevant dimensions were (1) the presence of a justices’ legitimizing rhetoric (versus the presence of judicial rhetoric devoid of any legitimizing content—that is, “non-legitimizing” rhetoric) and (2) the presence (or absence) of a story contextualizing the interview with a short vignette about the confirmation of Brett Kavanaugh to the Supreme Court; plus an additional baseline comparison condition, which provided the Kavanaugh vignette but no judicial rhetoric; plus a control condition providing no information (a randomization check is presented in Appendix 2). The legitimizing rhetoric versus non-legitimizing rhetoric dimension provided additional leverage on the specific effect of justices’ rhetoric on the outcomes of interest by allowing us to compare rhetoric that appears to be aimed at reinforcing legitimacy to regular interview fare that does not appear to be aimed at institutional maintenance (Glennon and Strother Reference Glennon and Strother2019). In other words, this study allowed us to assess whether specifically legitimizing rhetoric is uniquely influential or if all justice rhetoric influences public support for the Court. The dimension providing information (or not) about the Kavanaugh nomination provided a relevant political context of the kind that respondents will often encounter in real-world presentations of information about the Supreme Court (Strother Reference Strother2017; Hitt and Searles Reference Hitt and Searles2018). The true control and Kavanaugh-baseline conditions provided appropriate comparison groups for analysis.
In all treatment conditions, the respondents were presented with an excerpt from a real-life, televised interview with Justice Sonya Sotomayor. The excerpt preserved the question-answer format of the interview in an effort to keep the vignettes as true to life as possible, given the constraints of the survey experimental method. In this way, we aimed to extend beyond one limitation of Studies 1 and 2: the fact that in those studies the information presented to the respondents was presented as though it was repackaged by a newspaper reporter. Study 3, then, presented raw dialogue from an actual interview. Additionally, in Studies 1 and 2, the rhetoric was attributed to Chief Justice Roberts; in Study 3, we used Justice Sotomayor. In the baseline treatment and in the Kavanaugh + rhetoric treatments, the respondents were presented with information briefly describing the rancorous battle to confirm Kavanaugh to the Supreme Court. Following treatment, the respondents were asked (1) how well, in general, the words “political” and “legal” describe the Supreme Court and (2) the legitimacy battery. The full text of the vignettes and question wording are presented in Appendix 1.Footnote 11 Analyses are restricted to those who passed an attention check.Footnote 12
We began by estimating the effects of treatment on views of how political the Supreme Court is. Figure 5 depicts the findings graphically (point estimates and 95 percent CIs). First, as expected, legitimizing rhetoric had a substantively large and highly significant negative effect on views of how “political” the Court is. Next, we found a smaller, but still significant, effect of non-legitimizing justice rhetoric on views of how political the Court is. Those in the Kavanaugh confirmation (with no rhetoric) condition, by contrast, are not distinguishable from those in the baseline condition in their views of how political the Court is. That is, information about the acrimonious confirmation battle alone appears to have been insufficient to change respondents’ views about the Court’s political nature. The final two treatments are designed to maximize external validity by providing some context (“news”) about the Court in addition to the justice rhetoric. In these treatments, we found that, when presented with information about the Kavanaugh confirmation and legitimizing rhetoric, the respondents viewed the Court as significantly less political, compared to those in the control group. However, the political evaluations of the Court by those reading about the Kavanaugh confirmation and non-legitimizing rhetoric were statistically indistinguishable from the control condition.
Effect of Justice Rhetoric on SCOTUS Political in Context of Kavanaugh Saga.
Next, we turn to the effects of treatment on Supreme Court legitimacy, which is our key outcome of interest. The findings are depicted graphically in Figure 6. First, we discuss the treatments without political context. Both the legitimizing rhetoric and the non-legitimizing rhetoric in the interview significantly increased the respondents’ assessments of the Court’s legitimacy, compared to the control group. The first finding here is consistent with our expectations. The second, while not inconsistent with our expectations, is perhaps surprising—why would Justice Sotomayor’s discussion of her upbringing in the Bronx increase regular people’s trust in the Supreme Court? This outcome, while consistent with positivity theory, is not robust to the real-world context of information about the Kavanaugh confirmation, which reduces confidence in the meaningfulness of the effect.
Effect of Justice Rhetoric on Legitimacy in Context of Kavanaugh Saga.
Moving on to the analysis of those treated with information about Kavanaugh’s confirmation to the Supreme Court, we found again that Justice Sotomayor’s legitimizing rhetoric significantly increases legitimacy compared to the control. In other words, the finding regarding the legitimizing rhetoric is robust across both the context-free and real-world news story context. However, the assessed legitimacy of the Court for those presented with information about Kavanaugh’s confirmation and the non-legitimizing rhetoric was indistinguishable from the control group. Put differently, in the context of a true-to-life news story about the Court, legitimizing rhetoric had its intended effect, while non-legitimizing rhetoric did not exert any discernible influence on people’s views of the Court. These findings are robust to the inclusion of “democratic values” as well as standard political and demographic measures in the models (see Appendix 3). Study 3, then, strongly reinforces the inferences drawn from Studies 1 and 2: justices can effectively shift public opinion in the Court’s favor by appealing to the American public.
DISCUSSION
In summary, we have found strong evidence supporting the claim that Supreme Court justices’ off-bench rhetoric can significantly reinforce public support for the Court. In doing so, we demonstrate for the first time that the well-established ability of political elites to directly influence public attitudes extends to members of the Supreme Court. This finding is robust across three original survey experiments, including one on a nationally representative sample, and across three very different contexts: rhetoric without context, rhetoric in the context of a Court decision, and rhetoric in the context of the Kavanaugh confirmation. Additionally, we note that this finding is robust to framing effects and that these effects hold after controlling for “democratic values,” partisanship, political ideology, and standard demographic characteristics. This finding is important, as earlier work found no evidence of effect on legitimacy from off-bench judicial rhetoric (Krewson Reference Krewson2019) nor from off-bench “expert” legal commentary (Simon and Scurich Reference Simon and Scurich2013) (see Table 2).
Summary of Results
Beyond simply showing that justice rhetoric can have this powerful legitimating effect, our research also contributes to key theoretical debates in the literature. Consistent with recent work by Strother and Gadarian (Reference Strother and Gadarian2019), we have found that the treatments that enhance the Supreme Court’s legitimacy also reduce the perception that the Court is “political.” While not dispositive, this is strongly suggestive that perceptions of the Court’s “political” nature constitute a key link between performance evaluation and Court legitimacy in the public mind. Second, our work has shown that the legitimating rhetoric of justices was ineffective among that subset of respondents who were randomly assigned to disagree with a Supreme Court decision. This finding runs contrary to positivity theory, which would have suggested that the respondents should not have been influenced by decision disagreement. This finding suggests instead that performance approval does matter when the public evaluates the Court. Based on the present study, we cannot say precisely why this is the case, but perceptions of how political the Court is might tell the story. That is, Study 2 shows that those who were assigned to agree with the Court evaluated the Court as significantly less political and significantly more legitimate independent of judicial rhetoric. This suggests that respondents may evaluate the Court’s political nature and its legitimacy (in turn) as a function of performance agreement.
Of course, like all survey-experimental research, our arguments about the effects of judicial rhetoric assume treatment (Barabas and Jerit Reference Barabas and Jerit2010). However, because media coverage of the Supreme Court is exceptionally sparse (Solberg and Waltenburg Reference Solberg and Waltenburg2015; Strother Reference Strother2017), many Americans will not be “treated” with information about the Court, whether decisions, confirmation hearings, interviews, or whatever else; citizens with strong interests in (and knowledge of) politics are much more likely to be treated than others. More precisely, many Americans are not treated directly with information from the media. Even so, recent research shows that individuals often receive political information from people in their social networks and that people treated with this “social information” do learn, though rather less compared to those who are treated directly by the media (Carlson Reference Carlson2019). Additionally, we measured our outcomes at only one point in time and, therefore, cannot say whether, or for how long, the treatment effects persist over time. Future work would do well to take this question up.
Taken together, the findings presented above present a strong case that Supreme Court justices’ off-bench rhetoric can significantly influence the Court’s legitimacy. Put differently, when justices “go public” to maintain the Court’s institutional standing and preserve its role in our politics, their efforts are not wasted. In sum, we find that judicial rhetoric can significantly reinforce Supreme Court legitimacy. Across three unique experimental studies and three samples, we found that, when respondents are exposed to justices’ rhetoric about the Court as an institution, they view the Court as less political and more democratically legitimate. These findings contribute to our understanding of the relationship between the Court and the public.
Our analyses support two key takeaways. First, justices can indeed “go public.” All three studies produce strong results indicating that citizens are likely to consider the Supreme Court less political and more legitimate when exposed to legitimizing rhetoric from a sitting justice. Previous work implies that the justices believe this to be true, and the present study provides robust evidence that strongly indicates that they are correct. Furthermore, and interestingly, we found evidence that legitimizing messaging works even when it is presented alongside contrary information, such as information or framing indicating that the Court is behaving “politically.” This demonstrates the considerable willingness of the public to respond positively to suggestions that the Court is in fact apolitical. Citizens want their Supreme Court to be apolitical, and they will accept that it is when encouraged to do so, even if simultaneously presented with conflicting accounts.
Second, and consistent with recent work, media coverage of the Supreme Court matters and has the potential to impact perceptions of the Court’s basic nature and its legitimacy (Linos and Twist Reference Linos and Twist2016; Bass Reference Bass2018; Hitt and Searles Reference Hitt and Searles2018). This finding is of particular importance in the current political climate of intense partisan polarization and the increasingly partisan nature of media coverage of the Court. This reality poses a potential ongoing threat to the Court’s legitimacy by describing the justices as behaving just like other partisan political actors. This finding might also suggest another important reality: the fact that the justices themselves may need to consider increasing the number of times they do “go public” in trying to describe an apolitical Court, if only to ameliorate the impacts of partisan or “game frame” media coverage that depicts the Court in ways that harm its legitimacy. However, we found that this legitimating effect is muted among those who disagreed with the Court’s decision. This finding points to the limits of the justices’ ability to engage in this sort of institutional maintenance.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/lsi.2020.38
