How can attorneys establish credibility and influence Supreme Court justices? Scholarship on judicial decision making identifies a variety of factors that influence appellate judges, such as ideological goals (Segal and Spaeth Reference Segal and Spaeth2002), strategic considerations (Epstein and Knight Reference Epstein and Knight1998; Bailey and Maltzman Reference Bailey and Maltzman2011), legal doctrine (Bailey and Maltzman Reference Bailey and Maltzman2008; Bartels Reference Bartels2009), and external pressures (McGuire Reference McGuire2004; Casillas, Enns, and Wohlfarth Reference Casillas, Enns and Wohlfarth2011). Such studies undoubtedly inform academics about judicial behavior, but they offer little practical guidance for Supreme Court advocates, who are generally unable to affect a judge’s ideology, existing legal precedent, or public opinion. Put simply, much of what we know about judicial behavior turns on things over which attorneys have little control. What is left then for attorneys? What can they do, if anything, to influence the Court? We believe they can use the legal brief, conditioned by their perceived credibility, to try to influence justices.
The legal brief is a primary vehicle by which lawyers seek to persuade appellate judges. As Judge Frank M. Coffin once stated: “I know of no other field of human endeavor where a limited number of pages of written and structured argument can be so decisive as in appellate decision-making. The heavy artillery of appellate practice is the brief” (Coffin Reference Coffin1994, 107; emphasis in original). The brief is a key tool lawyers use to communicate with judges. In this capacity, briefs have the potential to influence how judges think about a case and vote in that case (see, e.g., Wedeking Reference Wedeking2010).
Yet a brief’s influence likely is conditional on the brief writer’s perceived credibility in the eyes of the justices. Persuasion literature consistently shows that under most circumstances, highly credible sources are more persuasive than low-credibility sources (see, e.g., Pornpitakpan Reference Pornpitakpan2004). To that end, we believe the language attorneys use in their briefs signals their credibility—or lack thereof. Appellate briefs are supposed to avoid emotionally charged language (Magidson Reference Magidson1971; Scalia and Garner Reference Scalia and Garner2008). A brief containing overtly emotional language suggests that the attorney writing it lacks credibility. Conversely, a measured brief that contains objective and logical information highlights the attorney’s awareness of the Court’s function and suggests that she is a credible appellate advocate. Therefore, we believe, briefs should be less likely to persuade Supreme Court justices when they feature emotional language.
The data concur. Justices are more likely to vote for parties whose briefs eschew emotionally charged language. Specifically, a party that abstains from emotional language in its briefs is more likely to win justices’ votes while a party that employs more emotional language is less likely to do so. For petitioners, using minimal emotional language is associated with a 29% increase in their probability of capturing a justice’s vote. For respondents, the effect is even greater: using minimal emotional language is associated with a 100% increase in their probability of winning a justice’s vote. To be sure, the available data do not allow us to make strong causal claims; however, these correlations (along with a host of robustness checks) offer considerable support for our theory.
These results are important for at least three reasons. First, for legal practitioners, the findings offer a cautionary tale: Avoid overtly emotional language. Certainly, not every brief that avoids such language will lead to victory. Yet, by showing empirical correlations between victory and language, we offer pragmatic guidance to practitioners. Second, the findings inform our understanding of judicial behavior by highlighting a critical component of the decision-making process. That judicial decisions are associated with brief language emphasizes the importance of quality lawyering before the Court (see, e.g., Johnson, Wahlbeck, and Spriggs Reference Johnson, Wahlbeck and Spriggs2006). The finding also suggests the value of looking beyond policy matters to explain Supreme Court decision making (Baum Reference Baum2006). Indeed, the findings suggest that justices might rely on things such as heuristics and cognitive biases when interpreting messages from lawyers. Third, our findings suggest important implications for the role of persuasion in politics more generally. At least in the judicial context, argumentation appears to matter: Decision makers tend to side with advocates who make more credible arguments. The same may be true in nonjudicial contexts in which the written word plays a key role. As just one example, our results might carry over to notice and comment rule making in federal agencies (see, e.g., Yackee Reference Yackee2006).
The Importance of Legal Briefs to the Supreme Court
It is widely accepted that the written brief is critical to modern judicial decision making. The brief has been described as “the central feature of modern appellate practice” (Martineau et al. Reference Martineau, Sinclair, Solomine and Holland2005, 770). Briefs offer lawyers a prized opportunity to communicate directly with courts. A series of interviews with the chief justice and associate justices highlight the importance of briefs.Footnote 1 Consider the following comments from Chief Justice Roberts: “[We] may not see your strong case. It’s not like judges know what the answer is. I mean we’ve got to find it out” (quoted in Lacovara [Reference Lacovara2008, 285]). Similarly, Justice Scalia has stated that “the overarching objective of a brief is to make the court’s job easier. Every other consideration is subordinate” (Scalia and Garner Reference Scalia and Garner2008, 59). When discussing whether attorneys should focus more effort on brief writing or oral argument, Justice Ginsburg declares that the brief “is ever so much more important. It’s what we start with; it’s what we go back to” (quoted in Garner [Reference Garner2010, 136]).
Briefs serve at least three important functions (see, e.g., Black and Owens Reference Black and Owens2012, 94–97). First, they help justices overcome information constraints. Justices have preferences over law (getting the answer to the legal question “right”) and policy (setting conservative or liberal precedent) but cannot always forecast how their votes will translate into outcomes (Epstein and Knight Reference Epstein, Knight, Clayton and Gillman1999). Justices need information to make such determinations, and they receive it, primarily, from the briefs filed in cases. Chief Justice Roberts has gone so far as to state, “We depend heavily on the lawyers [writing briefs]. Our chances of getting a case right improve to the extent the lawyers do a better job” (Lacovara Reference Lacovara2008, 281). So, briefs are important because they provide justices with information to achieve their goals.
Second, briefs coordinate judicial attention on an issue or set of issues. The Court’s norm against issue creation (the “sua sponte norm”) constrains justices from rendering decisions on issues outside the record of a case (Epstein, Segal, and Johnson Reference Epstein, Segal and Johnson1996). Justices are expected to decide cases on the basis of the issues presented to them. Consequently, the arguments attorneys present in their briefs, along with the lower court records, can set the boundaries of the issues the Court will address. Briefs help identify the legal issues, approaches, and controversies within those boundaries and thereby make the decision-making process more streamlined and transparent.
Finally, briefs provide attorneys their only real opportunity to present their best arguments to the Court without interruption. Many judges formulate lasting impressions of a case from the briefs alone. Judge Paul R. Michel of the Federal Circuit once declared he is able to reach a decision in roughly 80% of cases just from reading the briefs (Reference Michel1998, 21; discussed in Corley [Reference Corley2008, 468]). While the importance of oral arguments is well established (Johnson Reference Johnson2004; Johnson et al. Reference Johnson, Wahlbeck and Spriggs2006), these proceedings contain limitations for an advocate hoping to persuade the Court. During oral arguments, justices often derail the attorneys with difficult or distracting questions. For example, in King v. Burwell (14-114), attorney Michael Carvin completed just one sentence before Justice Ginsburg jumped in to ask him a question. In Elonis v. U.S. (13-983), attorney John Elwood finished one opening sentence before he was asked a question. He was later interrupted by Justice Breyer four times before he sighed, exasperated, “I’m trying hard to give [my answer] to you.” The written brief allows attorneys to present their arguments without such interruptions. To be clear, we are not suggesting that oral arguments are unimportant. Rather, we believe briefs offer an invaluable mechanism for persuasion since they emphasize the party’s strongest points just as the attorney wants, in the direction the attorney wants.
Despite conventional wisdom that briefs are important, scholars surprisingly know very little about the conditions under which briefs influence judicial decision making. To be sure, legal scholars offer a number of suggestions on how to write effective briefs, but they largely ignore whether those recommendations empirically bear fruit. To wit: some legal scholars have engaged in descriptive analyses of briefs. For example, Coleman and Phung (Reference Coleman and Phung2010) examined nearly every brief filed on the merits to the US Supreme Court between 1969 and 2004 to determine which parties used the fewest and most complex words (see also Fischer Reference Fischer2009; Coleman et al. Reference Coleman, Phung, Carlson, Syed and Jivani2013). Other legal scholars have attempted to examine the influence of briefs through surveys and experiments. For example, Benson and Kessler (Reference Benson and Kessler1987) provided mock legal briefs to judges and research attorneys in the California Court of Appeal (Second District) in Los Angeles. Some of the briefs were written in plain English while others were written in “legalese.” The authors asked their survey respondents to rate which briefs they thought were more professional, prestigious, and persuasive. By a large margin, the respondents believed the briefs written in plain English were more persuasive (see also Lewis Reference Lewis2005; Flammer Reference Flammer2010). While interesting, these studies take place in sterile contexts and face threats to external validity. Some legal scholars have attempted to examine systematically whether the characteristics of briefs can lead to increased chances of victory. Long and Christensen (Reference Long and Christensen2011) regress a party’s success on the readability of its brief. Employing various readability scores (such as the Flesch reading ease and Flesch-Kinkaid readability scores), the authors find no correlation between increased brief readability and success.
Political scientists have begun to examine the role of briefs before the High Court. In one highly creative approach, scholars used plagiarism software to uncover evidence of how the Court “borrows” (i.e., plagiarizes) content from litigant briefs. Corley (Reference Corley2008) was the first to deploy this approach, finding that the Court borrows more language from briefs filed by experienced attorneys, ideologically friendly parties, and the solicitor general (see also Black and Owens Reference Black and Owens2012). Wedeking (Reference Wedeking2010) examines whether attorneys use frames strategically to influence the dimension on which the High Court evaluates cases. Wedeking measures the most important words in case documents (e.g., cert petitions, party merits briefs, and amicus briefs) using content analysis software and then factor analyzes them to reveal frames. He finds that the lower court’s use of a prevailing frame decreases the likelihood of petitioners and respondents using alternative frames. Perhaps more importantly, he finds that when a petitioner uses an alternative frame, its likelihood of winning increases; when a respondent uses an alternative frame, the likelihood of the petitioner winning decreases. Put simply, Wedeking’s findings show that briefs—and their composition—can influence party success. But while Wedeking’s study is among the first and finest to examine how briefs influence the Court, it does not tell us how attorneys might establish credibility within those briefs and thereby win justices’ votes.
A study by Long and Christensen (Reference Long and Christensen2008) takes up the question of brief credibility by exploring the use of intensifiers (i.e., words such as clearly, obviously, and very). Using a sample of federal and state appellate briefs, they find that more frequent use of intensifiers by appellants is associated with a decreased chance of prevailing on the merits. While the investigation is limited to civil cases and a small set of word choices—a total of 12 intensifier wordsFootnote 2 —this work suggests that the language used in legal briefs may be connected to a party’s ability to influence judges.
The Role of Brief Language
Written language is in many ways the coin of the legal realm, particularly at the appellate level. Courts communicate with each other, parties, attorneys, potential litigants, and outside actors in writing. Furthermore, the language used by appellate courts is of preeminent importance given their ability to set precedent. Indeed, the common law system in the United States elevates the language of a written opinion to a legally binding status when it carries the support of a majority of judges on a court. Subsequent courts may then be constrained or left to their own discretion by the language used in decisions that carry precedential weight. Thus, judges have significant incentives to be attuned to written language. This notion is supported by evidence that Supreme Court justices spend significant amounts of time drafting, fine-tuning, and negotiating over the content of majority opinions (Maltzman, Spriggs, and Wahlbeck Reference Maltzman, Spriggs and Wahlbeck2000). For our purposes, such attentiveness to language suggests that justices also likely have preferences regarding the language used in briefs and are responsive to variation in such language. But what type of brief language do justices prefer?
Judges and justices have been trained and socialized in the traditional “rule of law” approach, which values objective, logical argumentation (Massaro Reference Massaro1989). Justice Sotomayor’s confirmation hearing before the Judiciary Committee elucidates this idea. Responding to questions regarding President Obama’s comments about the desirability of “empathy” among judges, Sotomayor rebutted this notion, explaining, “Judges can’t rely on what’s in their heart. … It’s not the heart that compels conclusions in cases, it’s the law” (National Public Radio 2009). Since justices are trained to value dispassionate adjudication of legal questions, we suggest that, while there are various ways attorneys can establish credibility—many of which we account for below—we suspect that attorneys can earn credibility by avoiding overtly emotional language in briefs. Or, perhaps more aptly, they can lose credibility when they use emotionally charged language.
Centuries of scholarship on rhetoric and communication establish the importance of a sender’s credibility in the eyes of the message’s receiver. For example, Aristotle once declared that “we believe good men more fully and readily than others: this is true … where exact certainty is impossible and opinions are divided” (Rhetoric, 330 BC, bk. 1, chap. 2). As he saw it, persuasion was, in part, a function of the source’s credibility (or ethos) as much as the logic of the argument being made. In fact, ethos, he believed, was probably more important even than appeals to logic (logos) or to emotion (pathos). It gave the speaker the power to say “believe me because I am the sort of person whose words you can believe” (Halloran Reference Halloran1982, 60). Indeed, Justice Scalia argues that attorneys’ first objective in cases should be to show themselves “worthy of trust and affection” (Scalia and Garner Reference Scalia and Garner2008, xxiii).
Modern studies of persuasion, including several competing psychological models, support the contention that source credibility matters (e.g., McCroskey and Young Reference McCroskey and Young1981; Pornpitakpan Reference Pornpitakpan2004). For example, individuals may regard information about source credibility as a persuasive reason to accept the validity of an argument as they carefully process a message (Heesacker, Petty, and Cacioppo Reference Heesacker, Petty and Cacioppo1983). Alternatively, source credibility may function as a simple heuristic cue—or mental shortcut—that can help a message receiver make quick and efficient judgments while expending minimal mental energy (Petty and Cacioppo Reference Petty and Cacioppo1981). Heuristic cues, such as source credibility, may even bias more systematic cognitive processes (Maheswaran and Chaiken Reference Maheswaran and Chaiken1994).
Several classic works in psychology find that the persuasive influence of a message depends, in part, on the expertise and trustworthiness of the message’s source (Lasswell Reference Lasswell and Bryson1948; Hoyland and Weiss Reference Hoyland and Weiss1951; Hoyland, Janis, and Kelley Reference Hoyland, Janis and Kelley1953). Expertise concerns a receiver’s assessment of a speaker’s capacity to make correct assertions, while trustworthiness captures the degree to which a receiver believes statements made by a messenger are valid (Pornpitakpan Reference Pornpitakpan2004). Higher levels of expertise and trustworthiness mean that a source is perceived as more credible.
Certainly, in the context of litigation, an attorney’s prior experience and professional reputation convey expertise. Lawyers experienced before the Supreme Court (McGuire Reference McGuire1995) and lawyers with highly respected professional reputations (Wohlfarth Reference Wohlfarth2009; Black and Owens Reference Black and Owens2012) tend to be more successful. Yet, by the time a case reaches the Court, an attorney’s level of experience and reputation (or lack thereof) are fixed, at least in the short term. So, how can attorneys present a more credible message? They can write objective briefs that lack overt emotional appeals.
Imagine two briefs before the Court. One brief writer relies heavily on affectively charged language to make her argument. The other brief writer, however, uses very little of this language, opting instead for nonemotive language. Whereas the first brief contains words designed to elicit an emotional response from the reader, the second brief appears objective and professional. We argue that the nonemotive language in the second brief will present a more trustworthy and, therefore, credible message to the Court. In contrast, the first brief’s overly affective tone and emotionally manipulative statements will “turn off” the justices.
These concerns are not just hypothetical. Judges have stated as much. Judge Ruggero Aldisert explains that appellate judges expect lawyers to build their briefs around reasoned arguments (Aldisert Reference Aldisert1996; Arey Reference Arey2000). Emotional appeals, he argues, are more appropriate for trial lawyers in front of jurors. Direct appeals to logic and legal authorities most effectively enhance a lawyer’s credibility and ability to persuade. Appellate review leaves little room for emotional responses. Magidson (Reference Magidson1971) makes precisely this point. He states that “arguments in a brief should be devoid of excessive emotionalism” (187). Indeed, he argues that an objective brief can virtually lock the case for counsel when pitted against an emotional brief. “The more extravagant the hyperbole—if such is possible—the more persuasive a contrasting factual recitation becomes until it is not even necessary for opposing counsel to draw conclusions. He can leave them for the court to draw” (187). Supreme Court Rule 24.6 drives home the point: “A brief shall be concise, logically arranged with proper headings, and free of irrelevant, immaterial, or scandalous matter. The Court may disregard or strike a brief that does not comply with this paragraph.”
Scalia and Garner (Reference Scalia and Garner2008) go farther and declare that making appeals to judges’ emotions “is misguided. … Good judges pride themselves on the rationality of their rulings and the suppression of their personal proclivities, including most especially their emotions” (32). They go on to tell writers that while subtle appeals to emotion may be appropriate, overt appeals are not: “You may safely work into your statement of facts that your client is an elderly widow seeking to retain her lifelong home. But don’t make an overt, passionate attempt to play upon the judicial heartstring. It can have a nasty backlash” (32).
In sum, we argue that brief language can influence the extent to which a party’s message is persuasive to individual justices. Justices are attentive to language choices; indeed it is a vital component of their craft as interpreters of the law and policy makers. This attentiveness suggests that justices are likely also attuned to language used earlier in the decision-making process as well, such as the words used to convey arguments in briefs. We submit that justices are responsive to brief language because it conveys the credibility of the attorney, but also because it more generally connects with their socialization into the “rule of law” approach that devalues extraneous extralegal factors such as emotional appeals. Put simply, we believe briefs can influence justices, and they will be more likely to do so when they contain less rather than more emotional language. In short, we hypothesize that a justice will be less likely to vote for a party whose brief employs more emotional language.
Data and Measures
To test our hypothesis, we analyze how each Supreme Court justice voted in 1,677 orally argued cases decided during the Court’s 1984–2007 terms. For reasons that are apparent below, we focus on cases that included only a single initial merits brief submitted by each party (i.e., one petitioner brief and one respondent brief). Following others before us (e.g., Corley Reference Corley2008, n. 3), we confine our analysis to the parties’ initial merits briefs (i.e., we exclude any reply or supplemental briefs submitted by the parties). All told, our subset of cases includes about 85% of all cases decided during those terms, which means that our analysis remains highly generalizable. We analyze each justice vote in a case. Our dependent variable is dichotomous, indicating whether a justice voted for the petitioner (= 1) or the respondent (= 0).Footnote 3 Our expectation, again, is that justices are more likely to side with parties whose briefs utilize less emotional language and, therefore, appear more credible.
Emotional language in briefs. Our main covariates of interest reflect the language in the party briefs. In order to examine the language of the briefs, we employed the Linguistic Inquiry and Word Count (LIWC) program (Pennebaker and King Reference Pennebaker and King1999; Tausczik and Pennebaker Reference Tausczik and Pennebaker2010). LIWC is a textual analysis software package that uses a series of dictionary-based word lists to classify the content of text. LIWC analyzes “attentional focus, emotionality, social relationships, thinking styles,” and other features of language (Tausczik and Pennebaker Reference Tausczik and Pennebaker2010, 24). The program employs a word count strategy that searches text for words or word stems and then counts the number of words in a document that fall into one of a number of categories. The internal and external validity of LIWC has been established in a series of publications (see, e.g., Pennebaker and King Reference Pennebaker and King1999; Tausczik and Pennebaker Reference Tausczik and Pennebaker2010; Frimera et al. Reference Frimera, Aquinob, Gebauer, Zhu and Oakes2015), and it has also been used in judicial politics scholarship (Owens and Wedeking Reference Owens and Wedeking2011).
Our analysis focuses on LIWC’s affective language category, which includes a total of 919 words or word stems. Some representative examples include words such as outrageous, apprehensive, wonderful, and glorious.Footnote 4 As described above, we seek to examine whether the extent to which attorneys use emotional language is associated with their ability to garner justices’ votes. We downloaded our briefs as text files from Westlaw and then used LIWC to determine the percentage of emotional words in the argument section of each party’s brief. Petitioner brief emotion and respondent brief emotion measure the percentage of affective language words contained in the petitioner’s and respondent’s brief on the merits, respectively.
Case quality. Like virtually all nonexperimental research, we must be attentive to potential confounders. For us, the most threatening confound is case quality. That is, the underlying strength (or weakness) of a case could lead an attorney to use less (or more) emotional language in his or her brief. Case quality is also very likely correlated with a justice’s vote. Thus, if we fail to control for case quality, then we cannot exclude the possibility that case quality is the lurking confounder. To that end, we deploy two measures of case quality.
First, we use the lc disagreement variable from the Supreme Court Database. Dissenting—especially for lower court judges—is a costly activity (Epstein, Landes, and Posner Reference Epstein, Landes and Posner2013). The dissenting judge must expend the effort to write the dissent (a costly action for busy judges). She also knows that when she dissents, she adds to the workload of her just-as-busy colleagues and, in doing so, incurs collegiality costs. For judges on the federal circuits, this is not a trivial undertaking. Therefore, when circuit judges dissent, it signals something important about the case. That is, judges are more likely to dissent when the losing party presents a high-quality legal argument—when the benefits of dissent outweigh the costs.Footnote 5 Dissent noted in lower court takes on a value of one if the Supreme Court opinion notes the presence of a dissenting opinion in the lower court and zero otherwise.
In many contexts (but not here), the Supreme Court Database’s coding strategy is problematic because it does not code whether a lower court judge in fact dissented but, rather, whether the Supreme Court’s opinion mentioned that dissent. In our context, this “selection effect” is likely beneficial because it highlights those cases that seem to be of higher quality. That is, when the Court mentions a lower court dissent, it often does so to highlight the strong legal arguments on both sides of the case. In many instances, the Court goes so far as to support the lower court dissent. For example, in Planned Parenthood v. Danforth, 428 U.S. 52 (1976), the majority opinion stated, “We thus agree with the dissenting judge in the present case [below] and with the courts whose decisions are cited above.” The majority then went on to quote directly from the lower court dissent and elaborate on its reasoning. Writing for the majority in CSX Transport v. Georgia State Board of Equalization, 552 U.S. 9 (2007), Chief Justice Roberts wrote, “We do not find [one of the party’s] interpretation compelling. Instead, we agree with Judge Fay in dissent below.” He then quoted directly from Fay’s dissent. In short, when a lower court judge dissents in a case, it likely signals that the parties’ arguments are well balanced and objectively strong. Therefore, cases that observe a lower court dissent are likely to be cases with high-quality arguments on both sides.
Second, we examine the usage of legal authority in the litigant’s briefs. The more legal authority a side has, the stronger, on average, its case should be. To create these measures, we returned to the briefs and, in particular, the Tables of Authorities, which identify the various materials cited within each brief. For each side in each case, we used Westlaw and counted the total number of unique authorities listed in the Table of Authorities for all briefs in that side (including any amicus briefs). To account for the likely nonlinear nature of authority, we took the natural logarithm of these totals. Petitioner legal authority and respondent legal authority measure these values for each side.
Attorney quality. We also include a number of controls to ensure that any results we obtain are not being spuriously driven by omitted variables. One important control is attorney quality. It is possible, of course, that higher-quality attorneys simply make better arguments and that their quality rather than the language they use in the briefs explains their success. If the data show that emotional language is associated with a decreased likelihood of capturing justices’ votes, even after controlling for attorney quality, then we can have more confidence that the language of briefs plays an important and independent role in persuading the Court.
We measure and include a number of variables related to attorney quality to ensure that our results are robust. First, we control for whether the United States filed a brief for the petitioner or respondent in the case. Numerous studies demonstrate the influence of the solicitor general, who is widely considered the most prestigious, high-quality attorney before the Court (e.g., Wohlfarth Reference Wohlfarth2009; Black and Owens Reference Black and Owens2012). Four dichotomous variables, US petitioner, US respondent, US amicus for petitioner, and US amicus for respondent, equal one if the solicitor general participated in this manner and zero otherwise.
Next, we account for the status of the two parties. In so doing we follow a long line of scholarship tracing back to Galanter’s (Reference Galanter1974) seminal article on the “haves” versus the “have-nots.” More recent studies document that resource advantages make a difference throughout the Supreme Court’s decision-making process, from agenda setting (Black and Boyd Reference Black and Boyd2012) to the merits outcome (Black et al. Reference Black, Treul, Johnson and Goldman2011). We follow the coding scheme of Collins (Reference Collins2004, Reference Collins2007) and use the Supreme Court Database party codes to determine which status category each party fell into: poor individuals, minorities, individuals, unions or interest groups, small businesses, businesses, corporations, local governments, state governments, and the US government. The weakest category—poor individuals—is coded 1 whereas the strong category—the US government—is coded 10. The resulting variables are petitioner status and respondent status.Footnote 6
Additionally, we control specifically for the attorneys’ experience before the Court. Scholarship shows that more experienced attorneys tend to win their cases more often than “one-shotters” (e.g., McGuire Reference McGuire1995). To control for this dynamic, we counted the number of previous oral arguments before the US Supreme Court each attorney in the case delivered. To account for the nonlinear nature of experience (Black and Owens Reference Black and Owens2012), we used the natural logarithm of one plus each attorney’s experience. The variables for the petitioner and respondent are labeled as petitioner experience and respondent experience, respectively.
Oral argument. We also account for what happens during oral argument. Specifically, we account for the well-documented negative relationship between the justices’ question activity toward a side and her likelihood of voting for that side (e.g., Johnson et al. Reference Johnson, Black, Goldman and Treul2009; Epstein, Landes and Posner Reference Epstein, Landes and Posner2010; Black et al. Reference Black, Treul, Johnson and Goldman2011).Footnote 7 Questions for petitioner and questions for respondent equal the number of questions asked of each party, respectively, during oral arguments. While the effectiveness of a brief and oral arguments may be connected, we believe it is important to distinguish them for three reasons. First, the same legal arguments may be more or less effective depending on the medium through which they are delivered. Second, parties may use different strategies in briefs than in oral arguments. Third, as we stated above, attorneys are often derailed during oral argument and are unable to make their points with the same precision as a brief.
Amicus briefs. Next, we next control for petitioner amicus support and respondent amicus support. Collins (Reference Collins2008) argues that amicus briefs can help a party win. Whether they add new information to help a party or simply signal that one side has a stronger case, the presence of amici is correlated with party success. We use amicus data provided by Collins (Reference Corley2008), which we updated for the 2002–7 terms, and count the number of amicus briefs for each side in each case.
Brief readability. We also control for the readability of the briefs. Numerous scholars argue that clarity drives persuasion. Scalia and Garner (Reference Scalia and Garner2008) go so far as to say that all other qualities of briefs should be inferior to clarity. Supreme Court justices are generalists who lack time to dig through briefs to find the brief writer’s points. Parties must make their points clearly and quickly so justices can move on to their next tasks (Springer Reference Springer1984). As Deputy Solicitor General Michael Dreeben once stated, “briefs should also be as clear as possible and as short as possible. Appellate judges have a lot to do. They are not interested in reading any more words than they have to” (https://www.youtube.com/watch?v=EmeHI4TryIw). Given these concerns, it is possible that more readable (i.e., clearer) briefs are more influential. To measure this concept we follow Owens, Wedeking, and Wohlfarth (Reference Owens, Wedeking and Wohlfarth2013) and measure the Coleman-Liau index for each merits brief. The two inputs for the Coleman-Liau index are word and sentence length. Texts with longer words and sentences will have higher values, which indicate lower readability.Footnote 8
Ideology. Finally, we control for the ideological congruence between the justice and the party’s position in the case. To do so, we interact each justice’s ideological preferences with whether the lower court decision was liberal. The first variable, justice ideology, reflects the justice’s Segal-Cover score (Segal and Cover Reference Segal and Cover1989), which is the perceived liberalism of a justice at his or her time of nomination and ranges from zero (most conservative) to one (most liberal).Footnote 9 The second variable, lower court decision liberal, equals one if the Supreme Court Database codes the lower court decision as liberal and zero if it codes the lower court decision as conservative. We then multiply these two variables to create lower court decision liberal × justice ideology.
Methods and Results
Our dependent variable—whether a justice voted for the petitioner—is dichotomous, so we estimate a logistic regression model with robust standard errors clustered on each justice in our data.Footnote 10 Parameter estimates for the model appear in table 1. The model predicts just under two-thirds of the votes correctly and reduces prediction errors by approximately 21%.
Logistic Regression Parameter Estimates for Whether a Justice Votes for the Petitioner (Coded as 1) or the Respondent (Coded as 0)
*. p < .10.
**. p < .05.
Consistent with our expectations, using emotional language appears to damage an attorney’s chance of successfully capturing individual justices’ votes. The coefficient on petitioner brief emotion is negative and significant, showing a negative relationship between a justice’s vote for petitioner and the increasing percentage of emotional language in the petitioner’s brief. Conversely, the coefficient on respondent brief emotion is positive and significant, showing a positive relationship between a justice’s vote for the petitioner and the increasing percentage of emotional language in the respondent’s brief. And it is important to recall that we observe these results after controlling for features that might themselves explain heightened success.
To explicate the substantive nature of these results, we calculate predicted probabilities and display them in figure 1. The horizontal axis represents the percentage of emotional language contained in the party’s brief, with our analysis of petitioners on the left and respondents on the right. The vertical axis in both images represents the probability that a justice votes for the petitioner in the case. All other variables were held at their median values.
Effect of brief emotional content on the probability a justice votes for the petitioner. Point estimates are denoted by the solid lines. Shaded areas indicate 95% confidence intervals. The vertical ticks along the x-axis denote (jittered) values of the x-variable in the actual data. All other variables were held at their median values.
When the petitioner uses little emotional language in its brief (approximately the 10th percentile in our sample), a justice has a .61 [.58, .63] probability of voting for her. When, however, the petitioner uses significantly more emotional language—about the 90th percentile in our sample—that probability decreases to around .56 [.53, .58]—a relative decrease of about 8%.Footnote 11 Going from the sample minimum to the sample maximum yields a bigger substantive effect, reducing the probability of winning a justice’s vote from .63 [.60, .66] to .48 [.41, .56], which is a 24% relative decrease. Given that petitioners start out with a high probability of winning their cases before the Court, this strong negative relationship offers a cautionary tale for lawyers: Even when justices are predisposed to vote for the petitioner, the use of emotional language (and the accompanying loss of credibility) may have hazardous effects on the petitioner’s odds of ultimately receiving justices’ votes.
Looking, next, to the effect on respondents, the results appear even more damaging. Using, again, the 10th and 90th percentile values, we estimate a .56 [.53, .58] probability of a justice voting in favor of the petitioner when the respondent uses little emotional language compared with a .63 [.61, .66] probability when the respondent uses more emotional language.Footnote 12 This is a relative decrease of about 13%. Using the sample minimum and maximum values yields probabilities that a justice casts a vote in support of the petitioner of .52 [.48, .55] (low emotional brief) and .76 [.69, .82] (highly emotional brief), which correspond to a relative decrease of 46%. Once again, these results are cautionary, this time for respondent attorneys. The Court is already (likely) predisposed to rule against them. Giving the Court extra reason to do so by displaying a lack of credibility may be particularly hazardous. An emotionally charged brief can turn into ruin for the respondent.
Importantly, the association between emotional brief language and justices’ votes holds even after controlling for other features that likely explain party success. Figure 2 plots the substantive effect of all control variables from our model that are statistically significant. The support of the United States is among the largest factors. So too is the effect of ideology, with a justice more likely (unsurprisingly) to vote for parties who make ideologically compatible arguments. The number of questions at oral argument also correlates with a justice’s vote. Less impactful but still influential are party resources, attorney experience, and amicus support. Taken together, these controls suggest that briefs are not the sole criterion justices use in making decisions, but importantly, the independent effect of brief language plays a strong role even after accounting for other significant factors.
Effect of control variables on the probability a justice votes for the petitioner. Point estimates are denoted by the solid circles. Horizontal whiskers denote 95% confidence intervals. All other variables were held at their median values. “Low” values always indicate the 25th percentile value of that variable and “high” values always denote the 75th percentile value.
Given that the Office of the Solicitor General (OSG) is so successful (Black and Owens Reference Black and Owens2012), one might ask whether the results are driven by OSG attorneys making objective, reasoned arguments. To examine this dynamic, we refit our model and excluded the solicitor general as a party. More specifically, we removed cases in which the United States was involved as the petitioner or respondent and then reexamined the role of emotional language. Table 2 provides these results. The “Included” column simply reproduces our two main results from table 1. The “Excluded” column shows the results when we exclude all instances in which the United States was a party to the case. Importantly, the results we obtain are substantively identical: both of our brief emotion variables remain highly significant. And these results hold even while we control for the solicitor general filing amicus briefs. In short, our findings are not simply driven by observations in which the OSG and the high-caliber attorneys within it are parties to the case.
US Party Involvement
Note.—Coefficient estimates for supplemental analysis that excludes cases in which the United States is a party involved in the case. Both models include all control variables featured in the model reported in table 1 except, of course, the US petitioner/respondent dummies. The coefficient size change between model 1 and model 2 for the respondent emotion variable is not statistically significant (p = .62).
*. p < .10.
**. p < .05.
Confounder Checks: Attorney and Case Quality
The results we present above support our theory. Still, a skeptical reader might argue that these results are consistent with two alternative stories about attorney and case quality. We consider each in turn.
First, it could be that we are not measuring better briefs but rather better attorneys. That is, better attorneys simply write better briefs, and it is attorney quality that drives justices’ voting behavior, not brief content. In the above analysis, we address this concern head-on by including eight variables to tap into the concept of attorney quality. (We also examined it when removing the solicitor general.) A key advantage of these measures is that data for them are readily available for all cases included in our analysis. This is important, of course, as it provides us with maximum statistical power to detect a relationship between briefs and votes, should one exist. Unfortunately, other measures proposed by the literature lack this attribute. More specifically, Szmer and Ginn (Reference Szmer and Humphries Ginn2014) recently developed a measure of “attorney capability,” which is a factor score for a host of attributes likely related to attorney quality (e.g., oral argument experience, former law clerk). Given the data-intensive nature of generating these measures, their scores are, at present, available only for the 1993–2001 terms. Despite the limitation in years, we conducted an auxiliary analysis using the subset of cases for which the Szmer-Ginn scores are available. Table 3 reports results for this supplemental analysis.
Attorney Capability (Szmer and Ginn)
Note.—Coefficient estimates for supplemental analysis that includes Szmer and Ginn’s alternative measure of attorney quality. Both models include all control variables featured in the model reported in table 1 except those that are also included in the Szmer and Ginn factor analysis.
*. p < .10.
**. p < .05.
Our dependent variable is the same as in the previous analysis. The model excludes those controls that were already included in the Szmer-Ginn factor analysis (e.g., attorney experience, affiliation with the OSG), but it includes everything else from our main model (e.g., amicus support, ideology). Cell values represent t-statistics for the variables indicated in the table rows. The separate columns are used to illustrate the effect of our main variables of interest in a model that excludes the additional measures (col. 1) and then in a model that includes the additional measures (col. 2).
These results provide ample evidence that brief content still matters, even after accounting for attorney quality. Both variables for petitioner and respondent brief content remain correctly signed and statistically significant. Although we do observe some minor attenuating of the coefficient size between the two models, neither of these differences is statistically significant. Perhaps more importantly, we also performed one additional series of tests: we compared whether the coefficient estimates in the subset models were significantly different from the coefficient estimates presented in our full model reported in table 1. This comparison is potentially the most important since the underlying question is whether the effect of emotional language decreases when we have controlled for additional aspects of attorney quality. None of the pairwise comparisons we evaluated provided even a hint of such evidence (i.e., p > .20).
The second alternative we consider is the potentially confounding role of case quality. By this potential account, the underlying legal strength of a case drives the language used in the briefs (see, e.g., Long and Christensen Reference Long and Christensen2013)—specifically, their emotive content. Lawyers with weaker cases would utilize emotional language—perhaps unconsciously or even desperately—to make up for their weaker cases. Conversely, lawyers with strong cases would avoid such language.
As an initial matter, we are skeptical about exactly why this dynamic would be at play only for attorneys in weak cases. Perhaps they believe it might “salvage” their cases somehow. But if they believe such language can have a positive effect, it is unclear why they employ this strategy only in weak cases. In other words, if they believe emotional language can resurrect a bad case, surely they must also think it can help them in stronger cases as well. In short, we doubt that attorneys direct their use of emotional language to weak cases alone. And so we doubt that our results are affected by the spuriousness of case quality.
Nevertheless, others may disagree with our reasoning, so we address the case quality confounder just as one would any other variable: by controlling for it. In our main analysis, we offer three variables to tap into this important concept. Dissent noted in lower court identifies whether the Court mentioned a dissenting opinion in the lower court. If the Supreme Court’s majority opinion notes the presence of a dissenting opinion in the lower court, then there is evidence that the petitioner has a stronger case than if such an indicator is absent. Potentially more directly, petitioner legal authority and respondent legal authority quantify the number of authorities cited in the briefs for each side. All of these variables are statistically significant and signed in the expected direction. And, as we illustrate above, the effect of emotional language in briefs holds even when we control for these dynamics.
Of course, our proxies for case quality are surrogates. Providing an ex ante measure of legal quality that is generalizable remains one of the most vexing empirical tasks for the law and courts community. Still, we were able to apply an approach drawn from Lindquist and Klein (Reference Lindquist and Klein2006), who demonstrate that legal factors influence the outcome of conflict cases before the Supreme Court. Those authors propose and measure three jurisprudential variables: the number of circuits supporting each side, the number of dissenting opinions opposing a side, and the prestige of judges who are on each side of the conflict. Each of these is shown to have a significant relationship in terms of predicting which side of a conflict prevails before the High Court. For our purposes, Lindquist and Klein’s measures offer a potential way to tap into different aspects of the legal quality of an argument.
Accordingly, we combined their three legal variables with our data on the emotional content of briefs. Just as in the case of attorney quality, however, a lack of overlap between our two samples results in significant data loss. In particular, Lindquist and Klein sample cases from 1985 to 1995, whereas our sample runs from 1984 to 2007. Additionally, given their substantive focus, they examine only federal circuit court conflict cases during that period. Our sample includes all types of cases. As a consequence of these differences, we could retain only 231 cases.
Our auxiliary analysis evaluates whether the legal quality factors measured by Lindquist and Klein are systematically related to our measures of brief emotional content.Footnote 13 That is, if case quality drives the emotional content in briefs, there should be a strong correlation between emotional content and these legal quality factors. Conversely, if case quality does not predict the emotional content in briefs, our argument that briefs matter remains sound.Footnote 14
We regressed petitioner brief emotion and respondent brief emotion on the measures of legal quality for each side in the case. Table 4 reports the results. Each column shows either the petitioner’s or the respondent’s brief. As the table makes clear, we find virtually no evidence that aspects of legal quality influence the emotive content of a litigant’s brief. The only statistically significant variable is respondent team size, which is positive and statistically significant at the .10 level in the model that explains petitioner brief emotion. This suggests that as the number of circuits supporting the respondent’s side increases (i.e., the petitioner has the weaker case), the amount of emotional content in the petitioner’s brief also increases.
Legal Quality (Lindquist and Klein)
Note.—Coefficient estimates for supplemental linear regression of brief emotion on legal quality variables (Lindquist and Klein Reference Lindquist and Klein2006).
*. p < .10.
**. p < .05.
Nevertheless, we do not believe this one finding does much to undermine the strength of our main empirical results. The substantive effect of the variable is quite small. Going from the modal/minimum value of zero circuits supporting the respondent’s side to the maximum of eight, which is over four standard deviations above the mean, results in only a 0.63% increase in the predicted emotive content of the petitioner’s brief. This corresponds to just four-tenths of a standard deviation in the petitioner emotion measure. And, as the anemic R 2 values for the models indicate, these variables explain hardly any of the variation observed in our brief content variables.
As in all nonexperimental empirical analyses, we must contend with the risk posed by confounding variables that are correlated with both our main independent variables of interest and our dependent variable. Two prominent threats to our theoretical and empirical account are attorney quality and case quality. The foregoing analyses scrutinize our results with the best available measures to determine whether the influence of brief content is spuriously driven by either of these factors. Our results withstand these tests.
Of course, if we are wrong and attorneys do use emotional language only in weak cases, our results still offer important insights. That is, even if attorneys tend to use emotional language in weak cases—a claim with which we (and our data) disagree—our results suggest that their efforts fail. Justices remain unfazed. Stated simply, either increased emotional words lead to diminished credibility in the eyes of the justices (which we suspect is the case) or they do not help a party escape a likely loss in a bad case. Either way, using emotional words is a losing enterprise for attorneys. And this, of course, follows the guidance of those who argue that the best practice is to avoid such language.
Discussion and Conclusion
One treatise author states that brief writing “is all art and no science” (Lacovara Reference Lacovara2008, 282). Perhaps. Our data, though, show that justices are less likely to side with briefs that employ emotional language. We argue that this finding suggests that such language decreases an attorney’s perceived credibility, a key component of persuasion. Justices respond to briefs that are less emotional, we believe, because they convey more credibility. And though the data cannot show a causal relationship, lawyers should nevertheless be wary of making their written arguments through overtly emotional language.
Importantly, these findings support the more general notion that litigant briefs influence Supreme Court justices. They also shed light on why. The results suggest that the language attorneys choose when crafting arguments plays an important role in determining a party’s ability to win the votes of US Supreme Court justices. We have presented evidence that these findings join those documenting the influence of quality lawyering (see, e.g., McGuire Reference McGuire1995; Johnson et al. Reference Johnson, Wahlbeck and Spriggs2006) and provide yet another mechanism by which attorneys are in a position to influence policies set by the Court. Our findings are also suggestive with regard to lower courts in the federal and state judicial hierarchies. Lower courts typically handle much larger caseloads, hold oral arguments in a smaller percentage of cases, and receive fewer amicus briefs compared to the US Supreme Court. Consequently, lower court judges tend to rely more heavily on briefs filed by the parties than do US Supreme Court justices. Therefore, if we are correct that emotive brief language influences judicial decision making in the High Court, this effect may be even stronger in lower courts.
Beyond our contention that brief language conveys credibility is a broader argument that the above analysis suggests that justices have preferences over brief language. Their legal training predisposes them to value objective, measured language while devaluing emotionally charged language. We suspect that as in their views on the appropriateness and usefulness of methods of textual interpretation, justices vary in the extent to which they are amenable to emotional words in briefs. These expectations for brief language may be rooted in the judicial decision-making process. For example, when reading, judges may anticipate written opinions and how the arguments made in the briefs will translate into policy. Previous work demonstrates that this is not strictly a mental exercise but, rather, that justices actually “borrow” the language in parties’ briefs for use in final opinions and that their proclivity to do so varies at the individual level (Corley Reference Corley2008). Future work could consider whether emotional language in briefs is connected to the likelihood that a brief’s arguments and specific language are used by justices in written opinions.
Of course, we do not argue that attorneys should avoid all emotional language. That would likely result in a boring brief no one would read. Rather, our argument is that attorneys should not make overt emotional appeals. It may be appropriate, as Scalia and Garner (Reference Scalia and Garner2008) say, to note that a party happens to be an elderly widow seeking to retain her lifelong home. But to go farther and claim that she is an “innocent victim of a heartless system that lacks compassion” is simply too much. Subtlety seems to be the key. Indeed, Shephard and Cherrick (Reference Shephard and Cherrick2006) argue that “the subtle emotional force of the brief expresses itself through the advocate’s fair factual statement, accurate analysis of the law, and compelling legal reasoning. The brief should be written in an objective tone with no flashy displays of adjectives intended to incite the court or enrage the opponent. An appellate court reacts against blatant emotional arguments” (161).
While we are generally satisfied with our approach, the study, like any, is not without limitations. Computer-formatted briefs are available only starting in the mid-1980s. As a result, our analysis necessarily focuses on relatively recent terms. Though this enhances our contemporary generalizability, it precludes examining how brief content has evolved across the Court’s far longer history. Additionally, as our analysis is nonexperimental, we are unable to establish that such language causes parties to win or lose justices’ votes. As we note above, it could be that the use of nonemotive language is correlated with something else that, itself, is strongly correlated to victory. This will always be a concern. Still, we have endeavored to measure and control for these concerns. We looked at attorney quality, case quality, the position of the solicitor general, ideology, and amici participation. That we continue to find results reassures us of the influential role of briefs. And, even if lawyers employ emotional language only in bad cases, the fact that such language does not resurrect their credibility in the eyes of the justices tells us something important about why not to use it.
Finally, our findings highlight the advantages of using textual analysis to illuminate the relationship between briefs and outcomes on the Court, echoing this message from other works (see, e.g., Corley Reference Corley2008; Wedeking Reference Wedeking2010; Long and Christensen Reference Long and Christensen2011). Moving forward, this approach could be applied to other questions of interest, such as testing the influence of other rhetorical strategies in litigant briefs. Our efforts also demonstrate the general importance of persuasive language in the political process and offer a promising avenue to study this language. Persuasion is a central component of policy making in many institutional contexts, and persuasive language appears in numerous forms, such as presidential rhetoric, lobbying activity, legislative debate, and comments on administrative rule making. The systematic study of this language and its effects offers an opportunity to enhance our understanding of these political processes. Examining the influence of parties in notice and comment rule making is one avenue. Scholars might also consider the use of language when appealing to expert versus generalist tribunals. At present, the textual analysis employed here suggests that, by eschewing emotionally charged language in their briefs, lawyers before the Supreme Court may be able to increase their odds of garnering the five votes needed to prevail on the merits.
Alternative Dependent Variable: Number of Votes
Negative Binomial Regression Parameter Estimates of Number of Votes for the Petitioner
*. p < .10.
**. p < .05.
LIWC Word List
Note that an asterisk (*) denotes a wild card of any length. For example, accepta* will match on acceptance or acceptable and bastard* will match on bastards or bastardly.
abandon*, abuse*, abusi*, accept, accepta*, accepted, accepting, accepts, ache*, aching, active*, admir*, ador*, advantag*, adventur*, advers*, affection*, afraid, aggravat*, aggress*, agitat*, agoniz*, agony, agree, agreeab*, agreed, agreeing, agreement*, agrees, alarm*, alone, alright*, amaz*, amor*, amus*, anger*, angr*, anguish*, annoy*, antagoni*, anxi*, aok, apath*, appall*, appreciat*, apprehens*, argh*, argu*, arrogan*, asham*, assault*, asshole*, assur*, attachment*, attack*, attract*, aversi*, avoid*, award*, awesome, awful, awkward*, bad, bashful*, bastard*, battl*, beaten, beaut*, beloved, benefic*, benefit, benefits, benefitt*, benevolen*, benign*, best, better, bitch*, bitter*, blam*, bless*, bold*, bonus*, bore*, boring, bother*, brave*, bright*, brillian*, broke, brutal*, burden*, calm*, care, cared, carefree, careful*, careless*, cares, caring, casual, casually, certain*, challeng*, champ*, charit*, charm*, cheat*, cheer*, cherish*, chuckl*, clever*, comed*, comfort*, commitment*, compassion*, complain*, compliment*, concerned, confidence, confident, confidently, confront*, confus*, considerate, contempt*, contented*, contentment, contradic*, convinc*, cool, courag*, crap, crappy, craz*, create*, creati*, credit*, cried, cries, critical, critici*, crude*, cruel*, crushed, cry, crying, cunt*, cut, cute*, cutie*, cynic, damag*, damn*, danger*, daring, darlin*, daze*, dear*, decay*, defeat*, defect*, defenc*, defens*, definite, definitely, degrad*, delectabl*, delicate*, delicious*, deligh*, depress*, depriv*, despair*, desperat*, despis*, destroy*, destruct*, determina*, determined, devastat*, devil*, devot*, difficult*, digni*, disadvantage*, disagree*, disappoint*, disaster*, discomfort*, discourag*, disgust*, dishearten*, disillusion*, dislike, disliked, dislikes, disliking, dismay*, dissatisf*, distract*, distraught, distress*, distrust*, disturb*, divin*, domina*, doom*, dork*, doubt*, dread*, dull*, dumb*, dump*, dwell*, dynam*, eager*, ease*, easie*, easily, easiness, easing, easy*, ecsta*, efficien*, egotis*, elegan*, embarrass*, emotion, emotional, emotions, empt*, encourag*, enemie*, enemy*, energ*, engag*, enjoy*, enrag*, entertain*, enthus*, envie*, envious, envy*, evil*, excel*, excit*, excruciat*, exhaust*, fab, fabulous*, fail*, faith*, fake, fantastic*, fatal*, fatigu*, fault*, favor*, favour*, fear, feared, fearful*, fearing, fearless*, fears, feroc*, festiv*, feud*, fiery, fiesta*, fight*, fine, fired, flatter*, flawless*, flexib*, flirt*, flunk*, foe*, fond, fondly, fondness, fool*, forbid*, forgave, forgiv*, fought, frantic*, freak*, free, freeb*, freed*, freeing, freely, freeness, freer, frees*, friend*, fright*, frustrat*, fuck, fucked*, fucker*, fuckin*, fucks, fume*, fuming, fun, funn*, furious*, fury, geek*, genero*, gentle, gentler, gentlest, gently, giggl*, giver*, giving, glad, gladly, glamor*, glamour*, gloom*, glori*, glory, goddam*, good, goodness, gorgeous*, gossip*, grace, graced, graceful*, graces, graci*, grand, grande*, gratef*, grati*, grave*, great, greed*, grief, griev*, grim*, grin, grinn*, grins, gross*, grouch*, grr*, guilt*, ha, haha*, handsom*, happi*, happy, harass*, harm, harmed, harmful*, harming, harmless*, harmon*, harms, hate, hated, hateful*, hater*, hates, hating, hatred, hazy, heartbreak*, heartbroke*, heartfelt, heartless*, heartwarm*, heaven*, heh*, hell, hellish, helper*, helpful*, helping, helpless*, helps, hero*, hesita*, hilarious, hoho*, homesick*, honest*, honor*, honour*, hope, hoped, hopeful, hopefully, hopefulness, hopeless*, hopes, hoping, horr*, hostil*, hug, hugg*, hugs, humiliat*, humor*, humour*, hurra*, hurt*, ideal*, idiot*, ignor*, immoral*, impatien*, impersonal, impolite*, importan*, impress*, improve*, improving, inadequa*, incentive*, indecis*, ineffect*, inferior*, inhib*, innocen*, insecur*, insincer*, inspir*, insult*, intell*, interest*, interrup*, intimidat*, invigor*, irrational*, irrita*, isolat*, jaded, jealous*, jerk, jerked, jerks, joke*, joking, joll*, joy*, keen*, kidding, kill*, kindly, kindn*, kiss*, laidback, lame*, laugh*, lazie*, lazy, liabilit*, liar*, libert*, lied, lies, likeab*, liked, likes, liking, livel*, lmao, lol, lone*, longing*, lose, loser*, loses, losing, loss*, lost, lous*, love, loved, lovely, lover*, loves, loving*, low*, loyal*, luck, lucked, lucki*, luckless*, lucks, lucky, ludicrous*, lying, mad, maddening, madder, maddest, madly, magnific*, maniac*, masochis*, melanchol*, merit*, merr*, mess, messy, miser*, miss, missed, misses, missing, mistak*, mock, mocked, mocker*, mocking, mocks, molest*, mooch*, mood, moodi*, moods, moody, moron*, mourn*, murder*, nag*, nast*, neat*, needy, neglect*, nerd*, nervous*, neurotic*, nice*, numb*, nurtur*, obnoxious*, obsess*, offence*, offend*, offens*, ok, okay, okays, oks, openminded*, openness, opport*, optimal*, optimi*, original, outgoing, outrag*, overwhelm*, pain, pained, painf*, paining, painl*, pains, palatabl*, panic*, paradise, paranoi*, partie*, party*, passion*, pathetic*, peace*, peculiar*, perfect*, personal, perver*, pessimis*, petrif*, pettie*, petty*, phobi*, piss*, piti*, pity*, play, played, playful*, playing, plays, pleasant*, please*, pleasing, pleasur*, poison*, popular*, positiv*, prais*, precious*, prejudic*, pressur*, prettie*, pretty, prick*, pride, privileg*, prize*, problem*, profit*, promis*, protest, protested, protesting, proud*, puk*, punish*, radian*, rage*, raging, rancid*, rape*, raping, rapist*, readiness, ready, reassur*, rebel*, reek*, regret*, reject*, relax*, relief, reliev*, reluctan*, remorse*, repress*, resent*, resign*, resolv*, respect, restless*, revenge*, revigor*, reward*, rich*, ridicul*, rigid*, risk*, rofl, romanc*, romantic*, rotten, rude*, ruin*, sad, sadde*, sadly, sadness, safe*, sarcas*, satisf*, savage*, save, scare*, scaring, scary, sceptic*, scream*, screw*, scrumptious*, secur*, selfish*, sentimental*, serious, seriously, seriousness, severe*, shake*, shaki*, shaky, shame*, share, shared, shares, sharing, shit*, shock*, shook, shy*, sicken*, sigh, sighed, sighing, sighs, silli*, silly, sin, sincer*, sinister, sins, skeptic*, slut*, smart*, smil*, smother*, smug*, snob*, sob, sobbed, sobbing, sobs, sociab*, solemn*, sorrow*, sorry, soulmate*, special, spite*, splend*, stammer*, stank, startl*, steal*, stench*, stink*, strain*, strange, strength*, stress*, strong*, struggl*, stubborn*, stunk, stunned, stuns, stupid*, stutter*, submissive*, succeed*, success*, suck, sucked, sucker*, sucks, sucky, suffer, suffered, sufferer*, suffering, suffers, sunnier, sunniest, sunny, sunshin*, super, superior*, support, supported, supporter*, supporting, supportive*, supports, suprem*, sure*, surpris*, suspicio*, sweet, sweetheart*, sweetie*, sweetly, sweetness*, sweets, talent*, tantrum*, tears, teas*, tehe, temper, tempers, tender*, tense*, tensing, tension*, terribl*, terrific*, terrified, terrifies, terrify, terrifying, terror*, thank, thanked, thankf*, thanks, thief, thieve*, thoughtful*, threat*, thrill*, ticked, timid*, toleran*, tortur*, tough*, traged*, tragic*, tranquil*, trauma*, treasur*, treat, trembl*, trick*, trite, triumph*, trivi*, troubl*, true, trueness, truer, truest, truly, trust*, truth*, turmoil, ugh, ugl*, unattractive, uncertain*, uncomfortabl*, uncontrol*, uneas*, unfortunate*, unfriendly, ungrateful*, unhapp*, unimportant, unimpress*, unkind, unlov*, unpleasant, unprotected, unsavo*, unsuccessful*, unsure*, unwelcom*, upset*, uptight*, useful*, useless*, vain, valuabl*, value, valued, values, valuing, vanity, vicious*, victim*, vigor*, vigour*, vile, villain*, violat*, violent*, virtue*, virtuo*, vital*, vulnerab*, vulture*, war, warfare*, warm*, warred, warring, wars, weak*, wealth*, weapon*, weep*, weird*, welcom*, well, wept, whine*, whining, whore*, wicked*, willing, wimp*, win, winn*, wins, wisdom, wise*, witch, woe*, won, wonderf*, worr*, worse*, worship*, worst, worthless*, worthwhile, wow*, wrong*, yay, yays, yearn*
LIWC Word List Robustness
A review of the above list reveals a number of terms that have a technical meaning in law or otherwise are likely to arise in an unemotional context in the discussion of a case.Footnote 15 Examples of the former include fault, harm, murder, supreme, and weapon. Examples of the latter would be any one of the various expletives or similarly offensive terms that would be included in briefs only as quotations. To ensure that our results were not being driven by inclusion of these words, we reviewed each of the words or word stems to determine if they fell into either of these categories. Three of us performed this task. We then recalculated our brief emotion variables in LIWC iteratively using dictionaries that excluded words identified by one (137 exclusions), two (77 exclusions), or three (40 exclusions) of us as being problematic. Finally, with these new measures in hand, we reestimated our statistical model for these three sets of variables. We obtain substantively identical results: the presence of emotional language reduces the likelihood that a side wins. Table A2 reports coefficients for the four different regressions. P-values for all coefficients are .01 or smaller (two-tailed test).
Brief Emotion
Oral Argument Robustness
We control for the influence of oral argument on a justice’s vote by including measures for the number of questions asked by the Court of both the petitioner’s and respondent’s attorneys. A more direct measure of oral argument quality is arguably the grades given by Justice Blackmun to each of the attorneys during their performances. Unfortunately, these data are available only for a limited number of terms in our analysis (i.e., 1984–93), which account for only 40% of the observations. Table A3 reports the coefficients for two models estimated on this subset of observations. Model 1 is a baseline model that includes all of the covariates in the article but does not include the Blackmun grades. We estimate this model to see if the same initial relationship between brief emotion and justice voting exists in this subset of the data. Model 2 includes all of these same variables plus the Blackmun grades.
Blackmun Grades
*. p < .10.
**. p < .05.
As the table indicates, we find a significant relationship for respondent brief emotion, and this relationship continues to hold even after controlling for the grade Blackmun awarded the respondent’s attorney. We fail to find a significant baseline effect for petitioner brief emotion, which has a two-tailed p-value of .16. Given the small sample size, however, we are not especially concerned about this somewhat weaker finding (see the next section for the reasoning behind this conclusion).
Model Specification Robustness
Table A4 reports t-statistics for a number of alternative model specifications discussed in the article. The top four rows examine treatment of the standard errors for our model. Of the eight results reported, only one—petitioner emotional content in the case-clustered model—fails to achieve significance at the equivalent of the .05 two-tailed level (i.e., a t-statistic with an absolute value of 1.96 or greater). The equivalent two-tailed p-value for this t-statistic is .15 (two-tailed). The bottom two rows present results for different parameterizations of justice-level effects. The article text reports a pooled model. As the table makes clear, our key results are robust to either random or fixed effects at the justice level.
Brief Emotion
Sample Power Analysis
In the text, we report that we attempted a supplemental analysis that reestimated our main model and included a number of indicators of legal quality. Owing to a lack of data availability for the legal quality measures, however, this analysis is performed on a small subset of our data, in particular, just 2,047 out of 14,888 observations, which is about 14%. When we estimate our baseline model, which is what we report in table 1, and do not include the legal quality controls, we fail to find a significant effect for either petitioner brief emotion or respondent brief emotion. Thus, when we include the legal quality variables and reestimate the model and continue to fail to find an effect for our variables of interest, this offers no insight into whether legal quality is spuriously driving our results.
Our belief is that our failure to find a significant relationship on this subset is merely a function of a lack of statistical power—as opposed to evidence that our main results are sensitive to omitted variables. To verify this, we returned to our full sample of 14,888 observations and took a random sample of 2,047 observations, which is how many we have for the legal quality subset analysis. We then reestimated our main model and looked to see if we recovered a statistically significant result for the brief emotions variables. We then repeated this process 5,000 times. If we are consistently unable to find a significant result with this sample size, then that is supportive that it is statistical power and not something else driving the null results we get in the legal quality subset. If, by contrast, we are usually able to still find results similar to those we report in the text, then that suggests that it is something other than sample size.
Across the 5,000 iterations of our analysis, we found a significant relationship only 17% of the time. Thus, in over 80% of our samples, we obtain null results. We interpret this to be strongly supportive of the sample size explanation. To further verify this, we repeated this process but did so while changing the sample size to be smaller (500 observations) and larger (5,000 and 10,000 observations). Consistent with our account, when we decrease sample size, the rate at which we find significance goes down. With 500 observations, we find significant results just 5% of the time. And with larger samples, we find significant results much more often: 38% (5,000 observations) and 82% (10,000 observations).
