%20-%20https://sketch.com%20--%3e%3ctitle%3e3ECBBF50-FD08-412A-943D-2CFC5E025A16%3c/title%3e%3cdesc%3eCreated%20with%20sketchtool.%3c/desc%3e%3cg%20id='Stylesheet'%20stroke='none'%20stroke-width='1'%20fill='none'%20fill-rule='evenodd'%3e%3cg%20transform='translate(-926.000000,%20-668.000000)'%20id='BUTTONS'%3e%3cg%20transform='translate(100.000000,%20597.000000)'%3e%3cg%20id='BTN-/-RIGHTS'%20transform='translate(816.000000,%2065.000000)'%3e%3cg%20id='ICON-/-Rights'%20transform='translate(10.000000,%206.000000)'%3e%3cpath%20d='M13.5752841,0%20L1.34943182,0%20C0.59375,0%200,0.586666667%200,1.33333333%20L0,22.6666667%20C0,23.4133333%200.620738636,24%201.34943182,24%20L17.6505682,24%20C18.40625,24%2019,23.3866667%2019,22.6666667%20L19,5.36%20L13.5752841,0%20Z%20M13.5752841,1.89333333%20L17.0838068,5.36%20L13.5752841,5.36%20L13.5752841,1.89333333%20Z%20M1.34943182,22.6666667%20L1.34943182,1.33333333%20L12.1988636,1.33333333%20L12.1988636,5.36%20C12.1988636,6.10666667%2012.7926136,6.69333333%2013.5482955,6.69333333%20L17.6235795,6.69333333%20L17.6505682,22.6666667%20L1.34943182,22.6666667%20Z'%20id='Shape'%20fill='%23595959'%20fill-rule='nonzero'%3e%3c/path%3e%3ccircle%20id='Oval'%20stroke='%23595959'%20stroke-width='1.5'%20fill='%23595959'%20cx='9.5'%20cy='13.5355339'%20r='5.5'%3e%3c/circle%3e%3cpolyline%20id='Path-2'%20stroke='%23FFFFFF'%20stroke-width='1.5'%20transform='translate(9.500000,%2013.035534)%20rotate(45.000000)%20translate(-9.500000,%20-13.035534)%20'%20points='8%2016.0355339%2011%2016.0355339%2011%2010.0355339'%3e%3c/polyline%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e)
Research on decision making by the U.S. Supreme Court has demonstrated that rules and procedures—such as the rule of four in certiorari voting (Boucher and Segal 1995; Caldeira, Wright, and Zorn 1999), the norm of opinion assignment (Maltzman and Wahlbeck 2004), the norm that a majority of the justices must support an opinion for it to be considered precedent (Epstein and Knight 1998; Maltzman, Spriggs, and Wahlbeck 2000), and the order of voting at conference (Johnson, Spriggs, and Wahlbeck 2005)—influence the choices justices make. The implication is that the rules and norms of the Court's decisional process provide information to help justices understand the consequences of their choices.
This spate of research, however, has generally ignored the most visible element of the Court's decisional process—oral arguments. Although a handful of studies show that justices gather information from these proceedings (Benoit 1989; Cohen 1978; Johnson 2001, 2004; Wasby, D'Amato, and Metrailer 1976), comparatively little is known about how oral arguments affect their choices. This lack of knowledge has led to significant differences of opinion over the extent to which oral arguments influence justices' decisions. Some scholars suggest that what transpires here is mere window dressing, and there is no indication it “regularly, or even infrequently, determines who wins and who loses” (Segal and Spaeth 2002, 280). Others, by contrast, note that these proceedings “come at a crucial time” in the process and can “focus the minds of the justices and present the possibility for fresh perspectives on a case” (O'Brien 1996, 275).1
Justices, themselves, offer contradictory statements about the effectiveness of these proceedings. Whereas Chief Justice William Rehnquist and Justices William Brennan and William Douglas point out the importance of oral argument, Justice Sandra Day O'Connor and former Chief Justice Earl Warren downplay its relevance (O'Brien 1996, 274–85; Rehnquist 1987, 277; Stern and Gressman 1993, 732).
We contend that oral arguments can influence Supreme Court justices' decisions by providing information relevant for deciding a case. Indeed, during these proceedings justices seek information in much the same way as members of Congress, who take advantage of information provided by interest groups and experts during committee hearings to determine their policy options or to address uncertainty over the ramifications of making a particular decision (Austen-Smith and Wright 1994, 29). In so doing, oral arguments can help justices come to terms with what are often complex legal and factual issues. As Justice Blackmun suggests, “A good oralist can add a lot to a case and help us in our later analysis of what the case is all about. Many times confusion [in the brief] is clarified by what the lawyers have to say” (Strum 2000, 298). These proceedings thus have the potential to crystallize justices' views or to move them toward a particular outcome (e.g., Wasby, D'Amato, and Metrailer 1976; Johnson 2004).
To plumb the role oral arguments play for the Court we analyze newly discovered and unique archival data—evaluations made by Justice Harry Blackmun of the arguments presented by attorneys who participated in these proceedings. Specifically, these notes include substantive comments about each attorney's arguments and a grade for their presentation. For example, in Florida Department of State v. Treasure Salvors (1982) Blackmun wrote 10 substantive comments about the argument made by the respondent's attorney and then noted that, “He makes the most with a thin, tough, case.”2
Blackmun used a set of cryptic abbreviations in his notes. Specifically, here, he wrote, “He makes t most o a thin, tough case.”
Focusing on the grades Blackmun assigned each attorney, our analysis proceeds in several steps. First, we outline why, theoretically, political decision makers must assess the credibility of information they gather and how Supreme Court justices can gather information during oral arguments. Based on this theoretical outline, we analyze the grades Blackmun assigned attorneys to determine whether conventional indicators of attorney credibility (e.g., litigation experience) correlate with their oral argument grades, and whether Blackmun's grades can plausibly be seen as measuring the quality of oral arguments. Finally, and more importantly, we determine whether justices' final votes on the merits are influenced by what transpires at oral arguments. Here we show that Justice Blackmun's colleagues are considerably more likely to vote for the litigant whose attorney offered more compelling oral arguments, even after controlling for alternative determinants of their votes.
By establishing a causal link between oral arguments and justices' votes, we make two contributions to the literature. First, we shed new light on how information transmitted to the Court during these proceedings affects decisions justices make. Although recent work by Johnson (2001, 2004) advances our understanding of how justices use information they obtain from oral arguments, we take his findings a step further by offering systematic evidence of exactly how justices evaluate these arguments and whether they directly influence decisions. This analysis is therefore the first study to demonstrate a causal relationship between oral arguments and justices' votes, and thus it reconciles the significant “differences of opinion as to the effectiveness of oral argument” present in the literature (Walker and Epstein 1993, 104).
Second, we add a new piece to the puzzle concerning how information affects choices made by political actors more generally. Our data allow us to examine what existing analyses have been unable to consider: how decision makers' explicit evaluations of information influence their decisions. Indeed, although the literature is replete with research on the efforts by decision makers to obtain information (e.g., Bartels 1986; Huber and McCarty 2001; Krehbiel 1991; Nemacheck 2001; Rogers 2001), few studies (e.g., Rahn 1993) examine how actors evaluate information transmitted to them. Previous studies also have been unable to assess whether actors who perceive information as being more credible or useful are more likely to be influenced by it than those who view it as less credible. Rather, scholars generally find observable data (an interest group's report of lobbying a member of Congress, for example) and then simply draw inferences about the effect of the information provided (the member's voting behavior). Our data go beyond such indirect measures by allowing us to examine the intermediate step—how an institutional actor evaluates information—because Blackmun systematically assessed the quality of information by grading the attorneys' arguments.
INFORMATION AND ELITE DECISION MAKING
Information plays a vital role in politics. It is not overreaching to say that the possession and appropriate use of information can mean the difference between political success and failure. In fact, political actors cannot determine which course of action will foster the outcomes they prefer unless they have sufficient information about the likely effects of alternative choices available to them. As Lupia and McCubbins (1998, 1) put it, “Reasoned choice, in turn, requires that people know the consequences of their actions.” The problem for political decision makers is that information is not always plentiful and it can be costly to obtain; they therefore often face uncertainties about which choices will lead to the distributional consequences they most desire. In short, information affects decisions because it can influence which choices political actors deem most compatible with their preferences over outcomes.
While gathering information, decision makers must assess its credibility because the efficacy of information provided to an actor depends on the credibility of the source in the eyes of the recipient. As Austen-Smith (1993, 800) puts it in his examination of Congress, “the extent to which any information offered … is effective depends on the credibility of the lobbyist to the legislator in question. Such credibility … depends partly upon how closely the lobbyist's preferences over consequences reflect those of the legislator being lobbied, and on how confident is the legislator that the lobbyist is in fact informed.”
Although Austen-Smith discusses the importance of an informant's credibility, his analysis does not contain data to measure legislators' evaluation of the information they receive. In fact, although congressional scholars have assessed the impact of information on decision making by examining the committee system (Gilligan and Krehbiel 1987), lobbying by organized interests (Austen-Smith and Wright 1994), and legislative hearings (Diermeier and Fedderson 2000), these scholars do not analyze how members of Congress assess the credibility of information. Even those who have analyzed the congressional analog to the Supreme Court's oral argument—expert testimony provided at legislative hearings (Diermeier and Fedderson 2000, 52)—have not documented this causal link. As such, although we know information is important, there is a significant gap in our understanding of how actors assess what they receive.
Supreme Court Oral Arguments
There are good reasons to expect that litigants' arguments, including the information presented at oral arguments, can affect Supreme Court justices' decisions. Justices often face uncertainty, and they need information about a case and the law in order to set policy in ways that will promote their goals. It is in this context that lawyers appear before the Court and attempt to provide the justices with information that will help their client's cause. They do so by trying to provide “a clear presentation of the issues, the relationship of those issues to existing law, and the implications of a decision for public policy” (Wahlbeck 1998, 783).
Although the justices often come to oral arguments after reading the written briefs and the lower court record, these proceedings themselves provide additional and relevant information to the Court (Johnson 2001, 2004). In fact, Johnson (2004, 5) demonstrates that justices often “seek new information during these proceedings” to help them reach decisions as close as possible to their desired outcomes. Others corroborate many of Johnson's findings with in-depth case studies (see, e.g., Benoit 1989; Cohen 1978; Wasby, D'Amato, and Metrailer 1976). Additionally, Wasby et al. (1992) find that oral arguments focusing on the procedural posture of a case have led to many of the Court's per curiam dispositions (see also Schubert et al. 1992).
Justices themselves substantiate the value of oral arguments for providing relevant information to them. For instance, Chief Justice Rehnquist explains that “if an oral advocate is effective, how he presents his position during oral arguments will have something to do with how the case comes out” (1987, 277, emphasis in original). Justice Brennan agrees: “I have had too many occasions when my judgment of a decision has turned on what happened in oral argument …” (Stern, Gressman, and Shapiro 1993, 732). Brennan further claims that, while not controlling his votes, oral arguments helped form his substantive thoughts about a case: “Often my idea of how a case shapes up is changed by oral argument …” (Stern, Gressman, and Shapiro, 732).
Beyond public statements, Justices Powell's and Blackmun's oral argument notes are replete with examples of how information from these proceedings helped them decide cases. For instance, in United States v. 12 200 Foot Reels of Film (1973) Justice Powell wrote, “[A]rgument was helpful, especially as a summary of previous law—read transcript.” Again, in EPA v. Mink (1973) Powell notes that Assistant Attorney General Roger C. Cramton provided an “excellent argument (use transcript if I write).” Similarly, after the respondent's argument in Jensen v. Quaring (1985), Justice Blackmun indicated that “This simplifies things for me.” As these examples indicate, information from oral arguments can influence how justices view a case.
Hypotheses
Based on the argument that information from credible sources helps political actors make decisions, combined with the argument that Supreme Court oral arguments provide information to justices, we test a number of hypotheses regarding the factors that affect Justice Blackmun's evaluation of attorneys' arguments. The first set of hypotheses focuses on verifying that his grades are based on the quality of the substantive arguments presented to the justices. We are especially interested in showing that these grades are not simply a function of Justice Blackmun's ideological proclivity to prefer one attorney's position over another position. Second, and more importantly, we use this evaluative measure to determine whether oral arguments influence the final votes of Blackmun's colleagues.
Probing the Quality of Oral Arguments
As we argue above, it is widely recognized that for information to be effective decision makers must perceive the source of the information to be credible or reliable (see, e.g., Austen-Smith 1993; Crawford and Sobel 1982; Farrell and Rabin 1996; Lupia and McCubbins 1998). The credibility of an information source hinges in part on whether the recipient believes the sender to be well informed and candid on the subject of the communication. The reason why is intuitive: if the receiver considers the sender to be ill-informed, then any information conveyed is likely to be discounted as being possibly inaccurate or misleading (Austen-Smith 1993).
In the context of the Supreme Court, a key indicator of credibility is the litigating experience of a lawyer, especially the extent to which he or she has appeared before the Court in the past. Indeed, one of the long-standing ideas in judicial politics is that repeat players, by virtue of factors including experience and resources, are more likely to enjoy litigation success (see, e.g., Galanter 1974; Vose 1959). McGuire (1993a, 1993b, 1995, 1998) finds that attorneys who litigate before the Court more frequently than their competitors are more likely to provide higher quality arguments and, as a result, are more likely to prevail in their case. As such, we hypothesize:
Litigating Experience Hypothesis. Attorneys with more experience arguing before the Court at oral arguments will earn higher evaluations from Justice Blackmun.
Solicitor General Hypothesis: Attorneys from the Solicitor General's office are more likely to earn higher marks from Justice Blackmun.
Federal Government Attorney Hypothesis: Federal government attorneys, other than attorneys from the Solicitor General's office, should receive higher grades from Justice Blackmun than other attorneys.
Elite Law School Hypothesis: Attorneys educated at more prestigious law schools are more likely to earn higher grades from Blackmun.
Former Clerk Hypothesis: Former Supreme Court clerks are more likely to earn higher marks than are attorneys who are not former clerks.
Washington Elite Hypothesis: Private attorneys from Washington, DC are more likely to earn higher evaluations than are attorneys from outside the beltway.
Law Professor Hypothesis: Law school professors who appear before the Court should garner higher grades than non-law-school faculty.
Amicus Attorney Hypothesis: Attorneys who participate at oral arguments on behalf of interest groups should garner higher grades at oral arguments.
Ideological Compatibility Hypothesis: Attorneys who present arguments ideologically closer to Justice Blackmun are more likely to earn higher marks for their oral arguments.
Explaining Final Votes on the Merits
Although it is important to understand how Justice Blackmun evaluated arguments, this does not answer our principal question: whether the quality of lawyers' oral arguments affects the likelihood justices will vote for the clients the attorneys represent. By taking this next step we are able to draw a direct link between the information attorneys present and the justices' votes.
Before proceeding, however, we must discuss our focus in this section. Clearly, the hypotheses in the previous section are directed at Justice Blackmun's behavior. We cannot, however, simply regress Blackmun's votes on his evaluation of the attorneys' arguments because of the inherent endogeneity we would face. That is, it is possible Justice Blackmun assigned higher grades to litigants for whom he anticipated voting. As such, we need to find another way to test whether votes are affected by the quality of oral arguments, as indicated by Blackmun's grading of attorneys. Although there are several solutions, we chose to examine the influence of oral arguments on all of Justice Blackmun's colleagues, thereby excluding him from the analysis.3
Another way around this problem would have been to examine Justice Blackmun's votes through the use of an instrumental variable regression. But this approach requires us to find a variable(s) that is (are) highly correlated with the quality of oral argumentation but uncorrelated with his vote in the case. We currently have no such variables that meet these criteria.
First, if oral arguments are integral for the Court, justices' decisions should be affected by the quality of arguments presented during these proceedings. Indeed, if the justices evaluate information from some attorneys as being more credible and therefore more reliable, they should be more apt to use it when deciding how to act. Thus, we hypothesize that:
Oral Argument Hypothesis: Justices will have a higher probability of voting for the litigant whose attorney provides better oral arguments.
Policy Preference Hypothesis: Justices are more likely to side with the attorney whose position is closer to their personal policy preferences.
The Conditional Effect of Oral Advocacy Hypothesis: Justices who are ideologically further from an attorney's position will be influenced less strongly by the quality of oral argumentation than justices who are ideologically closer to that attorney's position. Likewise, the positive relationship between a justice's ideological proximity to a litigant and his or her voting for that litigant will be weaker when the opposing counsel provides more compelling arguments.
While a number of factors may indicate that justices need additional information to decide a case, one of the most pertinent is a highly complex set of legal issues (Bailey, Kamoie, and Maltzman 2005; Maltzman, Spriggs, and Wahlbeck 2000). In fact, cases that come to the Supreme Court often focus on several issue dimensions, as well as on more than one constitutional or statutory question. It is these cases where lawyers, who often spend years researching the case, have more information than the justices, even after briefs have been submitted. Because of the need for information in complex cases, the quality of arguments should have a greater effect than in less complex cases.4
Case complexity should affect the extent to which the justices take oral advocacy into consideration, as manifested in their votes, but it should not affect Blackmun's evaluation of the quality of oral argument. Thus, we include case complexity in the vote model but not in Blackmun's evaluation of attorney arguments. For case complexity to affect Blackmun's evaluation of attorneys, he would have to evaluate all attorneys arguing in complex cases more highly (or lower) than attorneys in noncomplex cases. Instead we argue that justices, facing information asymmetry, will weigh highly credible information more heavily. The results in Table 2, however, do not differ much if we also include the complexity variable.
Conditional Effect of Information Need Hypothesis: The effect of oral arguments on justices is conditional on the complexity of the case under consideration.
DATA AND METHODS
To test the above hypotheses we analyze the grades Justice Blackmun assigned to attorneys during oral arguments in a random sample of 539 cases decided between 1970 and 1994.5
We used docket number as our unit of analysis, and over this time period the Court decided 3,755 cases with oral argument (full opinion, per curiam, judgment of the Court, or equally divided vote). Our data therefore represent about a 14% sample of the population of cases.
Note that our data include nine cases where Blackmun's case file contained more than one set of oral argument notes due to a reargument. In our first model, we include the grades from both arguments, but in the outcome model we obviously only include one observation for each justice in each case, and we use the data on the reargument to measure the quality of the oral argumentation. The results for the outcome model do not change if we instead drop reargued cases from the analysis. We also analyzed all cases where Blackmun had grades at both the arguments and the rearguments for an attorney. We found that a majority of the grades stayed the same or were actually lower on the second argument. This suggests that attorneys do not necessarily provide higher quality arguments at their second appearance in a case.
We included in the first model all attorneys who receive a grade, but excluded from the second model cases where Blackmun did not assign a grade to both the appellant's and the appellee's attorney. We do so because we must compare both attorneys' grades to assess the effect of oral advocacy on the justices' votes. One reason he may have failed to give grades in a particular case is that he may not have been fully engaged with the argument. For instance, in Local No. 82, Furniture & Piano Movers, Furniture Store Drivers, Helpers, Warehousemen & Packers v. Crowley (1984), Blackmun did not assign a grade to Mark D. Stern, the respondent's attorney. He wrote in his notes, “I am sleepy and drowsed off. Hope I was not observed by spectators or Rehnquist [who sat next to Blackmun].”
Dependent Variables
Oral Argument Grade
Using Justice Blackmun's grades as a dependent variable in the first model requires us to consider two key issues. First, it is possible that the grades in Blackmun's notes do not reflect his evaluation of the quality of the substantive arguments presented by the attorneys during orals, and are simply based on the attorney's rhetorical performance. Content analysis of Blackmun's oral argument notes for 70 cases (13% of the sample) demonstrates that it is the former and not the latter. Specifically, we coded each sentence in his notes for whether it discussed the substance of an attorney's argument or the presentation style of that attorney. We found that 95% (1,064) of the sentences in Blackmun's notes discuss the attorneys' substantive arguments, while only 5% (49) focus on stylistic comments about the presentation or on more general comments such as “bad argument.” These data overwhelmingly indicate Blackmun was concerned about the substance of arguments.
The second issue we must confront is that Justice Blackmun's grading system changed over the course of his tenure on the Court. In fact, he employed three different grading scales: A–F from 1970 to 1974; 1–100 from 1975 to 1977; and 1–8 from 1978 to 1993 (see Tables 1a–1c for the frequency with which he assigned grades to attorneys under each grading scheme as well as the mean and standard deviation for each scheme).7
The three different scales have similar distributions, as seen in measures of skewness, which assesses the degree of asymmetry, and kurtosis, which assesses peakedness. A kurtosis of 3 represents a normal distribution, and the A–F scale, 1–100 scale, and 1–8 scale, respectively, have a kurtosis of 3.4, 3.7, and 3.2. The respective skewness statistics for these three scales are −.39, .10, and .36. The negative value for the A–F scale indicates that a few more observations are at the low end of that scale, as compared to the other two.
To transform the alphanumeric scale into a numeric one, we converted an A to 95, an A− to 90, a B+ to 87, a B to 85, a B− to 80, and so forth. Occasionally, Blackmun assigned partial grades, specifically A−/B+, B−/C+, and C−/D; and we transformed them to 89, 79, and 69, respectively.
Our results are not sensitive to how we precisely measure these grades. Indeed, the results are largely comparable if we linearly transform the 1–8 scale into a 0–100 scale. These results and a replication dataset are available on the World Wide Web at http://www.polisci.umn.edu/faculty/tjohnson/.
Because the z-scores are on a continuous scale, we estimate an OLS regression. Additionally, because an attorney may appear multiple times before the Court we employ robust standard errors clustered on each attorney. We do so because, although our data contain 1,118 observations, only 867 different attorneys argued before the Court. Thus, clustering in this manner allows for errors to be correlated within a particular attorney across different cases.10
An alternative way to cluster would be on each case, which would allow the errors to be correlated across the different attorneys in the same case. The results are largely the same when doing so.
Justices' Vote
To evaluate the extent to which the quality of oral arguments affected the justices' decisions, we examine whether each justice, excluding Blackmun, voted to reverse the lower court decision. Using Spaeth (2004), we code such votes as 1, whereas votes to affirm are coded 0. Justices voted to reverse the lower court in 57.3% of the observations.
Independent Variables—Oral Argument Grades
Litigating Experience
Using McGuire's (1993a) definition of experience, we coded the number of times an attorney previously appeared before the U.S. Supreme Court at oral arguments.11
This measure of experience is well established in the literature (see, e.g., McGuire 1993b, 1995, 1998; Spriggs and Wahlbeck 1997; Wahlbeck 1998).
Because the log of 0 is undefined, we first added 1 to the number of prior appearances before the Court and then took its natural log.
Solicitor General
To determine whether the Solicitor General argued, we coded Justice Blackmun's oral argument notes as well as the Court opinions in either Lexis/Nexis or FindLaw to determine whether a title was attached to an attorney. Specifically, we looked for “Assistant Solicitor General” or “Solicitor General.” To account for the possibility that the Solicitor General provides higher quality arguments to the Court than other attorneys, we created two separate variables. First, if the attorney is the Solicitor General, we created a variable that is coded 1, and that is coded 0 for all other attorneys. We created a similar variable for Assistant Solicitors General; if they argue, it takes on a value of 1, whereas all other attorneys take on a value of 0. The Solicitor General constitutes 2.4% of the attorneys in our sample and Assistant Solicitors General appeared as the attorney in 13.0%.13
The literature on the success of the Solicitor General differentiates between the U.S. government appearing as an amicus curiae and as a litigant (see, e.g., Salokar 1992). We do not make such a differentiation because the imprimatur of the office will help the arguing attorney no matter the capacity in which he or she appears.
Federal Government Attorney
We searched Lexis/Nexis for any instances where the U.S. government is a party to the case and the Solicitor General or Assistant Solicitor General does not argue. Government attorneys are coded 1 and all other attorneys are coded 0. Approximately 12% of the attorneys in our sample represent the Federal Government but are not from the Solicitor General's office.
Attorney Attended Elite Law School
We obtained these data from Lexis/Nexis, Westlaw, or the Martindale Hubbell directory (the issue published during the year the case was argued). Attorneys who attended one of the elite law schools (Harvard, Yale, Columbia, Stanford, Chicago, Berkeley, Michigan, and Northwestern) are coded 1, whereas all other attorneys are coded 0.14
Although there are annual rankings of law schools, there are no rankings of elite law schools that span the long period of time during which Supreme Court advocates in our sample were trained. Although some may disagree with our identification of elite programs, one can be assured that the findings are not dependent on the exact specification of this variable. For example, we obtain the same result when we omit schools not routinely included in the recent top 10 (i.e., University of California, Berkeley) or when we add schools that are ranked highly today (i.e., University of Virginia, New York University, Duke, and the University of Pennsylvania).
Washington Elite
We coded the address for the arguing attorney as it appeared on the briefs submitted to the Court in each case. If an address was found in Washington DC, excluding federal government attorneys, this variable is coded 1. All other attorneys are coded 0. About 11% of the attorneys in our sample were private attorneys from Washington DC.
Law Professor
For this variable, we used the Martindale Hubbell directory for the year the case was argued, as well as the address listed on the briefs. If an attorney is listed as a professor at a law school, it is coded 1, whereas all others are coded 0. Nearly 2% of attorneys arguing before the Court were law professors at the time.
Attorney Argues for Interest Group
Sometimes the Court allows attorneys, beyond those representing the litigants, to appear at oral arguments. If the attorney appeared on behalf of an interest group as amicus curiae, we coded this variable as 1. All other attorneys are coded 0. Just over 1% of attorneys represented an interest group at oral arguments.
Former Court Clerk
We obtained these data from the Clerk's office at the U.S. Supreme Court. The list includes all clerks who worked at the Court from 1932 to 1991. Former clerks are coded 1, and all others are coded 0. Nearly 7% of the attorneys in our sample previously worked as a clerk for one of the justices.
Ideological Compatibility with Attorney
Finally, we measure the ideological compatibility of the attorney's position and Blackmun. Using the ideological direction of the lower court decision (Spaeth 2004), we determine whether an attorney represented the liberal or conservative position. Second, we employ Martin/Quinn (2002) scores to determine Justice Blackmun's year-to-year ideology over his entire Court career. Using a dynamic item response model with Bayesian inference, Martin and Quinn fit multivariate dynamic linear models to create measures of justices' ideology across time (for a full derivation of their procedure, see Martin and Quinn 2002). Larger values indicate that a justice is more conservative.
We matched our justice-specific ideology measure with the ideological direction of the attorney's argument, as described above. If an attorney argued for the liberal side in a case, we code this variable as the negative value of the Martin/Quinn (2002) score. Alternatively, if the attorney argued for the conservative side we used Blackmun's Martin/Quinn score. Higher values therefore indicate Blackmun is ideologically closer to the attorney's position. This variable ranges between −3.72 and 3.98, with a mean of .009 and a standard deviation of 1.06. This measure is consistent with work that examines the ideological relationship between justices and the Solicitor General (Bailey, Kamoie, and Maltzman 2005).15
If there is measurement error in our ideological distance variable, then Blackmun's evaluations of an attorney may be more heavily affected by ideological considerations than we report. We recognize that this measure is somewhat blunt, but current measurement technology does not offer a feasible alternative. Our proxy has been used in prior research (Sala and Spriggs 2004; Spriggs and Wahlbeck 1997) and is also analogous to a variable for the direction of the lower court decision because such a variable is a proxy for whether the petitioner sought a liberal or conservative Court outcome (e.g., McGuire 1995). Although there is some amount of error in our measure, we take comfort in how well it performs in our model that explains each justice's final vote on the merits. In that model, our measure of ideological distance correlates highly with the justices' vote on the merits. Because we do not expect the effect of measurement error to be significantly larger for the model explaining Blackmun's grades as compared to the one explaining votes, and because the measure of ideology works quite well in the model of votes, we infer that it is working reasonably well in the model explaining grades. In short, we do not think measurement error is masking any significant ideological bias in Blackmun's grading.
Beyond our variables of interest, we included Appellant Attorney as a control. Because the Court is predisposed to reverse lower court decisions (Palmer 1982; Provine 1980), we might expect Justice Blackmun to find petitioners' arguments more favorable. Thus, we include a dummy variable that equals 1 when the attorney represents the petitioner, and 0 otherwise.
Independent Variables—Final Vote on the Merits
Oral Argument Grade
We created a variable that compares the grades of the attorneys arguing each case. We did so by subtracting the appellee's grade, as measured by the z-score discussed above, from the appellant's grade. Larger values on this variable therefore indicate the appellant had the stronger oral argument.16
If more than one attorney argued on a side, which happens occasionally, we use the average of the grades earned by the attorneys on that side. The results do not differ if we instead use the maximum grade earned by the attorneys.
Ideological Compatibility with Appellant
We created a variable similar to Ideological Compatibility with Attorney, containing the Martin/Quinn (2002) score of each justice who sat during the time span covered by our sample. We first determined the ideological direction of both the petitioner and respondent based on Spaeth's (2004) measure of the ideological direction of the lower court decision. If the lower court made a liberal ruling, we assume the petitioner seeks a conservative outcome and the respondent seeks a liberal outcome from the U.S. Supreme Court. We then matched the Martin/Quinn measure with the ideological direction of argument we expect an attorney to make. If an attorney argued for the liberal side during a term, we coded this variable as the negative value of the justice's Martin/Quinn score. Alternatively, if the attorney argued for the conservative side we coded it as the justice's Martin/Quinn score. Higher values thus indicate a justice is ideologically closer to the attorney's position. This variable ranges from −6.15 to 7.15, with a mean of .473 and standard deviation of 2.55.
Ideological Compatibility * Oral Argument Grade
To test our expectation that the effects of Oral Argument Grade and Ideological Compatibility are conditional on one another, we include a multiplicative term, Ideological Compatibility * Oral Argument Grade. We expect this variable to have a positive coefficient.
Case Complexity
To measure the complexity of a case, we conducted a factor analysis of the number of legal provisions in a case and the number of issues involved for all cases decided by the Supreme Court between the 1946 and 1999 terms.17
This measure, or a variation on it, is used widely in the literature (e.g., Hoekstra and Johnson 2003; Maltzman, Spriggs, and Wahlbeck 2000).
RESULTS
Before moving to the impact of oral arguments, we first consider the determinants of Justice Blackmun's evaluations of attorneys' oral presentations. We posit that attorneys with more litigating experience, better legal education and training, and greater resources will receive higher evaluations because such attorneys will offer the Court more credible and compelling arguments than less experienced or less resourceful attorneys (McGuire 1993a, 1998). The results in Table 2 provide support for this expectation and, importantly, do indicate that Blackmun's evaluations are a reasonable measure of the quality of attorney's oral presentations.
As predicted, attorneys with more prior litigating experience before the Court present better oral arguments. To determine how the coefficient of .262 for Litigating Experience translates into points on the grade scale, we multiplied the coefficient by the standard deviation of the unstandardized grade variable on the 100-point scale, which is 6.28. Doing so indicates that a 1-unit change in this variable increases a lawyer's grade by 1.6 points on the 100-point scale, and a lawyer who has the maximum level of experience in our data earns an average of 7 points more than those with the minimum experience. Additionally, attorneys from the U.S. government, especially those from the Solicitor General's office, provide more compelling oral advocacy. When the Solicitor General personally argues, his grade is 2.3 points higher on the 100-point scale than our reference category of attorneys located outside of Washington.18
Our finding with respect to the Assistant Solicitor General variable is confounded by collinearity with past litigating experience. The Solicitor General himself argued, on average, 34.1 previous cases and the Assistant Solicitor General averaged 14.9 prior arguments. In contrast, the average civilian attorney had appeared before the Court in only 1.3 cases. If we omit the experience variable, Assistant Solicitor General is statistically significant. The remaining two insignificant variables in this model are not affected by multicollinearity.
Other factors of credibility also affect the quality of oral advocacy. For instance, attorneys who attended elite law schools earn grades 1.3 points higher than other attorneys.19
Some have argued that affiliations with law schools may communicate ideological information to the Court (Byrne 1993). Although some law schools have liberal or conservative reputations, alumni do not select cases strictly on ideological grounds. Graduates of Harvard and Yale, for instance, systematically represent parties on both sides of the ideological divide.
Finally, we turn to our test of whether messages from attorneys who put forward arguments more compatible with Justice Blackmun's ideological preferences will receive higher grades. The coefficient for Ideological Compatibility with Attorney is appropriately signed and distinguishable from a null effect. Substantively, a 1 unit increase in ideological compatibility resulted in a .32-point change in Blackmun's evaluation of the attorney's information when comparing attorney's transmitting information close to Blackmun and those who were more distant. When Blackmun was as ideologically distant from a litigant as possible, as compared to when he was ideologically aligned with a litigant, a lawyer's score only increased by about 1.2 points. Thus, Justice Blackmun's evaluations of attorneys are not greatly influenced by his own ideological leanings.20
We also measured ideology using Segal-Cover scores, and the results are similar to those in Table 2. They are available, along with the replication data, on the World Wide Web at http://www.polisci.umn.edu/faculty/tjohnson/. The advantage of the Martin/Quinn (2002) scores is that they vary over time, and conventional wisdom and the data indicate that Justice Blackmun became more liberal the longer he sat on the Court.
There is a possibility that lawyers' might pitch their arguments to the median justice on the Court, which might lead Blackmun to award attorneys higher grades when he occupied that position. To investigate the possibility of strategic attorneys, we included a dummy variable in our analysis for whether Blackmun assigned higher grades to attorneys when he was the median justice on the Court (1978 and 1979) than in other years. The data do not indicate that he gave lawyers higher grades when he was the median justice; and the other results in the model do not change when we include these variables. This result reinforces our finding for ideological distance by demonstrating that Blackmun did not give attorneys grades that were higher when they were likely pitching their arguments directly to him.
Combined, these findings suggest that the credibility of the attorney plays a role in Justice Blackmun's grading scheme. Taking each variable in isolation, as we have done, artificially diminishes the effect of credibility, because attorney profiles usually comprise combinations of these attributes. Take, for instance, three hypothetical attorneys: a very credible Solicitor General; a private, but credible, Washington attorney; and a less credible non-Washington attorney.22
We gave the Solicitor General the following attributes: the maximum value of experience for SGs, past law clerk experience, and graduation from an elite law school. The private Washington attorney was given the following characteristics: average experience of a Washington-based attorney; attendance at an elite law school; but not a Supreme Court clerk. The less credible non-Washington attorney had no prior Supreme Court experience, did not attend an elite law school, and was not a Supreme Court clerk. We held ideology constant at its mean of zero for each attorney type. To calculate the expected grade level, we multiplied the product of each coefficient and the standard deviation on the 100-point grade scale (6.28) by the designated value. We then add the mean of the unstandardized grade (78.8) and the regression constant to this product to arrive at the expected grade.
Do Oral Arguments Affect the Justices' Votes?
Now that we have established Justice Blackmun's grades as a measure of oral argument quality, we turn to whether these arguments affect his brethrens' votes on the merits of a case. The results in Table 3 show that the justices do indeed respond to the quality of oral argumentation.23
By excluding Justice Blackmun, we decrease the possibility that the oral argument measure is tainted by Blackmun's anticipated position in the case. Although our first empirical model shows that Blackmun's grading of attorneys was largely not influenced by his ideological orientation, we nonetheless think it best to exclude him from this analysis. If we include him in the analysis, however, the results do not change.
It is possible that attorneys get higher grades in cases in which they have the “better” legal position; thus the relationship we show here could reflect the effect of the legal and factual circumstances of a case. We think that the effect is more plausibly a function of attorney arguments than case facts. First, cases that are placed on the Court's docket and are decided with an opinion are by their very nature difficult ones that do not result in one litigant clearly having the better side of the case. Additionally, all of the existing accounts of fact patterns are only able to focus on one issue area in their analyses (Richards and Kritzer 2002; Segal 1984), and extending such an approach to an analysis of all issue areas before the Court would be inherently difficult. We are willing to bear the cost of not including facts in our analysis so that we can produce an analysis for the role of oral arguments that is generalizable across issue areas. Nonetheless, we did attempt to test for this possibility in this model by using certiorari votes. Our intuition is that cases with unanimous cert votes should indicate that the appellant has a strong case, whereas minimum winning cert coalitions should indicate a case in which the litigants have equally balanced legal and factual claims. The data do not offer much support to either idea, and, for example, unanimous cert coalitions do not lead to the appellant's lawyer receiving a higher grade or the appellant being more likely to win. The data indicate, however, that appellants are less likely to win when there was a minimum winning cert coalition. We also tested whether the “closeness” of a case might affect the measure of argument quality. To do so we included variables for whether there was a dissent in the lower court or conflict among the lower courts. In the grade model, grades are not affected by either lower court dissents or conflict. In the outcome model, a justice is less likely to vote for the appellant if the Court granted certiorari to resolve a lower court conflict. However, lower court dissents have no effect on how a justice votes. Importantly, our variables of interest do not change with the inclusion of any of the aforementioned control variables.
The predicted probabilities are based on the model with all of the control variables (column 3 in Table 3). Also, we set all other variables at their mean (or mode for a categorical variable) and we set each interaction term at the product of the values of its two component terms.
We set the Oral Argument Grade variable, respectively, at its maximum and minimum values in this example, holding everything else constant at the mean (mode for a categorical variable).
This result is obtained even if we use a quasi-instrumental measure for argument quality, that is, the attorney's grade purged of the effects of the variables specified in the first model. Specifically, we created this quasi-instrument from the residual of the regression in Table 1.
The effect of Oral Argument Grade is robust to how we estimate the model. However, the effect of the interaction term, Ideological Compatibility * Oral Argument Grade is less robust. If we attempt to control for potential heteroskedasticity in ways other than we used in Table 3 (robust standard errors clustered on justice), Oral Argument Grade's coefficient and confidence interval remain largely unchanged, whereas the interaction term's confidence interval widens. Whereas the interaction term is statistically significant if we use a heteroskedastic probit model (where heteroskedasticity is allowed to be in the Oral Argument Grade variable), it is only marginally statistically significant (p=.07) if we use a logit model with robust standard errors that are not clustered on justices, or if we use a logit model with robust standard errors clustered on Court cases (and include fixed effects for the justices) (p=.23). In these models, the Oral Argument Grade variable remains positive and statistically significant. Finally, if we run these models without the interaction term, Oral Argument Grade remains positive and statistically significant.
Is this finding an artifact of justices being more responsive to arguments provided by lawyers who advocate positions consistent with their policy preferences? Our data show that the effect of the difference between the petitioner's and respondent's oral argument quality varies with the justice's ideological support of the lawyer's position. Nevertheless, even though the impact of oral arguments is statistically different depending on the justice's ideological predilections, the effect of Oral Argument Grade is positive and statistically significant through nearly the entire range of Ideological Compatibility.29
Oral Argument Grade is statistically significant for 97.9% of the data, and remains positive, but not significant, for values of Ideological Compatibility with Appellant less than −4.5. Specifically, we cannot rule out the null hypothesis that the oral argument grades do not matter for Justice Douglas when lawyers represent litigants advocating conservative outcomes. This result does not indicate that ideologically distant justices are never influenced by the quality of oral arguments. The data, for example, do show that when justices such as Rehnquist, Brennan, or Marshall encounter a litigant advocating a position with which they ideologically disagree, they are influenced by the quality of oral argumentation. What it does imply is that Justice Douglas (here the most ideologically extreme justice) is not statistically significantly affected when facing attorneys advocating a position he ideologically disfavored.
The Effect of Oral Advocacy Conditional on Justice Ideology
For instance, consider a justice who is ideologically opposed to the petitioner (specifically, a justice who is 1 standard deviation below the mean on Ideological Compatibility with Appellant). This justice has a 32.2% chance of supporting the petitioner when the respondent presents oral advocacy that is considerably better than the petitioner's. By contrast, the likelihood of voting for the petitioner's position increases to 47.6% when this justice encounters a petitioner who outmatches the respondent's attorney at orals.30
We set the value of Oral Argument Grade 1 standard deviation above its mean when the petitioner's lawyer was better and 1 standard deviation below the mean when the respondent's attorney was better. All other variables were set at their means (or modal values for categorical variables), and we set each interaction term at the product of its two component variables.
Let us now consider the effect of the ideological distance between a justice and a litigant. Holding other variables constant at their means (or modes for categorical variables), a 1 standard deviation shift from below to above the mean of Ideological Compatibility with Appellant alters the probability of a justice voting to reverse from .396 to .763. Figure 2 provides a graphical depiction of the influence of ideological distance, conditional on the difference in the quality of the attorneys' oral arguments. It shows that the ideological distance between a justice and a litigant has a pronounced influence on the justice's vote. As we predicted, the effect of ideological distance is also conditioned by the relative quality of the attorneys' oral advocacy. For instance, consider a justice who is supportive of the petitioner's position (Ideological Compatibility with Appellant = 3). Under this scenario, when the petitioner's attorney is better than the respondent's a justice has an 81.0% chance of voting for the petitioner. When the respondent's attorney is better this drops to 70.6%.31
For this example and Figure 2, we manipulated the value of Oral Argument Grade from 1 standard deviation above the mean to 1 standard deviation below it.
The Effect of Ideological Distance Conditional on the Quality of Oral Arguments
The final element of our story does not work as expected. We argued that when justices have a greater need for information they would be more likely to listen to the information presented to them. Our measure of this condition, legally complex cases (as seen in Oral Argument Grade Compatibility * Case Complexity), is statistically insignificant. One reason why this variable may not support our argument is that it may not tap information asymmetry as much as it taps an information environment that is difficult for both attorneys and justices.
CONCLUSION
This analysis makes a notable contribution to the literature on U.S. Supreme Court decision making and has implications for our understanding of other elite decision makers. By providing the first systematic evidence that what transpires at oral arguments affects justices' final votes on the merits we show that elite decision makers can be influenced by those presenting arguments to them. These findings thus speak to analyses that focus on how information affects the decisions of members of Congress, the president, and other political decision makers. Like Supreme Court justices, other elite decision makers often face uncertainty in deciding important issues before them. Our data suggest that there is room for individuals to provide credible information that can influence the policy choices they make.
Let us consider two specific contributions of this research for the U.S. Supreme Court. First, we add a new level of understanding about how oral arguments affect justices who sit on the U.S. Supreme Court. Recent research provides evidence that justices seek information during oral arguments that can help them reach policy outcomes consistent with their preferences (Johnson 2001, 2004). Specifically, Johnson finds that a significant minority of the main arguments in majority opinions emanate from discussions during oral arguments. This result is buttressed by anecdotal correlations between oral arguments and the justices' ultimate response to those arguments (Benoit 1989; Cohen 1978; Greenhouse 2004; Shullman 2004; Wasby et al. 1992; Wasby, D'Amato, and Metrailer 1976). Our findings make significant advances over these existing studies. Most notably, we demonstrate that the justices are more likely to vote for the litigant whose attorney provided higher quality oral advocacy (as measured by Justice Blackmun), even after controlling for ideological considerations. This result is new to the literature and indicates that oral arguments do indeed enter into Supreme Court decision making.
Second, our study builds on recent research that shows how the process of decision making on the Court affects outcomes. Legal realists and their descendants, attitudinalists, place little explanatory emphasis on the process of decision making at the Court. Rather, their principal focus is on the individual ideological orientations of the justices as they relate to the facts of particular cases. In recent years, scholars have recognized the significant role of the Court's decision-making rules, norms, and procedures in influencing Court outcomes (Epstein and Knight 1998; Hansford and Spriggs 2006; Maltzman, Spriggs, and Wahlbeck 2000; Richards and Kritzer 2002). The oral argumentation process, however, has largely been ignored in these latter studies. Indeed, only a handful of analyses over the last couple of decades address the function of oral arguments at the Court. Our results therefore show that, like other elements of the Court's procedures, oral arguments matter to the decision in a case.
In sum, we show that the justices find oral arguments to be an important part of the Court's decision-making process, and that the quality of arguments, as measured by Justice Blackmun, affects the justices' votes. This result is important, and it should help build Supreme Court scholars' understanding of how the process of decision making affects outcomes.
A previous version of this paper was presented at the 2004 annual meetings of the American Political Science Association, Chicago, IL, September 2–5. We thank Ryan Black, Darryn Beckstrom, and Justin Wedeking for their research assistance. We also thank Jeff Segal, Lee Epstein, and Sara Benesh for helpful comments. Johnson thanks the National Science Foundation (IIS-0324992) and the University of Minnesota Department of Political Science (through its MacMillan Travel Grant fund) for partially funding data collection.
Comments
No Comments have been published for this article.