Decision Making Using Advice

Experimental research has yielded several useful findings about the use of advice in decision making, much of which, we argue, is applicable to the relationship between a Supreme Court justice and a Supreme Court law clerk. First, using advice and combining the opinions of others to obtain that advice generally improves decision making (Yaniv Reference Yaniv2004a), and relying on evaluations of the source of the advice enhances efficiency (Chaiken Reference Chaiken1980; Mondak Reference Mondak1993b). Second, people tend to weight their own opinion more heavily than they weight others’ opinions, and the further advice is from their own preconceived notions in ideological space, the more discounted it becomes, especially for knowledgeable decision makers (Birnbaum and Stegner Reference Birnbaum and Stegner1979; Huckfeldt Reference Huckfeldt2001; Yaniv Reference Yaniv2004a, Reference Yaniv2004b). Third, people evaluate the sources of advice, considering whether or not the advice is biased, whether or not the source is an expert, and how far from their own preconception the advice seems to be (Birnbaum and Stegner Reference Birnbaum and Stegner1979; Huckfeldt Reference Huckfeldt2001). (They rely on experts more often, esp. when they are unbiased, but rely more heavily on even biased experts if they are in ideological agreement with them.)Footnote ³ Fourth, even knowledgeable decision makers use heuristics at least to some degree to aid in decision making, and sometimes those are based on certain characteristics of the source of advice, including perceived expertise and “likability” (Chaiken Reference Chaiken1980, 762), although high issue involvement leads decision makers to rely less heavily on such cues. Fifth, source credibility can matter greatly to the decision maker considering advice as a heuristic device. To wit, any message delivered by a credible source can constitute persuasive evidence for decisions involving low degrees of personal relevance (or when decision makers have little issue-specific information), while source credibility is part of the evidence for decisions involving high degrees of personal relevance (or for those holding high levels of information; Mondak Reference Mondak1990, Reference Mondak1993b).

The Case of Supreme Court Law Clerks and Certiorari Decision Making

Since Lazarus’s (Reference Lazarus1998) book (and before; see Rehnquist Reference Rehnquist1957), scholars and citizens have been interested in the degree to which law clerks influence their justices, usually for normative reasons. By Lazarus’s account, law clerks not only influence their justices (something Woodward and Armstrong [Reference Woodward and Armstrong1979] corroborate) but also produce the Supreme Court’s opinions. Wahlbeck, Spriggs, and Sigelman (Reference Wahlbeck, Spriggs and Sigelman2002) test that claim and show empirical evidence that law clerks have a stylistic influence over opinions written ostensibly by their justice. Ditslear and Baum (Reference Ditslear and Baum2001) demonstrate that the Court’s ideological polarization could be a consequence of increasingly ideological clerk hiring decisions, and Peppers and Zorn (Reference Peppers and Zorn2008) show that clerk ideology matters directly to the outcome on the merits (at least a little). We suggest that a focus on the stage at which clerks are most important, the certiorari stage, would be even more useful. Brenner and Palmer (Reference Brenner and Palmer1990) take such a tact using a list of votes in which Chief Justice Vinson followed his clerk’s cert recommendations kept by Vinson’s secretary. The availability of the Blackmun Papers, which includes the actual marked-up cert pool memos, means our investigation of clerk influence can go even further.Footnote ⁴

Specifically, we seek to determine the extent to which law clerks’ advice was used by Justice Blackmun in his decisions on cert. The institution of the law clerk was designed explicitly with workload reduction in mind, and the agenda-setting stage has often been its focus. Justice Lewis F. Powell Jr., credited with the genesis of the institution of Supreme Court law clerk in 1972, suggested that justices should pool their clerks to improve the agenda-setting process by providing edited, reliable information about the cases seeking Supreme Court review in a way that does not duplicate chamber effort (Ward and Weiden Reference Ward and Weiden2006, 45). Indeed, this certiorari stage is the most important reason that the law clerks continue to be used; their work in sifting through cert petitions is among their most important and helpful functions (Brenner and Palmer Reference Brenner and Palmer1990; Best Reference Best2002; Peppers Reference Peppers2006; Ward and Weiden Reference Ward and Weiden2006). That they produce a document at this stage—the cert pool memo—that has summary information as well as formal and specific recommendations to the justices provides further corroboration for our conceptualization of the law clerk as advisor (and ends up being the greatest advantage of a Supreme Court focus).Footnote ⁵ And the institution of law clerk itself has been professionalized, making reliance on clerk advice even more likely. Justices have long chosen graduates of the best law schools as clerks (Peppers Reference Peppers2006; Ward and Weiden Reference Ward and Weiden2006); today’s clerks have the added training of having clerked for a lower court judge as well (Best Reference Best2002; Peppers Reference Peppers2006; Ward and Weiden Reference Ward and Weiden2006).

Finally, that we are able to use a written record of the advice provided, signed by the advisor providing it, advantages our approach over scholars who sought to evaluate, for instance, the president’s advisors and their relative influence over executive decisions (e.g., Garrison Reference Garrison2001; Kassop Reference Kassop2013). In addition, it is very likely that this written advice is candid, given that the Blackmun Papers were the first to contain a full set of cert pool memos; their authors were writing only for the justices and not for posterity.Footnote ⁶

Two kinds of advice exist at this stage, though: the cert pool memorandum and the clerk markup. Although only the former has been so far considered in an empirical analysis (see Black and Boyd Reference Black and Boyd2011),Footnote ⁷ the literature on the role advice plays in decision making leads us to believe that the latter may be the more influential. We detail both pieces of information obtained by the justice, and we focus solely on Justice Blackmun since he is the only justice for whom the full complement of necessary information exists.

The Cert Pool Memo

As we know, some of the justices on the Supreme Court have chosen to join the Powell-created cert pool.Footnote ⁸ Clerks to those justices share the onus of reading and summarizing the thousands of petitions received by the Court each year. The result of that process is a formal document, the cert pool memo, which takes a consistent format regardless of author. The first page of every memorandum lists the parties, the lower court (and judges) that made the most recent decision on the case, and an issue categorization. The memoranda then universally include the following: (1) a detailed summary of the case; (2) the facts and proceedings below; (3) the contentions made by the petitioner and the respondent; (4) the clerk’s evaluation of the case; and (5) the recommendation by the clerk of what action should be taken. The Court directed clerks to include a recommendation as to cert as early as the second year the pool existed, suggesting that the justices welcomed this piece of advice (Ward and Weiden Reference Ward and Weiden2006). The justices (including Blackmun) also issued directives to the pool clerks from time to time, highlighting their desire for specific information from the memos, including, in Blackmun’s case, a discussion of dissenting opinions below and the names of the judges involved in the lower courts (Ward and Weiden Reference Ward and Weiden2006, 133).

Chambers Markup

In addition to this systematic pool memo, Blackmun (and most of the other justices) also requested that his clerks “mark up” the memo. In most cases, this markup included a handwritten paragraph directly on the cert pool memo, reacting to the information therein and ending with a recommendation as to cert. Blackmun’s clerk also added information on the memo’s author, including his last name, law school, and justice or judge for whom he clerked. While the memo has been said to be objective, the markup has been described as far more candid and justice specific (see, e.g., Ward and Weiden Reference Ward and Weiden2006, 130). This direct, specific advice is likely the reason for the markup in the first place (Ward and Weiden Reference Ward and Weiden2006). We distill both the memo author’s advice and Blackmun’s clerk’s advice into grant or deny from the 39 potential recommendations we encountered in these cases, enabling comparison of their independent effects.Footnote ⁹

Focusing on Justice Blackmun

We focus solely on Justice Blackmun for two reasons. First, Justice Blackmun had his clerks mark up the memos, so each memo in his papers contains both the recommendation of the cert pool memo writer and his own clerk’s. Second, Blackmun is the only justice with a complete set of publicly available papers at the cert stage. While we recognize this affects our ability to generalize to other justices on the Court, we sacrifice generalizability and its attending external validity for the ability to consider the differential effect of someone else’s clerk’s advice on a justice’s vote as compared with the effect of advice from a justice’s own clerk (i.e., internal validity). In order to be sure of the distinction we draw between a general clerk in the cert pool and Blackmun’s own clerk, we consider only those cases in which one of Justice Blackmun’s clerks was not the cert pool memo author. In addition to the empirical reality that we have markups only on Blackmun’s copies of the memo, we also feel most confident in the relationship between the theory and Justice Blackmun’s vote, given that we know that he requested exactly the sorts of information we might expect a decision maker seeking to evaluate advice might request. Importantly, Blackmun’s papers include both the memo recommendation and the recommendation of his own law clerk as to cert for both denials and grants.Footnote ¹⁰

But Justice Blackmun is clearly not the only justice who sought advice from clerks. As noted earlier, most of the justices require their clerks to mark up the cert pool memos (Peppers Reference Peppers2006), and nearly all of the justices have noted, either in internal Court correspondence or publicly, that they want advice from their clerks. One Rehnquist clerk tells Ward (Reference Ward, Peppers and Ward2012) that it was expected that he provide Rehnquist with his own views and a recommendation, and Rehnquist suggests that the impetus for the cert pool was so that justices could focus on the memo, alleviating the need to read the whole record (Ward and Weiden Reference Ward and Weiden2006). Best (Reference Best2002) deems it clear that the justices rely on their clerks at cert. As Schwartz and Lesher (Reference Schwartz and Lesher1993) say, the justices had to learn to rely on their staff, just like Congress does, when their workload increased. Justices outside the cert pool likewise found it necessary to delegate and sought clerk recommendations (Best Reference Best2002). Hence, while our singular focus on Justice Blackmun surely limits generalizability, there is little evidence to suggest that he is, himself, singular in his use of clerk advice.Footnote ¹¹

The existence of the pool, then, surely signified an institutional decision that the justices needed help (advice) in case selection, but the markups are particularly useful given the close relationship that exists between nearly every justice and her clerks. These advisors were particularly trusted to evaluate the cert pool memo and make a recommendation. Many former clerks speak very personally of their relationship with their justices (see, e.g., Bezanson Reference Bezanson, Peppers and Ward2012; O’Neil Reference O’Neil, Peppers and Ward2012; Ward Reference Ward, Peppers and Ward2012; Worthen Reference Worthen, Peppers and Ward2012). The personal, trusted assistant is just the right advisor to consider for our purposes and likely has analogues in other areas of government (e.g., presidential, vice presidential, and congressional chiefs of staff, among others).

Combining advice from the pool memo with the additional advice from Blackmun’s own clerks, then, we can not only measure the effect of the recommendation of the clerk writing the cert pool memorandum on Justice Blackmun’s vote (even after controlling for the case’s cert-worthiness) as Black and Boyd (Reference Black and Boyd2011) do, but we are also able to tap into the theoretical expectation that not all advice is equal, comparing the impact of the memo writer (weighted by the ideological distance between Justice Blackmun and her justice) with the effect of Justice Blackmun’s own clerk’s advice.Footnote ¹² Lazarus (Reference Lazarus1998) assumes that the cert pool memos carry great weight and suggests that it is possible that the ideological slant of the clerks had some effect, via their framing of the cases, on the way the justices viewed the potential cases. Indeed, this potential was what Rehnquist wrote about in 1957 in his article on the influence of law clerks (which touched off an ideological debate). This potential is corroborated by the fact that most of the justices, as mentioned above, considering the memos to be subject to manipulation by ideological clerks, had their clerks look them over and mark them up (and sometimes rewrite them), ensuring they were fair appraisals of the cases seeking review (Peppers Reference Peppers2006; Liptak Reference Liptak2008). Of course, others characterize these as the more objective piece of advice (Ward and Weiden Reference Ward and Weiden2006). Hence, while the cert vote may well be influenced by (or appear to be influenced by) the recommendation of the cert pool memo author (Black and Boyd Reference Black and Boyd2011), the justices are likely even more interested in the reaction to the pool memo of their own, most trusted advisor. We seek to test this idea for Justice Blackmun.Footnote ¹³

The Cert Literature

Scholars have long sought to explain and predict the Supreme Court’s decisions on certiorari, its agenda-setting behavior, and so we must control for alternative explanations for judicial votes on cert to isolate advisor effects (see, e.g., Provine Reference Provine1980; Palmer Reference Palmer1982; Caldeira and Wright Reference Caldeira and Wright1988; Brenner and Krol Reference Brenner and Krol1989; Caldeira Reference Caldeira1990; Krol and Brenner Reference Krol and Brenner1990; Perry Reference Perry1992; Boucher and Segal Reference Boucher and Segal1995; Benesh, Brenner, and Spaeth Reference Benesh, Brenner and Spaeth2002). The best of these studies considers both those cases granted cert and those cases denied cert, enabling its authors to fully model the decision by a justice to hear a given case (Caldeira, Wright, and Zorn Reference Caldeira, Wright and Zorn1999). Caldeira et al. (Reference Caldeira, Wright and Zorn1999) employ all those variables found to be determinants of cert by past research and combine them in a model that explicitly tests for the existence of sophisticated voting. They consider the United States as petitioner; lower court reversal; alleged and actual conflict; the presence of a civil liberties issue; amicus briefs for and against cert; whether a constitutional claim existed; whether there was dissent among the lower court judges considering the case; whether the lower court’s decision was conservative; the justices’ ideology; the ideological leaning of the Court; and the interaction between the leaning of the Court and the leaning of the justice. Controlling for all of these variables, as well as for the solicitor general’s view on the advisability of granting cert,Footnote ¹⁴ we seek to determine whether the role of advice ought be considered as well.

Operationalization of the Role of Advice

Relying on theoretical expectations as to the use of and evaluation of advice as well as the literature on certiorari, we model three stages: the memo writer’s recommendation as to cert, Justice Blackmun’s clerk’s recommendation as to cert, and Justice Blackmun’s vote on cert. This set of empirical analyses considers the real possibility, plaguing much research on the influence of clerks or other advisors, that the advisors (clerks) are simply making the recommendation that their boss (justice) would, driven by all the same influences as those driving the ultimate decision. Given the neat chronology of this series of decisions, we can be sure that influences are posited in the correct causal direction, and we are able to compare the influences on decision making across actors. We include, then, when relevant, ideological bias, source of advice, direction of the lower court’s decision, outcome prediction, and cert-worthiness. We detail each in this section.

First, it appears that decision makers generally suffer from a bias in favor of their own predilections, oftentimes discounting advice with which they disagree (Yaniv Reference Yaniv2004b). This is especially true when a decision maker has high levels of information or a personal interest in the subject; under those circumstances, they are less likely to rely on advice and more likely to make an independent decision. Using that logic, then, we expect that Justice Blackmun will be most influenced by advice coming from his clerk (with whom he likely shares ideology [see Ditslear and Baum Reference Ditslear and Baum2001]). In addition, to the degree to which Blackmun does consider the cert pool memo’s recommendation, he will be more likely to do so when he is closer to the authoring clerk’s justice’s ideology. Likewise, his clerk will likely consider the pool memo’s recommendation in light of the ideology of the justice for whom the author clerks. Not only does this fit the idea of bias of the decision maker, but it also fits with Calvert’s (Reference Calvert1985) analysis of biased advisors (see n. 3). We measure Ideological Disagreement using Martin-Quinn scores, calculating the absolute difference between Justice Blackmun and the memo writer’s justice in the term previous, such that higher numbers indicate greater disagreement (Martin and Quinn Reference Martin and Quinn2002). As distance increases, the advice provided by the clerk will be less influential on Blackmun’s clerk’s recommendation and, ultimately, on Blackmun’s vote.

In addition to an ideological bias, decision makers also consider the expertise of their advisors in making their decisions (Birnbaum and Stegner Reference Birnbaum and Stegner1979; Huckfeldt Reference Huckfeldt2001). This “source cue” is especially likely to influence decision makers with a need for efficiency (Mondak Reference Mondak1993a, Reference Mondak1993b). Considering the huge number of cert petitions received each year and the wide variety of topics addressed by them, it seems reasonable to assume that the justices would employ some shortcuts in ascertaining which petitions are worthy of review. Under this reasoning, we again expect Justice Blackmun to rely more heavily on his own clerks’ recommendations than on the recommendation of the clerks from other chambers, given that he will be more likely to know what has generated his own clerks’ advice than he is to know what generated the advice of clerks for other justices, and more likely to agree with it. Just as a person more likely values her own opinion given that she knows what biases and information went into its creation (Yaniv Reference Yaniv2004b), so too would the justices feel more comfortable using their clerks’ advice, given what they know about their level of knowledge and their ideological predispositions. Because of our singular focus on Justice Blackmun, we are able to include both the memo writer’s recommendation and Blackmun’s own clerk’s recommendation, via her marked-up pool memo. As noted earlier, we do not possess any other complete collection of marked-up pool memoranda for any other justice.

The clerks and the justice may well consider the lower court’s direction in their recommendation or decision, given the justices’ tendency to take cases in order to reverse them. We expect cases decided conservatively in the lower court to be more likely to attract a cert recommendation from Justice Blackmun’s clerk and a cert vote from Justice Blackmun than those decided liberally. The cert pool memo clerks should be less likely to recommend a grant under those circumstances, however, given that, overall, the justices in the cert pool during this time (including Justices Powell, White, Rehnquist, O’Connor, Scalia, Kennedy, Souter, and Thomas) were conservative. (We operationalize this measure as the directionality of the lower court decision, using Spaeth’s coding rules for liberal and conservative as our guide.)Footnote ¹⁵

In addition, outcome prediction—the looking forward to the likely outcome on the merits—should also matter to all of the decision makers. Measured as the distance between the justice and the Court’s median, this variable should be negatively related to cert. As the justice in question moves further from the Court’s median, he (or his clerk) will be less likely to vote for or recommend a grant.

None of these things, though, will alone drive either the recommendations of the clerks or Justice Blackmun’s cert vote. Indeed, the literature is replete with determinants of cert. We expect that aspects of the cases that contribute to their cert-worthiness will be an important predictor of the recommendation or vote at each stage in the advising process.Footnote ¹⁶

We know that decisions to decide matter greatly and that agenda setting is one of the major powers of the Supreme Court, one often credited for the Court’s unique policy-making position. We also know that setting the agenda, given the huge number of cases seeking review each year, is extremely time consuming.Footnote ¹⁷ And many who study clerks indicate that clerk influence is most likely felt during the cert process, with the clerks themselves corroborating this impression (see, e.g., Ward and Weiden Reference Ward and Weiden2006). That the research shows that using advice improves both the quality of decisions and the efficiency of decision making makes it quite reasonable for the justices to rely upon advisors to help them make good decisions on case selection. Ascertaining exactly how is our goal.

We evaluate the following expectations:

Data and Method

Using the Spaeth et al. (Reference Spaeth, Epstein, Martin, Segal, Ruger and Benesh2019) data for cert grants, along with the online archive of the Blackmun Papers (Epstein, Segal, and Spaeth Reference Epstein, Segal and Spaeth2007) for cert denials, we draw a sample of cert denials and cert grants in cases considered by the conference between October term (OT) 1986 and OT 1993.Footnote ¹⁸

We use the Blackmun archive for data on Justice Blackmun’s cert votes (taken from the docket sheets housed there) and for the cert pool memos with markup, also posted.Footnote ¹⁹ The Court’s advising mechanism unfolds sequentially. The memorandum writer provides a memo to the chambers of each member justice in the cert pool. One of the justices’ own clerks considers the memo, adds her comments, and makes a recommendation specific to her justice as to cert. The justice then must vote on the issue, taking into account all of the evidence before him—the pool writer’s memo, his own clerk’s markup, and the characteristics of the case at hand as detailed in the memo.

Because there are three stages in the cert decision, we estimate three models, one for each actor’s decision. Although all of the decisions are binary, a run-of-the-mill logistic regression is not well suited to this task given that, in our sample, the 1s (grants) are vastly overrepresented relative to their prevalence in the population. We would, as a result, get an inflated sense of the likelihood that cert would be granted. Instead, we use a rare events logistic regression, suggested by King and Zeng (Reference King and Zeng2001), which is designed to estimate models of case control designs, modified for our sampling focus (Court decisions rather than individual actor decisions).Footnote ²⁰

In order to do so, we first specify the true proportion of 1s in the population (King and Zeng call this number τ). From Wallander (Reference Wallander2014), we know that the Court granted certiorari to about 3% of cases over this period, so if we were modeling the Court’s decision to grant certiorari, we would set $\tau =.03$ . However, even though we know the aggregate values for the Court’s decision to grant certiorari, we do not know the exact values for each of the three actors involved. Further, we exclude cases that do not make the discuss list. Assuming that roughly 20% of the cases are discussed (Caldeira and Wright Reference Caldeira and Wright1990), that would mean that the proportion of grants among the discussed cases is .03/.20, or .15. Using this figure, we can estimate grant rates for the three relevant actors. For the memorandum writer, we know that of the 85% of actual denials, the memorandum writer recommends granting certiorari 19% of the time. Of the 15% of actual grants of certiorari, the memorandum writer recommends granting certiorari around 72% of the time. This means that we would expect the actual number of memorandum writer recommendations to grant certiorari in the population to be $.85\times .19+.15\times .72=.27$ . The same quantity calculated for Blackmun’s clerk is .24 and for Blackmun himself is .23. In other words, while the Court grants cert to 15% of these cases it discusses, the memo writer recommends a grant in 27%, Blackmun’s clerk recommends one in 24%, and Justice Blackmun votes to grant on 23% of petitions. We refer to these numbers as ${\hat{\tau }}_{\mathrm{MW}}$ , ${\hat{\tau }}_{\mathrm{BC}}$ and ${\hat{\tau }}_B$ , respectively.

If we were confident that these numbers were exactly right, we could specify each as τ in the relevant context and proceed with the case control analysis as suggested by King and Zeng (Reference King and Zeng2001). However, since we do not possess all of the data, we know these are only estimates, with associated sampling uncertainty. We will identify a particular model specification soon, but assume for the time being that we can estimate $\mathrm{Pr}\left(y=1|x,\tau ,\beta \right)=f\left(x\beta ,\tau \right)$ , where f(⋅) in this case represents the rare events logistic regression estimator. We can use this estimator to obtain the posterior distribution of β [ $p\left(\beta \right|x,y,\tau )$ ].Footnote ²¹ Rather than treating τ as fixed, we treat τ as stochastic and marginalize over its probability distribution:Footnote ²²

$p\left(\beta |x,y\right)=\int p\left(\beta |x,y,\tau \right)p\left(\tau \right)d\tau .$

To do this, we need a probability distribution over which to marginalize. Recall that τ is the proportion of 1s in the population. What we need to know is the overall probability of recommending or voting to grant, which can be calculated as $\mathrm{Pr}\left(\text{Recommendation}=\text{Yes}\right)=\mathrm{Pr}(\text{Recommendation}=\text{Yes}|\text{Cert Granted})\times \mathrm{Pr}\left(\text{Cert Granted}\right)+\mathrm{Pr}(\text{Recommendation}=\text{Yes}\left|\text{Cert Not Granted}\right)\times \text{Pr}\left(\text{Cert Not Granted}\right)$ . We estimate these remaining quantities with a Bayesian logistic regression model of each decision maker’s recommendation (deny or grant) on whether or not the case was actually granted cert (here, the independent variable). We use this to get the posterior distribution of $\text{p}(\text{Recommendation}=\text{Yes}|\text{Cert Not Granted})$ and $\text{p}(\text{Recommendation}=\text{Yes}|\text{Cert Granted})$ . We use these posteriors to induce a posterior distribution on τ by calculating the quantity for each draw from the two posteriors mentioned directly above. Thus, we marginalize over draws from the posterior distributions of τ for each decision maker in the appropriate model.

Pool Memo Writer’s Recommendation

Because the memo clerk’s recommendation comes first, only cert-worthiness and strategic considerations (with reference to her own justice) should matter to her, and because the cert pool memo is expected to be more objective, we expect the cert-worthiness of a case to matter the most at this stage. To that end, we adopt the modeling strategy suggested above and include as predictors the lower court direction, the memo writer’s justice’s distance to the Court median, and the case’s cert-worthiness. Following Black and Boyd (Reference Black and Boyd2011), we create a measure of the cert-worthiness of each case seeking review, operationalized as the predicted probability of a particular case obtaining review, estimated via a logistic regression including all known determinants of the cert vote.Footnote ²³ While there are potentially more flexible means for creating this variable relating cert grants to case characteristics (e.g., machine-learning techniques), we find that the logistic regression model performs sufficiently well that the value added of the extra complication is nearly zero.Footnote ²⁴

The first column of table 1 represents the posterior means of the coefficients from our first rare events logistic regression model (of the cert pool memo writer’s recommendation) along with the 95% highest density region.Footnote ²⁵ Here, we find, as expected, that cert-worthiness is influential while neither the direction of the lower court ruling nor the distance to Court median matters.Footnote ²⁶ Considering cert worthiness, its effect is .52, which is a substantively large effect as this effect is theoretically bounded in the range (−1,1).Footnote ²⁷ Substantively, court cases with maximal cert-worthiness have a probability of being recommended cert on average .52 higher than those with the lowest cert-worthiness scores (averaging over the original values of lower-court direction and distance to median). This suggests case-related considerations are vastly more important than strategic concerns related to the direction of the lower court’s decision or the ideological placement of the justice relative to the Court’s median for the memo clerk’s recommendation.

Table 1.

Results of Rare Events Logistic Regression Models

Note.— N = 625. Main entries are rare events logistic regression coefficients (i.e., posterior mean values) with 95% highest posterior density interval below in parentheses. Bold coefficients are those that have at least 95% of their posterior distribution on the same side of 0. This would be similar to identifying one-tailed significance at 5% in a frequentist model. Probability cutoffs for classification in PRE were those that maximized classification success across both 0s and 1s in the dependent variable. Predicted probabilities for PRE were calculated with coefficient posterior means and posterior means of cert-worthiness. PMC = pool memo clerk. PRE = proportion reduction in error. All distances are absolute values.

These findings are consistent with our first expectation. The only interesting effect (statistically or substantively) is the effect of cert-worthiness on the pool memo writer’s recommendation. Further, this suggests that the justices may reasonably view these memos as containing objective information, just as designed, since they appear to be largely uncontaminated by strategic considerations that might be relevant to the memo writer’s justice.

Blackmun’s Clerk’s Recommendation

Next, we model Blackmun’s clerk’s recommendation on cert, the results of which are shown in the second column of table 1. Many of our expectations are borne out here as well. First, strategic concerns matter for Blackmun’s clerk’s recommendation, evidenced by the statistical reliability of the coefficient on lower court direction. The effect of the lower court direction is −.19. This suggests that cases decided by lower courts in the liberal direction are less likely to be recommended for cert by Blackmun’s clerk. That said, not all strategic concerns are important. Blackmun’s distance to the court median has no discernible effect on the probability that his clerk recommends granting cert, demonstrating a potential lack of concern with the outcome on the merits.

Cert-worthiness and the pool memo recommendation matter as well. Since the effects of these variables are conditional and nonlinear, we present them in figure 1. Each panel of the figure represents a different condition of cert-worthiness. The upper-left panel considers cases with the lowest cert-worthiness; cert-worthiness increases as one moves right and then down in the figure. In each panel, the difference between the two lines represents the effect of the memo clerk’s recommendation. When the pool memo writer’s justice is proximate to Justice Blackmun, it is clear that the pool memo recommendation matters a great deal to Blackmun’s clerk. Note that even when cert-worthiness is low, a recommendation to grant cert from an ideologically proximate justice’s clerk results in an average first difference of around .35. That effect increases by a factor of 2 when the case is considered highly cert-worthy.

Figure 1.

Effect of cert-worthiness (increases from top left to bottom right), ideological distance between the pool memo writer’s justice and Blackmun, and the pool memo writer’s recommendation on the likelihood that Blackmun’s clerk recommends a grant.

The other interesting aspect of figure 1 is that the strength of the effect of the recommendation decreases precipitously as ideological distance increases. When the pool memo writer’s justice is ideologically distant from Blackmun and the case is not particularly cert-worthy, the pool memo writer’s recommendation has almost no effect (average first difference of around .10). That effect does increase with cert-worthiness, but even for highly cert-worthy cases, the effect of the pool memo recommendation is only around .3 when the pool memo writer’s justice is ideologically distant from Blackmun.

We can also consider the independent effect of cert-worthiness here. The effect here is .24, which is roughly half the size of the effect this same variable had on the pool memo writer’s recommendation (recall that that effect was about .52). Since cert-worthiness is only one of the pieces of information that Blackmun’s clerk uses, it is not surprising that its effect is smaller.

The results here, in concert with the results from the previous section, identify an interesting and predictable perceptual bias on the part of Blackmun’s clerk. While the memo writer is not shown to be influenced by strategic considerations, Justice Blackmun’s clerk still discounts the recommendation offered by the pool memo writer when his justice is ideologically distant from Justice Blackmun.

Blackmun’s Vote

Finally, we arrive at Blackmun’s vote on cert, which is obviously of greatest import to our analysis. We expect that Justice Blackmun will use the advice available to him, but he will do so differentially: he will rely much more on his own clerk’s recommendation than on that of the pool memo writer. Further, cert-worthiness should matter but less so for two reasons. First, the system of advice from the cert pool exists expressly because the justices do not have time to acquaint themselves sufficiently well with all of the information available on every case. Second, the advice coming from Blackmun’s own trusted advisor should be at least as important, if not more so, than his independent evaluation of a case’s cert-worthiness, given that Blackmun trusts his advisor, to whom he delegates the task of discerning to which cases he might vote to grant cert.

We recognize that as Blackmun evaluates all the evidence available to him, he may combine the pool memo writer’s recommendation and his own clerk’s recommendation in any number of ways. Initially, we estimated a model that allowed for a multiplicative relationship between Blackmun’s clerk’s recommendation, the pool memo writer’s recommendation, and the distance of the pool memo writer’s justice to Blackmun. We found that this model was not statistically different from one that does not include either the pool memo writer’s recommendation or the pool memo writer’s justice’s distance to Blackmun. Hence, we present results from the simpler model.Footnote ²⁸

Our model of Justice Blackmun’s vote is shown in the third column of table 1. A couple of interesting findings emerge. First, strategic considerations seem less important to Justice Blackmun’s calculus. The lower court direction, which had a reasonably large effect on his clerk’s recommendation, has no effect on Justice Blackmun’s own vote. It is also clear that outcome prediction—the ideological distance between Blackmun and the Court’s median—has no consistent effect either. Cert-worthiness has a similar impact on the justice as it had on his clerk. For Justice Blackmun, cert-worthiness has an effect of around .22. Highly cert-worthy cases have a probability of being granted cert of .22 higher than those that are highly uncert-worthy, based on the objective indicators.

Blackmun’s clerk’s recommendation has the biggest (and only other significant) effect on Justice Blackmun’s vote—an effect of .41. Blackmun is much more likely to vote for cert if his clerk recommends cert, and the effect of his trusted advisor is the strongest driver of his vote. To the extent that the Black and Boyd (Reference Black and Boyd2011) study is comparable to ours (and we would argue that they are aimed at fundamentally different goals), our findings suggest that their discovered effect of the pool memo writer on the cert vote is channeled through the advice of the justices’ own clerks. Once Justice Blackmun’s clerk distills the information into a recommendation, the pool memo recommendation has no further bearing on Blackmun’s decision. These results are consistent with our third expectation—that Blackmun will privilege his clerk’s recommendation over the recommendation of the pool memo author. In fact, Justice Blackmun discounts completely the pool memo writer’s recommendation.

In sum, this particular elite decision maker seeks and uses available advice, preferring the advice of his own advisor. That advice, in turn, matters independently and substantially to his vote choice. This is a finding that those studying advisors to presidents, for example, have been unable to uncover so far, because of a lack of data. We expect such a process may well operate in that and in other venues as well.

While we find the results of our study compelling, there are several caveats that are relevant regarding our results. First, as mentioned above, the generalizability to other justices is an open question. While we offer evidence that the other justices behave similarly in terms of the information they seek from their clerks, the extent to which they, like Blackmun, rely on their own clerk to the exclusion of the cert pool clerk is an empirical question. Second, there is relatively little information about the clerks themselves in the model.Footnote ²⁹ If particular clerks develop differential reputations for incisiveness and insightfulness in their recommendations as to cert, the extent to which their advice is heeded, either as cert pool memo author or as Blackmun clerk, may vary (and may even change over the course of their tenure, as they learn how better to influence the justices). Indeed, it may even be the case that there is variation across Blackmun clerks in terms of his acceptance of advice. However, given data limitations as well as the likely marginal influences these differences might represent, we argue that our model reasonably captures the effect of the average clerk. It may well be that there is more nuance to explore in this regard, but we leave that to future, more specific analyses of the individual clerks, which is beyond the scope of the data we collected.Footnote ³⁰

Conclusion

Students of the Supreme Court have spent much time considering the determinants of the vote to hear a given case, and for good reason, given how important this agenda-setting power the Court wields is. However, in all of this literature, the role of the Court’s advisors has been all but ignored (but see Black and Boyd [Reference Black and Boyd2011]). Indeed, in political science more generally decisions are modeled without explicit consideration of the advice that went into them. In this article, we offer an innovative theory of advice that helps explain the justices’ decisions over which cases to hear and realistically captures the process of decision making that takes place on the High Court, and we use an innovative methodology to carefully separate the influence of the clerk from the influence of the other known determinants of cert. The clerks in our story do not have undue influence, but they offer reasoned advice, which is relied on by their justices. This demonstrates that a justice of the Supreme Court, no less than most of us, seeking both accuracy and efficiency, uses the advice of trusted advisors in decision making. Had we information on markups for other justices, we do not doubt the advice given there would be preferred by justices to the pool memo as well. It is likely that nonpool justices, like Justice Stevens, were influenced by clerk advice as well. This is nothing like the portrait of rogue clerks attempting to influence their feeble justices presented by many critics of the law clerk institution (Woodward and Armstrong Reference Woodward and Armstrong1979; Lazarus Reference Lazarus1998). Instead, we argue, advice is often sought when important decisions are being made. Seeking out and using such advice is an indication of a smart, careful, busy decision maker, attempting to make better and more efficient decisions. We ought to consider that advice more often in explaining the resulting decisions.