A major focus of judicial politics research has been the extent to which the Supreme Court is constrained or influenced in its decision making by external forces such as public opinion or institutional pressure from the coordinate branches of government. This question of external constraint has perennially occupied students of the Court because it has profound implications for the long-standing debates over the Court’s role as a policy maker in the American political system (Dahl Reference Dahl1957; Segal Reference Segal1997; Keck Reference Keck2007; Clark Reference Clark2009). The extent of judicial independence, from the people and from their elected representatives, suggests the extent to which the Court has a free hand to impose its policy preferences on the country. Thus, the possibility of constraint also has implications for the “countermajoritarian difficulty,” as a constrained Court is perhaps less “deviant” in a democratic polity than an unconstrained Court (Bickel Reference Bickel1986; Hall Reference Hall2016). If, however, a key benefit of an independent judiciary is insulation from routine political pressures, then a highly constrained Court may not “act according to the law” but rather attend to “the will of the political branches of government” (Breyer Reference Breyer1995, 989). As such, answering this question has important empirical and normative implications for American politics, broadly considered.
Most scholarship in this area has found that the Supreme Court is in fact constrained by external forces, at least under some circumstances (Clark Reference Clark2009; Casillas, Enns, and Wohlfarth Reference Casillas, Enns and Patrick2011; Segal, Westerland, and Lindquist Reference Segal, Chad and Stefanie2011; Hall Reference Hall2014). For this reason, recent research (e.g., Peabody Reference Peabody2011) has largely shifted focus from the fact of constraint to the conditions under which the Court is more or less constrained and to the mechanisms by which such constraints might operate. Today, most scholars agree that the Court is constrained by some combination of its desire for institutional maintenance (Friedman Reference Friedman2009; Segal et al. Reference Segal, Chad and Stefanie2011; Hall Reference Hall2014) and rational anticipation of possible adverse reactions, including sanctions and backlash (Clark Reference Clark2009; Casillas et al. Reference Casillas, Enns and Patrick2011; Strother Reference Strother2016).
In many of these accounts, case salience is theorized to play a key role in the operation of external constraint on the Court, as adverse responses to judicial decisions are thought to be much more likely in salient than nonsalient cases. This expectation has been supported by members of the Court itself; an anonymous justice stated in an interview that “we read the newspapers and see what is being said—probably more than most people do…. We know if there is a lot of public interest; we have to be careful not to reach too far” (Clark Reference Clark2009, 973). Several leading theories of judicial constraint find empirical support for this observation. In one recent study of constraints on the Court, Bailey and Maltzman (Reference Bailey and Forrest2011) find that the Court responds to changes in preferences in the elected branches and that these effects are most pronounced in salient cases. In another study, Hall (Reference Hall2014) finds that the Supreme Court is constrained in salient cases—especially in its decisions that cannot be implemented by lower courts. Hall (Reference Hall2014) draws on a key theoretical distinction developed in his 2011 book, between “vertical” and “lateral” cases to test this theory. Hall (Reference Hall2011) argues that the Supreme Court has significant power to implement its decisions in vertical cases (those concerning criminal or civil liability) because lower courts, over which the Court has significant hierarchical control, directly implement such rulings. Lateral cases, in contrast, must be implemented by other (i.e., noncourt) actors, over whom the Court has little power. As such, he reasons that the Court will be constrained by popular preferences and by Congress in its lateral decisions, precisely because implementation of those decisions relies on nonjudicial actors. Ultimately, Hall concludes that fear of nonimplementation is indeed a crucial factor motivating Supreme Court responses to external pressures.
These findings stand in stark contrast to those of Casillas et al. (Reference Casillas, Enns and Patrick2011), who contend that the Court is constrained in its nonsalient cases, as it seeks to avoid activating latent opinion that might erode its institutional legitimacy, but not in its salient cases, in which the policy stakes are high. Similarly, Bartels (Reference Bartels2011) finds that justices are significantly more likely to vote ideologically in salient cases, suggesting that the justices are not strongly constrained in such cases. The basic premise of this view is that the Court should be most attentive to public opinion in nonsalient cases because the Court must maintain its reservoir of diffuse support by saving countermajoritarian decisions for a select few cases that are particularly important or influential.
Thus, leading accounts of the operation of external constraints on the Supreme Court reach diametrically opposed conclusions. However, in testing these theories scholars have uniformly relied on postdecision measures of case salience—but postdecision salience is clearly endogenous to the decision rendered by the Court. Furthermore, postdecision salience is theoretically distinct from the salience of the case that came before the Court (Strother Reference Strother2017). Reliance on an endogenous measure of a key explanatory variable casts doubt on existing findings and suggests the need to revisit work on this important question. Indeed, it is possible that empirical support for these divergent views exists precisely because of reliance on this problematic measure.
In this article, I take a first step in this direction by replicating two important studies on external constraint, those by Casillas et al. (Reference Casillas, Enns and Patrick2011) and Hall (Reference Hall2014). The replication analyses yield mixed results in terms of the arguments offered in the original studies: I find support for the claim that public mood influences the Court in its salient decisions (supporting Hall, contra Casillas et al.) but do not find support for the proposition that the Court is influenced by Congress in its salient decisions (contra Hall). But while the analyses here are mixed with respect to earlier research, they resolve some of the empirical disagreements among those works.
Theories of External Constraint on the Supreme Court
Scholars have been drawn to questions regarding the separation of powers, including judicial independence and constraint, because of the theoretical and normative importance of their implications for American democracy. Studies in this rich tradition of scholarship have often found that the Supreme Court is constrained, at least to some degree or under some circumstances (Mishler and Sheehan Reference Mishler and Reginald1993; McGuire and Stimson Reference McGuire and James2004; Clark Reference Clark2009; Bailey and Maltzman Reference Bailey and Forrest2011; Casillas et al. Reference Casillas, Enns and Patrick2011; Engel Reference Engel2011; Segal et al. Reference Segal, Chad and Stefanie2011; Hall Reference Hall2014; Marshall, Curry, and Pacelle Reference Marshall and Brett2014; Bryan and Kromphardt Reference Bryan and Christopher2016), although a few contest this finding (e.g., Segal Reference Segal1997; Sala and Spriggs Reference Sala2004; Giles, Blackstone, and Vining Reference Giles, Bethany and Richard2008; Owens Reference Owens2010). As a result, many recent works have sought to uncover the mechanisms by which such constraints operate (Clark Reference Clark2009; Casillas et al. Reference Casillas, Enns and Patrick2011; Segal et al. Reference Segal, Chad and Stefanie2011; Hall Reference Hall2014).
One frequently hypothesized mechanism is rational anticipation by the justices of sanctions from the coordinate branches. Several scholars have highlighted the many tools available to Congress, in particular, to sanction the Court: jurisdiction stripping, impeachment, reducing the Court’s budget, court packing, altering decision rules, refusing to raise judicial salaries, eliminating courts or judges, ignoring decisions, tampering with bench size, and so on (Rosenberg Reference Rosenberg2008; Clark Reference Clark2009; Engel Reference Engel2011). In light of the reality of these formal checks, many scholars working in the separation of powers tradition argue that the Court’s policy decisions are significantly influenced by the possibility of congressional override (Ferejohn and Shipan Reference Ferejohn and Charles1990; Harvey and Friedman Reference Harvey and Barry2006). As Epstein, Knight, and Martin (Reference Epstein, Knight and Andrew2001, 591) put it, “Why would justices who are preference maximizers take a position they know Congress would overturn?” Indeed, this possibility of override may even influence the Court’s constitutional decisions (Dahl Reference Dahl1957; Meernik and Ignagni Reference Meernik and Joseph1997). However, the empirical case for the rational anticipation model is thin (Segal and Spaeth Reference Segal and Harold2002; Sala and Spriggs Reference Sala2004; Segal et al. Reference Segal, Chad and Stefanie2011).
Segal et al. (Reference Segal, Chad and Stefanie2011) argue that congressional overrides are not a meaningful threat to the Court and that their mere possibility is therefore unlikely to exert much influence over judicial behavior (see also Owens Reference Owens2010). They posit instead that the Court is motivated by a concern for “institutional maintenance”—a collective desire to maintain their institutional power and legitimacy by avoiding decisions that might prompt an attack from Congress (see also Harvey and Friedman Reference Harvey and Barry2006; Keck Reference Keck2007; Keck and Strother Reference Keck and Logan2016). More important decisions may be more likely to garner congressional attention, including rebukes, if the Court is not careful (Lindquist and Solberg Reference Lindquist and Rorie2007; Hall Reference Hall2014). As an example, Lindquist and Solberg (Reference Lindquist and Rorie2007) argue that the Court is likely to be more sensitive to the preferences of other actors when it is considering a constitutional challenge to a federal statute. Additionally, if the justices are concerned about the possibility of sanctions from Congress, they may also be more attentive to public opinion in such cases (Mishler and Sheehan Reference Mishler and Reginald1993; McGuire and Stimson Reference McGuire and James2004; Hall Reference Hall2014).
Some recent works have suggested that the Court is directly responsive to changes in public opinion, although again the mechanism through which this potential constraint operates is subject to debate (McGuire and Stimson Reference McGuire and James2004; Giles et al. Reference Giles, Bethany and Richard2008; Epstein and Martin Reference Epstein and Andrew2010; Casillas et al. Reference Casillas, Enns and Patrick2011). Giles et al. (Reference Giles, Bethany and Richard2008) suggest that justices respond to changes in public opinion through a slow process of individual attitude change. They contend that Supreme Court justices are subject to the same social forces as is the public at large, and thus their beliefs and attitudes evolve in a similar fashion to those of the general public. Casillas et al. (Reference Casillas, Enns and Patrick2011) offer a different mechanism: justices’ desire to maintain institutional legitimacy leads them to strategically respond to public opinion. Casillas et al. develop an original approach to control for “social change” in an effort to isolate the effects of attitudinal change and external influence from public opinion, and they conclude that public opinion constrains judicial decision making in nonsalient cases (75–78, 84–86). Relatedly, Bartels (Reference Bartels2011) argues that because the policy stakes are highest in salient cases, justices are most likely to vote attitudinally in those cases.
Hall (Reference Hall2014) offers a very different picture of the operation of external constraints on the Court. He argues that because research suggests that public support for the Court is both high and resistant to change (Gibson, Caldeira, and Spence Reference Gibson, Gregory and Lester2003; Bybee Reference Bybee2010), and because sanctions on the Court are so rare (Segal and Westerland Reference Segal and Chad2005; Owens Reference Owens2010; Segal et al. Reference Segal, Chad and Stefanie2011), neither of the above mechanisms is likely to significantly constrain the Court. Hall (Reference Hall2014, 354) posits instead that the “justices’ concern for institutional maintenance may be partially rooted in a fear of nonimplementation” of their decisions (see also Friedman Reference Friedman2009). Recurrent nonimplementation, he argues, might reduce the power and legitimacy of the Court in the long run (see also Cross and Nelson Reference Cross and Blake2000; Owens 2010; Casillas et al. Reference Casillas, Enns and Patrick2011). Importantly, while it may be quite difficult for Congress or the president to formally sanction the Court (Clark Reference Clark2009), it is relatively easy for either one to ignore the Court’s decisions (Rosenberg 2008; Engel Reference Engel2011).
In short, these leading theories of external constraint on the Court hold that case salience is a key factor in the operation of constraint from both mass and elite sources (Bailey and Maltzman Reference Bailey and Forrest2011; Casillas et al. Reference Casillas, Enns and Patrick2011; Hall Reference Hall2014; Bryan and Kromphardt Reference Bryan and Christopher2016). In these accounts, justices’ desire to maintain their reserve of diffuse institutional support, along with their fear of nonimplementation of their decisions, causes them to alter their behavior (votes) under some conditions. Some scholars, such as Casillas et al. (Reference Casillas, Enns and Patrick2011), argue that this constraint operates in nonsalient cases; others, including Bailey and Maltzman (Reference Bailey and Forrest2011) and Hall (Reference Hall2014), argue that it operates primarily in salient cases—especially salient lateral cases (Hall Reference Hall2014).
The Importance of Case Salience
It has been frequently noted that the salience of a given case may influence justices’ thinking and behavior with respect to that case (Bailey and Maltzman Reference Bailey and Forrest2011; Bartels Reference Bartels2011; Collins and Cooper Reference Collins and Christopher2012; Black, Sorenson, and Johnson Reference Black, Maron and Timothy2013; Clark, Lax, and Rice Reference Clark, Jeffrey and Douglas2015). This observation is an important part of the basis for the competing theories of the operation of external constraints on the Court discussed above. Some argue that the Court should be most responsive to the public in nonsalient cases because it must maintain its store of institutional legitimacy by rendering countermajoritarian decisions only in a few particularly important cases. Others suggest, however, that constraint from both public opinion and Congress should operate most significantly on the Court in the context of salient cases, perhaps because its unpopular, salient decisions might not be implemented by noncourt actors or because adverse decisions in these cases may be especially likely to cause political backlash.
These studies, however, may not have adequately accounted for the influence of case salience on Court action in their models. Salience is a latent quality that some cases possess—or perhaps that all cases possess in different measure. In general, this latent quality speaks to the contemporaneous importance of a given case. This importance may be due to legal or political significance, controversialness, expected impact, or other factors (Epstein and Segal Reference Epstein and Jeffrey2000; Collins and Cooper Reference Collins and Christopher2012; Black et al. Reference Black, Maron and Timothy2013; Clark et al. Reference Clark, Jeffrey and Douglas2015; Strother Reference Strother2017). Scholars have adopted a number of strategies to try to proxy this latent quality of interest, including media coverage (e.g., Epstein and Segal Reference Epstein and Jeffrey2000; Collins and Cooper Reference Collins and Christopher2012; Clark et al. Reference Clark, Jeffrey and Douglas2015), interest group participation as amicus curiae (e.g., Maltzman, Spriggs, and Wahlbeck Reference Maltzman, James and Paul2000; Collins Reference Collins2011), and the level of engagement individual justices display in cases (Black et al. Reference Black, Maron and Timothy2013).
Because salience is a key factor theorized to influence the operation of external constraint on the Court, specification and measurement of salience are critical. Virtually all research in this literature relies on Epstein and Segal’s (Reference Epstein and Jeffrey2000) case salience measure. Epstein and Segal’s measure is media based: cases are coded as salient if they appeared on the front page of the New York Times the day after the decision was announced and as nonsalient otherwise. Numerous scholars have pointed out that this measure, though widely used, has significant shortcomings (Collins and Cooper Reference Collins and Christopher2012; Black et al. Reference Black, Maron and Timothy2013; Clark et al. Reference Clark, Jeffrey and Douglas2015). For example, the Epstein-Segal measure is overly conservative and yields too many false negatives (Clark et al. Reference Clark, Jeffrey and Douglas2015). This is the case in part because space on the front page of the New York Times is both scarce and in high demand, and thus a multitude of considerations likely influence the content of the front page (Collins and Cooper Reference Collins and Christopher2012). This data set draws on only one news source and thus may be systematically influenced by internal idiosyncrasies (Collins and Cooper Reference Collins and Christopher2012; Clark et al. Reference Clark, Jeffrey and Douglas2015), and it may magnify natural fluctuations in the news cycle (Boydstun Reference Boydstun2013). Relatedly, the Epstein-Segal measure is dichotomous—cases are either salient or nonsalient—which masks the wide variability in salience actually observed in cases (Collins and Cooper Reference Collins and Christopher2012; Black et al. Reference Black, Maron and Timothy2013; Clark et al. Reference Clark, Jeffrey and Douglas2015).
Most crucially, though, its use is problematic in application to judicial decision-making research because salience is measured after the case is decided, and thus the salience captured by this measure could not have caused or influenced the decision. As Clark et al. (Reference Clark, Jeffrey and Douglas2015, 42) put it, “While claims of causation are always suspect in observational research, measuring a treatment by proxy after the observed outcome is a particularly troublesome form of posttreatment bias. If decisions and related choices affect the salience measure (coverage of the decisions themselves), then even if salience itself affects decisions or choices, our estimate of that effect can be biased in an unknown direction.” Epstein and Segal’s measure, if it is to be validly used as a measure of salience that might influence judicial decision making, relies on the assumption that salience before a decision correlates highly with postdecision salience. The Epstein-Segal measure and other measures of postdecision media coverage or importance capture the salience of the Court’s decision; measures of predecision media coverage or importance, in contrast, capture the salience of the case when it came before the Court. This distinction is crucial because the salience of any given Court decision is clearly endogenous to the decision rendered by the Court. Put differently, in deciding a given case, the Court itself may make that case much more (or less) salient than it was before having been decided (Strother Reference Strother2017).
This theoretical distinction points to the importance of the strategy employed to proxy salience. Epstein and Segal’s (Reference Epstein and Jeffrey2000) salience measure is a dummy variable that takes on a value of 1 if the case appeared on the front page of the New York Times the day after it was decided and 0 otherwise. That is, this measure clearly captures the salience of Supreme Court decisions, but it is less clear that it captures the salience of the case that comes before the Court.
Scholars have sought to address this issue by developing measures of salience of cases as they come to the Court. Clark et al. (Reference Clark, Jeffrey and Douglas2015), for example, also rely on media coverage, although their data include all media mentions of Court cases in the first section of the New York Times, the Washington Post, and the Los Angeles Times. In addition to drawing from a wider range of media sources, their data also cover a much wider time span: they collect data on cases from 1 year before the case is decided to 1 year after, and most importantly, they differentiate between coverage at different stages of a case’s life, including between pre- and postdecision coverage. These data offer a significant improvement over the Epstein-Segal measure for studies of judicial decision making because they allow the researcher to restrict the measure of salience to predecision salience and thus to test the effect of the salience of the case as it came before the Court. In other words, this measure avoids the endogeneity problem of postdecision measures in that it only captures case salience before a given decision is handed down by the Court.Footnote 1
As noted above, measures of postdecision salience, such as the Epstein-Segal measure, if they are to be validly used to proxy the salience that might influence judicial decision making, rely on the assumption that postdecision salience and predecision salience correlate at a high level. Pre- and postdecision salience do not correlate highly, however (Strother Reference Strother2017). When predecision coverage and postdecision salience are both specified using media measures, they only correlate at .23 (using Clark et al. [Reference Clark, Jeffrey and Douglas2015] and Epstein and Segal [Reference Epstein and Jeffrey2000], respectively, specifying both as dummy variables).Footnote 2
This low level of correlation is likely due both to the overly conservative nature of the Epstein-Segal measure and to the fact that pre- and postdecision salience of cases differ in important ways. Substantively, the theoretical distinction between the latent concepts captured by these measures, along with the low level of correlation between those measures, casts doubt on findings in which salience was argued to significantly influence behavior at the Supreme Court but which relied on an ex post measure of salience. As such, theories of external constraint on the Court must be tested using chronologically appropriate measures of salience.
To this point I have argued that because salience is a key factor theorized to condition the operation of external constraint on the Court, specification and measurement of salience are critical. The specification relied on in many important studies in the literature on law and courts is inappropriate for studies of judicial decision making, in that some phenomenon X at time t clearly cannot have caused or influenced Y at time t − 1. In the following section, I replicate two important studies of external constraint on the Court that reached opposite conclusions regarding the role of case salience in the operation of constraints on judicial action. I do so using a chronologically appropriate measure of case salience in an effort to shed new light on long-standing disagreements over the contexts under which and the mechanisms by which external constraints influence (or not) Supreme Court decisions.
Empirical Replication
In this section, I replicate two important studies of external constraints on the Supreme Court—those of Casillas et al. (Reference Casillas, Enns and Patrick2011) and Hall (Reference Hall2014)—using a theoretically and chronologically appropriate measure of case salience. With the exception of the salience measures, all data were obtained from the original authors’ replication files. To reiterate, because pre- and postdecision salience differ significantly, and because accurately distinguishing between salient and nonsalient cases is crucial to these theories of external constraint on the Court, I use Clark et al.’s (Reference Clark, Jeffrey and Douglas2015) measure of predecision salience.Footnote 3
In their study, Casillas et al. (Reference Casillas, Enns and Patrick2011) operationalize Court output in the aggregate, as the percentage of liberal decisions each term among all decisions that reverse the lower court’s ruling, in salient and nonsalient cases, respectively (see also McGuire and Stimson Reference McGuire and James2004). As such, in their analyses the dependent variables are the percentages of reversals of lower-court decisions decided in the liberal direction among cases that appeared on the front page of the New York Times the day after they were decided and among those that did not appear on the front page of the New York Times the day after they were decided. As argued above, however, postdecision salience clearly could not cause or influence judicial behavior in the decision-making process.
Thus, in order to retest their theory, I first construct new dependent variable series in which I differentiate between cases that were salient before they were decided, on the one hand, and cases that were not salient before they were decided, on the other. This approach requires dichotomizing the salience measure, in that it requires the creation of two discrete series delineated by their “salience.” For this purpose, I code the most salient 15% of cases in Clark et al.’s (Reference Clark, Jeffrey and Douglas2015) data set as “salient” and all others as “nonsalient.” This proportion matches the proportion of cases coded as salient in the Epstein and Segal data. As a robustness check, I also construct dependent variable series in which salience is defined as any case at or above the median level of predecision salience, and nonsalience is coded as below-median salience.
After creating the new dependent variables, I replicate Casillas et al.’s (Reference Casillas, Enns and Patrick2011) study. The measure of public opinion is Stimson’s (Reference Stimson1999) variable Policy Mood, which incorporates data on public opinion from hundreds of survey questions to capture the public’s changing preferences on a single liberal-conservative dimension. Virtually all studies of the influence of public opinion on the Court rely on this measure (McGuire and Stimson Reference McGuire and James2004; Giles et al. Reference Giles, Bethany and Richard2008; Casillas et al. Reference Casillas, Enns and Patrick2011; Hall Reference Hall2014). Segal-Cover median scores are used to control for the justices’ ideology (Segal and Cover Reference Segal and Albert1989). A major contribution of Casillas et al.’s (Reference Casillas, Enns and Patrick2011) article was their novel approach to control for “social forces,” which have long been theorized to influence both justices and the public at large: they use Martin-Quinn scores (Martin and Quinn Reference Martin and Kevin2002), which provide a dynamic measure of a justice’s revealed preferences and rely on the two-stage least-squares (TSLS) estimation to get around the potential problem of circularity introduced by use of the Martin-Quinn scores. In brief, TSLS generates predicted values of the Martin-Quinn score by regressing these scores on a series of exogenous instruments, in this case, the social forces theorized to simultaneously influence the justices and the public: the overall liberalness of national policy, changes in the economy, unemployment, inflation, income inequality, and the crime rate (for an extended discussion, see Casillas et al. [Reference Casillas, Enns and Patrick2011, 78]). The first stage of the model isolates changes in the ideology measure from changes in the measure of public mood; the second stage uses the predicted values of judicial preferences to assess the influence of the independent variables on changes in observed ideological outcomes of judicial decisions. All variables are coded such that higher values correspond to liberal preferences and outcomes, and lower values correspond to conservative outcomes.Footnote 4 Analyses include all formally decided cases reversing lower-court decisions from the 1956 through 2000 Supreme Court terms. Table 1 reports the results of the second stage of these TSLS analyses from the original study and the replications.
Influence of Public Opinion on Salient and Nonsalient Supreme Court Decisions While Controlling for Attitudinal Change, 1956–2000

Note.— Results from two-stage least-squares regression models of decision making by the US Supreme Court: the dependent variable represents the change in the percentage of liberal decisions issued by the Supreme Court each term (among reversals only) in nonsalient and salient cases, respectively. Standard errors in parentheses. Columns 1 and 2 (labeled “Original”) replicate the output from table 2 in Casillas et al. (2011, 84). The N in col. 6 differs from the others because in some years no cases decided (among the reversals) were in the top 15% of salient cases. Similarly, the N in col. 3 differs because in 2 years, no cases (among reversals) were below the median salience level. IV = instrumental variable.
+ p < .10 (two-tailed tests).
* p < .05 (two-tailed tests).
** p < .01 (two-tailed tests).
*** p < .001 (two-tailed tests).
The analyses in table 1 indicate some significant differences in outcomes based on the salience measure employed. First, I note that the original study replicated perfectly (i.e., the original data and code produce the reported findings exactly). Second, when the models are reestimated using a theoretically appropriate measure of salience, the original finding does not hold. Rather, in the models using both the median-plus or top 15% thresholds for salience, I find that the Court may be somewhat constrained in its salient decisions, but I find no strong evidence that it is constrained in its nonsalient decisions. Notably, the marginally significant influence of public mood on Court outputs in salient cases is more immediate than has been previously found in time-series studies (Giles et al. Reference Giles, Bethany and Richard2008; Casillas et al. Reference Casillas, Enns and Patrick2011).
Turning first to the models presented in columns 3 and 4 in table 1, which operationalize salience as cases at or above the median level of salience, I find that short-term changes in Public Mood exert a marginally significant effect on overall Court liberalness in salient decisions (p = .063) but that no such effect is discernible in nonsalient decisions. Substantively, the model indicates that as public mood becomes more liberal, the proportion of liberal Court decisions in salient cases increases, net of controls: a one-unit (i.e., a 1-percentage-point) increase in public liberalness results in a 1.8% increase in liberal decisions among salient cases. Additionally, these models do not indicate any significant long-run effects of public mood on Court outputs.
Next, when considering only cases in the top 15% of the salience distribution as “salient” (table 1, cols. 5 and 6), we see again that the influence of Public Mood on decisions in salient cases is only marginally significant (p = .097) in the short term. Substantively, the model indicates that as public mood becomes more liberal, the proportion of liberal Court decisions in salient cases increases, net of controls: a one-unit increase in public liberalness results in a 3.8% increase in liberal behavior in its salient cases. The main difference between the two sets of replication analyses lies in the findings with regard to the long-term effects of Public Mood: in these models using the higher threshold for salience, the long-term effect of public mood is marginally significant (p = .091) in nonsalient cases (i.e., in cases outside the top 15% of cases receiving the most media attention). In this regard, the finding is more similar to the original Casillas et al. finding than to the other specified replication, discussed just above.
In summary, these analyses do not strongly support the contention that the Court is constrained in either its salient or its nonsalient decisions; indeed, we cannot conclude definitively that salient and nonsalient cases are statistically distinct.Footnote 5 However, the balance of evidence here suggests that the Court may be modestly influenced by public opinion when deciding salient cases. When the models are specified with chronologically appropriate measures of case salience, Casillas et al.’s (Reference Casillas, Enns and Patrick2011) theory of constraint in nonsalient cases receives limited empirical support.
With these findings in mind, I turn now to replication of Hall’s case-level analysis of constraints on Supreme Court decisions. As noted above, Hall theorizes that the Court is constrained by fear of nonimplementation of its decisions and that this concern turns on case salience. He draws on the theoretical distinction between vertical and lateral cases to test this theory, as discussed in the previous section. From this, he reasons that the Court will be constrained by popular preferences and by Congress in its lateral decisions, precisely because implementation of those decisions relies on nonjudicial actors. Case salience is crucial to his analysis because, in his words, “first, my findings may be biased if lateral cases are more likely to be salient to the public and, therefore, attract public criticism. That is, the justices may fear public criticism rather than nonimplementation. Therefore, it is important to control for Salience. Second, my theory suggests that the justices primarily fear nonimplementation in important cases. Therefore, I expect constraint to be strongest in salient lateral cases” (Hall Reference Hall2014, 357).
Like Casillas et al. (Reference Casillas, Enns and Patrick2011), though, Hall (Reference Hall2014) uses Epstein and Segal’s (Reference Epstein and Jeffrey2000) measure to capture salience, raising the problems of causality and chronology discussed above. I retest Hall’s theory to see whether it holds when using chronologically appropriate measures of salience. As above, all other data and code are obtained from Hall’s replication files.Footnote 6 Most of the variables are identical to those discussed in the previous section. Congressional Ideology is measured using Poole and Rosenthal’s (Reference Poole and Howard1997) Common Space score for the median House and Senate members each year, which are then averaged (Hall Reference Hall2014). The Lateral variable is a dichotomous indicator coded as 0 (i.e., vertical) if the case involves potential criminal penalties, potential monetary damages, or internal administration of lower courts; it is coded as 1 otherwise (Hall Reference Hall2014, 356). The dependent variable in this part of the study is the ideological direction of each Supreme Court decision (liberal decisions are coded as 1, conservative decisions as 0; decisions for which ideological direction could not be determined were dropped). The sample includes 6,035 merits decisions issued by the Court between 1956 and 2007. These models include natural court fixed effects to control for the influence of changes in the Court’s composition.
Hall’s original results, as well as my replication using the Clark et al. salience measure, are presented in table 2. Again, I begin by noting that Hall’s original findings (col. 1) replicate when using his data and code. Columns 2 and 3 present two complementary replications of Hall’s study, using alternative specifications of the chronologically appropriate salience measure. The model in column 2 uses the continuous specification of the salience variable; column 3 uses a dichotomized version of the continuous variable, which codes the top 15% of salient cases as salient and the rest as nonsalient so that the proportion of cases coded as salient mirrors the Epstein-Segal measure.
Differential Impact of Supreme Court Constraint on Decision Making

Note. Results from logistic regression models of decision making by the US Supreme Court: the dependent variable takes on a value of 1 if the ideological direction of the decision is liberal. Natural court fixed effects are omitted for presentation. Robust standard errors in parentheses. Column 1 replicates the output from table 1 in Hall (Reference Hall2014, 359).
+ p < .10 (two-tailed tests).
* p < .05 (two-tailed tests).
** p < .01 (two-tailed tests).
*** p < .001 (two-tailed tests).
There are important similarities as well as notable differences when comparing Hall’s original analysis to my reanalysis that are attributable to the salience measures. I focus here on the three-way interaction terms because these are central to Hall’s (Reference Hall2014) theory. In all three models, the three-way interaction between Salience, Lateral, and Public Mood is positive and statistically significant, indicating, consistent with Hall’s theory, that Public Mood influences judicial decisions in salient lateral cases. Note, however, that the size of this effect is considerably smaller in the new analyses compared to the original. Because this three-way interaction is one of the two chief quantities of interest here, I also present the findings graphically in figure 1 in order to facilitate interpretation. The graph on the left in figure 1 reconstructs Hall’s (Reference Hall2014) figure 1; the graph on the right replicates the original but uses the Clark et al. (Reference Clark, Jeffrey and Douglas2015) measure of predecision salience. It is clear that the two figures are broadly similar. The main differences are that the graph constructed from the model using predecision salience indicates that the predicted probabilities of liberal decisions are about 10% lower than in the original analysis at each point on the scale and that there are substantially larger confidence intervals around the point estimates.Footnote 7 Even so, I find that Hall’s argument regarding public mood is supported when using a chronologically appropriate measure of salience: public mood influences Supreme Court decision making in salient lateral cases. A Wald test indicates that the effects of Public Mood in salient decisions are significantly different in lateral versus vertical cases (p = .0065, one-tailed test; for comparison, Hall finds p = .001, one-tailed test).
Left, original presented by Hall (Reference Hall2014, 360, fig. 1). Right, replication that replaces the Epstein-Segal (Reference Epstein and Jeffrey2000) salience measure with the Clark et al. (Reference Clark, Jeffrey and Douglas2015) predecision salience measure.

My findings regarding the influence of Congress on the Court depart somewhat from Hall’s (Reference Hall2014) original findings, however. The three-way interaction between Salience, Lateral, and Congressional Ideology is significant in Hall’s analysis but is not significant in the replication model relying on the continuous measure of predecision salience, and it is only marginally significant (p = .076) in the model using the dichotomous measure of predecision salience. As above, the size of the effect is considerably smaller as well.Footnote 8 Figure 2 depicts the relationship graphically, again comparing Hall’s original finding to my replication. As above, the key difference indicated by the figures is the size of the effect of Congressional Ideology in lateral salient cases: moving from the lowest to highest values of congressional liberalness in Hall’s analysis increases the probability of a liberal decision by nearly 50 percentage points; the corresponding change in the replication analysis would increase the probability by fewer than 25 percentage points. Moreover, a Wald test indicates no significant difference in the influence of Congressional Ideology in salient lateral and salient vertical cases (p = .180, one-tailed test; for comparison, Hall finds p = .030, one-tailed test).
Left, original presented by Hall (Reference Hall2014, 360, fig. 2). Right, replication that replaces the Epstein-Segal (Reference Epstein and Jeffrey2000) salience measure with the Clark et al. (Reference Clark, Jeffrey and Douglas2015) predecision salience measure.

In summary, the present replication reveals mixed support for two leading theories of external constraint on the Court. Hall’s (Reference Hall2014) theory regarding the importance of implementation in the operation of external constraint on the Court’s decisions is partially supported. When I reexamine Hall’s empirical account using a chronologically appropriate measure of a key independent variable, case salience, I find strong support for the proposition that public mood influences the Court in its salient lateral decisions but very little support for the claim that congressional ideology influences the outcomes of those decisions. The theory of constraint in nonsalient cases advanced by Casillas et al. (Reference Casillas, Enns and Patrick2011), in contrast, receives limited empirical support in the replication analysis.
Discussion
I have argued that despite the theoretical and normative importance of the question of judicial independence, there remains significant scholarly disagreement as to when and how Supreme Court decisions are influenced by extrajudicial forces. Notably, most accounts of the operation of external constraints on the Supreme Court have been empirically tested using an endogenous measure of a key variable. In light of the possibility that this scholarly debate is compromised by reliance on this problematic measure, in this study I replicated two leading works in this literature in order to see how the competing perspectives on constraint fare when tested using a chronologically appropriate measure of case salience. The replication analyses support the contention that public opinion influences the Supreme Court’s decisions in its salient cases, especially its salient lateral cases. However, I do not find support for the claim that the Court’s merits decisions are much influenced by congressional ideology.
Taking both replication analyses presented here as a whole, I find support for the broad claim that the Court is sometimes constrained in its decisions in salient cases, but I do not find support for the claim that the Court is constrained in its decisions in nonsalient cases. With that said, the analyses presented here suggest that the Court may be somewhat less constrained than previous studies have suggested. Specifically, I found that the influence of public opinion in salient lateral cases was significant but that the size of effect on the Court’s decision was somewhat smaller—roughly 10% smaller—than Hall found in his study. Consistent with Hall’s theory, I find that the Court is much less constrained by public opinion in vertical cases compared to lateral cases. Additionally, I found limited support for the claim that Court outputs are influenced by congressional preferences. The effect of Congress’s preferences in salient lateral cases (the theorized institutional context in which they should matter most) was marginally significant in one model and insignificant in the other. Furthermore, I could not statistically distinguish the effects of congressional ideology between salient vertical and salient lateral cases. In summary, these replications generally support Hall’s (Reference Hall2014) argument with respect to the influence of public opinion, although not with respect to Congress, on Court outputs; they do not support the theory of constraint in nonsalient cases advanced by Casillas et al. (Reference Casillas, Enns and Patrick2011).
These findings suggest that the Court may be more independent from congressional preferences than we have often thought and, by extension, that rational anticipation of overrides is not the primary mechanism by which external forces influence the Court (Owens Reference Owens2010; Segal et al. Reference Segal, Chad and Stefanie2011; Hall Reference Hall2014). The Court appears to be more independent, that is, with respect to the decisions it renders on the merits of cases it decides to hear. This suggests that the primary site of congressional influence over Supreme Court decisions is at the certiorari stage: previous work has found compelling evidence that congressional preferences do influence the Court’s decision to hear a case in the first place and in that way directly influence the content of the Court’s work (Harvey and Friedman Reference Harvey and Barry2006, Reference Harvey2009). This suggests an important avenue for future research, as the mechanisms by which congressional preferences influence the Court’s certiorari decisions are not well understood.
That public opinion influences Court decisions in its salient—especially its salient lateral—cases has important implications for our understanding of the nature of judicial independence and the likely pathways by which constraints operate. The observed importance of predecision salience suggests that justices do indeed attend to public interest in cases, particularly when noncourt actors must implement the Court’s decision. This finding supports Hall’s (Reference Hall2014) contention that fear of nonimplementation of its decisions is an important source of judicial attention to public opinion, as is the potential reduction in public legitimacy that may accompany nonimplementation. These findings should reinforce our commitment to studying how judicial behavior varies across different institutional contexts.
Finally, numerous scholars have pointed out the significant inferential problems created by using an ex post measure of case salience as an explanatory variable for judicial decision making. The analyses presented here lend credence to that concern, as replication of leading studies of the operation of judicial constraint yielded significantly different results when using chronologically appropriate measures of case salience compared to the original studies. More broadly, the findings presented here should motivate scholars to reexamine other theories in the field that rely on case salience. Indeed, any study relying on salience to explain judicial behavior should be replicated and validated using chronologically appropriate measures. Of course, measures of postdecision salience are important and appropriate for a wide range of studies—the claim here is simply that they are suspect in their application to studies of judicial decision making.



