Multilevel Logistic Regression Models

We estimate a series of multilevel regression models to demonstrate the effect of our key ideological explanatory variable at the justice level (Newspaper Ideology Score). Our models predict the likelihood of a liberal vote by a judge in an Australian High Court case between 1995 and 2019, taking into account the “crossed” multilevel data structure, where each case is seen by several justices, the justices sit on many cases across different areas of law, and the Court is presided over by different chief justices (Gelman and Hill Reference Gelman and Hill2006). This nonnested multilevel data structure is modeled with random intercepts across four levels: the justice ( $n=21$ ), area of law ( $n=33$ ), chief justice ( $n=4$ ), and the case itself ( $n=530$ ).Footnote ¹² The subsequent incorporation of a justice-level measure of ideology should increase the precision with which we estimate a given judge’s probability of making a liberal decision on any given case.

Our model specification may be summarized as follows:

$\mathrm{Pr}(y_i=1)={\mathrm{Logit}}^{\mathrm{-1}}({\beta }_0+{\alpha }_{j\left[i\right]}\mathrm{Justice}+{\gamma }_{k\left[i\right]}\mathrm{Case}+{\delta }_{l\left[i\right]}\mathrm{Issue}+{\phi }_{m\left[i\right]}\mathrm{CJ})\text{,}$

where α _j , γ _k , δ _l , and φ _m are random intercepts for justices, cases, issue area, and chief justice, respectively. Each set of intercepts is normally distributed with group-level explanatory variables specified at each level. For example, justice-level intercepts are the result of a justice-level regression:

${\alpha }_j\sim N({\Psi }_0+{\Psi }_1{\mathrm{ideology}}_j,{\sigma }_{\alpha }^2)\text{.}$

The intuition for this model specification is that the justice-level ideology variable will improve our estimation of each justice’s likelihood of voting liberally in two ways. First, it adds information to the model through the pooled “fixed effect” coefficient. Second, it accounts for the clustering of decision-making at the justice level and enables an improved estimation of the justice-level intercepts (when compared with models without justice-level predictors).

In building our model, we make three simplifying assumptions on case selection, covariate inclusion, and specification of random intercepts. First, as indicated above, we analyze nonunanimous cases only. In the appendix we discuss this reasoning further and provide supplementary analysis showing that there is no substantive change to the relationship between ideology and vote direction when including unanimous cases in the analysis. Second, we make an inclusive choice in considering the potentially different dynamics of cases with varying numbers of justices hearing each case. In contrast with the US Supreme Court, the Australian High Court sits in a variety of panel combinations when hearing cases. In fact, only 194 of 504 nonunanimous cases were heard by a full seven-justice panel of the Court, the others being heard by combinations of three, four, five, and six justices. Given the large number of cases heard by subsets of the Court rather than en banc, we included the entire corpus of High Court merits decisions in order to provide a robust analysis of High Court decision-making.Footnote ¹³ Third, for the sake of parsimony, our models contain a limited number of coexplanatory variables at both the individual justice level and case level that have been shown to influence voting behavior in apex courts and that do not introduce potential posttreatment bias.Footnote ¹⁴

The justice-level control variables we include are justice specialization, lower court experience, and gender. Specialization refers to the justices’ areas of legal specialization, for example, criminal law. Multiple studies have demonstrated that a judge’s issue-area expertise is a “primary determinant” of ideologically consistent behavior and that issue-specific expertise amplifies an individual justice’s ideological propensities (Judd and Downing Reference Judd and Downing1990; Miller and Curry Reference Miller and Curry2009). Further, specialization may lead to a direct effect on voting patterns if specialists see proportionally more cases in their area of specialization than other justices (Hanretty Reference Hanretty2020).

While specialization is generally measured after the appointment, based on a proportion of cases heard or opinions written in a specific area of law in the court being studied (Brenner Reference Brenner1984; Maltzman and Wahlbeck Reference Maltzman and Wahlbeck2004), to avoid posttreatment effects we measure specialization as preappointment, that is, an indicator of specialization before the justice takes a seat on the High Court (Hanretty Reference Hanretty2020). We propose a new measure of specialization, a justice’s legal practice area before any judicial appointment, and triangulate this with an existing measure, lower court specialization, which allocates specialization based on the proportion of cases each justice sat on in specific areas of law before the justice’s apex court appointment (Hanretty Reference Hanretty2020). To capture a justice’s legal practice specialization, we examine the records of each justice’s legal practice as a barrister to capture up to three areas of specialization and map these onto our six-category taxonomy of broad policy areas. We measure a justice’s lower court specialization by coding the legal issues for each justice’s 30 most recent cases on a lower court and again mapping these legal issues onto our six-category taxonomy. Both forms of specialization are measured as binary scores of whether a justice has above-average experience in each field, with a maximum three specializations for each justice. Given that four of the justices in our data set did not sit on appellate lower courts, legal practice specialization provides a more comprehensive measure than lower court specialization, and thus we present the results from legal practice specialization only.Footnote ¹⁵

We also include preappointment experience as an independent variable. Studies indicate that apex court judges who have served on lower courts before their apex court appointment are more likely to be socialized into their judicial role and consequently more deferential to precedent and less inclined to vote ideologically (Epstein et al. Reference Epstein, Landes and Posner2013). We account for this potential preappointment confounder and measure experience by the number of years the justice sat on any lower court before their appointment to the High Court. Justices with no prior appellate experience are included in the analyses and scored as 0.Footnote ¹⁶ As a final justice-level control, we include gender in light of a recent Australian study evidencing the impact of attorney gender on judicial voting behavior, as well as the various studies demonstrating that holding all else equal, female justices are more likely to vote in a liberal direction than male justices (Peresie Reference Peresie2005; Boyd, Epstein, and Martin Reference Boyd, Epstein and Martin2010; Smyth and Mishra Reference Smyth and Mishra2014).

In addition to justice-level confounders, we include two case-level control variables that studies have demonstrated affect judicial voting behavior in the US Supreme Court, namely, lower court direction and lower court dissent (Cross and Nelson Reference Cross and Nelson2001; Epstein et al. Reference Epstein, Landes and Posner2013). Both variables measure details about the decision of the court from which the case was appealed to the High Court. Lower court direction measures the ideological direction of each case in the court below. Prior studies have suggested that ideological distance between the lower court decision and the justice’s ideology increases the propensity to vote to overturn the lower court decision (Epstein et al. Reference Epstein, Landes and Posner2013). The ideological direction of the lower court decision was determined by the outcome in the High Court. If the High Court allowed the appeal, the lower court’s decision direction is the opposite of the Court’s, and if the Court dismissed the appeal, the lower court’s decision direction is the same as the High Court’s. Lower court dissent measures whether the decision of the court below the High Court was unanimous or nonunanimous. Prior studies have shown that the presence of a lower court dissent not only increases the probability of apex court review, but it also increases the probability of a unanimous apex court decision (Epstein et al. Reference Epstein, Landes and Posner2013; Beim and Kastellec Reference Beim and Kastellec2014). Further, the presence of a lower court dissent indicates that case is highly ideologically salient, given the propensity for lower court dissent aversion (Beim and Kastellec Reference Beim and Kastellec2014). Lower court dissent, then, is a binary variable that denotes the presence or absence of a dissent in the decision of the court from which the case in the High Court was appealed.

Table 4 presents three models of liberal voting in the High Court. Model 1 includes random intercepts and newspaper ideology scores only. Given the random effects, this foundational model incorporates a substantial amount of information about the likely direction of voting from a given justice, and, along with the newspaper ideology score, this model provides a robust estimate of any given justice’s probability of casting a liberal vote in any given case. Model 2 expands model 1 by incorporating multiple preappointment justice-level measures that may confound the relationship between ideology and voting. Model 3 additionally incorporates case-level variables and is the most comprehensive model, including both case-level and justice-level variables.Footnote ¹⁷

Table 4.

Nonnested Multilevel Models with Random Intercepts: Determinants of Liberal Votes

Note.—Significance presents estimates from nonnested multilevel models. Random intercepts specified are Case Number (n = 501 in models 1 and 2; n = 429 in model 3), Primary Issue Sub Area (n = 33), Chief Justice Reign (n = 4), and Justice (n = 21). AIC = Akaike information criterion. BIC = Bayesian information criterion.

* p < .05.

** p < .01.

*** p < .001.

In both models 2 and 3, ideology remains positive and a statistically significant predictor of voting behavior, even in the presence of numerous possible confounders. Gender has no observable effect on rates of liberal voting in model 2, and although the standard errors are too large to achieve statistical significance, the coefficients for gender are negatively signed and suggest that female justices do not disproportionately cast liberal votes. While the effects of specialization in specific fields are not easily interpretable, as specializations are measured as dummy variables and are not mutually exclusive, justice specialization in preappointment legal practice does not affect the relationship between ideology and voting, with the coefficient for ideology remaining strong and positive in both models 2 and 3. However, they do suggest that, net of judicial ideology and other justice-level factors, voting in criminal, public law, and common law cases is disproportionately conservative. Similarly, there is no relationship between lower court experience and the probability that a justice will vote liberally in either model 2 or 3. Liberal lower court decisions have a negative effect on liberal voting at the justice level, while nonunanimous decisions in lower courts positively predict liberal voting at the justice level.

Figure 1B provides substantive meaning to our regression results by plotting predicted probabilities at the justice level (using output from model 3), with whiskers indicating upper and lower 95% confidence intervals. Postestimation simulation gives the probability of a justice voting for a liberal outcome, conditional on a linear combination of coefficient effects, including justice ideology. Figure 1A gives a simple scatter diagram for comparison. It aggregates justice votes to the proportion of liberal votes over the course of each justice’s High Court career on the y-axis, with justice level ideology on the x-axis. The following section explores the relationship between ideology and specific types of cases.

Figure 1.

Predicted probabilities with judge-level ideology score, all nonunanimous cases. A, Scatter plot of justice ideological position and aggregate proportion of liberal votes by that justice. Black initials, Labor (left-wing) appointees; gray initials, Coalition appointees (right-wing). B, Predicted probabilities of a liberal vote for 21 justices of the High Court of Australia (model 3), conditional on their preappointment ideology. Initials, individual justices (see table 3). Black initials, Labor (left-wing) appointees; gray initials, Coalition appointees (right-wing).

Estimating the Relationship within Specific Policy Areas

We expect that ideology will find expression in each subset of the law, not simply in areas that are analogous to civil-rights-based cases in other legal systems. To explore this claim, we use the same specifications from model 3 and estimate the effect of ideology conditional on policy area. Table 5 presents the estimates from a varying slopes nonnested multilevel model. With civil rights cases as the reference category, we find significant reductions in the gradient of the slope for each non-civil-rights category, indicating that ideological voting is strongest in the civil rights category. Yet, while ideology appears to be strongest in the area of civil rights and vulnerable persons, figure 2 (with conditional predicted probabilities derived from model 4) indicates that a strong ideology effect is also present in other areas of law, indicating that ideological behavior is not conditional on an entrenched bill of rights or a strong rights culture. Common law cases (which include traditional common law issues such as tort, contract, equity, and trusts) also show strong ideological effects. This is largely unsurprising: traditional common law issues frequently involve claims reflective of rights-based claims, with a strong party versus an “underdog.” In addition, these common law categories involve “judge-made law,” in which the judiciary itself develops the law through its own decisions, rather than legislation creating those causes of action. The result is judicial decision-making free of legislative constraints and open to ideological persuasion.

Table 5.

Ideology and Liberal Voting, by Policy Issue: Logit Mixed Model with Random Intercepts

Note.—n = 2,416. σ2 = 3.29. p-values < .05 in bold. CI = credible intervals.

Figure 2.

Predicted probabilities for the probability of a liberal vote in a nonunanimous case, by policy area. Gray shading, 95% confidence intervals.

Similarly, public law cases involve few formal constraints on judicial actors; the High Court has the power of judicial review and supremacy in matters of constitutional interpretation, and the political branches cannot overturn the Court’s decisions except by passage of a constitutional amendment (Australian Communist Party v. The Commonwealth [1951] HCA 5, [1951] 83 CLR 1 [March 9, 1951]). Conversely, numerous studies have examined the extent to which judicial voting behavior is affected by informal constraints imposed by the legislative or executive branches in public law matters (Segal Reference Segal1997; Jacobi Reference Jacobi2006; Segal, Westerland, and Lindquist Reference Segal, Westerland and Lindquist2011). While we cannot state whether the Court is responsive to various informal constraints of the political branches or not, our analysis suggests that matters of constitutional law are inherently ideological.

Similar results are evident in criminal law and procedure cases: ideology is a strong and positive predictor of voting behavior. This is unsurprising, as criminal cases are often comparable to civil rights ones, involving decisions on the rights of defendants, challenges to jury decisions, sentencing decisions and procedures, pretrial conduct of judges and police, and admittance of evidence. It is intuitive, then, that liberal ideology generally would predict liberal voting on criminal cases.

Economic relations involves issues such as corporate law, taxation, bankruptcy, and property law. Comparatively, US and Canadian studies find mixed results in economic cases (Segal and Spaeth Reference Segal and Spaeth2002; Staudt et al. Reference Staudt, Epstein and Weidenbeck2006; Ostberg and Wetstein Reference Ostberg and Wetstein2007). Nevertheless, in the Australian context, we find evidence of a strong relationship between judicial ideology and voting behavior in economic policy cases. This finding is consistent with Myers (Reference Myers2020), who examined High Court voting behavior in the context of economic cases involving the federal government. Myers found that justices behave differently toward the federal government as a litigant in economic cases depending on the strength of the economy, concluding that the “results present evidence that challenges the … assumption that judges are motivated by economic factors and innocent of political or ideological considerations” (Myers Reference Myers2020, 48).

Finally, the coefficient for the ideology score in procedure and ethics cases is positive but significantly lower than in civil rights cases (and the lowest among the six policy categories). This category of cases largely involves civil procedure and evidence cases—those issues that are constrained by technical rules of procedure and evidence. This suggests that in cases in which justices are ex ante constrained by technical rules, and the issues do not directly involve the substantive allocation of benefits and burdens, justices are less willing (or able) to act ideologically.

Ultimately, while prior studies have demonstrated that evidence of ideological voting is most likely in rights-based claims (Segal and Spaeth Reference Segal and Spaeth2002; Epstein et al. Reference Epstein, Martin, Quinn, Segal and Hanson2012), our results show that ideological behavior is not conditional on an entrenched bill of rights or a strong rights culture. Instead, we largely replicate findings from the United States and other countries where protection of the underdog is a classic predictor of liberal voting behavior in rights-based claims across the corpus of decisions. Even more significant is the fact that these results occurred with a very general ex ante measure of ideology—a measure that includes a justice’s ideological predispositions on issues including, but not limited to, rights—whereas equivalent US studies measure ideology based entirely on rights-focused newspaper reports.

Accounting for Uncertainty in Measuring Judicial Ideology

Thus far our modeling strategy assumes perfect measurement of preappointment justice ideology when estimating its effect on voting behavior. To our knowledge, this is true of virtually all tests of the attitudinal model using newspaper measures of ideology (cf. Zorn and Caldeira Reference Zorn and Caldeira2008), despite the acknowledged need to account for measurement error in explanatory variables (Treier and Jackman Reference Treier and Jackman2008) and the explicit incorporation of measurement uncertainty into endogenous votes-based models of judicial ideology (Martin and Quinn Reference Martin and Quinn2002). Given considerable variability exists between justices in terms of the number of newspaper articles and ideologically relevant paragraphs that contribute to each justice’s ideology score, measurement uncertainty is a serious issue that may undermine the reliability of our ideology scores.

To this end, we propagate the uncertainty around the measurement of ideology by deploying a two-step strategy: first, we obtain simple estimates of uncertainty for each justice’s ideology score; second, we incorporate those uncertainty estimates into a justice-level Bayesian estimation of the effect of ideology on voting with a measurement model component.

First, we provide estimates of uncertainty of the measurement of judicial ideology. Since our ideology measures for each justice are bound between 0 and 1, a simple approach to provide a reasonable source of measurement error might be the standard deviation for a proportion, $\mathrm{SD}=\sqrt{i(1-i)/n}$ , where i is the estimate of the ideology score for each justice and n is the number of ideological paragraphs coded for each justice. However, given the presence of moderate paragraphs in each ideology score, the standard error for the proportion would overestimate the variance in an ideology score where there is a large proportion of moderate paragraphs for each justice. For example, in simple numeric terms, the standard error for an ideology score with 10 moderate paragraphs (score: 0.5), 0 liberal paragraphs (score: 1), and 0 conservative paragraphs (score: 0.5) would be 0, since there is no variance from 0.5 in the sample. However, when applying the standard error for a proportion, the estimate of the standard error would be 0.16.

We account for the impact of moderate paragraphs on ideological variability with resampling, estimating standard deviations with 10,000 bootstrapped ideology estimates for each justice. Table 6 shows how our estimates of standard deviation increase because of both smaller sample size (see Dawson, for whom there were only 10 ideological paragraphs available) and ideological inconsistency. For example, Chief Justice Brennan had 9 conservative paragraphs, 2 moderate paragraphs, and 10 liberal paragraphs. His standard deviation is similar in size to Dawson’s, despite having more than twice the number of ideological paragraphs.

Table 6.

Uncertainty in Measurement of Judicial Ideology, by Justice, with Bayesian Estimates

Second, we estimate a Bayesian model, similar to model 3, correcting for the measurement error in the independent variable by including a measurement model of the observed ideological positions, varying according to the estimates of their standard deviations from the bootstrapping procedure. The model was implemented using the measurement error option within the brms package in R (Bürkner Reference Bürkner2017). In the appendix, we provide a simple alternative specification with explicit parameterization, showing substantively similar results.

The results are presented in table 7. We find that model 5 is comparable to model 3, with similar sign, magnitude, and significance for each coefficient. However, when accounting for the error in measuring ideology, we find that the estimation of the standard error increases from .25 to .41, a factor of approximately two-thirds. Despite this increase in the standard error, we do not find evidence for an attenuation of effect size or an estimate of the coefficient indistinguishable from 0.

Table 7.

Bayesian Estimates of Justice-Level Liberal Voting Rates

Note.—N = 2,458. The model was run using Hamiltonian Monte Carlo (with 4,000 iterations reserved for burn in) and contained four MCMC chains run for 1,000 iterations each after burn in. Columns 3 and 4 display the 95% credible intervals for each regression coefficient.