2.1 Study 1: public opinion prior to the 2016 election

We begin by establishing the extent to which judicial appointments are important to voters. To do this, we use survey data from the NORC Center for Public Affairs Research (NORC). The survey was conducted between 15 September 2016 and 18 September 2016 via telephone and as a web-based survey and had a sample of 1022.Footnote ⁹ The survey asked about the importance of 18 different campaign issues (e.g., healthcare, crime, and unemployment), including Supreme Court appointments. Each respondent was shown a random subset of nine of these issues and asked to rate their importance on a 5-point scale ranging from “not important at all” to “extremely important.” After dropping respondents not asked about the Supreme Court, we are left with a sample of 513 individuals. Of those asked about Supreme Court appointments, 61.4 percent rated the issue as either very or extremely important and the modal response was very important. This appears to be fairly consistent among both Democratic and Republican respondents, as the percentages of the two groups rating Supreme Court appointments as either very or extremely important are 63.6 and 67.8, respectively.

Although these percentages may seem high, these evaluations only result in Supreme Court appointments ranking 13th of the 18 issues included. Given this, these raw importance scores should be considered in relation to all the issues rated. For example, if a respondent rates all eight non-Supreme Court appointment items as “extremely important” and also rates the Supreme Court appointment item as “extremely important,” this is less meaningful than a respondent who, on average, views the eight non-Supreme Court appointment items as only “somewhat important” but views the Supreme Court appointment item as “extremely important.” Based on this reasoning, we create a relative importance of Supreme Court appointments measure. To create this measure, we take the meanFootnote ¹⁰ importance score across the eight non-Supreme Court issues and subtract that from the importance rating the respondent gave to the Supreme Court appointments question. As such, positive values indicate that the respondent considered Supreme Court appointments to be more important than the average issue, whereas negative values indicate that the respondent considered Supreme Court appointments to be less important than the average issue. The distribution of our relative importance of the Court measure is presented in Figure 1. The mean is $-0.15$, the standard deviation is 1, and the median is 0.

Figure 1.

Distribution of relative important of judicial nominees. Dashed vertical line represents mean value.

To explain the variation in the relative importance given to judicial appointments, we estimate an ordinary least squares (OLS) linear regression model. The regression predicts the importance of judicial appointments as a function of an individual's partisanship, demographic traits such as age, gender, and race, and the mode by which the survey was administered.Footnote ¹¹ To capture respondent partisanship, we create six dichotomous indicators from the 7-point scale that runs from strong Democrat to strong Republican. In our model, pure independents are the omitted reference category. The results of the OLS regression are presented in column 1 of Table 1.

Table 1.

OLS importance of judicial appointments

Standard errors in parentheses.

Pure independents are the omitted reference category for partisanship.

*${\rm p}< 0.05$, **${\rm p}< 0.01$, ***${\rm p}< 0.001$.

Table 1 shows that when compared to pure independents, each group of partisans perceives the relative importance of judicial appointments to be higher (${\rm p}< 0.11$, in all cases. Two-tailed test.). To highlight this effect, we plot the predicted values by partisanship in Figure 2. But, to further probe into the differences between the two parties, we conduct $F$-tests to determine whether the coefficients for the partisan groups are different. We find that strong Republicans perceived judicial appointments as significantly more important (${\rm p} < 0.05$, in all cases. Two-tailed test.) than other partisan groups. Specifically, the predicted relative importance scores are $-0.08$ for strong Democrats, $-0.56$ for pure independents, and 0.34 for strong Republicans. This suggests that although both Democrats and independents generally perceived Supreme Court appointments as less important than other issues, strong Republicans viewed the issue as more important than the average. Thus, our results are consistent with both of our expectations and the general media narrative of the time that judicial appointments would be perceived as more important for Republican partisans.

Figure 2.

Effects of partisanship on relative importance of judicial nominations. Dashed vertical line represents the mean relative importance of judicial nominations. From Table 1.

However, it is worth noting that although the NORC survey shows some partisan variation in the importance of Supreme Court nominations,Footnote ¹² it does not allow us to test for the role of political knowledge. Although we control for education, research suggests that education may be a poor proxy for the actual concept of political knowledge (Highton, Reference Highton2009). Moreover, the NORC data do not allow us to test how these considerations relate to vote choice, as there were no questions about presidential or Senate candidate preferences. Therefore, in Study 2 we use an alternative data set to examine the hypothesized link between Supreme Court nominee preferences and voting.

2.2 Study 2: 2018 Senate elections

To examine whether the Supreme Court is an important issue for voters in Senate elections, we use data from the 2018 Cooperative Congressional Election Study (CCES).Footnote ¹³ We focus on all 2018 Senate races that featured a major-party incumbent opposed by a candidate from the opposite major party.Footnote ¹⁴ This leaves us with a sample of over 16,000 verified votersFootnote ¹⁵ from 27 states. Our dependent variable is whether the respondent voted for the incumbent candidate. This takes the value 1 if they did and 0 if they voted for one of the challengers.

All respondents were asked whether they supported or opposed the confirmations of Supreme Court nominees Neil Gorsuch and Brett Kavanaugh. We use respondents’ answers to these two questions to measure congruence with the votes cast by their sitting senators. Specifically, we create indicators for whether a respondent was congruent with 0, 1, or 2 judicial confirmation votes.Footnote ¹⁶ This measure of congruence serves as our independent variable in determining the role that Supreme Court nominations play in vote choice.

Since these confirmation votes were highly polarized and only three senators in our sample did not vote the party line,Footnote ¹⁷ congruence is significantly correlated with shared partisanship ($r = 0.72$). But, although the votes of the senators generally follow the same pattern, there is still significant variation in voters’ preferences and how they line up with the votes cast by their senators. This is evident in Figure 3, which shows the distribution of congruence for each senator–voter partisan combination. When looking at partisans evaluating an incumbent senator from the same party, 27.60 percent of Democrats and 21.08 percent of Republicans disagree with one of their senator's confirmation votes. Conversely, when looking at partisans evaluating an incumbent senator from the opposite party, 28.04 percent of Democrats and 22.25 percent of Republicans agree with one of that Senator's votes. Altogether, 28.64 percent of the partisans in our sample side with the opposite party on at least one of the two confirmation votes. This is a nontrivial figure, especially when considering that we have restricted our focus to just verified mid-term voters, a group comprised of people who should be significantly more partisan and engaged than the average citizen.Footnote ¹⁸

Figure 3.

Distribution of congruence with Senator's confirmation votes by incumbent party and respondent party.

One assumption of our test of whether congruence with judicial confirmation votes influence a voter's decision to vote for or against the incumbent is that voters know how their incumbent voted on these judicial confirmations. In many cases, this assumption may not be met. Prior research has demonstrated that the public is by-and-large uninformed about political events (Delli Carpini and Keeter, Reference Delli Carpini and Keeter1996). And when looking specifically at knowledge of six 2005–2006 senate roll-call votes, Dancey and Sheagley (Reference Dancey and Sheagley2016) report that the proportion of correct responses ranged from 0.31 to 0.57. Thus, we anticipate any effects of congruence should be conditional upon political knowledge. The CCES included six objective knowledge questions: the party in the majority of the US House, the party in the majority of the US Senate, and the party identification of the respondent's current US Senators, US House member, and governor. We code each as 1 if the respondent gives the correct answer and then take the mean across the questions answered. These knowledge items have a high degree of scale reliability ($\alpha = 0.86$). The resulting variable ranges from 0 to 1 and indicates a rather high level of knowledge in our sample (${\rm mean} = 0.72$; ${\rm standard\; deviation} = 0.342$; ${\rm median} = 0.83$).Footnote ¹⁹

To show how congruence on these confirmation votes compared to congruence on other roll calls, we also include a variable indicating whether the respondent's position on abortion after 20 weeks lines up with the way the incumbent senator voted on a motion of cloture on a bill that would ban such procedures. This will allows us to put our findings into perspective, as abortion is a highly salient and partisan issue. And to account for more general policy and partisan congruence, we include measures for partisanship and ideological distance. Partisanship is derived from the respondent's identification on the standard 7-point scale and is coded in the direction of the incumbent's party. Ideological distance is the absolute value of the difference between the respondent's self-placement on the 7-point ideological scale and where the respondent places the incumbent on that scale. Given that each of these measures is correlated with our measure of congruence,Footnote ²⁰ we also interact each with knowledge so as to offer a more complete test of our hypothesis. Finally, we include a number of demographic controls and incumbent fixed-effects that account for the uniqueness of each of these 27 different contests.

Table 2 column 1 shows the results of a logistic regression model for all survey participants. The large, significant coefficients for both ideological distance and partisan congruence are as expected. Yet, even in the presence of these significant effects, the judicial nomination congruence variables are still positive and significant, and as anticipated, this relationship is conditional upon political knowledge. We plot the relationship between judicial nomination congruence, knowledge, and incumbent vote choice in Figure 4. Figure 4 shows that, on average as a respondents’ political knowledge increases, they are more likely to reward or punish their Senators based on whether or not they agree with how the Senator voted in the judicial confirmation hearings. To draw a substantive example, we examine how the probability of an individual with the mean level of political knowledge (0.72) voting for the incumbent Senator varies across the number of congruence judicial confirmation votes. For an individual who is congruent with both votes, the probability of voting for the incumbent is 0.691 while the probability of voting for the incumbent is 0.584 and 0.460 for an individual who is congruent with one or zero judicial confirmation votes, respectively. These results demonstrate not only that congruence with judicial nominees is associated with an individual's decision to vote for or against the incumbent candidate, but that the strength of this association is particularly substantive.

Figure 4.

Effect of congruence with judicial nominees and political knowledge on incumbent vote. Results from Table 2, column 1.

Table 2.

Logit regression: incumbent

Standard errors in parentheses.

*${\rm p}< 0.05$, **${\rm p}< 0.01$, ***${\rm p}< 0.001$.

To further demonstrate the substantive effect of judicial confirmation votes, Figure 5 plots the marginal effect of congruence with one or two judicial confirmation votes and a cloture vote to ban abortion after 20 weeks. The cloture vote on the 20 week abortion ban is a good benchmark to compare the effect of Supreme Court confirmation votes for two reasons. First, abortion is a silent issue that shapes individuals political behavior in significant ways (Abramowitz, Reference Abramowitz1995; Layman and Carsey, Reference Layman and Carsey1998). Second, although the cloture vote was procedural and the substantive issue never came to the floor, the vote did receive substantial media coverage.Footnote ²¹ Figure 5 demonstrates that although congruence with the abortion ban cloture vote did increase the probability of voting for the incumbent Senator, the substantive effect is much smaller than congruence with judicial nominees. For example, at the average level of political knowledge, the effect of congruence with one judicial nominee is 4.55 times larger than the marginal effect of congruence with the abortion cloture vote, whereas congruence with two judicial nominees is 8.55 times larger. Thus, it appears that congruence with judicial confirmation is particularly salient in voters’ decision-making calculus relative to other Congressional votes.

Figure 5.

Marginal effects of congruence and political knowledge on incumbent vote choice. Comparing judicial nominees to abortion ban cloture congruence. Results from Table 2, column 1.

2.2.1 2018 Senate elections: partisan differences

Next, we subset the data by the respondent's party and re-estimate the model presented in Table 2. This allows us to determine if Supreme Court nominations are more important for Republican or Democratic partisans. These results are presented in Table 2 columns 2 and 3 and in Figure 6. The large partisan differences in the likelihoods of voting for the incumbent are likely due to the fact that only five out of the 27 races included in our sample featured a Republican incumbent. But, even with these differences, the results still demonstrate that congruence with confirmation votes—conditional upon political knowledge—is a significant predictor of vote choice for both Democratic and Republican partisans. An examination of the changes in probability, however, reveals that the results are also somewhat consistent with the expectation that the Supreme Court matters more for Republican partisans. For example, at the mean level of political knowledge (0.72), the change in the probability of voting for the incumbent that results from shifting from zero to two congruent votes is similar for both parties (0.251 for Republicans and 0.216 for Democrats). But, it appears that Republicans are more concerned with having their incumbent Senator congruent with both confirmation votes rather than just one. Shifting from one to two congruence nominees produces a large shift for Republicans (0.172) whereas the shift is much smaller for Democrats (0.027).

Figure 6.

Effect of congruence with judicial nominees and political knowledge on incumbent vote choice by party. Results from Table 1, columns 2 and 3.

To further demonstrate the substantive effect of confirmation votes across partisan groups, Figure 7 plots the marginal effect of congruence on one or two judicial confirmations and compares that to the marginal effect of congruence on the 20 week abortion ban cloture vote by partisan groups. Figure 7 demonstrates that although congruence with the 20 week abortion ban cloture vote did slightly increase the probability of voting for the incumbent Senator for both partisan groups, the substantive effect of congruence on the 20 week abortion ban cloture vote is again much smaller than congruence with judicial nominees.

Figure 7.

Marginal effects of congruence and political knowledge on incumbent vote choice by party. Comparing judicial nominees to abortion ban cloture congruence. Results from Table 2, columns 2 and 3.

Thus, it appears that both Democrats and Republicans perceive judicial confirmation and judicial candidates as equally if not more important than other political issues. Figure 7 further highlights the fact that Republicans were particularly concerned with being congruent with both confirmation votes relative to Democrats who appear mostly to be motivated by having at least one congruent confirmation vote.

2.3 Study 3: primary vote experiment

Although the results presented thus far generally support our claims, the observational nature of the data used only allow us to show associations between the quantities of interest. Thus, in our final study, we use a conjoint experimental design (Hainmueller et al., Reference Hainmueller, Hangartner and Yamamoto2015) to isolate the connection between the importance of judicial nominations and candidate preference. Conjoint experiments have become increasingly popular in the study of vote choice and candidate evaluation (Hainmueller et al., Reference Hainmueller, Hopkins and Yamamoto2014; Franchino and Zucchini, Reference Franchino and Zucchini2015; Kirkland and Coppock, Reference Kirkland and Coppock2018; Badas and Stauffer, Reference Badas and Stauffer2019; Ono and Burden, Reference Ono and Burden2019). Furthermore, Hainmueller et al. (Reference Hainmueller, Hangartner and Yamamoto2015) make the case for the external validity of conjoint experiments by demonstrating that the results they produce predict real-world behavior.

Our experiment specifically told participants that they were engaging in a primary election in which they would be selecting a candidate from their own party to run in a general election context against the opposition party.Footnote ²² The use of a primary election in which both candidates were from the subject's own party allows us to determine whether partisans select candidates based on their focus on judicial appointments. This question is distinct from Study 2 which asks whether voters reward or punish incumbent candidates based on their judicial confirmation votes.

Our design follows the conventional candidate selection paradigm of voter choice conjoint experiments in which participants are provided with two completely randomized profiles and are asked to select the candidate they prefer.Footnote ²³ Randomized profiles are constructed using attributes and levels. Attributes can be conceptualized as variables researchers are interested in studying. For example, candidate gender is often used as an attribute in conjoint designs. That attribute consists of levels or values that are displayed to participants. In the example of the gender attribute, the levels are often either female or male.

The focus of our conjoint experiment is an attribute which lists the candidate's number one issue priority. For both Republican and Democratic candidates, this attribute could take on one of 13 values. Eleven of these attribute values (listed in the Supplementary appendix) were included for comparison and varied by party to better match the appropriate platform and to increase the realism of the experiment. The remaining two values conceptualize a candidate's focus on Supreme Court appointments. For Republican candidates, the attribute can indicate that the candidate's priority is “confirming qualified Supreme Court nominees” or “confirming conservative Supreme Court nominees.” For Democratic candidates, the attribute can indicate that the candidate's priority is “ensuring only qualified Supreme Court nominees are confirmed” or “blocking conservative Supreme Court nominees.” The differences in how candidates’ preferences are framed were done to increase the external validity of the study. At the time of our study, Donald Trump—a Republican—was president. And because the president nominates judicial nominees, it would not make sense that a Democratic candidate would be running on a platform to confirm liberal nominees. Therefore, our design allows us to determine whether ideological cues are required to activate support for candidates who prioritize the Supreme Court. As Sen (Reference Sen2017) demonstrates, the public is most concerned with political signals when deciding to support Supreme Court nominees. Thus, it may be expected that voters only respond to candidates who explicitly mention the Court in relationship to ideology. Our designs allow us to test this. In addition to our attribute indicating the candidate's number one issue priority, our conjoint experiment includes attributes on the candidate's age, gender, race, education, experience, recent media topics, chances of winning the general election, family life, and military experience.Footnote ²⁴

We fielded our conjoint experiment as part of a survey fielded on Amazon's Mechanical Turk (MTurk) between 25 January 2019 and 1 February 2019. To be eligible for participation, individuals had to be located in the United States and at least 18 years of age. Conducting research on MTurk has some potential limitations. Berinsky et al. (Reference Berinsky, Huber and Lenz2012) show that MTurk samples tend to not be representative of the United States population and thus the external validity of studies conducted on MTurk may be diminished. However, despite non-representative samples, many studies have demonstrated that MTurk produces consistent results that replicate across nationally representative samples (Thomas and Clifford, Reference Thomas and Clifford2017; Coppock, Reference Coppock2019). To further ensure data quality, we use the protocol developed by Winter et al. (Reference Winter, Burleigh, Kennedy and Clifford2019) to ensure participants are located in the United States and that each participant completed the survey only a single time. We recruited 779 participants who each viewed five sets of candidate profiles. Our sample consisted of 517 Democrats and 262 Republicans.Footnote ²⁵ Although Republicans are underrepresented, the repeated measures give us sufficient statistical power. Moreover, findings that liberals and conservatives on MTurk closely mirror liberals and conservatives in the general public (Clifford et al., Reference Clifford, Jewell and Waggoner2015) should help ease concerns about the generalizability of our results.

To test whether voters are responsive to a candidate's discussion of the Supreme Court, we compared marginal means (Leeper et al., Reference Leeper, Hobolt and Tilley2020). Due to the forced choice nature of our candidate selection experiment, the overall mean of a candidate being selected is 0.50. The marginal mean represents the probability of a candidate being selected conditional upon an attribute level being displayed. Means above 0.5 indicate that the particular attribute level increases support for a candidate while means lower than 0.5 indicate that the particular attribute level decreases support for a candidate. Due to the complete randomization of profiles, each attribute level is independent of all other attribute levels.

2.3.1 Republican participants

We first analyze the Republicans participants. The marginal means and their 95 percent confidence intervals are displayed in Figure 8. For ease of visualization, only the issue priority feature is displayed. Full results are available in the supplementary appendix. The results indicate that Republican participants are more likely to favor candidates who mention the Supreme Court. However, this effect is limited to candidates who explicitly mention confirming conservative Supreme Court nominees. Republican primary candidates whose number one issue priority is confirming conservative Supreme Court Justices are selected 57 percent of the time, or 7 percent more than the baseline (${\rm p}< {0}.05$, two-tailed test). Candidates mentioning confirming qualified Supreme Court nominees did not see an increase in support. Those candidates were selected 47.4 percent of the time, which did not significantly differ from the baseline of 50 percent. Furthermore, confirming conservative Supreme Court nominees was among the most popular priority for all Republican candidates and one of few that significantly increased support from the baseline.Footnote ²⁶ These results further validate our observational survey data, which demonstrated that Republican voters viewed the Supreme Court as an important issue. Additionally, these results support the general findings of Sen (Reference Sen2017) who found that the public were particularly concerned with judicial ideology when evaluating potential judicial nominees.

Figure 8.

Results for Republican participants.

2.3.2 Democratic participants

Now we turn to the Democratic participants. The results are presented in Figure 9. For ease of visualization, only the issue priority feature is displayed. Full results are available in the supplementary appendix. Contrary to what we found with Republican participants, Democratic participants do not respond positively to mentions of Supreme Court nominees. Democratic candidates whose number one issue priority was “ensuring only qualified Supreme Court nominees were confirmed” were selected 46 percent of the time, which did not vary significantly from the baseline of 50 percent. On the contrary, Democratic candidates whose number one issue priority was “blocking conservative Supreme Court nominees” were punished by Democratic participants and selected just 39.1 percent of the time, which represents a significant departure from the baseline (${\rm p}< {0}.05$, two-tailed test). Furthermore, blocking conservative Supreme Court nominees was the least popular issue priority for Democratic candidates. Although this result may be somewhat surprising, the survey experiment was conducted after President Trump and the Republican controlled Senate had confirmed both Neil Gorsuch and Brent Kavanaugh to the Supreme Court. Considering President Trump and the Republican Senate's success confirming two conservative nominees, it may be the case that Democratic participants saw this as a losing issue and that reflected poorly on the candidate. Moreover, the differences between a majority party (i.e., the Republicans) issue statements and a minority party (i.e., the Democrats) issue statements may have elicited different responses. That is, although our Republican treatments emphasized ideology and the positive forward movement of an agenda, the Democratic treatments necessarily involved obstruction that may have been viewed as partisan. This is important, as prior research indicates legislative behavior perceived as overly partisan or as contributing to gridlock is more likely to be punished (Carson et al., Reference Carson, Koger, Lebo and Young2010; Flynn and Harbridge, Reference Flynn and Harbridge2016). Yet, whatever the reason, these results do further validate our observational survey data which demonstrated that Democratic voters viewed the Supreme Court as a less important issue than the Republicans. In our conjoint experiment, the Supreme Court did not positively—and even negatively—affected candidate selection.

Figure 9.

Results for Democratic participants.