INTRODUCTION
Why and to what extent does the mass public view international courts as legitimate? Recent events in the United Kingdom can illustrate the significance of these attitudes. In November 2019, news media reported extensively on historic allegations of war crimes committed by members of the British armed forces in Iraq and Afghanistan and on an apparent failure by domestic authorities to properly investigate those incidents or hold perpetrators to account (BBC News 2017). These reports prompted renewed claims that the United Kingdom has proved unwilling or unable to prosecute war crimes committed by its military and that prosecution by the International Criminal Court (ICC) may be justified .Footnote 1
The United Kingdom has accepted ICC jurisdiction continuously since it was created under the 1998 Rome Statute and became operational in 2002.Footnote 2 To date, however, no case involving the United Kingdom has proceeded to a full investigation. One prolonged preliminary investigation was conducted in relation to alleged crimes (including murder, torture, and other forms of ill-treatment) committed by UK nationals between 2003 and 2008 during the Iraq conflict and occupation. The ICC prosecutor’s examination of this matter was terminated in 2006 but re-opened in 2014 following receipt of new information. In December 2019, the prosecutor indicated that the recent news reports would be relevant in deciding whether to open a full investigation into the allegations (ICC 2019). The prosecutor finally announced in December 2020 that no further action would be taken, but these events raise important questions (ICC 2020). How would the UK government respond to any ICC decision to open a full investigation into the actions of its citizens? Would it allow its citizens to be tried outside the UK’s military or civilian justice systems? The answers to these questions depend in part on the political costs—both domestic and international—of a failure to comply.
In this article, we investigate the extent to which the UK public ascribes legitimacy to the ICC. Would the public expect their government to cooperate and comply with an investigation that could lead to the prosecution of their fellow citizens by an international tribunal? Do they support the existence of an international institution with the authority to overrule prosecutorial decisions of their domestic government in that way? Put more generally, to what extent, and why, does the public view international courts as legitimate? This is an important question because a court with a high degree of legitimacy is more likely to see its decisions implemented. High public legitimacy creates pressure for compliance by increasing the social and political costs associated with noncompliance (Gibson and Caldeira Reference Gibson and Caldeira1998; Vanberg Reference Vanberg2005; Staton and Moore Reference Staton and Moore2011; Krehbiel, Reference KrehbielForthcoming). Although states—not individuals—are the central actors tasked with complying with international court decisions, public perceptions of legitimacy will inform state calculations on whether to comply or defect in the face of international court action. Perceived legitimacy is essential if the authority granted to a court on paper is to be recognized and acted on in practice.
As a relatively new international institution, the extent of the ICC’s legitimacy and authority remains uncertain. There are now 123 states party to the Rome Statute, but that list excludes some of the world’s most populous and powerful countries. The ICC’s own decision-making has recognized de facto limits on its authority. In 2019, it rejected a proposal from the ICC prosecutor to investigate allegations involving US personnel in Afghanistan, despite recognizing that the complaints were serious and within its jurisdiction.Footnote 3 The court concluded that an investigation was unlikely to succeed given the unwillingness of key parties to cooperate, and so it was not in the interests of justice to proceed.Footnote 4 The ICC’s Appeals Chamber overruled that decision in March 2020, allowing the prosecutor to proceed with a full investigation. This decision earned plaudits from victims’ advocates but was immediately denounced by (then) Vice President Mike Pence, who indicated that the United States “will take all necessary measures to protect our citizens from this renegade, so-called court” (Harrell Reference Harrell2019). The United States went on to impose sanctions and entry bans on ICC officials in September 2020 (Human Rights Watch 2020).
To date, only limited empirical research has been conducted regarding public legitimacy attitudes toward international courts. Little is known about the causes and dynamics of legitimacy for international courts, particularly when compared with their extensively studied domestic counterparts. Work on the legitimacy of international institutions has gained momentum in recent years (see especially the agenda-setting work in Hurd Reference Hurd2008 and Tallberg and Zürn Reference Tallberg and Michael2019), but much empirical research regarding international court legitimacy uses less-than-ideal measurements—for example, substituting trust as a proxy for legitimacy (Kelemen Reference Kelemen2012; Voeten Reference Voeten2013).
We seek to advance understanding by testing a novel theory of home-state interest. We propose that a substantial factor in the decision to confer legitimacy to an international court is whether it is perceived to be acting in the national interest of a respondent’s home country. Moreover, because this theory depends on the relation of the international institution vis-á-vis one’s home country, we expect the respondents’ nationalism to moderate this relationship. As a test of the theory, we focused on UK public attitudes to the ICC. We conducted a nationally representative survey experiment on 1,515 adults, randomly assigning respondents to one of four treatment conditions. Following treatment, respondents were asked a series of questions measuring legitimacy as both institutional support and compliance attitudes. Finally, respondents were also asked questions measuring two types of attitudes commonly associated with nationalism: pride in being British (patriotism) and British exceptionalism (chauvinism).Footnote 5
We found significant support for our national interest theory. Those exposed to the negative treatment (indicating that the ICC is antithetical to the national interests of the United Kingdom) ascribe less legitimacy to the court, while we found the opposite for those in the positive treatment group. Importantly, we were able to demonstrate that these findings are not the result of altered perceptions of domestic courts (a mechanism previously identified as influencing international court legitimacy). Additionally, we found that both types of nationalistic attitudes moderate this relationship where legitimacy is measured as institutional support—magnifying the effect of the negative treatment and reducing the effect of the positive treatment. However, we found differences in the way in which nationalism influences the relationship when legitimacy is operationalized as compliance attitudes.
By identifying a new factor influencing international court legitimacy independent of attitudes toward domestic courts, this study has important implications for politicians, practitioners, and activists who seek to shape public perceptions of the ICC or international courts more broadly. Our findings are particularly important as international courts become increasingly visible in the lives of ordinary citizens and create greater opportunity for (actual or perceived) conflict between international and domestic legal obligations. Finally, our study has implications for international courts themselves as they consider the potential institutional costs of taking action against states.
WHAT IS LEGITIMACY?
“Legitimacy” as a property of political institutions has both normative and sociological dimensions, which are both concerned with whether an institution has a justified right to rule. Normative legitimacy refers to the extent to which an institution satisfies some external or pre-determined standard such as democracy, justice, or efficiency (Buchanan and Keohane Reference Buchanan and Keohane2006; Cohen et al. Reference Cohen, Follesdal, Grossman, Ulfstein, Nienke, Harlan Grant, Andreas and Geir2018), whereas sociological legitimacy refers to the extent to which the institution is perceived to be legitimate by the public or other audiences. In this article, we are concerned with sociological legitimacy: beliefs about whether an institution is worthy of deference and ought to be obeyed.Footnote 6 Following Ian Hurd (Reference Hurd1999, 381; emphasis in original), we view legitimacy as “a subjective quality, relational between actor and institution, and defined by the actor’s perception of the institution.”
Legitimacy instills within an individual “the belief that rules and regulations are entitled to be obeyed by virtue of who made the decision or how it was made,” propelling cooperation, compliance, and acceptance of a decision, even when one disagrees with the outcome (Levi, Sacks, and Tyler Reference Levi, Audrey and Tyler2009, 354). For judicial institutions, legitimacy serves several important functions: bolstering court authority, encouraging compliance, and reducing the need for coercive enforcement.
The dominant contemporary approach to studying judicial legitimacy focuses on institutional loyalty as an indicator of legitimacy. It distinguishes between “specific support,” characterized as satisfaction with a court’s performance in particular cases, and “diffuse support,” a “reservoir of goodwill” that allows the institution to generate unfavorable outputs without serious threat to its authority. Legitimacy is equated (and frequently treated interchangeably) with diffuse (institutional) support. It is operationalized using a series of questions designed to measure respondents “institutional commitment,” characterized by an unwillingness to “accept, make, or countenance major changes in fundamental attributes” of the institution in question (Gibson and Caldeira Reference Gibson and Caldeira1992, 638), such as narrowing jurisdiction, increased political control, or complete abolition (Gibson, Caldeira, and Spence Reference Gibson, Caldeira and Lester2003).
While the nature of the relationship between diffuse and specific support is contested (see, for example, Bartels and Johnston Reference Bartels and Johnston2013, Reference Bartels and Johnston2020; Christenson and Glick Reference Christenson2015; Zilis Reference Zilis2018; Strother and Kushner Gadarian Reference Strother and Shana2019), Gibson’s dominant account suggests that diffuse support is built up over time through a series of favorable (or acceptable) rulings (Gibson and Nelson Reference Gibson and Nelson2014). Over time, levels of diffuse support will become robust in the face of individual unpopular decisions (Gibson, Caldeira, and Baird Reference Gibson, Caldeira and Baird1998). For newer courts, including the ICC and most international courts, a closer connection between diffuse and specific support might therefore be expected to hold than in more established contexts.
Still, not all scholarship on institutional legitimacy equates it with diffuse support. Drawing from social psychology, other scholars measure an internalized feeling of obligation that arises from perceptions of fairness regarding the processes that the authority uses to arrive at a decision (Tyler, Casper, and Fisher Reference Tyler, Casper and Bonnie1989; Baird Reference Baird2001; Tyler Reference Tyler2006; Tyler and Trinkner Reference Tyler and Trinkner2017). In so doing, they recognize that institutional support may not be the only, or even the most important, type of attitude connected with the overarching belief that an institution has a justified right to exercise authority over other actors. Rejecting the apparent conflation of legitimacy with institutional support opens the way for research that addresses more broadly the types of attitudes that legitimacy is understood to generate and likewise enables the study of aspects of institutional support that may be unrelated to legitimacy (Bartels and Johnston Reference Bartels and Johnston2013). Studies of legitimacy as manifested in “compliance attitudes” focus on whether an individual feels compelled to defer to an authority figure as an indicator of their recognition of that authority.
Few scholars have used these empirical tools to investigate legitimacy in relation to international courts. James Gibson and Gregory Caldeira investigated levels and predictors of diffuse support for the European Court of Justice (ECJ) in the 1990s. They found that both awareness of, and diffuse support for, the ECJ were low, and concluded that the process of building diffuse support by accumulating a “tally” of specific support that had been identified in relation to domestic courts could not operate where knowledge of the court’s outputs was so low (Caldeira and Gibson Reference Caldeira and Gibson1995, Reference Caldeira and Gibson1997; Gibson and Caldeira Reference Caldeira and Gibson1995, Reference Gibson and Caldeira1998). Instead, they conclude that the public forms attitudes toward the ECJ based on their general attitudes toward the European Union (EU).
More recent scholarship connects Gibson and Caldeira’s work to an expanding body of research about the legitimacy of, and public opinion on, international organizations and courts. However, that work often uses survey data measuring trust or confidence rather than operationalizing legitimacy as diffuse support or compliance attitudes. For example, Daniel Kelemen (Reference Kelemen2012) shows that the ECJ has relatively consistent and high trust relative to other EU and domestic institutions. Erik Voeten (Reference Voeten2013) finds that trust in international courts (the ICC, the ECJ, and the International Court of Justice) correlates with both trust in the international organizations with which they are associated and trust in domestic courts. Lisa Marie Dellmuth and Jonas Tallberg (Reference Dellmuth and Jonas2020) operationalize legitimacy with a measure of respondents’ confidence in four international institutions across four countries. They find that generalized social trust serves as an antecedent factor shaping a broader relationship that has been observed between legitimacy of domestic and international organizations.
While these studies all offer valuable insights, they do not measure legitimacy as we conceive it, which is as beliefs about whether an institution has a justified right to exercise authority over other actors. Those beliefs are conceptually distinct from both trust and confidence in an institution. Trust in governing institutions can be defined as a public belief that institutional actors (collectively or individually) have the right intentions and are competent to act in specific ways in specific situations (Jackson et al. Reference Jackson, Ben, Mike, Jouni, Sally, Sally, Rory, Maria and Todor2011). “Confidence” implies a related belief that institutions can perform the task(s) assigned to them. While trust/confidence and legitimacy may be connected (closely, even), they will not necessarily coincide. One can quite reasonably (1) believe that the ICC (or another institution) is well intentioned, competent, and able to perform its functions and (2) not feel any sense of loyalty or obligation to the court. Studies of the US Supreme Court provide empirical evidence of this distinction, showing that trust and confidence align more closely with measures of specific support than others measuring institutional loyalty (Grosskopf and Mondak Reference Grosskopf and Mondak1998; Gibson, Caldeira, and Spence Reference Gibson, Caldeira and Lester2003; Gibson Reference Gibson2011). Trust and confidence are thus imperfect measures of legitimacy beliefs, as we understand them.
We therefore have only limited understanding of legitimacy with respect to international courts. However, a handful of recent studies have investigated other public attitudes toward the ICC. Kelebogile Zvobgo (Reference Zvobgo2019) uses a survey experiment to test how elite cues, using “human rights” and “national interest” frames, affect US public attitudes regarding the prospect of the United States joining the ICC. She finds that these cues do shift attitudes, with the human rights frame prompting increased support for the United States joining the ICC, and the national interest frame (emphasizing the risk of US armed personnel being unfairly targeted for political reasons) reducing support. Notwithstanding the US government’s strong objection to allowing ICC jurisdiction, the public was shown to be receptive to ICC oversight when prompted with a treatment highlighting the United States’ historical commitment to international criminal tribunals.
Alan Simmons (Reference Simmons2017) also focuses on domestic attitudes toward accepting international court jurisdiction, finding that public attitudes to international courts are shaped by levels of chauvinistic nationalism rather than by knowledge and information. Using existing survey data, he finds that more nationalistic US respondents were more likely than their less nationalistic counterparts to oppose international criminal jurisdiction over US servicemen accused of human rights violations. Stephen Chaudoin (Reference Chaudoin2016) uses public opinion data on the ICC to measure the effectiveness of messaging by an anti-compliance alliance in Kenya after the ICC issued summonses to political leaders implicated in electoral violence. Members of this alliance cast themselves as “patriotic Kenyans in opposition to a patronizing international community,” and Chaudoin shows that support for the ICC (measured as happiness with the ICC’s process) dropped most dramatically in regions most strongly associated with the indicted politicians (566). Finally, Terrence Chapman and Stephen Chaudoin (Reference Chapman and Chaudoin2020) test support for a hypothetical ICC investigation using a survey experiment. Respondents exposed to a treatment describing an investigation regarding real events involving violence in their home country (Kyrgyzstan) were less likely to say that the investigations described were a good thing than those prompted to consider investigations elsewhere. Furthermore, the effect of the treatment was stronger the closer a person lived to the events in question.
THEORY AND HYPOTHESES
The central motivation for our project is to discern the nature and dynamics of public legitimacy afforded to an international court. We offer a novel theory of international court legitimacy and test it using measures of legitimacy that are consistent with those used in domestic courts scholarship. Our theorizing builds both on earlier works concerning domestic court legitimacy and more recent—but increasingly diverging—scholarship that has begun to identify mechanisms that shape public attitudes toward international institutions and courts. Our approach therefore allows us to increase knowledge of international court legitimacy and bridge subfield boundaries to identify how the dynamics surrounding domestic and international court legitimacy overlap and diverge.
Of course, international courts are further removed from the lives of ordinary citizens than domestic courts. This distance leads us to expect distinctive dynamics to shape public attitudes to international courts. First, socialization to—and familiarity with—international courts will be weak in comparison with domestic legal institutions. For domestic courts, existing research suggests that legal socialization and exposure to legitimating symbols associated with courts contributes to their perceived legitimacy among the general public. We expect any pre-existing attitudes to international courts, to the extent that there are any, to be less crystalized and more malleable than for domestic courts given the public’s relatively limited exposure to international courts, along with the general inattention to politics evident among the public (Carpini and Keeter Reference Carpini and Keeter1997; Tesler Reference Tesler2015; Trinkner and Tyler Reference Trinkner and Tyler2016; Tyler and Trinkner Reference Tyler and Trinkner2017). Under these circumstances, we expect elite signals to play an important role in shaping public support or opposition for an international court (Jacobs and Shapiro Reference Jacobs and Shapiro2000; Lenz Reference Lenz2012).
However, not all elites are created equally. James Druckman (Reference Druckman2001) underscores the importance of source credibility on accepting information, an insight recently confirmed by research investigating the consequences of former President Donald Trump’s attacks on support for the US Supreme Court (Nelson and Gibson Reference Nelson and Gibson2019). Ultimately, in-group cues are likely to be more persuasive than out-group cues, often because they are viewed as being more credible (Druckman Reference Druckman2001; Bolsen, Druckman, and Cook Reference Bolsen, Druckman and Fay2014; Mason Reference Mason2018). Given limited public awareness, we anticipate that national or subnational elite cues will seek to draw connections between an international court and domestic concerns with which the mass public can more easily engage. Erik Voeten (Reference Voeten2020) provides examples of such discourse, highlighting cases where populist backlashes against international courts have drawn on existing domestic mobilization narratives concerning property rights and the identity of the polity. Stephen Chaudoin (Reference Chaudoin2016) highlights how both pro- and anti-compliance groups mobilized domestic political support in response to ICC action against political candidates in Kenya.
Hypothesis 1: years of formal education completed will be positively correlated with legitimacy afforded to the ICC
To investigate these dynamics in relation to the ICC, we first test the extent to which some key determinants of domestic court legitimacy—educational attainment and knowledge of the court—shape the ICC’s legitimacy. Consistent with our preregistration plan, Hypothesis 1 is observational in nature and non-causal. Given limited socialization and public knowledge, we expect that public legitimacy of any international courts will generally be lower than for domestic courts. However, we theorize that formal education is likely positively correlated with the ICC’s legitimacy, in line with findings in the domestic courts literature (Gibson, Caldeira, and Baird Reference Gibson, Caldeira and Baird1998; Benesh Reference Benesh2006; Gibson and Caldeira Reference Gibson and Caldeira2009a, Reference Gibson and Caldeira2009b). Our purpose here is exploratory since it is unclear how far we should expect this relationship to hold for an international court or even across domestic courts. Increased education seems less likely to bolster legitimacy for an unpopular court, while it might trigger a distinctive critical response to international courts (Krehbiel, Reference KrehbielForthcoming). To build our basic understanding, we investigate this relationship in an international context.
Hypothesis 2: offering factual information about an international court will increase the legitimacy afforded to the institution
Hypothesis 2 is related to Hypothesis 1 but goes beyond formal education into more specific knowledge about a court. Evidence from multiple domestic courts in the United States and Europe indicates that those who “know” a court are more likely to afford it legitimacy (Gibson, Caldeira, and Baird Reference Gibson, Caldeira and Baird1998; Gibson and Caldeira Reference Gibson and Caldeira2009b). Explanations for this relationship suggest that increased awareness brings familiarity with legitimizing symbols that courts use to signal their distinctiveness from other political institutions and reinforce the “myth of legality” around their work (Gibson and Caldeira Reference Gibson and Caldeira2009a, 7–9). Gibson and Caldeira (Reference Gibson and Caldeira2009a) note that legitimizing symbols may trigger pre-existing institutional loyalty and signal that the institution sits above “normal” politics. However, it is again unclear whether the same relationship should hold for international courts, and the available evidence is inconclusive.Footnote 7 Without pre-existing loyalty to an international court, we expect this mechanism to be weaker or to operate through a more general loyalty to the rule of law. Testing the strength of this relationship will provide practical insight into whether raising awareness is an effective strategy for activists seeking to bolster public support for a court and increase the costs for elites of noncompliance. Hypothesis 2 thus tests whether the relationship found in domestic contexts holds for an international court.
Hypothesis 3a: increasing the perception that an international court is acting in the interest of one’s home country will increase the legitimacy afforded to the court; Hypothesis 3b: increasing the perception that an international court is hostile or acting antithetically to the interests of one’s home country will decrease the legitimacy afforded to the court
Our remaining hypotheses focus on dynamics specific to international courts. We theorize that perceptions of “home state interest” will serve as an important pathway through which elites can influence public legitimacy afforded to international courts. We hypothesize that the extent to which international courts are viewed as legitimate will depend upon the extent to which people view the institution as advancing or hindering the interests of their home country. Where elite rhetoric is purposely directed toward influencing legitimacy beliefs, it constitutes an example of what Jonas Tallberg and Michael Zürn (Reference Tallberg and Michael2019) refer to as “legitimation” and “deligitimation” practices: activities designed to bolster or undermine the legitimacy of an institution. These are broad labels, capable of referring to a range of activities undertaken by different actors for diverse goals. For example, courts engage in legitimation practices when—as discussed by Gibson and Caldeira (Reference Gibson and Caldeira2009a)—they deploy legitimization symbols to bolster beliefs that courts are worthy of respect and deference. Tallberg and Zürn (Reference Tallberg and Michael2019) highlight other examples directed toward informed or specialized audiences, including activity intended to demonstrate the proper exercise of authority (for example, through the use of fair procedures) or to undermine those messages. However, we expect that where non-judicial elites use legitimation or delegitimation practices to influence general public attitudes to a little-known international court, they will do so in blunter and simpler terms.
Narratives that frame international court activity as aligned with, or inimical to, national interests hold considerable promise for actors wishing to bolster or undermine an international court’s legitimacy. They offer the public an accessible framework for evaluating a distant and poorly understood institution against more familiar concerns. They can also be deployed across diverging national contexts, since the potential for tension between national interests (however defined)Footnote 8 and international court activity will arise in any state that accepts or contemplates international court jurisdiction. We expect the effectiveness of these signals to be further reinforced by in-group biases. The human propensity to privilege an “in-group” over others means that elite signals emphasizing a threat to domestic affairs (“us”) brought about by an “outside” organization can be especially effective movers of mass attitudes toward the institution in question. Conversely, elite signals can serve to counteract in-group bias by emphasizing areas of alignment between an outsider and the in-group.
Turning specifically to the ICC, we can see how conflicting elite cues can compete for public attention. Some governments, advocates, and NGOs actively work to persuade more states to accept ICC jurisdiction, while several of the world’s most populous and powerful states—including the United States, India, China, and Russia—continue to decline to do so. Other states that have accepted ICC jurisdiction have reconsidered their commitments when faced with their practical consequences. For example, both Burundi and the Philippines withdrew from the Rome Statute after the ICC opened investigations into events involving them. The ICC’s attempts to prosecute the heads of state of Kenya and Sudan prompted particular objections among African states. In Kenya, ICC indictments of senior politicians prompted pro- and anti-compliance groups to mobilize around their electoral campaigns (Chaudoin Reference Chaudoin2016). South Africa and the Gambia both gave notice of their intention to withdraw from the Rome Statute in 2016, before rescinding that notice the following year.
In summary, our theory argues that elite rhetoric is part of a mechanism that affects international court legitimacy: shaping individuals’ beliefs about whether the ICC works in the national interests of their home country and, in turn, affecting the ICC’s legitimacy. This mechanism is significant because high legitimacy would boost the ICC’s authority by raising the social and political costs of noncompliance. Although states—not individuals—are the central actors tasked with complying with international court decisions, public perceptions of legitimacy will inform state calculations on whether to comply or defect in the face of international court action. We test for this mechanism with our Hypothesis 3, the “home-state interest hypothesis.”
Hypothesis 4a: high nationalism moderates the observed relationship between national interest and legitimacy by magnifying the effect of our negative treatment; Hypothesis 4b: high nationalism moderates the observed relationship between national interest and legitimacy by reducing the effects of our positive treatment
Finally, because our theory depends on how people perceive the ICC vis-á-vis their home country, we expect nationalism to heavily moderate this relationship. This follows from an expectation that individuals who exhibit nationalistic attitudes will be more responsive to potential threats to national interests from international court jurisdiction and more skeptical of international oversight of state actions than those who do not exhibit nationalistic attitudes. These expectations follow from our belief that national interest framing is effective for elites seeking simple heuristics to shape attitudes among an inattentive public, in part because it can activate or counteract in-group bias. Since nationalism is a form of in-group bias, we expect nationalistic individuals to be more responsive to domestic elite signals indicating that the ICC acts contrary to national interest and less responsive to cues indicating that the ICC’s activities are aligned with the national interest than non-nationalistic individuals. These expectations are consistent with Alan Simmons’s (Reference Simmons2017) theory and finding that chauvinistic nationalism makes individuals less likely to agree that US armed forces should be subject to international judicial institutions when accused of human rights violations.Footnote 9 Hypotheses 4a and 4b test this relationship.
DATA AND METHODS
To test the theory, we conducted a survey experiment focusing on attitudes toward the ICC in the United Kingdom. A survey experiment was chosen because it allows an evaluation of the theory in a tightly controlled environment that emphasizes internal validity while still maintaining a moderate degree of external validity. This medium allowed us to directly manipulate the mechanism (perceptions of national interest) to see the effect on our primary dependent variable (legitimacy toward the ICC), thus allowing us to test causality. Moreover, we were able to directly measure both the mechanism and dependent variables, allowing us to conduct more advanced statistical analyses such as mediation and sensitivity analyses. Finally, the survey experiment permitted the pretesting of treatments prior to data collection, which ensured that the treatments were moving the theorized mechanism of interest and not alternate mechanisms. To put this idea another way, a survey experiment was selected because it allowed us to artificially manipulate the independent variable of interest, measure these effects, and look at how these effects influence the dependent variable of interest.
The United Kingdom is the focus of our study for several reasons. Most importantly, it has accepted ICC jurisdiction and, at the time when the experiment was conducted, was the subject of a preliminary examination by the ICC to determine if members of the UK armed forces should be prosecuted for alleged war crimes committed in Iraq. This scenario allowed us to develop a negative treatment incorporating information about the ICC’s actual activities with respect to the respondents’ home country and highlighting the risks associated with these investigations. Although our experiment relied on deception, conducting it in the United Kingdom let us significantly reduce the amount of deception required. Moreover, among the short list of countries that have been subject to ICC investigation to date, the United Kingdom represented a pragmatic choice since it has high Internet usage rates, making it possible to field an Internet-based survey to a diverse sample at a manageable cost and without the need for translation services.Footnote 10
The survey was administered by Lucid, which provides a quota-based nationally representative sample that has previously been shown to mirror traditional probability-based samples, at least in the United States, and is increasingly being used to make descriptive inferences (Coppock and McClellan Reference Coppock and McClellan2019; Armaly, Reference ArmalyForthcoming). A total of 1,515 respondents took part in the study. Although recent research has highlighted concerns about a decline in respondent attentiveness on Lucid (Aronow et al. Reference Aronow, Joshua, Lilla and John2020; Peyton, Huber, and Coppock Reference Schatz, Ervin and HowardForthcoming), our survey included an embedded attention check with the goal of addressing this issue. It identified and removed seventy-nine inattentive respondents, but otherwise reassured us that respondents were actively engaging with survey questions.Footnote 11 The analysis that follows is based on the 1,436 respondents who successfully passed the attention check.
Upon consenting to participate, respondents were randomly assigned to one of four groups: a pure control, a baseline treatment, and negative- or positive-valenced treatments. Those in the baseline condition were exposed to neutral, factual information about the ICC, including its age, location, and function, as shown in Figure 1. For the positive and negative treatment groups, the baseline information was supplemented with content that framed the court’s activities as either aligned or in conflict with the United Kingdom’s national interests respectively.Footnote 12 The positive and negative treatment texts are presented in Figures 2 and 3. Finally, those in the pure control condition were exposed to no text and advanced directly to the questions designed to measure attitudes toward the ICC, the UK Supreme Court (UKSC), and various demographics.
Baseline treatment text as presented to study participants.
Positive treatment text as presented to study participants.
Negative treatment text as presented to study participants.
The treatments were designed with the goal of replicating elite signals outside the experimental context. They incorporate non-partisan elite cues signaling support or opposition to the ICC from a cross-party parliamentary committee (chosen to limit the scope for party identification to influence respondent attitudes). The treatments also include an explanation from the elites as to why the ICC is (or is not) in the national interest, which contextualizes and rationalizes their argument by reference to values and practices that they associate with national interests. These include human rights, justice, and deterrence in the positive treatment, and procedural unfairness, prosecutorial bias, and military effectiveness in the negative treatment.
Including a rationale for the elite message enhances the external validity of the treatments since it is very unlikely that such signals will exist without some sort of (even brief) rationalization behind them. Additionally, it recognizes that members of the public are unlikely to have a clear or fixed idea of what is in the “national interest” independently of elite signals in a particular context. This feature is also consistent with experimental treatments used by Zvobgo (Reference Zvobgo2019), who included references to “fairness” in national interest frames that were designed to influence attitudes toward the ICC.
The treatments therefore prompt respondents to associate the ICC with positive or negative values, which are associated with national interest. This raises the question of whether the treatments also prompted changes in those wider attitudes, and, if so, whether those changes distorted the experimental findings. In short, we accepted that the treatments would affect attitudes other than perceptions of national interest, including general affect, or perceptions of the ICC’s fairness. Our focus on a criminal court with jurisdiction over the worst international crimes means that it would be impractical—even in a controlled experiment—to present its work in terms that do not also engage considerations of justice and fairness. The key question for our purposes is whether any shift in perceptions of national interests can be distinguished from change to those other attitudes. The pretest and a manipulation check included in the study indicated that the treatments successfully shifted the respondents’ perceptions of national interest with respect to the ICC and that this effect can be distinguished from any effect on those broader attitudes toward the court. For more discussion of these points, see the “extended analysis” section in the supplementary material.
We measured our dependent variable—the ICC’s legitimacy—using a set of questions based on those developed by Gibson and his collaborators to measure the US Supreme Court’s legitimacy viewed as diffuse support (see, for example, Gibson, Caldeira, and Spence Reference Gibson, Caldeira and Lester2003). These questions have become ubiquitous in the US Supreme Court literature (Gibson Reference Gibson2011) but, to our knowledge, have only been used to measure legitimacy in relation to international courts (specifically, the ECJ) by Gibson and Caldeira (Reference Gibson and Caldeira1995, Reference Gibson and Caldeira1998) and Caldeira and Gibson (Reference Caldeira and Gibson1995, Reference Caldeira and Gibson1997). These questions operationalize legitimacy beliefs as a general willingness to defend an institution from meaningful alterations in its structure or function. Specifically, our questions asked respondents the extent to which they agreed with statements including: “If the [COURT] started making decisions that most people disagree with, it might be better to do away with the [COURT] altogether” and “The right of the [COURT] to decide certain types of controversial issues should be reduced.” Full question wording can be found in the supplementary material. This legitimacy battery created an additive index ranging from zero to one, with higher values indicating greater diffuse support for the institution. Below, we refer to these questions as “diffuse support measures.” Additionally, in line with our pre-registered plan, we collected data using a series of questions designed to measure legitimacy as compliance attitudes—that is, the extent to which an individual feels an internalized sense of obligation to the demands of the ICC. In the analysis below, we refer to the second set of questions as “compliance measures”.
Following these questions about the ICC’s legitimacy, respondents were asked a parallel set of questions designed to measure legitimacy beliefs with respect to the UKSC. These questions allow us to test a significant observable implication of our theory. Since we theorize that national interest perceptions will influence international court legitimacy only, we do not expect to see variation across treatment groups with respect to the legitimacy of the UKSC. Moreover, these questions served as an important check to ensure that our treatments targeted only attitudes toward the ICC and did not manipulate attitudes toward courts more generally. Finally, inclusion of questions about the UKSC additionally permitted us to check the extent to which our findings support the conclusions offered by Voeten (Reference Voeten2013) on the importance of domestic court legitimacy for international court legitimacy.
The final section of the survey included demographic and other attitudinal questions for use in randomization checks. This section also included questions designed to measure two attitudes commonly associated with nationalism—pride in being British and British exceptionalism. Measuring both attitudes allowed us to evaluate the extent to which the relationship between national interest and legitimacy is moderated by a positive evaluation of one’s own country (patriotism) and negative evaluation of others in comparison with it (chauvinism).Footnote 13 Recognizing this distinction allowed us to seek a more nuanced understanding of the role of nationalistic attitudes in relation to the mechanism of interest to us. The experimental design is modeled in Figure 4.
Experimental design.
FINDINGS
In this part, we present our analysis using the measures of legitimacy as diffuse (institutional) support.Footnote 14 Overall, we found substantial support for our theory. First, we found that the mean level of diffuse support afforded to the ICC within the control (untreated) group was relatively low when compared to that afforded to the UKSC. For example, the mean of the factored battery of questions measuring diffuse support among those randomly assigned to the control condition is 0.42, compared with 0.51 for the UKSC’s legitimacy (p = 0.000). This discrepancy becomes more pronounced when looking at specific questions. For example, 66 percent of respondents indicated that they agreed or strongly agreed that it might be better to do away with the ICC if it started making decisions that most disagree with, compared to only 48 percent who thought the same for the UKSC (p = 0.000). The individual question breakdowns can be found in Table 1, which is consistent with our expectations that legitimacy for domestic institutions would be greater than for the ICC.Footnote 15
Percentage of unweighted respondents in the control group that responded either “agree” or “strongly agree” to questions used to make the diffuse support index
Source: Lucid Study, April 29 – May 7, 2019.
Notes: Unweighted means among those in the control condition. * indicates reverse coding.
To test Hypothesis 1, we regressed the diffuse support afforded to the ICC by the 345 respondents in the pure control condition on education and a variety of control variables, including institutional support toward the UKSC, educational attainment, ideology, Brexit vote, and income. The results are presented in Figure 5. Contrary to Hypothesis 1, we found that education is not a significant predictor of the ICC’s legitimacy, measured as diffuse support. We did not find a relationship parallel to that found with respect to domestic court legitimacy.
Correlates of the ICC’s legitimacy (diffuse support measure) showing that domestic court legitimacy is a strong correlate of the ICC’s legitimacy but that education is not (full model is presented in the supplementary material).
Although not consistent with our hypothesis, this result is not entirely surprising. As we indicated earlier, the processes that are believed to produce this relationship in a domestic context (greater education/knowledge being associated with increased socialization to a court) are less clearly applicable for international courts. This finding indicates that these mechanisms do not operate in relation to the ICC, and further research is therefore needed to identify whether and how socialization processes affect the legitimacy of international courts. However, our observational analysis does confirm one relationship that is expected based on prior research. In line with the findings of Voeten (Reference Voeten2013), we found that domestic court legitimacy is a very strong predictor of the ICC’s legitimacy. The full regression table is presented in the supplementary material.
Next, we investigated the effects of offering factual information about the court in a neutral manner. Outside of the experimental environment, this would be equivalent to an international court itself, activists, or elites undertaking a public relations campaign in an attempt to raise visibility and awareness about the court. Hypothesis 2—that offering this information will increase the legitimacy of the court—is confirmed in the “baseline” bar in Figure 6. As is clear, providing factual information in a neutral manner significantly increases the diffuse support afforded to the court. In this respect, at least, the ICC’s legitimacy appears to follow the pattern we would expect also for a domestic court.
Average treatment effects of the three experimental conditions.
However, there is an important caveat to this finding. Although the treatments were pretested to ensure that they correctly manipulated the primary mechanism in our theory (perception that the ICC is aligned with, or opposed to, the United Kingdom’s national interest), the factual baseline treatment also appears to have affected this underlying mechanism. Pre-testing indicated that our positive and negative treatments shifted perceptions of the national interest in the intended direction, while the baseline treatment had no statistically significant effect on those perceptions. In the study, however, the results of a manipulation check embedded within the experiment indicate that the baseline treatment also influenced perceptions of national interest.
This difference emerged following a change between our pretest and survey. The pretest used a battery of questions designed to measure perceived national interest, but the study relied on just one of those questions (the one that proved most sensitive in pretesting) to measure respondents’ views on whether the ICC does work in the national interest of the United Kingdom. The results are shown in Figure 7. While the negative and positive treatments successfully made people less and more likely respectively to think that the ICC works in the United Kingdom’s national interest, the baseline treatment also moves attitudes such that it is statistically indistinguishable from the positive treatment. This finding might also be a result of our case selection. It seems possible that legal socialization—either within the United Kingdom’s strong rule-of-law culture or in general—means that a criminal court will not be easily perceived neutrally with respect to national interest. We hope that future research will explore these questions further to consider the legitimacy of other international courts and legitimacy attitudes in countries with different rule-of-law traditions and legal socialization practices.
Manipulation check showing that all the treatments manipulate the primary mechanism of interest.
Next, we sought a direct test of our home-country interest theory that perceptions of national interest affect peoples’ willingness to afford legitimacy to an international court. The results presented in Figure 6 show strong support for this theory. For both the positive and negative treatment groups, we found a substantial gap between the level of diffuse support they exhibit and the control group. Specifically, exposure to our treatment reduced the legitimacy of the ICC by approximately six percentage points, while the positive treatment increased legitimacy by around seven percentage points, producing a thirteen-point gap between these groups. When we swap out the treatment group assignment in our model with the manipulation check question (asking respondents how much they agree that the ICC does work in the United Kingdom’s national interest), we found that a move from “strongly disagree” to “strongly agree” results in a 30 percent increase in diffuse support
However, our theory specifies that this mechanism will affect only international court legitimacy, so a successful test also requires demonstration that the treatment did not affect attitudes to the UKSC. This is demonstrated in Figure 8, which shows that, although our treatments affect the ICC’s legitimacy, they do not affect the UKSC’s legitimacy. This indicates that we have identified a pathway of international court support that is independent of domestic court support, and the treatments did not manipulate attitudes toward legal institutions in general. Finally, as a robustness test, we performed a mediation and sensitivity analysis that further supports our theory. These can be found in the supplementary material.
Treatment effects on the UKSC’s legitimacy showing that our treatments do not affect domestic court legitimacy (diffuse support measure).
To this point, we have failed to confirm Hypothesis 1 (that education predicts the ICC’s legitimacy), but we have confirmed Hypothesis 2 (providing factual information increases the ICC’s legitimacy) and Hypotheses 3a and 3b (that perceptions of national interest matter for an international court’s legitimacy). We now turn to the final set of hypotheses and ask how nationalism affects this relationship. As noted above, we measured nationalism using questions designed to elicit two types of national affinity: national pride (patriotism), which is expressed as a positive evaluation of a respondent’s own country, and exceptionalism (chauvinism), which is expressed as a positive evaluation of one’s own country relative to others. For the pride measure, respondents were asked: “How proud are you of being British?” with the response options: “very proud,” “somewhat proud,” “not proud at all,” or “I am not British.” To measure belief in British exceptionalism, we asked respondents: “Which of the following is closest to your view of the UK’s standing in the world: The UK stands above all other countries, the UK is one of the greatest countries along with others, or there are other countries that are better than the UK.” We interacted and regressed the treatment condition with each measure of nationalism on diffuse support for the ICC. The results are displayed in Figure 9.
Nationalism influences treatment effects on the ICC’s institutional support across treatment groups.
Our findings support Hypothesis 4a: those who exhibit greater nationalism according to both our measures do “double-down” when exposed to a signal that the ICC is antithetical to their nation’s interest. Specifically, those expressing low pride report a mean legitimacy value of 0.40, while those exhibiting high pride report a mean of 0.33 (p = 0.01; one-tailed). For exceptionalism, those exhibiting low belief in British exceptionalism report a mean of 0.39, whereas exceptionalist individuals are at 0.33 (p = 0.03; one-tailed).
Within the positive treatment condition, we also found support for Hypothesis 4b across both measures of nationalism. In both cases, the positive treatment had less effect on respondents who manifested nationalistic attitudes than on those who did not. Mean diffuse support among those with low exceptionalism is 0.53, while for highly exceptionalist individuals, it is 0.40 (p = 0.000). When pride replaces exceptionalism, these means become 0.53 and 0.45 respectively, which is still significantly different, despite the smaller gap (p = 0.00). Consistent with our theoretical expectations, treatment effects are therefore heavily moderated by an individual’s expressed level of nationalism. Table 2 summarizes the outcomes of our hypothesis testing with respect to legitimacy measured as diffuse support.
Summary of hypothesis testing for diffuse support measures
COMPLIANCE ATTITUDES
We then moved to the questions designed to measure legitimacy as compliance attitudes (as an “internalized feeling of obligation”). Answers to these questions were added together to form an additive indexFootnote 16 ranging from zero to one, with higher values indicating a greater feeling of obligation. These compliance attitudes speak directly to the ability of the UK elites to ignore a decision by the ICC, which has significant implications for both domestic and international politics. Descriptively, the respondents reported greater feelings of obligation than diffuse support for the ICC. For example, among respondents in the control condition, the mean feeling of obligation toward the ICC is 62 percent compared to just 42 percent who are willing to defend it from alterations.Footnote 17 When compared to the UKSC, there is a greater feeling of obligation toward it than to the ICC but only slightly (69 percent compared to 62 percent). Experimentally, however, average treatment effects are similar to those found in relation to the diffuse support measure examined earlier, as shown in Figure 10. Finally, switching out treatment assignment with the embedded manipulation check shows that a move from “strongly disagree” to “strongly agree” (that the court does work in the United Kingdom’s national interest) results in a 51 percent increase in compliance attitudes.
Mean compliance attitudes across treatment groups showing a significant difference from the control condition.
The expected moderating effects of nationalism are not found with respect to compliance attitudes as they were with respect to diffuse support, and we found differing effects in response to the two types of nationalistic attitudes. For the pride (patriotism) measure, we found no significant moderating effects within any treatment group, as shown in the panel at right in Figure 11. Exceptionalism does have strong effects but not in the ways we predicted. For example, those respondents in the control condition who expressed highly exceptionalist attitudes have a mean value of 0.73, whereas those who expressed low exceptionalism have a significantly lower mean of 0.62 (p = 0.00; one-tailed). This pattern holds across all the treatment groups: those who view the United Kingdom as superior to other countries consistently report a higher internalized feeling of obligation to the ICC. This pattern is statistically significant for three of the four groups, with the exception being the positive treatment condition (p = 0.27; one-tailed).
Different conceptualizations of nationalism result in different moderating effects of the treatments.
One possible explanation for this unexpected finding is that British exceptionalism is connected with an internalized commitment to the rule of law (that is, the belief that what makes Britain exceptional is its commitment to upholding the rule of law). However, the negative findings regarding exceptionalists in the positive treatment condition, though not statistically significant, suggest that this is not the case or, at least, that it is not very malleable. Another potential explanation is that we are seeing the result of a ceiling or floor effect of compliance attitudes felt toward the ICC. Although our analysis is concerned with within-group variation based on nationalism, comparing between groups reveals a roughly consistent pattern. At least among those who view the United Kingdom as rising above all other countries, it appears difficult to change their feelings of obligation felt toward the ICC. Regardless, this is an interesting finding that should be investigated in future scholarship in other contexts. To verify the robustness of our findings, we conducted a series of subsequent analyses, including a mediation and sensitivity analysis. This further analysis can be found in the supplementary material.
DISCUSSION
To summarize, we have found as expected that, although the legitimacy afforded to the ICC is lower than that afforded to the UKSC, the amount of legitimacy afforded to it depends heavily on the extent to which the public perceives the court as being aligned with the United Kingdom’s national interests. Moreover, we have found that the provision of basic information about the ICC increases respondents’ perceptions of its legitimacy. Finally, consistent with our theory, nationalism is shown to heavily moderate the connection between perceived national interest and the legitimacy afforded to the ICC—at least where legitimacy is operationalized as diffuse support. Nationalism seems to cause a “double down” effect when respondents are exposed to negative information indicating that the ICC works against the national interest, reducing the effects of positive information indicating that the ICC’s work is in the national interest. Education level did not correlate with the ICC’s legitimacy in the way we would have expected if the dynamics shaping international courts mirrored those found with respect to domestic courts.
These are promising results for the theory, but we must also recognize their limitations. The current experiment was designed as a preliminary test conducted within a single country and in relation to one court. Further testing will be required to establish whether our theory can be generalized beyond that context and become a more general theory of international court legitimacy. In particular, our project focused on a young court that has jurisdiction over highly salient and emotive events. Drawing an analogy with the findings on domestic court legitimacy, we might expect that the legitimacy of international courts is most vulnerable to attack in the first decades of operation but might become more stable in the face of attack over time. Future research could test this finding comparatively (for example, with respect to the three regional human rights courtsFootnote 18 ) or via a longitudinal study of a single court. In either case, the findings would have significant real-world relevance for those working with, or opposed to, international courts as well as advancing understanding of the development of international court legitimacy. Comparative research might also consider whether the mechanism we have identified operates also in courts with more technical or less salient jurisdiction. Additionally, because the manipulation check in our survey indicated that there was statistically significant movement in the perception of national interests within the baseline condition, future scholarship should further scrutinize the relationship between providing neutral, factual information and judicial legitimacy of international courts.
Even with these limitations, our findings have theoretical, empirical, and practical implications that justify further research. From a theoretical perspective, we have shown that international court legitimacy is influenced by factors parallel to those affecting domestic court legitimacy and others that do not affect domestic court legitimacy. This overlap suggests that scholarship on both international and domestic legitimacy is enhanced to the extent that the conceptualization and measurement of legitimacy is consistent across both types of courts. However, recent scholarship in these two areas has diverged, making it difficult to integrate the insights of scholars of domestic and international courts. When scholars first attempted to understand the public legitimacy of international courts, the central challenge came from a lack of public awareness of what were then obscure institutions. More recently, the reason for the divergence is that the scholarship on international courts has on occasion (and of necessity) relied on imperfect measures and existing survey data. The increasing prominence of international courts and familiarity with experimental research methods in political science offer the opportunity to overcome these weaknesses and develop studies that accurately measure legitimacy and allow comparison across treatment conditions. We recommend that future research on court legitimacy take advantage of these developments to ensure that knowledge accumulation develops coherently and to allow dialogue between related areas of scholarship.
By manipulating perceptions of national interest, our experiment has demonstrated in a controlled environment the practical significance of our theory. We have shown that public perceptions of the legitimacy of international courts depend to a significant extent on perceptions of their alignment or conflict with national interests and that these perceptions can be shaped by outside cues. This creates an opportunity for those opposed to an international court’s jurisdiction to undermine its legitimacy and reduce the social and political costs of noncompliance with a disagreeable decision. Advocates of international courts and the courts themselves can respond to such threats through public information programs and outreach to build familiarity with a court’s role or to communicate competing narratives about the relationship between the court’s work and national interests. In other words, our findings suggest that attitudes toward the ICC are not crystallized and remain malleable.
CONCLUSION
Public legitimacy is an important attribute of governing institutions, especially for courts and even more so for international courts. High levels of legitimacy can reinforce a court’s authority, promote compliance, and raise the political and social costs of noncompliance. Legitimacy allows a court to issue unfavorable rulings that are still respected, viewed as authoritative, and implemented. Despite the growth of international courts and their geographical and substantive reach in recent decades, scholarship to date has made limited progress in understanding the extent to which the public ascribes legitimacy to these courts and the factors that shape legitimacy beliefs.
This article has offered several contributions to scholarship in this area. We have successfully tested the novel theory that perceptions of national interest are a significant determinant of international court legitimacy, generating findings with significant practical significance for actors wishing to support or challenge the authority of international courts. Moreover, we have shown that nationalistic attitudes strongly affect this relationship by reducing the effects of positive information and magnifying negative information. It is our hope that this study will generate more research in this important, yet understudied, domain.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/lsi.2021.21
