2. The debate over alliance interdependence

We follow Press (Reference Press2005, 10) in defining credibility as “the perceived likelihood that the threat will be carried out if the conditions that are supposed to trigger it are met.” Since we focus on credibility within alliances, we consider the perceived likelihood that a country will intervene in the defense of its allies if they are attacked. Like other aspects of reputation, credibility is therefore in the eye of the beholder. We focus on beliefs about intervention because they are the essence of alliances that promise active military support.

In turn, we define interdependence as the degree to which the perceived credibility of a state’s commitments to different alliances are correlated. This definition is agnostic as to the sources of interdependence, which vary.Footnote ² We equate interdependence with correlations as a broad initial cut, and turn to mechanisms with other tests.

There are several reasons why credibility might be interdependent. The simplest is that allies might base credibility assessments on aggregate partner attributes. Allies might consider partners’ internal characteristics, such as domestic politics and regime type (Gartzke and Gleditsch, Reference Gartzke and Gleditsch2004; Leeds et al., Reference Leeds, Mattes and Vogel2009), or situational characteristics such as relative power.

Alternatively, a large body of scholarship argues that credibility is interdependent because state behavior reveals information about interests and willingness to fight. In this telling, defending one ally will lead other allies to view the partner’s commitment as credible, while failing to support one’s partner or backing down in a crisis will negatively affect observers’ credibility perceptions (Crescenzi et al., Reference Crescenzi, Kathman, Kleinberg and Wood2012; Harvey and Mitton, Reference Harvey and Mitton2016; Dafoe et al., Reference Dafoe, Zwetsloot and Cebul2021). Schelling (Reference Schelling1966, pg. 55) argued explicitly that the U.S. made many Cold War commitments and had to stick to them because “our threats are interdependent.” The strongest version of this claim has few bounds, especially in policy and journalism. For example, some observers argued that U.S. withdrawal from Afghanistan would undermine U.S. credibility around the world (Schmitt and Cooper, Reference Schmitt and Cooper2021).

Other scholars advance a more modest claim: that credibility is conditionally interdependent. Allies may be more likely to draw inferences from how their partner behaves toward allies in similar situations. This similarity may stem from a variety of sources, including the stakes involved, the identity and strength of the adversary, and geography (Snyder, Reference Snyder, Jervis and Snyder1991; Gibler, Reference Gibler2008; Weisiger and Yarhi-Milo, Reference Weisiger and Yarhi-Milo2015; Lupton, Reference Lupton2020). Henry (Reference Henry2022) likewise argues that allies look for evidence that their partner shares their interests, rather than unconditional displays of loyalty to other allies.

A related body of research suggests that security interests vary systematically by region. For example, Buzan and Wæver’s (Reference Buzan and Wæver2003) “regional security complex theory” suggests that “security interdependence is normally patterned into regionally based clusters” (p. 4) because geographic proximity leads to repeated interactions and relational patterns. Danilovic (Reference Danilovic2002) argues that the inherent credibility of great powers’ deterrent threats varies by region due to distinct regional interests. A state’s capability to project power may vary by region as well, whether due to the composition and location of its military forces or adversaries’ capabilities (Bak, Reference Bak2018).

Similarly, perceptions of credibility may be correlated within regions because allies with similar regimes and cultures cluster and shared democracy and cultural similarity predict alliance reliability (Leeds, Reference Leeds2003; Lee et al., Reference Lee, Huygens and Mitchell2025). This may be particularly relevant for the perceived credibility of a democratic patron like the United States, given that democracies are likely to defend their democratic allies, especially given high stakes (Clare, Reference Clare2013). One might thus expect credibility to be highly interdependent within regions, but not necessarily across regions, owing to shared environments, shared allied characteristics and common overarching interests in each region.

A third strand of scholarship advances multiple reasons why credibility might not be interdependent at all. Situations may be too different to draw inferences about a partner’s inherent credibility from its behavior toward another actor (Hopf, Reference Hopf1994; Schmidt, Reference Schmidt2025). In this telling, observers are instead likely to rely on their assessments of a state’s interests in whatever is at stake, rather than its past behavior in other circumstances (Press, Reference Press2005). Moreover, a partner’s efforts to defend one ally may even undermine other allies’ perceptions of that partner’s credibility, insofar as those efforts expend resources that could defend them instead (Krebs and Spindel, Reference Krebs and Spindel2018; Kim and Simón, Reference Kim and Simón2021; Kim et al., Reference Kim, Byun and Ko2024). Finally, Mercer (Reference Mercer1996) argues that psychological biases from in-group/out-group dynamics mean allies will rarely see a partner’s displays of resolve as evidence of its intrinsic resolve, while adversaries will rarely see displays of irresolution as evidence of intrinsic irresolution. Rather, those observers will explain these events in terms of the situation.

These arguments make distinct predictions about the prevalence of correlations in alliance credibility, the distance at which credibility is correlated, and whether changes in the credibility of one alliance cause changes elsewhere. The strongest interdependence arguments suggest that most of a state’s alliance commitments should have correlated credibility, even across substantial geographic and political distances. Conditional interdependence arguments predict that some alliances with similar characteristics are correlated, especially when they are closer. No interdependence arguments claim that few alliances have correlated credibility, and credibility will be uncorrelated across and even within regions. Table 1 summarizes these implications.

Table 1.

Summary of arguments about interdependence between alliance commitments

Table 1 Long description

Arguments are listed with three attributes: correlated alliances, across regions, and causal. For strong interdependence, correlated alliances are many, across regions is yes, and causal is yes. For conditional interdependence, correlated alliances are some, across regions is weak, and causal varies. For no interdependence, correlated alliances are few or none, across regions is no, and causal is no.

Debate between these perspectives continues in part because systematic evidence is lacking. There is a variety of rich case-study and survey research on these questions, but generalizing from such evidence is challenging. Furthermore, many existing general measures of credibility are ill-suited for addressing interdependence.

3. Measuring credibility

Scholars have measured alliance credibility in six ways. Each measure has uses, but also pertinent shortcomings for assessing whether credibility is interdependent.

Some early work used treaty duration as a proxy for credibility and commitment (Gaubatz, Reference Gaubatz1996; Reed, Reference Reed1997). Alliance treaties can remain active but lack credibility, however. Some scholars argue that the Rio Pact is hardly an alliance at all (Rapp-Hooper, Reference Rapp-Hooper2020), despite being an active defense treaty according to the ATOP database (Leeds et al., Reference Leeds, Ritter, Mitchell and Long2002).

Due to these concerns, later work used alliance treaty violation to infer credibility (Leeds and Anac, Reference Leeds and Anac2005; Nieman and Gibler, Reference Nieman and Gibler2023). Violation demonstrates ex post that a commitment was not credible in a specific instance, but using it to measure credibility beforehand presents significant difficulties. The most important concern is that, like audience costs (Schultz, Reference Schultz2001), alliance violations are subject to partial observability and strategic selection. States often design alliance obligations to minimize the risk of violation (Benson, Reference Benson2012; Chiba et al., Reference Chiba, Johnson and Leeds2015), and opportunities to violate a treaty in response to a challenge may only arise when credibility is weak (Smith, Reference Smith1996). Furthermore, treaty violation is rare. Some statistical models attempt to correct for these issues (Nieman and Gibler, Reference Nieman and Gibler2023), but selection models often employ instrumental variable specifications that are vulnerable to commonplace violations of the exclusion restriction (Lal et al., Reference Lal, Lockhart, Xu and Zu2024) or model misspecification (Rainey and Jackson, Reference Rainey and Jackson2018).

The emphasis on treaty violation and comprehensive data from the Alliance Treaty Obligations and Provisions (ATOP) project led scholars to focus on treaty design as a source of credibility. Researchers sought to capture commitment by considering conditions on intervention (Benson, Reference Benson2012; Chiba et al., Reference Chiba, Johnson and Leeds2015), issue linkages (Poast, Reference Poast2013), and peacetime cooperation (Leeds and Anac, Reference Leeds and Anac2005; Alley, Reference Alley2021). Treaty content is usually static, however, while credibility is dynamic. Moreover, what is written down may not occur in practice.

Recently, scholars have turned to signals of support (Fuhrmann and Sechser, Reference Fuhrmann and Sechser2014; McManus and Nieman, Reference McManus and Nieman2019; Blankenship, Reference Blankenship2020). These works argue that alliance partners reveal commitment by either sinking costs or tying their hands. Taking costly actions such as deploying troops or making public statements of reassurance reveals commitment because a leader who was less willing to honor their alliance commitments would not do such things. Signals are closer, but are still an indirect and potentially noisy measure of credibility, given potential misperceptions (Jervis et al., Reference Jervis, Yarhi-Milo and Casler2021). The signals one state sends may miss their intended audience or have unintended consequences. Measuring credibility with signals themselves may therefore introduce substantial measurement error. Most importantly, the signals states send are not credibility itself, so signal measures risk conflating credibility with its predictors.

Finally, while not focused on alliance credibility in particular, some scholars treat dispute behavior as a proxy for credibility. One might expect actors that are perceived as credible to be more successful in deterring challengers from initiating disputes and more likely to win their disputes (Schultz, Reference Schultz2001; Weisiger and Yarhi-Milo, Reference Weisiger and Yarhi-Milo2015). Thus, credibility might be interdependent to the extent that a state and its allies face repeated challenges over time and/or simultaneous challenges across space. However, like other behavioral measures, dispute behavior is at best an indirect proxy for credibility. Moreover, it is difficult to say whether dispute behavior is a cause or effect of credibility. States and their allies may be targeted for disputes when challengers expect less resistance, raising questions about whether displays of irresolution reduce perceived credibility or low perceived credibility contributes to more challenges.

Measuring credibility is difficult, and all of these approaches are worthy efforts to proxy for a critical concept. Unfortunately, none measures credibility in a way that matches the concept directly or facilitates tests of interdependence.

4. Our credibility measure: expert ratings

We address measurement challenges by drawing on expert knowledge of how foreign policy elites perceived their alliance partners. Expert surveys have a long history in political science and are particularly useful for measuring concepts like credibility that require deep domain and contextual knowledge. For instance, Braumoeller (Reference Braumoeller2013) used expert surveys to classify the foreign policy orientation and ideology of great powers. In our case, experts are well-suited to capture the nuances of what credibility means to different allies and thus capture beliefs in a more granular and varied way than prior measures.

Our survey asked experts to classify the credibility of defensive alliances in the ATOP data (Leeds et al., Reference Leeds, Ritter, Mitchell and Long2002). Credibility refers to the extent to which states expected support in war from their alliance partners, or, for wartime coalitions, expectations that partners would continue to fight according to their obligations. We focus on defensive alliances because these treaties contain crucial obligations of support in war, unlike neutrality or non-aggression pacts.

We programmed 289 unique surveys for ATOP defense pacts in Shiny apps, and experts received up to three links for alliances on which they had expertise. Supplementary Appendix section 1 includes sample links and screenshots of several surveys. With help from research assistants, we matched a wide range of experts to specific alliances based on their publications and background. Experts either had meaningful background on a specific alliance or on the foreign policy of key members. Our expert pool included diplomatic historians, political scientists with case-specific expertise, and think tank members. To complement the general search and find experts who had written on specific alliances, we also examined the editorial boards of major history journals, book reviews featured in the American Historical Review dating back to 2000, and back issues of both major history journals and major security studies journals dating back to 2000. We likewise distributed a general version of our survey that indexed links to all alliance surveys to members of the American Historical Association, the Society for Historians of American Foreign Relations, the British International Studies Association, and H-Diplo via their discussion boards.Footnote ³

This initial wave of the survey generated 54 total submissions, and 28 submissions on U.S. alliances. Of the experts surveyed on U.S. alliances, 17% responded. Among our U.S. submissions, we received eight for NATO, four for Japan, three for South Korea and Israel, two for the Philippines and Taiwan, and one for all other alliances, including ANZUS, Spain, Pakistan, Thailand, the Rio Pact, and Iran. While experts could rate multiple alliances, only one did in our U.S. sample.

Our criteria for including U.S. alliances in the survey were to include all of those listed as defense pacts in the ATOP data for the duration they were in force. Those countries with defense pacts include four allies–Israel, Iran, New Zealand, and Taiwan–whose defense pacts with the United States were abrogated before the end of our sample period. Of these four, Israel, New Zealand, and Taiwan have continued to have some expectation of wartime support from the United States–New Zealand because it remains under Australia’s protection through ANZUS, Taiwan because of the Taiwan Relations Act, and Israel because of U.S. practice. Thus, we extend the timeframe for inclusion into the sample for Israel, New Zealand, and Taiwan. We exclude informal allies that never appear in the ATOP defense pact data (e.g., Saudi Arabia).

Due to the small pool of potential respondents for any alliance, we did not collect detailed demographic data on respondents. Given the cognitive burden of rating an alliance’s perceived credibility in each year, we streamlined each survey to minimize attrition. Moreover, even simple data on gender, occupation, and age could compromise respondent anonymity in small expert pools. Email correspondence, however, indicates that respondents came from all over the world, including the United States, allies, and other countries. This reduces potential American bias in the ratings (Colgan, Reference Colgan2019).

Our smaller expert pool is an unavoidable consequence of analyzing alliances. Similar efforts such as the Varieties of Democracy project recommend many raters per case, but start with a much deeper pool of country experts. Many alliances had less than five living experts, the minimum recommended by V-Dem. Given response rates, using this rule would restrict our data to only a few U.S. alliances.

We asked respondents to focus on the prevailing beliefs among the relevant foreign policy elite. There are often differences of opinion among a state’s foreign policy elites, but we emphasized the dominant consensus, should one exist. Crucially, this highlights the perceived credibility of the alliances by the actors themselves, rather than perceptions of the experts.

While our survey has responses for many alliances, our analysis of interdependent credibility focuses on U.S. defense pacts. We do this because we have ratings for every U.S. alliance after 1945. Furthermore, much of the policy and scholarly debate around interdependent credibility examines the vast U.S. alliance network (Kim and Simón, Reference Kim and Simón2021; Henry, Reference Henry2022; Kim et al., Reference Kim, Byun and Ko2024).

The aggregation in our credibility measure varies by alliance size and type. For bilateral and trilateral alliances (e.g., ANZUS), we asked respondents to measure each member’s perception that the other member or members would honor their obligations of support in war. Multilateral alliances–those with four or more members–were more challenging, because asking experts to rate credibility perceptions for every treaty member would have created enormous cognitive burdens and produced inconsistent coverage within treaties. For multilateral asymmetric alliances which had a clear alliance leader in the form of one state that was much more capable than the others, we asked experts to rate the leader’s commitment and whether members could expect support from at least one alliance member. In our U.S. data, this applies to NATO and the Rio Pact. For multilateral alliances without a leader, which are absent in the U.S. data, expert respondents assessed whether at least one alliance partner would provide promised support in war. This accounts for the critical distinction between multilateral and bilateral alliances (Poast, Reference Poast2010), because while group and single-country perceptions differ, we measure partner-level perceptions in both cases. It is also important because many existing studies of alliance credibility only analyze bilateral treaties (Leeds et al., Reference Leeds, Mattes and Vogel2009; Nieman and Gibler, Reference Nieman and Gibler2023). The downside is that we only ask for aggregate ratings of multilateral alliances because it would not be possible to have comprehensive expert coverage for all of their members.

The key survey measure rates an alliance actor’s credibility on a scale from one to four. A score of one indicates “an alliance in name only,” where partners had no expectation of support. Two indicates a partner was “unsure” of support. Three marks years when actors were “fairly sure” that their partner(s) would support them in wartime, and four when alliance actors were “very sure.” We asked respondents to mark the initial level and then only mark years when that credibility changed, using the four-point scale to ensure that responses were roughly consistent across respondents.Footnote ⁴

There are at least three conceptual objections to this approach. First, while we ask respondents for the prevailing view in the foreign policy elite of an alliance actor at that time, there may be some retrospective element to the ratings. Respondents’ knowledge of what happened in the near or distant future may influence their views of how policymakers viewed the alliance at the time. But while hindsight brings some potential risks, it also allows more information to come to light about the inner workings of the alliance and how policymakers viewed credibility, which experts with deep historical knowledge are ideally suited to evaluate.

Second, we collected ratings of perceived credibility for partners, not adversaries. Credibility applies to both partners and adversaries, however. We did not ask about adversaries because identifying an adversary for every alliance is difficult and adversaries can shift over time. To the extent an adversary’s assessment diverges from those of the partners, this limits the data’s utility for exploring research questions about adversary behavior. Nevertheless, the focus on ally perceptions makes the data well-suited to assessing alliance interdependence, especially in terms of reassurance.

Third, while our scale is relatively simple, experts will perceive it differently. Individual respondents may place distinct thresholds between the different outcome levels.Footnote ⁵ These commonplace perceptual differences are sometimes called “differential item functioning” (Marquardt, Reference Marquardt2020). Differences across alliances, including multilateral and bilateral treaties, further increase this concern.

We therefore address distinct expert perceptions and combine responses onto a common scale, using a hierarchical IRT model. In IRT models, observed data reflect an underlying latent concept, and the model reduces noise and estimates different weights to measure that concept. In our application, expert ratings are the observed data, and high ratings reflect high credibility, just as many correct test answers reflect higher ability. Using an IRT model to measure credibility accounts for expert and alliance-specific differences in expert perceptions. Other popular measures of political ideology (Clinton et al., Reference Clinton, Jackman and Rivers2004) and democracy (Marquardt and Pemstein, Reference Marquardt and Pemstein2018) that face similar challenges in harmonizing perceptions of latent concepts also use IRT models to aggregate expert ratings.

In our ordered logistic IRT model, we estimate latent U.S. credibility in each alliance-year observation based on the expert ratings.Footnote ⁶ The credibility estimate in each year depends on three inputs. First, the expert ratings themselves. Second, an expert-specific discrimination parameter lets the difference between levels of our scale vary for each expert submission across all the years and partners they rated. Crucially, there are no regional patterns in these parameters, which means that our findings are not driven by common expert approaches within regions. Third, we adjust for potential shifts in the level or meaning of credibility across different alliances with alliance-specific intercepts. We model dynamics in the model with a Student-t prior on latent credibility, which allows sharp changes over time (Reuning et al., Reference Reuning, Kenwick and Fariss2019). This approach fits the observed data much better than a dynamic IRT model with a random walk prior.

The hierarchical model has two critical properties. First, it accounts for differences in how experts perceive the scale and shifts in the differences in levels across alliances, including potential distinctions between multilateral and bilateral treaties. Second, the model pools information across alliances by giving key parameters a common distribution, which makes alliances with fewer expert ratings less likely to be outliers. Alliance-year observations with fewer expert ratings shift closer to the overall mean. This regularization, which is visible in comparing the raw ratings data and the model estimates for partners such as Iran, is important because we have uneven depth in our geographic coverage.

Our final measure is the posterior median of the latent credibility parameter in the IRT models, rescaled to range between 0 and 1.Footnote ⁷ The resulting measure of credibility has several useful properties. First, they directly match our definition of credibility. Second, these measures have some face validity in changes over time and how they correlate with other indicators. Crucially, we also find equivalent results with median credibility ratings.

5. Expert ratings data

First, we present the expert ratings of U.S. credibility. In Figure 1, faint lines mark the individual submissions, and the black line summarizes the median rating. We use the median because it is often a more accurate summary than the average when expert coders give different responses (Lindstädt et al., Reference Lindstädt, Proksch and Slapin2020). For alliances with only one submission, the median and submission lines overlap.

Figure 1.

Ratings and median credibility rating for U.S. alliance partners. The faint lines show individual expert ratings, and the black line marks the median.

Figure 1 Long description

US Credibility Ratings: 1949-2023. Small multiple step line graphs for Australia, Iran, Israel, Japan, New Zealand, North Atlantic Treaty Organization, Pakistan, Philippines, Rio Pact, South Korea, Spain, Taiwan, Thailand. Purpose text in the figure: Ratings and median credibility rating for U. S. Alliance partners. Axes and scale used in each small graph. Horizontal axis label: Year. Tick labels: 1960, 1980, 2000, 2020. Vertical axis tick labels: (3) High credibility, (2) Moderate credibility, (1) Low credibility, (0) Alliance in name only. Line identification and style. Each small graph contains multiple faint step lines and one thicker step line. Comparative summary across partners. Most thicker lines spend long stretches at (2) Moderate credibility. Several thicker lines include segments at (1) Low credibility. One thicker line includes a segment at (0) Alliance in name only near the end of the timeline. Data points listed as coordinate pairs from the thicker step line in each small graph. Australia: (1960, 3), (1960, 2), (2020, 2). Iran: (1960, 2), (1960, 3), (1980, 3), (1980, 2), (1980, 3), (2000, 3). Israel: (1960, 2), (1980, 2), (1980, 3), (2000, 3). Japan: (1960, 2), (1980, 2), (1980, 1), (1980, 2), (2000, 2), (2000, 1), (2000, 2), (2020, 2). New Zealand: (1960, 3), (1960, 2), (1980, 2), (1980, 1), (2000, 1), (2000, 2), (2020, 2). North Atlantic Treaty Organization: (1960, 2), (1980, 2), (2000, 2), (2020, 2), (2020, 0). Pakistan: (1960, 2), (1960, 1), (1980, 1), (1980, 2), (1980, 1), (2000, 1), (2000, 2), (2020, 2). Philippines: (1960, 2), (1980, 2), (1980, 1), (2000, 1), (2000, 2), (2020, 2), (2020, 1), (2020, 2). Rio Pact: (1960, 2), (1980, 2), (1980, 1), (2000, 1), (2000, 2), (2020, 2). South Korea: (1960, 2), (1980, 2), (1980, 1), (2000, 1), (2000, 2), (2020, 2). Spain: (1960, 3), (1960, 2), (1980, 2), (1980, 1), (2000, 1), (2000, 2), (2020, 2). Taiwan: (1960, 3), (1960, 2), (1980, 2), (1980, 1), (2000, 1), (2000, 2), (2020, 2). Thailand: (1960, 2), (1980, 2), (1980, 1), (2000, 1), (2000, 2), (2020, 2).

These ratings have some face validity. Across several ratings, U.S. credibility toward Japan is high and steady over time. Perceived U.S. credibility in NATO, which is bundled across all members of the multilateral alliance, is generally high during the Cold War. It then drops slightly at the end of the Cold War and plunges during the Trump administration, recovering under Biden. U.S. credibility also falls dramatically in South Korea during the Carter administration, which publicly considered withdrawing American forces. U.S. credibility to Taiwan bottoms out after the U.S. abrogated a formal treaty commitment and recognized the People’s Republic of China, and has slowly recovered since. Iranian perceptions of U.S. credibility drop around the fall of the Shah. Pakistani expectations of U.S. support are volatile and highly contingent on geopolitical context.

Due to potential differences in perceptions of the scale across experts, IRT estimates are likely more reliable than the raw data. We present the IRT ratings in Figure 2. The ratings are generally similar in when credibility shifts, but the levels depend somewhat on the expert discrimination parameters. For instance, the average rating for Spain moves toward the median value because there is little data. NATO also has greater uncertainty due to wider expert disagreement.

Figure 2.

IRT estimates of credibility for U.S. alliance partners. Error bars mark the 90% credible interval.

Figure 2 Long description

Multiple line graphs depicting US credibility ratings for different countries and organizations from 1949 to 2023. Each graph represents a specific entity: Australia, Iran, Israel, Japan, NATO, New Zealand, Pakistan, Philippines, Rio Pact, South Korea, Spain, Taiwan and Thailand. The horizontal axis is labeled ′Year′ and spans from 1949 to 2023. The vertical axis is labeled ′IRT estimates′ and ranges from 0 to 1, representing credibility ratings. Error bars indicate the 90 percent credible interval for each data point. Australia shows stable ratings around 0.75. Iran fluctuates between 0.25 and 0.5, with a peak near 1980. Israel′s ratings rise steadily from 0.5 to 0.75 after 1980. Japan remains stable around 0.5. NATO shows fluctuations, peaking near 0.75 in 2020. New Zealand′s ratings are stable around 0.75. Pakistan fluctuates between 0.25 and 0.5, with a drop in 2000. Philippines shows a gradual decline from 0.5 to 0.25. Rio Pact fluctuates around 0.5. South Korea shows a decline from 0.75 to 0.5. Spain′s ratings rise from 0.25 to 0.5 after 2000. Taiwan fluctuates between 0.5 and 0.75. Thailand shows a decline from 0.75 to 0.5 after 1980. The graphs illustrate varying trends, with some entities showing stable ratings and others displaying significant changes. The purpose of the graphs is to show differences in credibility ratings over time for US alliance partners.

Our expert ratings also correlate with other measures in sensible ways, as Figure 3 shows. Figure 3 presents scatterplots measuring the correlation between credibility ratings and prior measures. High scores on the latent measure of McManus and Nieman (Reference McManus and Nieman2019), along with more U.S. troops (Allen et al., Reference Allen, Flynn and Martínez Machain2022), leader visits, statements of support (Blankenship, Reference Blankenship2020), and military exercises (D’Orazio, Reference D’Orazio2013) in a year, are all correlated with greater credibility. UN voting distance is also correlated with credibility, as greater UN distance overlaps with low credibility (Bailey et al., Reference Bailey, Strezhnev and Voeten2017). Presidential visits, statements, and S-scores are less correlated with credibility, however, S-scores are essentially uncorrelated.

Figure 3.

Correlations between expert credibility measure and other proxies for alliance commitment and credibility. Points mark alliance-year observations, and a linear smooth shows the correlation. Latent support, total troops, and UN voting distance are averaged for multilateral alliances, while we sum exercises, statements, and presidential visits.

Figure 3 Long description

Seven scatter plots arranged in a grid under the heading, Expert Ratings and Other Credibility Measures. Latent Support scatter plot. Horizontal axis label, Latent Support, with tick labels 0, 1, 2, 3. Vertical axis label, Credibility IRT estimate, with tick labels 0.00, 0.25, 0.50, 0.75. A fitted line slopes upward. Log Total Troops scatter plot. Horizontal axis label, Log Total Troops, with tick labels 0, 2, 4, 6. Vertical axis label, Credibility IRT estimate, with tick labels 0.00, 0.25, 0.50, 0.75. A fitted line slopes upward. Military Exercises scatter plot. Horizontal axis label, Military Exercises, with tick labels 0, 20, 40, 60. Vertical axis label, Credibility IRT estimate, with tick labels 0.00, 0.25, 0.50, 0.75. A fitted line slopes upward. Pres. Visits scatter plot. Horizontal axis label, Pres. Visits, with tick labels 0.0, 2.5, 5.0, 7.5, 10.0, 12.5. Vertical axis label, Credibility IRT estimate, with tick labels 0.00, 0.25, 0.50, 0.75. A fitted line slopes upward. S Score Kappa scatter plot. Horizontal axis label, S Score Kappa, with tick labels 0.00, 0.25, 0.50, 0.75, 1.00. Vertical axis label, Credibility IRT estimate, with tick labels 0.00, 0.25, 0.50, 0.75. A fitted line slopes upward. Statements scatter plot. Horizontal axis label, Statements, with tick labels 0, 5, 10, 15. Vertical axis label, Credibility IRT estimate, with tick labels 0.00, 0.25, 0.50, 0.75. A fitted line slopes upward. UNGA Distance scatter plot. Horizontal axis label, UNGA Distance, with tick labels 0, 1, 2, 3, 4. Vertical axis label, Credibility IRT estimate, with tick labels 0.00, 0.25, 0.50, 0.75. A fitted line slopes downward.

Although our measure correlates with existing measures, it adds value in three ways. First, some large shifts in credibility have little to do with the indicators in prior measures, and are often the result of specific events or changes in leadership. U.S. credibility toward New Zealand collapsed due to disagreements over port access for nuclear-armed vessels, for example. Second, all these measures have substantial unexplained variation in credibility. The direction of the correlations is as we should expect, but there is ample variation around the general trend, as observations with similar values on objective indicators can have dramatically different credibility values in our measure. Third, most of these variables outside of S-scores and UN voting distance are factors that might influence credibility, but are not themselves measures of credibility. Moreover, S-scores are poorly correlated with perceived credibility, while UN voting distance is a measure of foreign policy similarity rather than credibility.

In addition to the conceptual advantages, our measure has more complete coverage of alliances and years for the United States. The McManus and Nieman (Reference McManus and Nieman2019) latent measure ends in 2012 and the statement data in 2010 (Blankenship, Reference Blankenship2020). Furthermore, Taiwan is missing in some data because it is not a COW system member and South Korea did not join the UN General Assembly until 1991, which limits coverage of some interest-based measures.

6. Is credibility interdependent?

We now use our credibility measure to examine interdependence across U.S. alliances. Unlike some prior work, we focus on inferring interdependence, not adjusting for autocorrelation to make unbiased inferences about a covariate. To do so, we must account for three forms of correlation. These include contemporaneous spatial correlations of credibility between different alliances, persistent spatial correlations over time, and correlations over time within alliances. We estimate all three correlations but focus on spatial correlations in the same year because the three arguments make different predictions about how much interdependence persists as distance increases. Strong interdependence arguments suggest meaningful correlations across long distances, while conditional and no interdependence theories expect weak or absent correlations at great distances.

In testing, we are especially wary of inadvertently biasing results by fixing temporal and spatial lags in a model specification or hard-coding decay functions (Crescenzi et al., Reference Crescenzi, Kathman, Kleinberg and Wood2012; Weisiger and Yarhi-Milo, Reference Weisiger and Yarhi-Milo2015). We therefore use Gaussian process regression to estimate correlations in U.S. alliance credibility. These models are a flexible mix of traditional linear models and machine learning (Prati et al., Reference Prati, Chen, Montgomery and Garnett2023),Footnote ⁸ and scholars often use them to examine spatial interdependence in continuous distances (McElreath, Reference McElreath2016; Gill, Reference Gill2021; Aksoy and Carlson, Reference Aksoy and Carlson2022; Chen et al., Reference Chen, Garnett and Montgomery2023). We estimate correlations between alliance observations in time and physical space because failing to account for one type of interdependence will impact our estimates of the other (Cook et al., Reference Cook, Hays and Franzese2023).

The key intuition behind our approach is that alliance-year observations of credibility become more dissimilar and perhaps uncorrelated as geographic and temporal distance increases. Observations within an alliance are perfectly similar in geography, but differ in time. Observations of different alliances have different locations and become even less similar as the gap in years increases. Because the rate at which correlations decay is uncertain, we estimate it instead of fixing proximity.

The model estimates correlations between all the alliance-year observations in our data based on their year, latitude and longitude. Each observation has a unique intercept based on its geographic position and year, and the model estimates correlations between these alliance-year intercepts. The most important choice is how we group these latent parameters of the Gaussian process.

We report results from a model wherein all observations could be correlated depending on their distance from one another in both space and time. A second model we report in the appendix groups responses by year, estimating correlations between allies within years. The model we report here has the most complicated possible structure but fits far better than simpler models because it encapsulates the full correlation structure. Alternative structures lead to pathological behavior in cross-validation comparisons of model fit.

Initial correlations between the closest possible data points, whatever their level, can decay to zero or remain as distance increases. This decay depends on two parameters that control the covariance function of the Gaussian process. One parameter sets the overall variance and another controls the scale of the process—the distance at which correlations matter. A key choice is the kernel encoding some prior beliefs about the shape of input functions. We use a squared exponential kernel because it encodes a smooth prior that is otherwise very flexible (Prati et al., Reference Prati, Chen, Montgomery and Garnett2023). This allows correlations in credibility to persist over long distance in time and space or drop off very quickly.

Our outcome variable is the median credibility rating from the IRT model, rescaled between 0 and 1. This bounded outcome has a lumpy distribution, so we use ordered beta regression for the likelihood (Kubinec, Reference Kubinec2023). We then fit models that use a dummy indicator of whether the alliance is bilateral, the log of U.S. troops present in the partner, U.S. changes in presidential party, partner democracy, and s-scores to predict mean credibility. We use these measures because they have the best temporal and cross-sectional coverage. In addition to these baseline covariates, we added the aforementioned Gaussian process term or kernel based on an observation’s latitude, longitude, and year.Footnote ⁹ In the appendix, we provide a formal presentation of this Bayesian ordered beta regression with a constructed kernel that allows for non-independent errors based on time and space alongside some covariates outside the kernel.

8. Why is credibility correlated within regions?

While the preceding results suggest that alliance credibility is correlated within regions, they provide limited insights into why this pattern exists. The correlations could reflect changes in credibility itself, the diffusion of U.S. actions, or common factors. We unpack these possibilities with two additional tests, results for which are in sections five and six of the Supplementary appendix.

First, we test whether these correlations might follow from allies reacting to U.S. behavior in the same way. We do this with three measures of U.S. behavior: (1) whether the United States had backed down or won in recent militarized interstate disputes; (2) U.S. performance during alliance performance opportunities; and (3) U.S. alliance treaty terminations that were in violation of the alliance’s terms. The results suggest that past actions have a small role in credibility correlations. However, effect sizes are small and sensitive to model specification.

Second, we use spatial regression models to test direct relationships between credibility values (autocorrelation) and common unobservables (spatial error). The spatial tests are critical because different spatial models reflect different processes. Contrary to strong interdependence arguments, which argue that credibility changes in one alliance impact credibility elsewhere, we find little autocorrelation. Instead, nonzero spatial error suggests that our Gaussian process models pick up correlated unobservables within regions. This and small effects of U.S. actions suggest that, following Press (Reference Press2005), observed credibility correlations reflect region-specific assessments of U.S. capabilities, interests, and reliability given distinct threats.

9. Discussion and conclusion

Our findings support conditional credibility arguments. Alliance credibility is strongly correlated among close allies but weakly correlated beyond regions. This implies that alliance credibility is interdependent, but not on a global scale.

There is therefore little evidence for strong claims about interdependent or independent alliance credibility. We find little evidence of meaningful correlations in credibility across regions or spatial autocorrelation. As a result, upholding credibility in one region will have at best a marginal relationship with credibility in more distant alliances. But nearby allies do exhibit meaningful correlations due to spatial error.

The results have policy implications as well. Policymakers who are considering reducing alliance commitments can expect that lost credibility in one treaty matters most to proximate partners. But downgrading U.S. commitments in one region might not lead perceived credibility in other regions to decline. The results thus suggest some optimism that prioritizing alliance commitments in one region will not come at the expense of a tarnished reputation elsewhere. To the extent that support for strengthening or maintaining the status quo in U.S. alliances draws on fears that allies make inferences about American credibility across regions, our findings suggest this concern may be misplaced.

Our study has some limitations. For one, there is heterogeneity in how different partners within multilateral alliances like NATO perceive credibility, and our measure does not capture those differences. Moreover, we focus on U.S. allies, and other countries with fewer commitments may exhibit different patterns of interdependence and independence. In particular, one might object that two features may limit the generalizability of these findings beyond the United States. The first is that U.S. alliances span the globe, which may make interdependence especially important. Yet given that our findings suggest that interdependence is strongest at the regional level, the results should apply to other countries with multiple allies in the same region. Second, U.S. alliances are highly asymmetric and many are far from the U.S. homeland. This could make allies particularly attuned to changes in U.S. credibility and how the United States behaves toward nearby partners.

Future research should therefore broaden the scope of this analysis beyond the United States, or consider interdependent credibility within multilateral U.S. alliances. Scholars could also use our credibility measure to predict other aspects of U.S. partners’ foreign policy. Others could also tackle the causal mechanisms behind interdependence and clarify their relative weight. Researchers should expand and integrate work on the psychology of key actors as they do this.

Credibility is conditionally interdependent within regions. Upholding credibility matters most for proximate partners. Leaders need not fight for allies everywhere, but nor can they reduce commitment without undermining their credibility among nearby allies.