Hostname: page-component-74656f9884-54qdv Total loading time: 0 Render date: 2026-11-10T21:46:58.053Z Has data issue: false hasContentIssue false
  • English
  • Français

Do Treaties Constrain or Screen? Selection Bias and Treaty Compliance

Published online by Cambridge University Press:  31 October 2005

JANA VON STEIN
JANA VON STEIN
Affiliation:
University of Michigan
Rights & Permissions [Opens in a new window]

Abstract

Much recent research has found that states generally comply with the treaties they sign. The implications of this finding, however, are unclear: do states comply because the legal commitment compels them to do so, or because of the conditions that led them to sign? Drawing from previous research in this Review on Article VIII of the IMF Treaty (Simmons 2000a), I examine the problem of selection bias in the study of treaty compliance. To understand how and whether international legal commitments affect state behavior, one must control for all sources of selection into the treaty—including those that are not directly observable. I develop a statistical method that controls for such sources of selection and find considerable evidence that the unobservable conditions that lead states to make the legal commitment to Article VIII have a notable impact on their propensity to engage in compliant behavior. The results suggest that the international legal commitment has little constraining power independent of the factors that lead states to sign.

Information

Type
FORUM
Information
American Political Science Review , Volume 99 , Issue 4 , November 2005 , pp. 611 - 622
Copyright
© 2005 by the American Political Science Association

Are international agreements only a reflection of states' preferences, or can they also alter leaders' interest in pursuing a particular course of action? In recent years, a number of international relations and legal scholars have sought to answer this question by examining whether states abide by the international legal commitments they make. Much of this literature has found that states generally comply with the treaties they sign, whereas enforcement problems are minimal (Chayes and Chayes 1995; Young 1994). As others have noted, however, compliance does not by itself demonstrate that international law constrains state behavior in meaningful ways. Downs, Rocke, and Barsoom (1996) argue that a state's decision to sign a treaty is endogenous to its expectations about future compliance. Consequently, compliance data alone do not tell us whether states abide by the treaties they sign because the legal commitment compels them to do so, or because they sign treaties that do not require significant departure from what they would have done in the absence of the treaty. To even begin to overcome this problem, one must first control for the basis of state selection (Downs, Rocke, and Barsoom, 383).

Theoretically and empirically, this insight is of central importance to the study of international institutions. Any theory of treaty compliance must recognize that institutional design is at least in part endogenous: states are only likely to invest their time and resources in agreements with which they have at least some interest in complying. This means that we must also think about how the conditions that lead states to sign the agreement affect their postsigning behavior. Moreover, much of our reasoning must be expressed in counterfactuals: if the institution constrains state behavior, then it must be the case, all else equal, that a signatory would have engaged in compliant behavior less had it not signed, and/or that a nonsignatory would have engaged in compliant behavior more had it signed.

Empirical research on treaty compliance—both qualitative and quantitative—must also account for endogeneity and selection effects. This article explores the implications of these problems for the latter type of empirical research. If states sign international agreements only when certain conditions are present, examining whether signatories engage in compliant behavior more than do nonsignatories does not enable us to distinguish whether the behavior is attributable to the agreement itself, or to the conditions that led them to sign (Przeworski and Vreeland 2000, 387). One important way of mitigating this problem is by including in one's statistical analyses variables that control for the factors that affect both the decision to sign and the subsequent compliance. Yet, if some of these factors are unobservable, standard regression techniques will continue to yield biased results of the treaty commitment's effect.

Drawing from research in this Review on Article VIII of the International Monetary Fund (IMF) Treaty (Simmons 2000a), this article examines the problem of selection bias in the study of treaty compliance. I develop a statistical method that allows one to estimate the treaty commitment's effect on state behavior independent of all sources of selection—including those that cannot be directly measured. I find strong evidence that the unobservable factors that lead states to sign Article VIII significantly increase their propensity to engage in compliant behavior. The results with regard to nonsignatories are less conclusive, but suggest that the unobservable factors that lead states not to make the treaty commitment decrease their propensity to engage in compliant behavior. Failing to control for the sources of selection leads one to overstate considerably the effect of an Article VIII commitment on compliant behavior. Indeed, the international legal obligation appears to have little constraining power independent of the factors that lead states to sign.

ARTICLE VIII COMMITMENT AND COMPLIANCE: PREVIOUS FINDINGS AND THE PROBLEM OF SELECTION BIAS

States that sign Article VIII of the IMF Treaty commit, among other things, to keeping the current account free from restriction. This entails allowing residents to use national currency or obtain foreign currencies to remunerate nonresidents for international transactions and permitting nonresidents who have obtained the national currency through current international transactions to use or transfer those balances (Edwards 1985, 390–93). Governments may wish to restrict the current account to mitigate balance-of-payments problems, or to support developmental goals that favor certain types of transactions (exports, capital inflows) over others (imports, capital outflows) (Simmons 2000a, 820). The Fund generally views these as undesirable practices that distort economies and hinder development (Edwards, 425–26).

Official IMF policy stipulates that while members may at any time inform the Fund that they accept the obligations of Article VIII, it is desirable that they “satisfy themselves that they are not likely to need recourse” to current account restrictions in the foreseeable future.1

Executive Board Decision 1034- (60/27), (IMF Transitional Arrangements, Articles VIII and XIV).

In practice, the Fund exercises significant discretion over the accession process. During annual consultations, it first encourages members that have not assumed Article VIII status to decrease or eliminate restrictions on the current account. Once a member has done so, the Fund usually then urges it to make the treaty commitment (Simmons 2000b, 581). In this manner, although the decision to sign ultimately lies in the hands of national authorities, the Fund's Executive Board has been fairly successful at imposing its preference that a member not sign Article VIII until it has eliminated current account restrictions significantly or entirely (Edwards 1985 404, 422–23).

States cannot rescind an Article VIII commitment formally, and the IMF does not provide direct rewards for signing or punishments for not signing (Simmons 2000a, 823). Why, then, do states accept the treaty obligation? Simmons (819–21) argues, “Article VIII commitment is one way in which governments may seek to enhance their credibility to markets that doubt their ability or willingness to maintain current account policy liberalization…The acceptance of treaty obligations raises expectations about behavior that, once made, are reputationally costly for governments to violate.” In this interpretation, by signing Article VIII, governments attempt to signal their policy intentions by tying their hands—that is, by creating reputational costs that they will suffer ex post if they renege.2

See Fearon 1997 for game-theoretic models of signaling foreign policy interests using ex post or ex ante costs.

This implies that signatories will be more likely to engage in compliant behavior, ceteris paribus. The analytical problem this poses, however, is that the ceteris paribus upon which the comparison hinges is unlikely to hold in practice: the IMF encourages countries it believes are ready to do so to sign, and the clearest indicator of such readiness is a low or null level of restrictions. If states sign only when certain conditions are present, it is difficult to distinguish whether signatories engage in compliant behavior more than do nonsignatories because of the agreement itself or because of the conditions that led them to sign (Przeworski and Vreeland 2000).

The intuition behind this problem can be clarified via a comparison from the field of medicine. Imagine that, to test the effectiveness of a new treatment, doctors ask sick people and healthy people to choose whether to take the drug, and then compare the health of those who took it with those who did not. In all likelihood, the sick will have opted to take the treatment in the hopes of being cured, whereas the healthy will have chosen not to do so because of potential side effects. If medical researchers attempt to draw conclusions about the drug's effectiveness by comparing the two groups' health, they will be unable to decipher whether the differences are attributable to the treatment or to the disease itself. Instead, of course, medical researchers test treatments by placing sick patients randomly into two groups—one that receives the treatment and another that is given a placebo. They can then draw unbiased conclusions about the drug's effectiveness because they have two groups that are exactly alike, except that only one has received the treatment.

Just as it is not possible in the hypothetical medical example to determine the drug's effectiveness by comparing the health of sick people who chose to take the treatment with that of healthy people who opted not to take it, it is not possible in the Article VIII case to draw conclusions about the treaty commitment's constraining effect by comparing the restriction behavior of signatories to that of nonsignatories. Indeed, doing so does not tell us whether the observed behavior is attributable to the international legal commitment or to the underlying characteristics/conditions that lead states to sign or not sign. Yet in the Article VIII case, as in much social science research, we do not possess the experimental control that medical researchers do. We cannot create a control group of states that possess the attributes of nonsignatories but sign, or a control group of states that possess the attributes of signatories but do not sign. As a result, it is very unlikely that we will find two states that are alike in every way, except that one has signed and the other has not (Przeworski and Vreeland 2000, 386–87).

Hence, we are faced with a violation of one of the fundamental assumptions of classical regression theory: random selection. One important way of mitigating this problem is by controlling for the factors that affect both selection into the treaty (let us call this the selection equation) and the extent of compliant behavior (let us call this the outcome equation). Simmons makes important efforts to do so. Even when controlling for these sources of selection, she finds an Article VIII commitment to have a substantively large and statistically significant effect on restriction behavior. Indeed, signatories are up to 27% less likely to restrict the current account than are nonsignatories (Simmons 2000a, 830–31).

If, however, some unobservable factor(s) also leads states to sign and affects compliant behavior, estimates of the legal commitment's impact will continue to be biased.3

Simmons (2000a, 829) recognizes this possibility, but does not control for it.

In some instances, controlling for observed variables can increase the bias (Achen 1986; Przeworski and Limongi 1993). Indeed, although many of the conditions that lead states to sign agreements or undertake policies can be measured, some are unlikely to be measurable. Przeworski and Vreeland (2000, 387) and Vreeland (2002, 124) suggest, for example, that “political will” may affect a government's decision to enter an IMF program as well as its behavior subsequent to entering, but that this variable cannot be directly measured. Other examples of such unobservables include “trust” and “negotiation posture” (Vreeland 2003, 5–8, 52–54).

What unobservable factor(s) might affect commitment to and compliance with Article VIII? The IMF repeatedly has stated that by signing, a country “gives confidence to the international community that it will pursue sound economic policies.” Similarly, Article VIII status is viewed by many as a “fundamental indicator of ‘good standing’ in the Fund.”4

See IMF, “Zambia Accepts Article VIII Obligations,” Press Release No. 02/26, 20 May 2002; and Shiraz Sidhva, “India Completes Key Reform of Currency,” Financial Times, 21 August 1994, page 4, London Edition.

A government's commitment to sound economic policies and/or desire to demonstrate “good standing” in the Fund are not directly observable attributes. Yet, these factors are likely to play a key role in determining a state's propensity to engage in compliant behavior and to accept Article VIII status. More specifically, because governments that place greater value on liberal economic policies and/or demonstrating “good standing” in the Fund are probably less likely to restrict, and those governments are probably also more likely to sign Article VIII, standard regression techniques are likely to overstate the extent to which being a signatory decreases the propensity to restrict. Conversely, because governments that place little value on liberal economic policies and “good standing” in the Fund are probably more likely to restrict, and those governments are probably also less likely to sign, standard regression techniques are likely to overstate the extent to which being a nonsignatory increases the propensity to restrict.

As I demonstrate formally later in this article, the result is that standard regression techniques are likely to overstate the impact that a legal obligation to Article VIII has on restriction behavior, attributing to it the unobservable factors that lead states to sign or not sign and to engage in compliant behavior. Ideally, one would measure these unobservable attributes/conditions and include them in one's analyses. Because it is not likely that all sources of selection can be measured, we must instead adjust our statistical techniques (Przeworski and Vreeland 2000; Vreeland 2002, 2003).

WHO SIGNS? A CLOSER LOOK AT PATTERNS OF COMMITMENT AND COMPLIANCE

The previous section discusses why selection effects are likely to be present in the Article VIII case, and how one might expect them to affect estimates of the treaty commitment's impact on compliant behavior. I now turn to the empirical record and conduct a number of preliminary graphical and statistical analyses to determine whether there is evidence of selection and/or endogeneity. The data are yearly observations for up to 133 IMF members from 1967 to 1997 (Simmons 2000a). The dependent variable of interest, Restrict, equals one if a state placed restrictions in year t, and zero otherwise. The sample contains 1,354 signatory-observations (of which 350 restrict the current account) and 1,746 non-signatory-observations (of which 1,326 restrict the current account). Starting with those states that have not yet signed Article VIII but eventually sign, I calculate the average number of states placing restrictions as a function of the number of years remaining until signature. Next, for states that have already signed, I calculate the average number of states placing restrictions as a function of the number of years since signature. Finally, for states that never sign, I calculate the average number of states placing restrictions. Figure 1 displays these calculations, along with confidence intervals to assess the degree of variation in restriction behavior.

The Percentage of States Placing Current Account Restrictions as a Function of the Number of Years to and since an Article VIII Commitment

As Figure 1 demonstrates, a notable change in current account behavior takes place approximately 4 years prior to an Article VIII commitment: the percentage of countries placing restrictions decreases sharply from 70% to 31% and reaches levels considerably lower than those attained at any other point prior to signing. Immediately following the treaty commitment, the percentage of states placing restrictions continues to decrease; approximately 2 years after signing, however, the percentage of states placing restrictions increases somewhat. It is also of note that states that eventually sign or have already signed are always less likely to place restrictions than those that never sign (p<.05). This suggests that there is something inherently different about Article VIII signatories, even decades before they sign. These preliminary observations neither confirm nor disconfirm the existence of unobservable sources of selection. They do, however, provide evidence that changes in restriction behavior precede signing, which suggests that selection into Article VIII is not random and that the treaty commitment is at least in part endogenous. A closer examination of restriction patterns is therefore in order.

To investigate with greater precision the patterns described previously, I examine three questions statistically. First, does the restriction behavior of states that are close to making an Article VIII commitment differ from their behavior long before signing? Clearly, one should expect to observe decreases in restriction levels as a state approaches the treaty commitment: the IMF encourages countries it believes are ready to do so to sign (Simmons 2000a, 820), and the clearest indicator of such readiness is a low or null level of restrictions. Second, does the restriction behavior of states that are close to signing Article VIII differ from the behavior of states that have already signed? If states are essentially behaving like signatories in the years leading up to an Article VIII commitment, this suggests that states may sign because they have reached low restriction levels, and not the opposite. Finally, do observable factors account fully for patterns of restriction behavior as a state approaches the treaty commitment?

To answer these questions, I conduct a probit analysis of the probability of current account restrictions.5

See Simmons 2000a (833–34) for a description of the variables. For simplicity, I focus on the full models in Simmons 2000a (825, 830). I first replicate Simmons's results, which are based on a logit model. I then utilize a probit model because the estimator employed later in this article relies for its starting values on the Heckman probit model. The logit and probit models yield similar results.

The independent variables include those used by Simmons 2000a (including the variable Article VIII, which equals one if a state has signed Article VIII, and zero otherwise) as well as two new independent variables. The first captures patterns of restriction behavior as a state approaches the treaty commitment. We must therefore determine at what point a nonsignatory should be considered “close to making an Article VIII commitment.” Figure 1 suggests that a substantial change in restriction behavior begins approximately 4 years prior to signing. Accordingly, I create the variable Lead 4, which equals one if a state will sign Article VIII in the next 1 to 4 years, and zero otherwise.6

Robustness checks suggest that the general result of a considerable decrease in the probability of restrictions in the years leading up to an Article VIII commitment holds across several codings of the Lead variable.

Because we are interested here in assessing restriction behavior as a state approaches an Article VIII signature, it is not entirely clear how one should code the year in which the state signs, henceforth referred to as t′. The most straightforward procedure is to create a second variable, Year of Signature, which equals one in year t′ and equals zero for all other observations.7

This variable should be interpreted as the “added effect” of being in the year of signature because the Article VIII variable also equals one in year t′.

I also include controls for temporal dependence.8

Beck, Katz, and Tucker (1998) suggest that to control for temporal dependence in binary time-series-cross-section (BTSCS) data, the analyst use either a spell identification variable plus three splines, or a series of dummy variables marking the number of years since the last “event” (i.e., restriction). The data used in this article make the spline solution problematic for two reasons. First, the distribution of the spell identification variable is highly right-skewed. The STATA BTSCS routine (Tucker 1999) places the knots at the 25th, 50th, and 75th percentiles of the variable's distribution: 0, 0, and 7 years since the last restriction. Two of the terms are therefore identical. A second problem is that the spell identification variable is distributed very differently for signatories than for nonsignatories, making it difficult to control for different patterns of temporal dependence in each group using the splines. This becomes particularly problematic when I implement the selection model later in this article, as separate outcome equations are estimated for signatories and nonsignatories. The second solution proposed by Beck, Katz, and Tucker (1998) is preferable here because it is more flexible, and more adaptable to the skewed data distributions and different patterns of temporal dependence present in my data. A series of likelihood ratio tests suggests that dummies marking 0 and 1 years since the last restriction belong in the model, whereas dummies for subsequent years do not. To control for linear patterns of temporal dependence and to calculate the probabilities for Figure 2, I include the variable marking the number of years since the last restriction as well. Using temporal dummies rather than splines slightly decreases the Article VIII coefficient in the standard probit analysis, but not in a notable manner—the variable remains both statistically and substantively significant. In another analysis, I estimated the selection model outcome equations (see Table 2 and Figure 2) using splines rather than the temporal dummies. The results predict a slightly (6%) higher marginal effect for states that restricted in the previous year. Yet that analysis also predicts the international legal commitment to have an even smaller impact for states that are in their second year of current account liberalization than does the analysis using the temporal dummies. The results of the additional analysis are available from the author or at www-personal.umich.edu/~janavs/apsr.html.

Table 1 displays the results of the probit analysis.

The results reveal four interesting patterns. First, states that are within 4 years of signing are 18% less likely (p<.001) to place restrictions than are other nonsignatories, ceteris paribus.9

I use Clarify (Tomz, Wittenberg, and King 2001) to estimate predicted probabilities and condfidence intervals for the standard probit models in this paper. I use Gauss to estimate predicted probabilities and confidence intervals for the selection model. With both programs, I hold all other independent variables at their mean and vary the independent variable(s) of interest.

This suggests that states are not selected randomly into Article VIII—a finding that is sensible, given what we know about the Article VIII accession process, but that can have important consequences for the conclusions one draws about the effect of the treaty commitment on compliant behavior. Second, restriction behavior during the 4 years leading up to an Article VIII commitment is undistinguishable from that of states that have already signed (Wald test p-value = .865). This provides preliminary evidence that Article VIII status and compliance are at least in part endogenous. Third, states are considerably less likely to restrict in the year of signature than at any other time (p<.001), all other observable variables being equal. This highly “virtuous” behavior during the year of signature appears to be a reflection of the fact that the Fund has generally not allowed members to move to Article VIII status and at the same time be placing restrictions (Edwards 1985, 422–23).

Finally, the statistical significance of the Lead 4 variable suggests that some important patterns of variance in the dependent variable are not explained by the observable variables. Lead 4 may be proxying a number of unobservables that cause states to sign Article VIII and affect restriction behavior. Imagine, for example, that for one reason or another, a country's leaders “convert” to the IMF's economic orthodoxy. Their newfound commitment to liberal policies and to establishing/maintaining “good standing” in the Fund leads to a considerable change in current account policy, and, approximately 4 years later, to an Article VIII commitment. The Lead 4 variable may be proxying this unobservable variable(s).

An important caveat must be raised with regard to the findings discussed previously. The argument can be made that the observed decrease in restrictions as a state approaches an Article VIII commitment suggests not that states sign because they have reached low or null restriction levels, but instead that they cease to restrict the current account in the few years before signing because they are concerned about establishing in advance good postsigning reputations. The latter logic is plausible, and indeed the results displayed in Table 1 alone do not allow us to distinguish between the two interpretations. However, a number of additional considerations suggest that the former and not the latter process is at work.

First, if the legal obligation did generate reputational costs that restricting nonsignatories would suffer after signing, one would expect the Fund to place greater emphasis on the importance of making a legal commitment to Article VIII, in an attempt to encourage nonsignatories to eliminate restrictions. Yet, the IMF's practices are much more suggestive of a selection process: the organization first focuses on promoting current account liberalization, and generally does not emphasize the international legal commitment until restrictions have been reduced significantly or eliminated. Second, if we believe that the observed shift in restriction behavior is attributable to states' desires to establish good postsigning reputations, we would not expect their restriction behavior to deteriorate after signing. Indeed, such behavior might expose them to criticism of being on good behavior solely in order to acquire Article VIII status. Yet as Table 1 indicates, states are considerably less likely to restrict in the year of signature than in subsequent years (p<.001). It is difficult to believe that a state's good behavior before signing would carry it very far in the eyes of markets if it reimposed restrictions after committing to Article VIII.

Third, if postsigning reputational concerns affected restriction behavior, one would expect them to have an impact only during the period leading up to and following the treaty commitment. As a result, the restriction behavior of states that are far from signing but eventually sign should not differ systematically from that of states that never sign. Yet the empirical record reveals long-term differences in the two groups' restriction behavior, suggesting that there is something fundamentally different about signatories, even long before they sign.10

In an additional analysis, I reestimated Model 1 and included a variable that equals one if a state is more than 4 years away from signing but eventually signs, and zero otherwise. The results confirm that states that are far from signing but eventually sign are significantly less likely (p<.001) to restrict the current account than are states that never commit to Article VIII. The results of that analysis are available from the author or at www-personal.umich.edu/~janavs/apsr.html.

This is more suggestive of a selection process whereby a certain “type” of state will assume the treaty obligation. Finally, if the reputational process were at play, one would expect Article VIII to constrain state behavior independent of the sources of selection. As we shall see in subsequent sections, however, there is little evidence that these constraining effects are present.

A STATISTICAL MODEL OF TREATY COMMITMENT AND COMPLIANCE

The previous section provides evidence of the endogeneity of Article VIII. To understand why this is problematic for statistical inferences about the treaty commitment's effect on restriction behavior, it is necessary to examine why standard regression techniques are likely to yield bias, and how the procedures originally proposed by Heckman (1976, 1979) and adapted to the particular observability problem described in the following present a solution. The Heckman probit model, which controls for sample selection when the outcome equation dependent variable is dichotomous (van de Ven and van Praag 1981), is common in political science research (e.g., Berinsky 1999; Lemke and Reed 2001). This model is appropriate for cases in which one observes the outcome of interest only for the selection group: for instance, one only observes whether states implement IMF austerity measures for the group of states that have committed to such measures (Vreeland 2002).

The Article VIII case presents a different partial observability problem: one only observes the restriction levels of Article VIII countries if they sign, and one only observes the restriction behavior of nonsignatories if they do not sign. If we believe that states that have signed differ in important ways from those that have not, then a signatory's decision on whether to restrict should be thought of as being fundamentally different from that of a nonsignatory. A decision to restrict is for a signatory a decision to not comply with a commitment, whereas for a nonsignatory, the issue of compliance is not part of a leader's calculus. It is therefore necessary to estimate three (rather than the standard two) equations: an equation determining selection into Article VIII, a noncompliance equation for signatories, and a restriction equation for nonsignatories. The techniques explained in the following and derived more fully in the Appendix do this.

Let the equation that determines selection into Article VIII be

where

is a state's latent propensity to sign; 1 is an (nS + nN)×1 vector of ones; nS and nN denote the number of observations for signatories and nonsignatories; θ is the baseline propensity to sign; w is an (nS + nN)×k matrix of covariates that affect the probability of signing; γ is a k×1 vector of coefficients; and μ denotes the unobservable factors that determine a state's propensity to sign. I have conjectured that μ includes unobservables that also make states less likely to restrict the current account, such as commitment to sound economic policies and/or desire to demonstrate “good standing” in the Fund. We do not observe

. Rather, we observe z, an (nS + nN)×1 vector in which an element equals one if a state has signed Article VIII and zero if it has not.

Let the equation for whether a signatory places restrictions be

where 1 is an nS×1 vector of ones; αS is a signatory's baseline propensity to restrict; xS is an nS×k matrix of covariates that affect the probability that a signatory will restrict; βS is a k×1 vector of coefficients; and εS denotes the unobservable factors that determine a signatory's propensity to restrict, which I have hypothesized to be negatively correlated with μ in equation (1). We do not observe

. Instead, we observe yS (which equals zero if a signatory does not place restrictions and one if it does) only if z equals one.

Let the equation for whether a nonsignatory places restrictions be

where 1 is an nN×1 vector of ones; αN is a nonsignatory's baseline propensity to restrict; xN is an nN×k matrix of covariates that affect the probability that a nonsignatory will restrict; βN is a k×1 vector of coefficients; and εN denotes the unobservable factors that determine a nonsignatory's propensity to restrict, which I have hypothesized to be negatively correlated with μ in equation (1). We do not observe

. Rather, we observe yN (which equals zero if a nonsignatory does not place restrictions and one if it does) only if z equals zero.

Now let us examine the standard probit model and why it may yield biased estimates of the effect of an Article VIII commitment on restriction behavior. A standard approach is to estimate the following:

where 1 is an (nS+nN)×1 vector of ones; αN is a nonsignatory's baseline propensity to restrict; z is an (nS + nN)×1 vector whose elements equal one for signatories and zero for nonsignatories; and αS-αN, as previously defined, is the difference between a signatory's and a nonsignatory's baseline propensity to restrict. xS and xN are pooled together, forming an (nS+nN)×k matrix of covariates that affect both signatories' and nonsignatories' propensity to restrict; and β is a k×1 vector of coefficients.

As Heckman (1976, 1979) and others have shown, whether equation (4) will yield biased results, as well as the direction of the bias, in a function of the relationship between the unobservables that lead states to restrict and sign and the unobservables that lead states to restrict and not sign. In equation (4), any part of ε that is correlated with μ will be attributed to z. That is, standard techniques will attribute to being a signatory the unobservable shocks that affect both a state's propensity to restrict and its propensity to sign. Consider first what must be true if equation (4) is to yield unbiased results: the unobservables ε that affect a state's propensity to restrict are unrelated to the unobservables μ that affect its propensity to sign:

Here, equation (4) attributes no part of ε to z. It produces consistent estimates of αS and αN, hence yielding unbiased estimates of the treaty commitment's effect.

Next, consider the case in which the unobservables ε that affect a state's propensity to restrict are positively correlated with the unobservables μ that affect its propensity to sign:

Standard techniques will attribute to z any part of ε that is correlated with μ. Because signatories on average have higher values of μ than do nonsignatories, and because in equation (6) ε is positively correlated with μ, standard techniques will overestimate αS, signatories' baseline propensity to restrict, and underestimate αN, nonsignatories' baseline propensity to restrict. As a result, standard techniques will in this case understate the treaty commitment's effect.

Finally, consider the case in which the unobservables ε that affect a state's propensity to restrict are negatively correlated with the unobservables μ that affect its propensity to sign:

Signatories on average have higher values of μ than do nonsignatories, and in equation (7), these factors are negatively correlated with the unobservables ε that affect a state's propensity to restrict. As a result, standard techniques will underestimate αS, signatories' baseline probability to restrict, and overestimate αN, nonsignatories' baseline probability to restrict. Equation (4) will in this case overstate αS-αN, the treaty commitment's effect on restrictions. Because unobservables such as commitment to sound economic policies and/or desire to demonstrate “good standing” in the Fund that are thought to make states less likely to restrict are also thought to make them more likely to sign Article VIII, I hypothesize that the Article VIII case falls into this category.

To estimate the effect of Article VIII on restriction behavior independent of selection, I derived a likelihood function based on equations (1) through (3).11

See Heckman 1976, 1979; and van de Ven and van Praag 1981 for further details. The Appendix contains the statistical proofs, the likelihood function, and an explanation of the identification restrictions. The STATA code is available from the author or at www-personal.umich.edu/~janavs/apsr.html.

The estimator developed here has two benefits in addition to controlling for the unobservable sources of selection. First, because it estimates the outcome equations for signatories and nonsignatories separately, it does not assume that the independent variables affect the restriction behavior of the two groups in the same manner (i.e., that βS = βN). This alone does not necessitate controls for selection (a series of interaction terms would suffice), but given that I already intend to employ a selection model, the separate estimation of βS and βN provides additional flexibility. Second, the estimator developed here allows the outcome equations to contain different columns of xS and xN. If we believe that those states that have made a treaty commitment differ in important ways from those that have not, it is also likely that some independent variables affect one group's decision to engage in compliant behavior and not the other's. This is therefore an additional source of flexibility in statistical estimation.

An important concern remains with regard to the estimator I use. The selection equation examines why states sign Article VIII; hence, the estimates should be based on the independent variables' effects before and when states sign, but not after. Because once an Article VIII commitment is made, it cannot be rescinded, survival analysis techniques that focus on the spell of time until signing occurs are necessary for the selection equation (Simmons 2000a, 823). Yet, the selection model requires a probit model for the selection equation. This problem can be circumvented by creating a dummy variable which equals one for all observations after year t′, and zero otherwise. When this dummy variable is included in the probit equation, the estimated coefficients, standard errors and z-scores of the independent variables are based only on the values of the independent variables before or in year t′ (i.e., prior to signing and in the year of signing). This makes it possible to estimate a probit model in the selection equation while still accounting for the nature of the data.12

To control for temporal dependence, I include a variable marking the number of years since the state joined the IMF (which is precisely the Cox function's “time until failure”) as well as three cubic splines (Beck, Katz, and Tucker 1998). I employ this approach because differences in patterns of temporal dependence among signatories and nonsignatories do not pose a problem here, and because the variable marking the number of years since joining is almost perfectly normally distributed.

To ensure that the transformation from a Cox to a probit model is not accounting for the differences between my results and Simmons's, I confirm that the two models yield comparable estimates.13

Using the coefficients generated by the probit analysis, I calculate for each independent variable the “relative risk” of signing; that is, the ratio of the predicted probability of signing when there is a one-unit increase in the independent variable to the predicted probability of signing before the one-unit increase. These “relative risks” can be directly compared with the Cox model coefficients. All other variables are held at their mean. The results are available from the author or at www-personal.umich.edu/~janavs/apsr.html.

RESULTS

The results of the statistical analysis controlling for selection appear in Table 2. For comparative purposes, Table 2 also displays the standard probit model's estimates. The results provide strong evidence of selection effects. Indeed, a likelihood ratio test that the joint effect of the correlation coefficients ρS and ρN equals zero is highly significant (p<.001), suggesting that the selection model employed here maximizes the likelihood function significantly better than do methods not controlling for selection. ρS is negative and highly statistically significant (p<.001), indicating as hypothesized that the unmeasured conditions that lead states to commit to Article VIII make them considerably less likely to restrict the current account. As conjectured, ρN is negative, providing evidence that the unobservable factors that cause states not to sign make them more likely to restrict the current account. ρN falls short of conventional levels of statistical significance (p = .130), and therefore one cannot reject the null hypothesis that there are no selection effects for nonsignatories. Note, however, that standard regression techniques will yield biased results as long as selection effects are present for at least one of the two groups.14

Consider the case in which negative selection effects exist for the signatory group but not for the nonsignatory group: ρS<0 and ρN = 0. Suppose that the biased (standard probit) αS coefficient = 1, while the unbiased αS coefficient = 2 (the numerical values are hypothetical, but their ordering makes sense substantively). Because we are assuming here that there are no selection effects for nonsignatories, suppose that in both the standard and selection models, αN = 2. The standard probit model would estimate αS − αN = −1. Controlling for selection, however, αS − αN = 0. Therefore, standard techniques would yield biased estimates of αS − αN even if ρN = 0. Clearly, the stronger the selection effects for both groups (i.e., as ρS and ρN→−∞), the more standard techniques will overstate (in the negative direction) αS−αN. Yet, relatively large negative values of ρS will also lead one to overstate the extent to which signing decreases a state's propensity to restrict even if ρN = 0.

That selection effects are stronger and more systematic for signatories may also be sensible given how the Article VIII accession process works. Because it is generally the Fund that urges members to sign (Simmons 2000b, 581), noncompliant “types” are unlikely to be willing or able to attain the low or null restriction levels necessary to be approached by the IMF and “encouraged” to sign. On the other hand, because the Fund cannot obligate members to commit to Article VIII (Gold 1988, 227), some compliant “types” will choose for one reason or another to delay accepting Article VIII status. Hence, it is likely that there are fewer noncompliant “types” that have signed than compliant “types” that have not signed. In other words, the IMF is probably more successful at screening out bad apples than it is at forcing good apples to sign. This apparent “asymmetrical selectivity” may explain why selection effects are stronger and more systematic for signatories than for nonsignatories.

Another important test of Article VIII's independent effect is whether—all else equal—the international legal commitment has a strong negative impact on the probability of restrictions (Simmons 2000a, 830). The techniques implemented here make possible the estimation of such counterfactuals by holding constant for all other conditions—including those that cannot be directly measured. This involves two steps. First, take the average nonsignatory and estimate its probability of restricting when it is exposed to the conditions—observable and unobservable—to which nonsignatories are exposed. To do so, I estimate the probability of restrictions as predicted by the selection equation and the nonsignatories' outcome equation, using the mean values of the nonsignatories' independent variables. Second, take the same average nonsignatory and determine what its probability of restrictions would have been had it signed. To do so, I use the same mean values of the nonsignatories' independent variables and estimate the probability of restrictions as predicted by the selection equation and the signatories' outcome equation. We now know the probability of current account restrictions for two countries that are alike in every way, except that the latter has made a legal commitment to Article VIII and the former has not. The difference between these probabilities yields the marginal effect of an Article VIII commitment, independent of selection. Figure 2 displays these marginal effects, as well as those produced by Simmons's (831) standard probit model, as a function of time elapsed since the last restriction.

The Marginal Effect of an Article VIII Commitment on the Probability of Current Account Restrictions: Selection and Standard Probit Models

The previous analysis yields two important results. First, failing to control for the unobservable sources of selection consistently overstates the effect of an Article VIII commitment on restriction behavior. Indeed, selection bias accounts for between 31% and 95% of the standard probit model's estimated effect of the legal commitment on a state's propensity to engage in compliant behavior. Second, once one controls for the unobservable sources of selection, the treaty obligation is found to have only a limited independent effect on state behavior. For states that are in their first year of current account liberalization, the legal commitment does constrain: signatories are 13% less likely than are nonsignatories to restrict the current account, ceteris paribus (p<.05). Subsequently, however, the treaty commitment's effect virtually disappears. By the second year of liberalization—when the standard probit model estimates an Article VIII commitment to matter the most, making signatories 23% less likely to restrict the current account than nonsignatories (Simmons 2000a, 831)—methods controlling for selection suggest that the legal commitment has virtually no independent effect on state behavior.

The results also confirm that estimating the impact of the observable variables separately for signatories and for nonsignatories significantly maximizes the likelihood function (p<.05). A series of Wald tests indicates the following. Volatility in the terms of trade and GDP growth affect restriction behavior in approximately the same manner for signatories and nonsignatories. Increases in the balance of payments as a proportion of GDP have a stronger negative effect on the probability of restrictions for nonsignatories than for signatories. Article VIII signatories that use Fund credits are more likely to restrict than are nonsignatories that use credits, ceteris paribus. Increases in reserves as a proportion of GDP appear to decrease the probability of restrictions for signatories (though not at standard levels of statistical significance), whereas they increase the probability of restrictions for nonsignatories (p<.05).15

This result is somewhat perplexing. However, it is consistent with Simmons's (2000a) findings. Reestimation of the model without this variable in the outcome equations does not change the results notably.

The most considerable difference between the two groups concerns patterns of temporal dependence. All else equal, nonsignatories that are within one year of their last restriction are notably more likely to place restrictions (p<.05) than are signatories that are within one year of their last restriction. After the first year, however, the difference is no longer significant.

CONCLUSION

This article has demonstrated that selection effects can have important consequences for the conclusions we draw about the impact of treaty commitments on state behavior. I have shown that the unobservable conditions that lead states to sign Article VIII of the IMF Treaty make them considerably more likely to engage in compliant behavior. I have also found evidence, albeit less conclusive, that the unmeasured factors that cause states not to commit to Article VIII make them less likely to engage in compliant behavior. Failing to control for selection effects leads one to overstate considerably the extent to which the treaty obligation affects states' restriction behavior. Indeed, if the conditions that led a state to sign change, a legal commitment to Article VIII appears to have little constraining power.

Although this study has examined one article of one treaty, it presents a methodological challenge to scholars conducting empirical research on treaty compliance more broadly. It demonstrates that a fundamental part of understanding whether and why states comply lies in understanding what drives behavioral change as a state approaches a treaty commitment, and how changes in those conditions affect subsequent compliance. For scholars employing quantitative methods, the central implication of this article is that in order to obtain unbiased estimates of the treaty commitment's impact on state behavior, statistical methods that control for the unobservable sources of selection are very likely to be necessary. For scholars using qualitative methods, the chief implication is that it is important to consider not only the extent of compliant behavior both after and well before signature but also what drives the decision to sign (or not sign) and determines the extent of compliant behavior.

My findings also have some interesting substantive implications for how we think about treaty commitment and compliance. The results cast doubt on the argument that an Article VIII obligation serves as a constraining mechanism that raises the reputational costs a state will pay if it reneges (Simmons 2000a, 819). Why, then, do states sign? That is, if the decision to commit is largely endogenous to expectations about future compliance (Downs, Rocke, and Barsoom 1996), why do states sign at all? Article VIII may instead serve as a screening device. In this conception, Article VIII status does, as Simmons (819–21) argues, signal future policy intentions to markets, which possess incomplete information. However, the mechanism at work is different from that suggested by Simmons. Signing Article VIII enables leaders credibly to signal their intention to engage in compliant behavior in the future not because the legal commitment generates ex post reputational costs for noncompliance, but because the ex ante costs of becoming a signatory are high enough to deter noncompliant “types” from signing. If the political capital and effort (formal or informal) necessary to become a signatory are sufficiently costly ex ante, states are likely to comply because the requirements for entry effectively screen compliant “types.”

This article's findings may lead some to adopt the bleak view that—at least with regard to Article VIII of the IMF Treaty—international institutions do little or nothing to promote compliant behavior. I believe the evidence points toward a different interpretation. In the Article VIII case, a central role of the Fund appears to lie not in advocating the legal commitment itself, but in promoting—both before and after signature—the conditions that lead states to make treaty commitments and to engage in compliant behavior. Another fundamental role lies not in monitoring and punishing defectors, but in using formal and/or informal requirements for entry to screen potential signatories. Different international cooperation problems call for different institutional solutions (Koremenos, Lipson, and Snidal 2001), and it is not the claim of this article that all international institutions fulfill functions similar to those I have identified in the Article VIII case. Under what conditions do international institutions play these roles rather than others? In what circumstances is the prospect of signatory status sufficient to compel states to become compliant “types?” These questions point to interesting new avenues of thinking about the nature of interactions between states and international institutions.

APPENDIX: DERIVATION OF THE LIKELIHOOD FUNCTION

Let

;

. Let Φ2 denote the standard bivariate normal cumulative distribution function.

For signatories, the probability of not restricting is

For signatories, the probability of restricting is

For nonsignatories, it is straightforward that the probability of not restricting is

and that for nonsignatories, the probability of restricting is

The likelihood function is as follows:

In order for equation (12) to be identified, x must contain at least one variable not contained in w, or w must contain at least one variable not contained in x. The derivations of equations (10) and (11) and the STATA code for equation (12) are available from the author or at www-personal.umich.edu/~janavs/apsr.html.

I am greatly indebted to Kenneth Schultz and Jeffrey Lewis for their invaluable assistance on every stage of this research. I also thank Beth Simmons for data and excellent comments; and Jeffrey Smith, Rob Salmond, Chad Rector, Barbara Koremenos, Dan Hopkins, Geoffrey Garrett, Lisa Blaydes, Neal Beck, Matthew Baum, and Nigel Ashford for very helpful comments. All remaining errors are my own. I gratefully acknowledge research support from the UCLA International Institute, the Institute on Global Conflict and Cooperation, the Institute for Humane Studies, and the Burkle Center for International Relations.

References

Achen Christopher. 1986. The Statistical Analysis of Quasi-Experiments. Berkeley: University of California Press.
Beck Nathaniel, Jonathan Katz, and Richard Tucker. 1998. “Taking Time Seriously: Time-Series-Cross-Section Analysis with a Binary Dependent Variable.” American Journal of Political Science, 42 (October): 1260– 88.Google Scholar
Berinsky Adam. 1999. “The Two Faces of Public Opinion.” American Journal of Political Science 43 (October): 1209– 30.Google Scholar
Chayes Abram, and Antonia Handler Chayes. 1995. The New Sovereignty: Compliance with International Regulatory Agreements. Cambridge: Harvard University Press.
Downs George, David Rocke, and Peter Barsoom. 1996. “Is the Good News about Compliance Good News about Cooperation?” International Organization 50 (Summer): 379– 406.Google Scholar
Edwards Richard. 1985. International Monetary Collaboration. Dobbs Ferry, NY: Transnational Publishers.
Fearon James. 1997. “Signaling Foreign Policy Interests: Tying Hands versus Sinking Costs.” The Journal of Conflict Resolution, 41 (February): 68– 90.Google Scholar
Gold Joseph. 1988. Exchange Rates in International Law and Organization. Chicago: American Bar Association.
Heckman James. 1976. “The Common Structure of Statistical Models of Truncation, Sample Selection and Limited Dependent Variables and a Simple Estimator for Such Models.” Annals of Economic and Social Measurement 5 (4): 475– 92.Google Scholar
Heckman James. 1979. “Sample Selection Bias as a Specification Error.” Econometrica 47 (January): 153– 62.Google Scholar
International Monetary Fund. Archives S424: Transitional Arrangements, Articles VIII and XIV. Various years.
Koremenos Barbara, Charles Lipson, and Duncan Snidal. 2001. “The Rational Design of International Institutions.” International Organization 55 (Fall): 761– 99.Google Scholar
Lemke Douglas, and William Reed. 2001. “War and Rivalry among Great Powers.” American Journal of Political Science 45 (April): 457– 69.Google Scholar
Maddala G. S. 1983. Limited-Dependent and Qualitative Variables in Econometrics. Cambridge: Cambridge University Press.
Przeworski Adam, and Fernando Limongi. 1993. “Political Regimes and Economic Growth.” Journal of Economic Perspectives 7 (Summer): 51– 69.Google Scholar
Przeworski Adam, and James Vreeland. 2000. “The Effect of IMF Programs on Economic Growth.” Journal of Development Economics 62 (August): 385– 421.Google Scholar
Simmons Beth. 2000a. “International Law and State Behavior: Commitment and Compliance in International Monetary Affairs.” American Political Science Review 4 (December): 819– 35.Google Scholar
Simmons Beth. 2000b. “The Legalization of International Monetary Affairs.” International Organization 54 (Summer): 573– 602.Google Scholar
Tomz Michael, Jason Wittenberg, and Gary King. 2001. “Clarify: Software for Interpreting and Presenting Statistical Results.” http://gking.harvard.edu/clarify (June 27, 2003).
Tucker Richard. 1999. “BTSCS: A Binary Time-Series Cross-Section Data Analysis Utility.” http://www.vanderbilt.edu/rtucker/programs/btscs/. (June 27, 2003).
van de Ven Wynand, and Bernard van Praag. 1981. “The Demand for Deductibles in Private Health Insurance: a Probit Model with Sample Selection.” Journal of Econometrics 17 (November): 229– 52.Google Scholar
Vreeland James. 2002. “The Effect of IMF Programs on Labor.” World Development 30 (January): 121– 39.Google Scholar
Vreeland James. 2003. The IMF and Economic Development. Cambridge: Cambridge University Press.
Young Oran. 1994. International Governance: Protecting the Environment in a Stateless Society. Ithaca, NY: Cornell University Press.
Figure 0

The Percentage of States Placing Current Account Restrictions as a Function of the Number of Years to and since an Article VIII Commitment

Figure 1

Results of Analysis of Current Account Restrictions as a State Approaches an Article VIII Commitment

Figure 2

Results of Analyses of the Probability of Current Account Restrictions

Figure 3

The Marginal Effect of an Article VIII Commitment on the Probability of Current Account Restrictions: Selection and Standard Probit Models

Submit a response

Comments

No Comments have been published for this article.