In October 2008, African heads of state capped off an eight-day summit with the announcement that they would create a new free trade zone. The ambitious new plan would combine 26 countries that were already in three existing regional agreements—the South African Development Community (SADC), the Common Market for Eastern and Southern Africa (COMESA) and the East African Community (EAC). But the SADC was already suffering severe international criticism for its failure to constrain one of its members, Zimbabwe, either politically or economically. A month later, COMESA canceled a proposed meeting to establish a customs union for the second time in six months, ostensibly because of the instability in Zimbabwe.Footnote 1 The EAC had already fallen short of its ambitions for a common market, despite several meetings and agreements. “I am not sure whether we can make the EAC common market to be operational—as a matter of fact, a common market is a big exercise indeed, though writing it is very simple”, one EAC official said.Footnote 2
This example highlights the fact that there is often a vast difference between writing rules that could theoretically provide the framework for a functioning agreement, and an agreement that is likely to function in practice. Regional economic organizations (REOs)Footnote 3 are found all over the world, and many of those agreements have highly legalized language. Yet there are many types of REOs—highly ambitious but potentially ineffective ones, such as the proposed African Free Trade Area mentioned above; ambitious and effective ones in Europe and North America; and unambitious and ineffective ones that exist in name alone, such as the Latin American Integration Association, the Mano River Union and the Economic Community of the Great Lakes Countries, to name a few. If writing a legal framework is not sufficient to create a functioning trade agreement, what does lead to successful trade agreements? Given that some regional agreements—such as the European Union (EU) and the North American Free Trade Agreement (NAFTA)—are quite effective in promoting trade and fulfilling their goals, why are some countries unable (or unwilling) to negotiate or implement trade agreements that work on all the dimensions that those agreements purport to encompass?
Research on this topic has tended to suffer from one of two problems. Most commonly, scholars who evaluate the effects of REOs treat them all as like units. Many empirical studies simply operationalize REO membership as a dichotomy (a country is either in one or it isn't) and ignore the relative capacity of different agreements. The second problem is that studies that do address variation across different REOs tend to single out one aspect of the agreement—either trade volumes, technical aspects of the agreement or institutional structure—and use that sole dimension to assess the overall effectiveness of the REO. However the effectiveness of a trade agreement comes from the interaction of several of these variables, and looking at just one aspect of a REO is rarely enough to provide a complete picture of its effectiveness.
We argue that the conditions that produce such a variety of agreements (and agreement effectiveness) are exogenously determined. The presence of outside options—that is, trading opportunities beyond the REO that exceed the potential for trade within the proposed REO at the time of the agreement's formation—will lead to an agreement that does not meet the goals it sets out for itself. States will have little incentive to use the institutions they develop, or even to set up (potentially) effective institutions in the first place. By contrast, when prospective members of a proposed REO conduct a relatively high share of their trade with one another, the agreement is much more likely to meet the goals it sets for itself, as there are clear gains to be made through the process of institutionalization. When there are few prospects for trade at the start, enhanced institutionalization is unlikely to help.Footnote 4 We test these arguments using a new dataset collected from expert surveys on the many attributes that make for a successful agreement (Gray and Slapin Reference Gray and Slapin2012). The expertise of officials in the trade agreements themselves, as well as external observers at the multilateral level, can shed much light on the actual functioning of these REOs, beyond what the treaties and trade flows alone can tell us. Thus, our measures of REO effectiveness across a variety of dimensions capture the de facto workings of the institutions and better capture the externalities that can be directly attributed to the REO itself. We show that many different measures of institutionalization do not capture expert assessment of effectiveness. The degree to which the largest member of an organization trades with other members, relative to its world trade, is associated with a well-functioning agreement.
This article proceeds by first discussing the literature on regional economic agreements and legalization. We then lay out our theoretical expectations regarding the success of REOs. We argue that a relative lack of outside options (as modeled by the ratio of trade in the proposed agreement to world trade at the outset of the agreement's existence) produces agreements that can meet their goals. Next we describe our expert survey dataset on the capacity of REOs across numerous dimensions, and use that dataset to explore our theoretical expectations. We then demonstrate that the existing measures of legalization bear little relation to expert assessments of REO effectiveness across a variety of dimensions. The final section concludes.
Beyond Legalization to Effectiveness
The number of regional, bilateral and multilateral trade agreements has increased dramatically since the 1990s, and they have become increasingly important with the failure of the Doha rounds of multilateral trade talks.Footnote 5 In the General Agreement on Tariffs and Trade (GATT), Article 24 allows REOs to be set up as an exception to the most-favored nation rule that members must uphold, as long as those REOs complement the multilateral trading system. To that end, much of the initial literature on the formation of regional trade agreements focused primarily on their economic effects in the context of multilateralism (Panagariya Reference Panagariya1999). Since the boom in popularity of REOs coincided with the Uruguay Round, and the greater institutionalization of the multilateral agenda with the transformation of the GATT to the World Trade Organization (WTO), most researchers at the time were concerned with whether REOs would have a trade-diverting effect that could potentially derail multilateralism (Viner Reference Viner1950; Bhagwati Reference Bhagwati1991; Bhagwati, Greenaway and Panagariya Reference Bhagwati, Greenaway and Panagariya1998; Krueger Reference Krueger1999). Others focused on the rise in REOs as a result of strategic interaction at the international level—namely, that REOs are created in response to the creation of other REOs (Baccini and Dür Reference Baccini and Dür2012; Panagariya and de Melol Reference Panagariya and de Melol.1992; Mansfield and Milner Reference Mansfield and Milner1997; Mansfield Reference Mansfield1998; Oye Reference Oye1992; Yarbrough and Yarbrough Reference Yarbrough and Yarbrough1992; Serra Reference Serra1997).
Much of the work on the systemic- or domestic-level determinants of regional agreements focuses on the depth of integration that countries choose to set as a goal (Milner Reference Milner1997; Haggard Reference Haggard1997; Grieco Reference Grieco1997). These explanations do not, however, address whether the agreements in question achieve the level of integration they intended, or explain the emergence of many ineffective trade agreements. Indeed, the Grossman-Helpman models assume that every agreement signed would, by definition, be effective at promoting intra-REO trade, but presumably any increase in trade would be counted as effective (Grossman and Helpman Reference Grossman and Helpman1994, Reference Grossman and Helpman1995). A REO could provoke some increase in trade among its members, but that increase may fall short of the original levels or target dates that were envisaged.
Only recently have researchers begun to study REOs as institutions in and of themselves, applying the theories from the literature on international organizations to them (Fernandez Reference Fernandez2000; Foroutan Reference Foroutan1998; Fratianni and Pattison Reference Fratianni and Pattison2001; Holmes Reference Holmes2005; Kono Reference Kono2007; Nitsch and Sturm Reference Nitsch and Sturm.2004; Urata Reference Urata2002). However, many of the empirical studies treat membership in REOs as a dichotomy—either a country is a member of a REO, or it is not (for example, Mansfield and Reinhardt Reference Mansfield and Reinhardt2008; Büthe and Milner 2008). Indeed, a country could be a member of several REOs (as Kenya is a member of SADC, COMESA and the EAC), or belong to one dysfunctional one (Mauritania in the Arab Maghreb Union) or one highly functional one (Germany in the EU) and receive the same empirical treatment. Clearly, there are vast differences between these different possible scenarios, and researchers are only beginning to try to weigh REOs according to their differing characteristics (Crawford and Fiorentino Reference Crawford and Fiorentino.2005; Stinnett Reference Stinnett2007; Kim Reference Kim2008; Hicks and Kim Reference Hicks and Kim2012; Mansfield and Reinhardt Reference Mansfield and Reinhardt2009; Haftel Reference Haftel2007).
The issue of effective institutional design has long been a concern of researchers on international cooperation (Koremenos, Lipson and Snidal Reference Koremenos, Lipson and Snidal2001). Many have argued that deeper legalization—that is, treaties and procedures that put the functioning of the institution in the hands of third parties, rather than leaving them at the mercy of heads of state—makes for more effective forms of cooperation (Keohane, Moravcsik and Slaughter Reference Keohane, Moravcsik and Slaughter2000; McCall Smith Reference McCall Smith2000). Empirical tests of these propositions have tended to focus primarily on the design of the treaty itself and the language employed (Koremenos Reference Koremenos2005; Rosendorff Reference Rosendorff2005). Furthermore, these studies have primarily examined multilateral (or the more ambitious of the regional) institutions, such as the WTO (Rosendorff Reference Rosendorff2005) or the EU (Moravcsik Reference Moravcsik1998)—organizations that make available a relative abundance of data. However, a comprehensive test of the effects of legalization would examine not just organizations with high levels of legalization and documentation, but also those with medium or low levels. Indeed, the most frequently studied organizations represent only a small fraction of the myriad of organizations in the world today.
An additional stumbling block facing much research on REOs relates to the common proxies for effectiveness. Much of the REO-specific research in economics and political science only looks at observable levels of increases in trade, without evaluating whether those increases map the intended levels of trade promotion within a REO, or controlling for how much of that trade can be directly attributed to the REO, rather than to other factors. Many researchers conflate an organization's competence with the treaty's level of legalization or with the presence of a dispute settlement mechanism (DSM) (Koremenos, Lipson and Snidal Reference Koremenos, Lipson and Snidal2001; Koremenos Reference Koremenos2005; McCall Smith Reference McCall Smith2000). Some have argued that in addition to making rulings, a well-functioning DSM can play an important informational role to members of the REO (Li Reference Li2000; Dai Reference Dai2005; Chapman Reference Chapman2007; Kono Reference Kono2007; Fang Reference Fang2008).
But the simple presence of a DSM does not necessarily mean it functions as one would expect based on the language in the treaty. Many REOs—such as SADC, Mercosur and COMESA—have highly legalized DSMs that are scarcely ever used in practice. These DSMs may not be used because the politics of the relationships among countries are such that members feel pressure not to use the channels of public dispute settlement, particularly when powerful countries are potentially at fault. Anecdotal evidence abounds that even some of the most highly legalized treaties fail to function well in practice. Many countries, sidestep their own regional DSMs, preferring instead to seek redress through the WTO's court (Lacarte-Muró and Gappah Reference Lacarte-Muró and Gappah2000; Busch Reference Busch2007; Bown and Hoekman Reference Bown and Hoekman2005).
Lastly, much of the language used to create these under-performing regional DSMs is lifted almost directly from organizations that do function well. Take Article 30 of the courts of COMESA and the EAC in Africa, which establishes the relationship between national courts and the DSM: “National Courts or tribunals may refer a matter to the Court or request the Court to give a preliminary ruling on the application or interpretation of the Treaty or the validity of regulations, directives and decisions of the Common Market.” Though the exact language differs, this borrows heavily from the EU's preliminary ruling procedure (Article 234 of the Consolidated European Treaties). This article is notable because it allows national courts to refer questions about the compatibility of national law with European obligations to the European Court of Justice, an unprecedented move that was critical for EU legal integration (Stone Sweet and Brunell Reference Stone Sweet and Brunell1998; Alter Reference Alter1998). Presumably, national governments in Africa have adopted preliminary rulings in language alone, since in practice national governments are far from constrained by the regulations of the REOs of which they are members. Indeed, Jo and Namgung (Reference Jo and Namgung2012) discuss how many DSMs follow “legal boilerplates” that are passed along across organizations.
Exit Options, Legalization and Effectiveness
While good institutions, such as legalized DSMs, may enhance trade in well-functioning REOs such as the EU or NAFTA, simply writing down the language of good institutions on paper is insufficient to produce effective trade agreements. We argue that exogenous factors present in the trading environment when states first decide to form the REO drive the later functioning of these REOs. Many have demonstrated that when at least one state within a proposed organization can make a credible threat of exit—or has a reasonable option for trade beyond the organization—the organization may be less effective (Bednar Reference Bednar2007; Odell and Eichengreen Reference Odell and Eichengreen1998). At the very least, bargaining within the organization will be significantly different from these compared to scenarios with few credible outside options (Binmore Reference Binmore1985; Muthoo Reference Muthoo1995; Voeten Reference Voeten2001; Slapin Reference Slapin2009; Fang and Ramsey Reference Fang and Ramsey2010). Large states with good alternatives for trade beyond a potential REO can make demands on the other potential members. These other members must comply with the demands of the state with outside options to prevent it from exiting the agreement.Footnote 6 If, however, member states have less to gain from trade outside the proposed organization than within it, we can anticipate that they will follow through on the promises they make at the time of the REO's formation. If the largest state has lucrative opportunities for trade beyond its neighbors, the incentive to develop a strong regional market is smaller.
There are many reasons why states may have significant outside options. If there is little potential for trade within a region—due to factor endowments, market size or other barriers—the incentive to establish an effective regional agreement is small, and the costs of setting up a functioning regional agreement will outweigh the benefits (Mattli Reference Mattli1999). Opportunities for trade beyond a potential regional arrangement could include anything from trading bilaterally with other countries outside the region to engaging more heavily in multilateral trade under the auspices of the WTO. Whereas Mattli (Reference Mattli1999) makes a similar argument with respect to all the participating states in an agreement, here we focus on the incentives for the largest market in the region. Since that market is a central player, without whom the agreement would not be successful, its options are pivotal to the success or failure of the agreement. The exit (and entrance) costs of smaller states are controlled in part by the desires of the larger ones in an agreement (Gruber Reference Gruber2000). For example, after Mercosur's Economic Partnership Agreement negotiations with the European Commission (EC) stalled due to the EC's unwillingness to meet Brazil's requests on agriculture, Uruguayan leaders privately expressed their intention to negotiate their own preferential trade agreements (PTAs) separately from Mercosur. However, they were pressured by Brazil not to do so.Footnote 7
In short, when the largest state (or states) conducts a significant amount of trade with its neighbors, a REO is likely to be successful.Footnote 8 Since Germany, for example, cannot credibly threaten to leave the EU in favor of better opportunities elsewhere, the EU is a relatively well-functioning organization. Even in a moderately well-functioning organization such as Mercosur, Brazil can credibly threaten to renegotiate terms, in part because of its numerous trading opportunities with the rest of the world, particularly North America. Similarly, SADC did not really function as an economic agreement until South Africa consented to join in 1994. South Africa's exports—as well as its domestic market—were always acknowledged to be the key economic drivers of that arrangement, and it is widely acknowledged that the agreement would have been defunct without its participation. This leads us to our first hypothesis:
Hypothesis 1: Where the largest state in a proposed agreement has many options for trade beyond the members of the proposed agreement (many exit options), agreements will be unable to accomplish their goals.
Furthermore, exit options will also influence the performance of the institutions created by the agreement. If there are gains to be made from trade, it is worth member states’ while to build effective institutions to further enhance these opportunities. On the contrary, even when a DSM is strong on paper, a state with good outside options can simply threaten to ship its goods elsewhere if the DSM goes against its wishes. Thus if there is one state whose participation in the REO is vital for the REO's existence, and which has significant outside options, the DSM may never be used at all. The states can gain more by effectively using the institutions they have designed than by seeking opportunities beyond the REO. Thus we do not dispute the role that strong institutions can play in enhancing trade. However, institutions that are strong on paper are not a sufficient condition for enhanced trade. Member states must have sufficient incentives to use the institutions they write down on paper. Many trade organizations feature what appear to be impressive DSMs, but it would be considered impolitic to actually use those courts.
Hypothesis 2: Where the largest state in a proposed agreement has many options for trade beyond the members of the proposed agreement (many exit options), DSMs will be ineffective.
The next section considers rival explanations for the effectiveness of trade agreements.
Alternate Hypotheses
As discussed above, a prevailing view in the existing literature is that REOs’ effectiveness can be judged by design factors—that is, the strength of the institutions within a REO, such as the courts set up by the member states (Hicks and Kim Reference Hicks and Kim2012; McCall Smith Reference McCall Smith2000). The work on the informational role of international organizations also indicates that organizations with strong DSMs written into the treaty would do a better job of promoting trade cooperation. These theories can both be tested by examining the levels of organizations’ legalization as written in the treaties. If those theories are correct, a higher level of legalization on paper would translate into an organization that is more effective across a variety of dimensions. Similarly, champions of institutional design would claim that other features, such as the proposed level of integration among member states, should have a strong impact on the agreement's overall effectiveness. However, that effect could cut both ways—either a higher level of planned integration would mean that the members set the bar too high, and it would be harder to meet their goals, or the setting of high goals would indicate the member states’ motivation to conclude a successful agreement, and the expectation would be negative. Others propose that the presence of escape clauses in agreements leads to greater flexibility, though those predictions center on the probability that an agreement will be formed and that cooperation will occur in those conditions, not necessarily that an agreement will achieve its goals (Bagwell and Staiger Reference Bagwell and Staiger2005; Rosendorff and Milner Reference Rosendorff and Milner2001). Regardless, we will test all of these arguments in the next section.
Many other external dimensions—such as the relative size of the potential members—provide the context for forming a REO. If one large state dominates, commitment to a regional agreement is likely lower because large states, in general, are less dependent on trade due to their larger internal market (Alesina and Wacziarg Reference Alesina and Wacziarg1998; Alesina, Spolaore and Wacziarg Reference Alesina, Spolaore and Wacziarg2000; McCall Smith Reference McCall Smith2000). Where asymmetry among proposed member states is low, however, one might expect all participating states to benefit equally from enhanced trade among them, which could allow for deeper cooperation among states (Slantchev, London and Stone Reference Slantchev, London and Stone2008). Therefore, such a group of states might create an agreement that could meet its own goals.
Other work also examines the effect of domestic politics on REOs (Grossman and Helpman Reference Grossman and Helpman1994; Rickard Reference Rickard2010; Baccini and Dür Reference Baccini and Dür2012; Reference BacciniBaccini forthcoming). The influential Grossman-Helpman (Reference Grossman and Helpman1994) model describes trade policy as the result of the interaction of policy makers and lobbyists who represent the industries in which a country's factor endowments are concentrated; the eventual policy benefits organized (more than unorganized) interests. Grossman and Helpman predict that a preferential trading arrangement will be formed if its benefits to exporters outweigh the losses incurred by import-competing industries and average voters (Grossman and Helpman Reference Grossman and Helpman1995). This model and subsequent work (for example, Eichengreen and Frankel Reference Eichengreen and Frankel1995; Krishna Reference Krishna1998) predict that trade-diverting REOs, the benefits of which go to a concentrated few at the expense of general welfare, are more likely to be established than trade-creating ones. We can test this hypothesis by evaluating whether agreements that promote a rise in trade among members also meet their goals more generally.
Finally, it is also possible that highly functional states tend to form better (and more effective) trade agreements. States with a higher domestic capacity and greater resources might be expected to carry those competencies to the supranational level as well; conversely, states with low domestic governance and resources would be more likely to form ineffective agreements.
Data and Analysis
We have collected a new dataset on 24 distinct characteristics of REOs around the world, as well as an overall scoring of whether the REOs are strong or weak. This article examines six of these 24 dimensions. The data collection process involved conducting expert surveys, a practice that is widely used in political science to measure latent concepts that are otherwise difficult to operationalize with observational data (Laver Reference Laver1998; Whitefield et al. Reference Whitefield, Vachudova, Steenbergen, Rohrschneider, Marks, Loveless and Hooghe2007; Hooghe, Marks and Wilson Reference Hooghe, Marks and Wilson2002; Benoit and Laver Reference Benoit and Laver2006; Marks et al. Reference Marks, Hooghe, Steenbergen and Bakker2007; Busch, Reinhardt and Shaffer Reference Busch, Reinhardt and Shaffer2009). As with all surveys, our data represent only a snapshot in time and cannot capture temporal variation in the performance of REOs. This is a well-acknowledged drawback of survey methods, but it does not limit the insights that can be gained from such data. Expert surveys have been shown to offer substantial leverage in operationalizing parties’ political positions, in situations where their own manifestos are suspect and little other available data, such as voting records, can be leveraged. This situation mirrors that of the functioning of regional trade agreements: founding documents can only be trusted to a limited extent, and little other data on their actual progress are available. All responses were given anonymously and represent individual views. They in no way represent the views or opinions of the respondent's institution or member state.
Our sample includes 25 respondents, many of whom filled out the survey for more than one REO, leading to a total of 3,924 individual data points encompassing 34 different agreements.Footnote 9 Though the number of total respondents may seem small, it should be noted that expert surveys generally have relatively few respondents. For example, the Chapel Hill expert survey on party positions in Europe obtained 98 survey responses (34 percent of the total solicited)—and parties are far more visible and public than regional agreements (Steenbergen and Marks Reference Steenbergen and Marks2007; Marks et al. Reference Marks, Hooghe, Steenbergen and Bakker2007). It is surprisingly rare to find experts who are able to comment on the many aspects of the actual day-to-day workings of REOs. Perhaps as a function of the previously discussed tendency for academics to think of REOs as like units, relatively few academics have expertise on the details of individual REOs. The majority of the responses came from practitioners, specifically the WTO's Regional Trade Agreements unit and the EU's Directorate General for Trade, which employs the main trade negotiators for the EU. Respondents included issue area and regional specialists. We attempted to balance the representation from the developed and the developing world by soliciting email responses from experts from Southeast Asia and Africa, and from in-country visits to Zambia, Botswana, South Africa, Peru, Argentina and Uruguay, as well as to various organizations in Washington, DC, the WTO offices in Geneva and the Directorate General for Trade in Brussels. The final sample includes one respondent from Africa, six from Latin America and 18 from Europe. To eliminate the biases that respondents might have in evaluating their own organization, we excluded observations in which respondents scored their own institution.
Our survey instrument asked participants to rank the REOs on which they felt comfortable commenting along a simple scale, from one to 10, for each different dimension.Footnote 10 The respondents were also asked to report the importance of each dimension to the particular REO, from 1 to 10.
The WTO's website lists 190 bilateral and regional agreements that have been reported to the organization. Of that list, however, only 34 agreements are more than just bilateral arrangements. Our survey listed those 34 REOs as prompts, and we also encouraged respondents to comment on any additional REOs that they felt should be included or felt qualified to comment on. Respondents were encouraged to only score dimensions they felt qualified to evaluate, meaning that participants did not necessarily score every dimension for every REO they chose to rank. In aggregate, respondents at the EU coded attributes of 25 different REOs, while respondents at the WTO coded a total of 30 different REOs. Unsurprisingly, the better-known REOs received more responses; the EU received 445 data points, while the European Free Trade Agreement (EFTA), Association of Southeast Asian Nations (ASEAN), Caribbean Community (CARICOM) and NAFTA received between 223 and 278 entries. On the low end, a few bilateral FTAs were scored, but of the larger REOs, the Mano River Union, the East African Community and the Arab Maghreb Union received the lowest number of respondents (between 8 and 39 total). The appendix lists the REOs used in this analysis. The standard deviations around these responses are relatively small (1.3, range 1–10), indicating a relatively high degree of inter-rater reliability and a substantial degree of expert consensus on the performance of these agreements.
Although some responses scored bilateral agreements, for the purposes of this analysis we are restricting our sample to include only multilateral REOs of neighboring states that have (or could potentially have) a secretariat or DSM. This contrasts with many other studies of REOs (McCall Smith Reference McCall Smith2000; Hicks and Kim Reference Hicks and Kim2012). Although bilateral agreements would not necessarily be excluded from our theory—since they represent possibilities for interstate cooperation with minimal levels of institutionalization—we feel the theory is most applicable to REOs, since they tend to offer the greatest opportunities for institutionalization and have the most potential to touch on a broad scope of issues. Thus we limit our empirical analysis to the REOs that notify to the WTO as more than bilateral agreements. It should be noted that even within this sample, there is substantial variation on every dimension coded.
Empirical Tests
Our first task is to show the relationship between the pre-existing measures of legalization and our survey results. Table 1 shows the correlations among the variables used in this study; an index of legalization based on treaty language was taken from McCall Smith's study of REOs (McCall Smith Reference McCall Smith2000). The McCall Smith study codes the level of formal legalization of these DSMs, realized through five different categories: whether third-party review and ruling were provided for, whether permanent judges were appointed, who has standing and what the potential punishments might be. McCall Smith took the level of legalization of the DSMs as a necessary (though perhaps not sufficient) indication of how well the agreements functioned as a whole. It is important to note that these codings were derived from the language of the agreements themselves, not necessarily from how those agreements worked in practice.Footnote 11 The original McCall Smith study coded only 12 of the regional agreements in our dataset. To retain as many cases as possible, we have taken the McCall Smith coding rubric and applied it to the remaining REOs in our data. All results hold when using only the limited subset of agreements coded in the original McCall Smith study.
Correlation Matrix of Dimensions of REOs, with Exit Options and Legalization
Many of the correlations are to be expected; experts’ scores on regional trade agreements that can meet their own goals map closely to agreements that promote intra-member trade and open markets in goods (r = 0.76). The correlation of those variables with expert scores of a well-functioning DSM is much weaker (r < 0.60 for all variables). The McCall Smith legalization index, which focuses on the strength of the DSM as written in the treaties, does not correlate with our experts’ measure of DSM effectiveness (r = −0.15) and is only weakly related to the degree to which the REO enhances members’ market access to goods (r = 0.07). This should serve as an initial indication of the practical distinction between a highly formalized DSM and the way these agreements function in practice. By contrast, the exit options of the largest member in an agreement are strongly correlated with an agreement's ability to achieve its own goals, the functioning of the DSM and trade in goods.
We now present further evidence that legalization on paper is not sufficient to produce well-functioning agreements. Our dependent variables, all measured using our expert survey, capture an agreement's ability to achieve its own goals, its ability to meet its goals multiplied by the ambition of those goals, its ability to enhance trade, the degree to which the REO provides its members market access to goods and how effective dispute settlement is in practice. We use these dependent variables to address the many ways in which REOs might potentially work. The first four dependent variables focus on our first hypothesis, while models using the DSM dependent variable relate to our second hypothesis. Examining expert assessment of the actual performance of these de jure elements of regional organizations gives us greater purchase on the factors driving effectiveness across these dimensions.Footnote 12
We rescale the survey responses on each dimension, so that they range from −4.5 to 4.5, and multiply this rescaled score on a particular dimension with its salience. Experts were asked to rate how important each dimension was for the agreement in question. The new variable has a low value when a dimension is not salient for an organization. It is highly positive when an organization scores high on a dimension it cares about, and it is highly negative when it scores poorly on a dimension it cares about. For example, the Andean Community scores relatively low on services (a rescaled score of −1), as does the South Pacific Trade and Economic Co-Operation Agreement (SPARTECA) in East Asia (−4). While trade in services is rated as an important part of the Andean Community's original charter (7 on the salience dimension), it is rated as less integral to SPARTECA's original goals (1 on the salience dimension). Thus the Andean Community receives a lower overall score on that dimension (−7) than does SPARTECA (−4), since it fell short of achieving a goal that was important in its original charter. Similarly, the EU's dispute settlement score was high both in terms of its overall ranking and the importance of dispute settlement to the EU charter (4 and 9.2, respectively, which translates to an overall score of 36.8), whereas the West African Economic and Monetary Union's (UEMOA) dispute settlement organ ranks poorly despite being an important goal for that organization (−4 and 8.25, respectively, which translates to an overall rescaled score of −33). We also ran the analyses without weighting by salience and the results remain unchanged. The unweighted results for the primary models are reported in the appendix.Footnote 13
We begin by examining the performance of the design variables alone. Do the legal institutions, as written in the treaty, explain the effectiveness of REOs? Table 2 shows the results. Results in Column 1 pertain to an organization's ability to meet its own goals; in Column 2, to an organization's ability to meet its own goals, multiplied by the ambition of those goals; in Column 3, to the trade promoted by the REO; in Column 4, to the market access in goods provided by the REO; and in the final column, to the functioning of its DSM. For our independent variables, we use our expanded McCall Smith measure of legalization as well as our expert survey assessment of the ease of invoking escape clauses. Ideally, we would have a measure of escape clauses that, like the McCall Smith measure, captures the agreement's de jure flexibility. However, like many other features of these agreements, on paper the escape clauses of many agreements are all but indistinguishable. Other existing measures of escape clauses do not cover many of the agreements in our data (Kucik and Reinhardt Reference Kucik and Reinhardt.2008). Thus we use our expert survey measure of the effectiveness of escape clauses to capture this element of institutional design and its relationship to the functioning of the organization.Footnote 14
Agreement Design as a Determinant of Effectiveness
Note: dependent variables are aggregated expert survey rankings of the ability of a REO to promote its own goals; of that ability multiplied by an agreement's ambition; of intra-REO trade; of market access for goods; and of the effectiveness of the DSM. OLS regression. Standard errors in parentheses. **p < 0.05, ***p < 0.01.
The legalization variable does not achieve statistical significance in any of the five regressions. It is also of little substantive importance. A one standard deviation change in the legalization measure (1.67 on a scale from 0 to 4) produces barely any change at all in the dependent variables. The escape clause variable has a greater impact. It achieves statistical significance in all but one regression and shows a negative relationship with all of our dependent variables. Where escape clauses are easily invoked, the trade agreement is generally less effective. Substantively, the effect of this variable is stronger as well. A one standard deviation increase in the escape clause variable leads to approximately a half standard deviation decrease in all our dependent variables, except Trade.
To reiterate our hypotheses, we anticipate that arrangements in which states were given a feasible exit option when the REO was negotiated will be less effective in meeting their goals and in creating functioning institutions. In Table 3 our dependent variables are the same as in Table 2, but to analyze our theory, we operationalize “exit options” by taking into account the possibilities for trade within the region versus trade with the outside world. As trade opportunities with the rest of the world improve relative to local opportunities, newly proposed REOs are likely to be less effective. To capture the options facing members of a proposed REO at the time of an agreement's formation, we calculate the percent of trade that the largest member (that is, the member with the highest GDP) conducts with the members of a potential REO as a share of its trade with non-members, calculated for the year prior to an agreement's initiation.Footnote 15 Thus agreements whose members trade relatively little with their proposed partners just prior to the agreement's inception have a high level of exit options, such as the Mano River Union (0.007), the Melanesian Spearhead Group (0.0007) and the Pacific Island Countries Trade Agreement (0.002). Since trade complementarities do not exist among these countries, they profit more by trading with the rest of the world. By contrast, agreements such as the EU (0.52) and NAFTA (0.30) had more to gain at the outset from creating effective agreements with countries with whom they already traded substantially.
Design Determinants of Effectiveness, with Exit Options
Note: dependent variables are aggregated expert survey rankings of the ability of a REO to promote its own goals; of that ability multiplied by an agreement's ambition; of intra-REO trade; of market access for goods; and of the effectiveness of the DSM. OLS regression. Standard errors in parentheses. **p < 0.05, ***p < 0.01.
The legalization variable is not statistically significant and has a negative sign; if anything, more on-paper legalization is associated with less effective organizations across all dimensions when controlling for exit options. By contrast, the exit options variable is statistically significant and positively associated with organizations that function better across all the dimensions measured here. For example, comparing an agreement whose key member has an average level of exit options in the sample—such as Mercosur, where Brazil's ratio of intra-member trade to trade with the rest of the world is 0.08—to an agreement that has fewer exit options, such as the EU (0.52), produces an increase in the agreement's ability to meet its own goals by 35 points on our scale. This lends substantial support to the predictive power of exit options, even in creating institutions that function as they should on paper. The appendix includes several alternate specifications for robustness checks. We run the same models using Kucik and Reinhardt's measure of agreement flexibility (Kucik and Reinhardt Reference Kucik and Reinhardt.2008) rather than our measure of escape clauses. These regressions have a smaller number of observations due to their limited coverage—their coding centers primarily on bilateral agreements, not REOs—but the main results still hold. We also run the same model with Haftel's binary coding of whether a given REO has a standing tribunal, and whether that tribunal's decisions are binding (that is, whether member states will accept the arbitration of a third party) (Haftel Reference Haftel2012). Our central results still hold.
These results present preliminary evidence for our hypotheses, but we must now control for alternative explanations. We next examine our first hypothesis (that exit options lead to less effective agreements) against the alternative hypotheses outlined above. As before, we measure the effectiveness of agreements with the survey variables: (1) the ability of agreements to meet the goals they set, (2) that ability multiplied by the ambition of the agreements, (3) the ability of the agreement to increase trade and (4) the provision of market access to goods. To operationalize the alternative hypothesis (that effective agreements are a function of the economic asymmetry of member states), we use a variant of the Herfindahl-Hirschman Index as described by McCall Smith (Reference McCall Smith2000). This index is calculated for each REO by summing the squares of each country's GDP as a share of the aggregate GDP of the REO, then subtracting the inverse of the number of countries in the agreement from this figure.Footnote 16 This measure ranges from 0 to 1 (the mean here is 0.25, with a standard deviation of 0.24); a value of 1 represents a trade agreement in which at least one trading partner is a far larger market than the other members. Thus, a value of 1 would mean that at least one country dominates the agreement, such as SAFTA (0.85) and the Commonweath of Independent States (CIS) (0.80), where India and Russia are the central actors, respectively. Members of trade agreements at the lower end of the scale have greater symmetry, including the Organization of Eastern Caribbean States (OECS) (0.007), Central American Common Market (CACM) (0.04) and COMESA (0.05). We also control for the total logged GDP of an agreement, as well as the average GDP per capita of agreement members, on the assumption that larger and wealthier agreements may perform better. Finally, these models include a variable that captures the level of trade diversion promoted. Recall that the Grossman-Helpman models predicted that agreements resulting from substantial lobbying from domestically competitive sectors should be more effective. We can capture this argument by including a variable from Haftel's (2012) dataset on trade among all members.Footnote 17 The results are presented in Table 4.
Exogenous Determinants of Effectiveness
Note: dependent variables are aggregated expert survey rankings of the ability of a REO to promote its own goals; of that ability multiplied by an agreement's ambition; of intra-REO trade; of market access for goods; and of the effectiveness of the DSM. OLS regression. Standard errors in parentheses. *p < 0.10, **p < 0.05, ***p < 0.01.
The first two models demonstrate that our finding for exit options on the Ability and Ability x Ambition variables holds when controlling for asymmetry. The next two models also include the GDP and Trade controls. Our sample size decreases due to missing values on the trade diversion question, but the Exit Options variable remains significant. The final three models present the results of the full model with different dependent variables—increased trade, market access to goods and effectiveness of the DSM. Again, the results hold, although the Exit Options variable just barely misses the conventional threshold for statistical significance in the regressions with market access to goods and DSM as the dependent variables (p = 0.11 and p = 0.13, respectively). As further robustness checks, we calculated Cook's Distance for the models in Table 4 and dropped the most influential cases. See the appendix for these models.
Partial regression plots: effect of legalization and exit options (controlling for GDP per capita)
If the alternate hypotheses were valid, including the variables listed above should negate the previously observed significance of the exit options variable. But those variables are not even consistently statistically significant on their own. The asymmetry variable has the expected sign, such that higher disparities in income among potential REO members are associated with agreements that do not work well. However the relationship is only statistically significant in one model, in which the dependent variable is expert assessment of overall trade promotion within the REO. GDP per capita is also correctly signed—wealthier organizations perform better—and statistically significant in two of the four models in which it is included. As before, though, our exit options variable is statistically significant (at the p < 0.01 level) in almost every model. Moving from the mean to the maximum of the exit options variable (roughly, from the scores of the CIS to those of NAFTA) would give an agreement a huge jump in ability (42 points on our scale).
As a final test of our hypotheses, we include design and exogenous determinants of REO effectiveness in the same model. Table 5 presents regressions of our dependent variables on the exit options variable, the recoded McCall Smith variable (to capture legalization) and average GDP per capita—the best-performing control variable from the models in Table 4. Once again, the exit options variable most consistently explains the dependent variables. It is statistically significant in four of the five models, and borderline significant at the p < 0.1 level in the fifth. To visually display the effects of these variables of primary theoretical interest tested against one another, we present partial regression plots in Figure 1. The plots show the lack of effect of de jure legalization and the strong positive effect of Exit Options and GDP per capita. They also help alleviate fears that our results are driven by outliers or highly influential cases.
Design and Exogenous Determinants of Effectiveness, with Exit Options
Note: dependent variables are aggregated expert survey rankings of the ability of a REO to promote its own goals; of that ability multiplied by an agreement's ambition; of intra-REO trade; of market access for goods; and of the effectiveness of the DSM. OLS regression. Standard errors in parentheses. *p < 0.10, **p < 0.05, ***p < 0.01.
Conclusion
This article has begun to open up the black box of regional trade agreements. We have gone beyond the current literature by examining systematic differences within the REOs that now mark the global economic landscape. We examine the entire range of REOs, from those that are highly effective to those that are little more than paper agreements, against an array of different variables. Moreover, we have developed a theoretical framework for understanding when and why agreements prove to be effective. By demonstrating the gap between legalization and actual effectiveness, we have also shown that high levels of legalization do not necessarily have their intended consequences, which addresses long-stated concerns about the outcomes of legalization (Martin and Simmons Reference Martin and Simmons1998; Kahler Reference Kahler2000). Of course, legalization is only one type of institutional design. Organizations have many design features that may impact effectiveness. Further research that codes various elements of institutional design could explore this potential relationship more fully.
Many have argued that membership in regional economic agreements is associated with positive outcomes for countries, including direct investment (Milner and Büthe Reference Milner and Büthe2008) and reduced volatility (Mansfield and Reinhardt Reference Mansfield and Reinhardt2008). Some of those outcomes are attributed to the rule of law that those agreements are supposed to provide. This article has shown that agreements’ legal frameworks do not automatically make them effective. It may, however, still be the case that investors or other economic actors may perceive or anticipate the rule of law, even if it is not actually put into practice.
These findings have important implications not only for scholarship in the area of economic integration, but also for policy work. Many of the trade policy makers with whom we spoke in the course of conducting our expert survey were quick to acknowledge the wide variation not only across different REOs, but also the frequently observed gap between the commitments announced in summits and the actual fulfillment of those commitments. This survey should provide a consistent metric for assessing how these REOs work in practice.
Furthermore, these findings should be useful to researchers concerned with actual levels of integration among states. Our theory and data demonstrate that—even at the bargaining stage—states may have preferences for levels of integration that are not expressed in the language of the treaty or the structure of the bureaucracy. By modeling the environments within which states negotiate on integration, we have provided a contextual explanation that accounts for variation in the shape of regional agreements, as well as in their relative effectiveness. This allows us to answer questions not only about when deeper integration is preferred, but why agreements take the particular forms they do. This perspective sets our research apart from the many studies that attempt to account for variation in levels of economic integration but cannot explain the different types of integration that we observe.
Nevertheless, work still needs to be done to understand the existence of so many ambitious, yet poorly functioning REOs. Why do states bother to write ambitious agreements at all if they know the terms of the agreements are unlikely to ever be met? While a complete answer is beyond the scope of this article, it remains a point worth exploring. If all potential member states know that the terms of an agreement are unlikely to ever be fulfilled, there is little cost in negotiating and writing an ambitious document. As demonstrated earlier, the language can even be directly borrowed from existing functioning treaties. While writing an agreement involves some costs, such as setting up and traveling to an international meeting, they are relatively small. Moreover, they may be negotiated because the signatories feel that this what the international community wants to see. These possibilities all require separate theorizing and testing. Regardless, it is clear that the incentive structure behind forming ineffectual REOs is theoretically distinct from the drive to create REOs that meet their goals, and this article has taken a step toward elucidating the logic of the latter.
Appendix
List of REOs
Effectiveness with Kucik & Reinhardt Flexibility Variable
Unscaled Dependent Variables
Legalization, Flexibility and Exit Options
Dispute Settlement and Exit Options
Exclusion of Outliers (ECCAS)
Exclusion of Outliers (EU)
Robustness Checks: Control for Proposed Level of Integration, Full Model with Raw Scores
Descriptive Statistics for Dependent Variables
