1. Introduction
The international investment agreement (IIA) universe has become a fixation for policy-makers, civil society, and scholars across disciplines. The global proliferation of bilateral investment treaties (BITs) and preferential free trade agreements (PTAs) with investment components has created a web of more than 3500 treaties, most of which offer private investors the possibility to launch direct claims for treaty violation against sovereign states (UNCTAD, 2019). As a result, the IIA regime has become one of the most litigated and controversial areas of international law with close to 1000 disputes (UNCTAD, 2019). These disputes, at times, challenge sensitive areas of public policy-making from tobacco control to renewable energy policies and can lead to multi-billion dollar decisions (or ‘awards’ as they are known in arbitration) against host states (UNCTAD, 2015).
In consequence, citizens have protested against investment arbitration, states have denounced or renegotiated their investment treaties, and policymakers across the globe have begun to rethink the system, culminating in ongoing multilateral reforms under the auspices of the United Nations. Scholars have been equally captivated by investment agreements. Political scientists have asked why states sacrificed their sovereignty (Poulsen, Reference Poulsen2015), economists have investigated whether these treaties yield the investment-promoting benefits they promise (Bellak, Reference Bellak2015; Desbordes, Reference Desbordes2016; Egger et al., Reference Egger, Erhardt and Masllorens2019; Pohl, Reference Pohl2018; Sauvant and Sachs, Reference Sauvant and Sachs2009; Yackee, Reference Yackee2008, Reference Yackee2010), and legal scholars have studied the burgeoning investment arbitration case law that interprets and applies these agreements (see e.g. Brown, Reference Brown2011).
In spite of the intense interest IIAs have sparked and the research they have motivated, important aspects of the IIA universe still remain unknown or understudied. Scholars have focused attention on readily available English language treaty texts of major capital importing and exporting states leaving the IIA practice of entire regions, mostly in the developing world, underexplored. Furthermore, although coding efforts by international organizations and researchers have elucidated important general content variations among IIAs, treaty language particular to specific countries or of interest to particular research questions have remained outside of their purview. As a result, the collective understanding of the IIA universe is patchy, prone to misconceptions, and partially biased in favor of English-language treaty practice of dominant states.
To address these shortcomings, this article introduces the Electronic Database on Investment Treaties (EDIT). EDIT seeks to facilitate informed public discourse, evidence-based reform, and academic study by providing unprecedented access to the most extensive set of the IIA texts to date. EDIT stands out among existing investment treaty resources for five reasons: it is (1) comprehensive, (2) uniform, (3) machine-readable, (4) annotatable, and (5) freely accessible to researchers and the public.Footnote 1
First, EDIT is the most comprehensive database on IIA full texts to date. Relying on more than 3474 IIAs available texts, it contains more agreement full texts than any other database.Footnote 2 In the process of creating EDIT, we contacted national agencies and ministries and uncovered hitherto unavailable full texts shedding new light on the treaty practice of states, e.g. in the Middle East and North Africa, which are underrepresented in other datasets. In addition, we adopt a more inclusive definition of IIAs that not only includes BITs and PTAs, but also Friendship Commerce and Navigation (FCN) treaties, which have been coined the ‘first investment agreements’ (Vandevelde, Reference Vandevelde2017). As a result, EDIT offers the most comprehensive overview of the IIA universe yet.
Second, EDIT is uniform insofar as it contains an English version of each treaty. While EDIT includes more non-English texts than any other investment law database, we realized that Arabic, Russian, and German texts may not be accessible to all users. Therefore, we provide an English translation of all non-English treaty texts. While the treaty's original language is available in the database and constitutes the authentic text, the standardized translation allows users to search for key provisions such as ‘fair and equitable treatment’ or ‘full protection and security’ across the entire database irrespective of the original language. We thereby hope to make the content of understudied non-English texts more widely accessible.
Third, EDIT is machine-readable. Existing databases often contain imaged texts that can be read by humans but not searched or analyzed by computers. EDIT now renders all IIAs machine-readable. We went through the painstaking process of digitizing texts through optical character recognition (OCR) and manually corrected ensuing conversion errors. We also standardized the structure of each treaty through Extendable Markup Language (XML) and Extensible Hypertext Markup Language (XHTML) that allows us to distinguish between preambles, articles, and annexes. As a result, users can now search through all agreements more efficiently. In addition, EDIT allows for the machine-processing of text-as-data to support the emerging field of computational legal research.
Fourth, EDIT can be annotated to map the content features of treaties, meaning that leveraging the XML structure of EDIT documents, users can identify and mark content features of investment agreements directly in the text. We use a transparent and automated keyword-based approach to identify a first layer of content features that are likely to be of general interest to users and researchers. In addition, we anticipate that users will validate, refine, and add to this content mapping over time through their own annotations. In other words, EDIT allows crowd-sourcing the content analysis of IIAs.Footnote 3 Furthermore, while the authors and the World Trade Institute, in collaboration with its research partners,Footnote 4 are committed to regularly updating EDIT, we hope this crowd-sourcing approach will help build a broader community of users that will expand and maintain the database.
Fifth, EDIT is a free-of-charge academic resource primarily aimed at researchers. Several IIA databases are only available to paid subscribers, thus limiting access to investment treaty knowledge and expertise. UNCTAD's investment policy hub stands out as an invaluable and free resource for developing countries and the wider public. Yet, its text versions so far do not always match the quality of commercial providers and it is difficult for researchers to search and access its text in bulk. EDIT is not meant to compete with these databases, rather, it seeks to complement them by broadening access to IIAs and by facilitating innovative text-as-data research.
The primary purpose of this article is to introduce the EDIT database. In addition, we also offer a range of new descriptive insights about the investment treaty universe to showcase EDIT's utility for legal scholars, political scientists, and economists. A better understanding of the IIA universe is crucial in all three disciplines. While we anticipate that legal researchers will use EDIT to search for key terms, its unprecedented scope also provides new avenues for empirical scholarship, e.g. by studying the evolution of key clauses such as ‘most favored nation treatment’ provisions or by investigating the interaction between treaties and investment arbitration. Moreover, its machine-readable texts can fuel new computational legal analyses of investment treaty content (Alschner et al., Reference Alschner, Pauwelyn and Puig2017). Political scientists, in turn, can use EDIT to investigate the determinants of investment treaty design by exploring, for example, why some content features proliferate while others decline. Finally, to economists EDIT offers multiple ways to model the design of IIAs in order to assess its effects, e.g. on investment flows and other economic factors.
This article is structured as follows. First, it outlines in more detail the added value that EDIT can provide. Second, it introduces specific EDIT features. Third, it showcases interdisciplinary applications based on EDIT to address diverse questions across multiple levels of analysis ranging from the global to the treaty provision level.
2. Why Another IIA Database?
EDIT goes beyond existing IIA databases. UNCTAD, for example, administers a free and extensive database and full text repository of IIAs. By May 2020, it included information on 3712 IIAs. However, the full texts of 486 IIAs are missing (around 13% of the total), leaving 3226 available texts. The large majority of the available texts are provided in English and the files are offered in pdf format, partly as scanned images and partly as machine-readable texts.Footnote 5 Other existing public international databases, such as the United Nations Treaty CollectionFootnote 6 and ICSID's ‘Investment Laws of the World’ (the latter only in printed loose-leaf), also provide selected full texts of investment treaties, without aspiring to include all existing IIAs.
In addition, several commercial databases have been created, which are only available to paid subscribers. Kluwer Arbitration ‘BITs’ features around 1400 treaties supplied by the Penn State Institute of Arbitration.Footnote 7 Similarly, Oxford University Press (OUP) ‘Investment Claims’ includes around 1850 BITs.Footnote 8 However, both are limited to agreements in English-language. More recently, another commercial database (Jus Mundi) includes around 2500 IIAs, in several languages.Footnote 9 Some of the above databases also rely on investment obligations in PTAs, but lack a comprehensive coverage.
A third group of datasets have focused on coding specific provisions in investment treaties and trade agreements instead of providing the full text. In BITSEL (‘BIT Selection Index’), eleven key elements have been coded manually relying on more than 1,500 BITs and more than 100 PTAs with investment chapters (Chaisse and Bellak, Reference Chaisse and Bellak2015). In DESTA (‘Design of Trade Agreements’), the coding includes seven of the most relevant provisions with respect to non-discrimination, market access, and investor-state dispute settlement (ISDS), from approximately 1000 PTAs signed between 1947 and 2018 (Dür et al., Reference Dür, Baccini and Elsig2014). PluriCourts, the Centre for the Study of the Legitimate Roles of the Judiciary in the Global Order at the University of Oslo, has been coding investment treaties invoked in investor-State arbitration cases as a part of a larger project on an investment treaty arbitration database (PITAD – PluriCourts Investment Treaty Database).Footnote 10 Finally, the IIA Content Mapping Project, a collaborative initiative between UNCTAD and several universities worldwide to code the features of IIAs, has categorized more than 120 provisions for around 2500 treaties found in UNCTAD's database.Footnote 11
Together these databases are extremely helpful, but they also suffer from several limitations. First, they lack texts of agreements currently in force, particularly older agreements and those signed by developing countries. In addition, these databases tend to focus on English treaty texts. Even though the most common language of IIAs is English (used in around 64% of the IIAs), more than 633 treaties are only available in another language (Polanco Lazo et al., Reference Polanco Lazo, Desilvestro and Bazrafkan2018). Several existing databases include IIAs exclusively in English, and the ones that include treaties in another language usually do not provide an English translation of the official agreement. Not accounting for these biases may affect the validity of research findings based on existing databases.
Second, some databases suffer from lack of easy access or may not be user-friendly. Commercial databases allow sophisticated text searches but are only available to subscribers. Furthermore, they typically discourage, prevent, or prohibit bulk download of texts or web scraping, which curtails the ability of researchers to run text analysis software across agreements. Publicly available databases are more accessible, but they typically contain inferior search functions and poorer data formats. Texts are often stored as image or photographic presentations. This limits the full exploitation of the data for research purposes as texts need to be converted into a machine-encoded text formats (e.g. using Optical Character Recognition (OCR)) to carry out textual analyses using computational approaches.
Third, datasets of manually coded content features are useful, but have significant shortcomings, too. They operate at a high level of abstraction and might not always detect subtle and important differences. For example, knowing that a treaty contains a clause on ‘expropriation’ is helpful, but even such ‘standard’ provisions in investment treaties can vary substantially. Indeed, even the most fine-grained manual coding efforts can never be fully satisfactory, as country-specific practices, new arbitral interpretations, and evolving research questions constantly expand the list of potentially meaningful treaty variation. Moreover, manual content mapping tends to focus on key words found in the main body texts of treaties, but does not generally account for the interaction of obligations with exceptions or additional annexes or protocols or schedules that may alter or qualify the content of obligations.
In short, there is need for a database that addresses these shortcomings and limitations. EDIT fills this gap by providing the most comprehensive and uniform database of IIAs in a machine-readable format that allows for efficient search, text-as-data analysis, and content feature annotation.
3. Introducing EDIT
As mentioned, five features set EDIT apart from other IIA databases. EDIT is (1) comprehensive, (2) uniform, (3) standardized and machine-readable, (4) annotatable, and (5) free. We explain each characteristic in more detail in this section.
3.1 Comprehensive
EDIT is comprehensive in two ways. First, it contains more full texts of bilateral investment treaties than any other database. Second, it adopts a more inclusive definition of what constitutes an IIA to account for the diversity of investment treaties beyond BITs and integrates these treaties into a single database.
How does EDIT determine what an IIA is and whether it is included in the database? We define an IIA as a treaty concluded under international law between two or more states, territories, or economies,Footnote 12 which, in whole or in part, contains substantive obligations to protect and/or liberalize foreign investment either generally or sector specific. This notion includes bilateral investment treaties, investment chapters of preferential trade agreements and regional investment agreements (RIAs).Footnote 13 A few IIAs included in EDIT are not listed in UNCTAD or any other database, and consist mainly of Friendship, Commerce, and Navigation Agreements (FCNs), which have been referred to as the ‘first BITs’ (Vandevelde, Reference Vandevelde2017). At the same time, only those investment-related obligations in PTAs that fit our definition are included in EDIT; we thus exclude general framework agreements counted by UNCTAD as a treaty with investment provisions, since they do not themselves contain specific investment protection or liberalization provisions. Our definition also excludes a number of agreements that have been listed in some existing databases; for example, we do not count the ICSID Convention as an IIA.Footnote 14 As the purpose of EDIT is to identify past or current existence of IIA texts, terminated and renegotiated IIAs are also included in the database. We also excluded some ‘duplicate treaties’, meaning agreements that are listed as separate treaties, whereas in reality these had identical texts.Footnote 15
With above definition in mind, we began compiling IIA full texts and metadata. First, we identified a list of existing public and private databases, both digital and physical, as well as government webpages and official websites of international organizations (e.g. United Nations, World Bank, OECD) where information on investment agreements can be found. In the end, we obtained a list of 146 different websites of ministries, agencies, and public institutions. UNCTAD's IIAs Navigator proved particularly useful as the organization regularly surveys UN member states for their IIA practice and thus possesses some of the most accurate meta data and most advanced full text repository. Using these different websites, we generated a new list of agreements that would fit our definition. Where databases differed on meta-data information, we cross-checked these.
Second, based on this close to comprehensive list of agreements, we set out to collect the full texts. When an agreement's text was unavailable on one of our 146 target websites, we contacted governments directly (through their respective ministries or agencies competent on investment issues) in order to obtain a copy of the official text of the agreement. On occasion, local personal contacts in each country were instrumental to obtain texts (Polanco Lazo et al., Reference Polanco Lazo, Desilvestro and Bazrafkan2018). Through this process, we were able to include treaty texts into EDIT that cannot be found in any other database.
EDIT strives to cover the full texts of as many IIAs as possible. Aside from the main texts, it also includes preambles, footnotes, annexes, protocols, schedules, and side-letters. When it comes to PTAs, i.e. agreements that cover investment alongside many other issues, EDIT includes investment chapters, but also all other relevant parts that relate to investment protection or liberalization such as chapters on trade in services, financial services, taxation, and general exceptions.
By May 2020, the number of IIAs included in EDIT amounted to 3617, categorized as 3239 BITs and 378 other IIAs. Of that number, we have a total of 3474 available texts meaning that we do not have access to texts of 143 IIAs that we know to exist (‘‘Missing IIAs’’).Footnote 16 Table 1 illustrates the two types of categories of treaties found in EDIT (BITs and Other IIAs). It also reports the total count of signed treaties that are/have been in force (Force) and those that have not yet entered into force (NoForce). Amendments and replacements are also included in the available agreements. The remaining columns provide the count and percentage of treaties in force for which no text could be found (Miss) and a count of treaties that are missing and are pending (MissForce)
EDIT database

Note: Authors’ calculations based on EDIT database.
3.2. Uniformity
IIAs come in different languages. While most IIA databases focus primarily on English texts, EDIT covers a greater amount of non-English texts. Table 2 provides an overview of the different languages.
Most common languages

Note: N and N% indicate the number and percentage of treaties available in the original language
About 63% of the IIAs found in EDIT are written officially in English or have a complementary translation into that language. But some treaties are only available in another language, notably French (around 10% of the treaties) and Spanish (around 7% of the treaties), followed by Russian, German, and Arabic (each one representing around 2% of the treaties). All other languages represent less than 1% of the universe of treaties, although considered altogether represent around 10% of available texts.
To make these texts uniformly accessible and to facilitate text-as-data research across the entire corpus, we decided to provide an English version for each treaty in EDIT. All IIAs that were not found in English were translated, using machine and manual translation. The former was feasible with the help of customized machine translation software.Footnote 17
We first translated all non-English treaties in French and Spanish (that constitute the largest part of non-English treaties) using the tailor-made software for IIAs. For this purpose, two machine translators were built (‘MOSES’), using Joint Research Centre (JRC) texts from the EU, and ‘parallel texts’ of IIAs that contain Spanish/English or French/English versions in the original IIAs. Overall, 2000 sentences were drawn upon for tuning. The outcome of this machine translation provided high accuracy in short texts and common languages. MOSES was further improved with data from the UNCorpus (11.6 million sentences). MOSES translation was then compared with Google Translate using BLEU (Bilingual Evaluation Understudy), an algorithm for evaluating the quality of text which has been machine-translated from one natural language to another with different precision (Papineni et al., Reference Papineni, Roukos, Ward and Zhu2002). For all other language translations, we relied on the general machine translation engines (Google Translate, DeepL), with manual corrections by team members, research assistants, or volunteers fluent in those languages.Footnote 18
3.3 Standardization and Machine-Readability
Aside from the translation exercise, we also did extensive manual work to convert texts into a standardized digital format. This crucial step allows EDIT users to engage in more effective search tasks, opens the door for computational content analysis techniques, and enables the annotation of IIA content.
The conversion towards machine-readability involved several steps. If original files were non-machine readable (e.g. scanned or image pdfs), we digitized the text using OCR software. This process can create errors (e.g. an ‘i’ is recognized as ‘l’). With the help of research assistants, we went through all texts to correct such errors as rigorously as possible.
Once texts were machine-readable, we standardized them into a common format and structure. We store texts in Extensible Markup Language (XML), which allows augmenting text with mark-ups for (1) meta data, (2) layout and text structures, and (3) additional content annotation.Footnote 19 Each XML contains information characterizing the IIA (e.g.. its parties, year of signature). This is followed by the text of the agreement, which we split into its subcomponents, i.e. its preamble, body, and annexes. Furthermore, treaty texts are split up into articles, paragraphs, and footnotes (when they exist) to capture document structure. Researchers can subsequently use this information to search exclusively in preambles or specific paragraphs. We also use this detailed document structure for the automated content annotation of IIAs.
3.4 Content Annotation
From researchers to policymakers, the users of EDIT are likely interested in specific IIA content features. For example, they may want to investigate all provisions dealing with ‘fair and equitable treatment’ or ‘most favoured nation treatment’. In contrast to existing manually coded datasets that keep full text and content information separated, EDIT's XML format allows ‘tagging’ or labelling specific parts of the text according to content features.
We acknowledge that such a content annotation cannot be comprehensive, since it is not feasible to foresee all the specific features that a user may be interested in. Therefore, EDIT offers a first layer of content features, which registered users can expand on through their own annotation.
To implement this general layer of content features, we opted for an automated keyword-based tagging (based on Alschner and Skougarevskiy, Reference Alschner and Skougarevskiy2016a). We systematically examined and compared treaty texts to identify recurrent expressions of core investment law concepts, e.g. the words ‘full protection and security’ and ‘complete protection and security’ indicate a protective norm of the same type. On that basis, we created a list of keywords and their corresponding content features. We then used that list to automatically tag treaty articles that contained a keyword with the content feature. To improve accuracy and distinguish between categories that share overlapping keywords, we used article titles (when they exist) to limit the scope of the search. This allowed us to distinguish a health reference in the preamble from a health-related general exception for instance. This general annotation of IIA facilitates the search for specific content features and will allow users to further refine and expand our categorization through future manual annotation.
3.5 A Free Academic Resource for Research
EDIT is a free of charge academic resource to support and facilitate research on international investment agreements. EDIT overcomes a limitation of some datasets that are of high-quality, but hidden behind a paywall. EDIT thereby seeks to expand access to IIA full texts. Moreover, EDIT is designed to facilitate scholarship and innovative research, in particular the computational analysis of IIAs that treats these texts as data (Gentzkow et al., Reference Gentzkow, Kelly and Taddy2017; Grimmer and Stewart, Reference Grimmer and Stewart2013). As a result, EDIT complements, but does not seek to duplicate or compete with, other available resources on investment law.
4 Research Applications of EDIT
EDIT aims to facilitate investment treaty research across disciplines. In this section, we situate EDIT in current scholarship in international law, international relations, and economics, before showcasing the new descriptive insights that can be gained from it.
4.1 EDIT and International Law Research
We anticipate that most international lawyers will use EDIT as a searchable full text database. EDIT will provide these users with quick access to the agreements that contain the key words or content features they are interested in. The database will thereby enable practitioners and researchers to answer questions such as ‘which agreements contain references to sustainable development?’, ‘which treaty included the first counterclaims clause?’, ‘which agreements include provisions on investment facilitation?’, or ‘which treaties use identical wording on expropriation?’.
Beyond one-off searches, EDIT's comprehensiveness and machine-readability can support a wide range of doctrinal, empirical, and computational legal research agendas. It can help investigate trends (‘How has the content of most favored nation clauses evolved over time?’);Footnote 20 legal policy questions (‘What treaty design features or language is correlated with losing or winning an investment claim?’); and large-scale content patterns (‘How consistent are a country's investment agreements’). For example, researchers have used early versions of EDIT to show that treaty design is increasingly diverging rather than converging (Alschner and Skougarevskiy, Reference Alschner and Skougarevskiy2016a); the close similarity between the Trans-Pacific Partnership (TPP) investment chapter and prior IIAs concluded by the same signatory countries (Polanco Lazo and Gómez Fiedler, Reference Polanco Lazo and Gómez Fiedler2017); the partial alignment of G20 countries treaty practice with their Guiding Principles for Global Investment Policymaking (Polanco Lazo, Reference Polanco Lazo2017); or that three out of four citations in investment arbitration awards reference decisions rendered under a treaty that is 50% or more dissimilar than the treaty currently interpreted (Alschner, Reference Alschner, Fauchald, Behn and Langford2020).
The perhaps greatest contribution of EDIT, however, is to help make the unknown known. Legal scholars tend to focus on true positives – legal acts that could happen and did happen. For example, they study the use of general exceptions in IIAs through the arbitral decisions that interpreted these exceptions (Henckels, Reference Henckels, Bartels and Paddeu2018; Keene, Reference Keene2017); these are illustrative, but not comprehensive. Scholars should also study false negatives – legal events that ought to happen, but did not. We can learn as much about the role of general exceptions in IIAs by looking at the cases where they should have been discussed – because the underlying treaty contained one such exception and the facts warranted its invocation – as by studying those decisions that did interpret them. Yet, while it is easy to spot true positives, it is hard to identify false negatives. EDIT can help in this respect. By offering a comprehensive repository of investment treaty texts, researchers can easily identify which treaties contain general exceptions and compare these to the awards rendered thereunder to investigate the whole universe of relevant observations and render the unknown known (Alschner and Hui, Reference Alschner, Hui, Sachs, Coleman and Johnson2019).
4.2 EDIT and International Relations and Economics Research
In international relations research, EDIT can support and weigh in on research debates related to the design determinants, evolution, and impact of investment treaties by offering a more comprehensive and machine-readable dataset.
When it comes to exploring and explaining the variation in treaty design, the BIT literature has made important advances in recent years by relying on manually coded data. There is evidence that for instance after economic downturns or civil wars, countries are more likely to sign BITs (Billing and Lugg, Reference Billing and Lugg2019; Simmons, Reference Simmons2014). Further, autocratic regimes are not shy in opting for bilateral investment treaties to bolster their political survival (Arias et al., Reference Arias, Hollyer and Rosendorff2018; Mazumder, Reference Mazumder2015). EDIT will allow us to tease out in more detail the types of treaties that are signed by these leaders. Scholars have also investigated what factors are responsible for specific design choices by focusing on dispute settlement clauses (Allee and Peinhardt, Reference Allee and Peinhardt2010) or on the degree of a treaty's legalization (Abbott et al., Reference Abbott, Keohane, Moravcsik, Slaughter and Snidal2000; Manger and Peinhardt, Reference Manger and Peinhardt2017). EDIT can complement such existing work by analysing in more depth the linguistic subtleties that vary between agreements or by comparing computational and manual text coding methods. It can also help further develop, operationalize, or refine concepts, such as ‘delegation’ or ‘policy space’ based on their varied textual expressions (Blake, Reference Blake2013). By providing a more refined description of treaty variation, EDIT also opens new opportunities to study the determinants that contribute to such variation. It can be used to identify what states had a greater influence over the design and authorship of the agreement (Lugg et al., Reference Lugg, Lund, Allee and Elsig2019). The recourse to a comprehensive database allows us to detect in such events the trade-offs negotiators make within and across treaties.Footnote 21 In addition, by providing full texts of BIT from hitherto understudied regions, such as the Middle East and North Africa, EDIT can help tease out regional specificities. For example, using EDIT Morr Link and Yoram Haftel found that states with an Islamic Legal tradition systematically opt for more informal dispute settlement arrangements in their agreements (Link and Haftel, Reference Link and Haftel2019).
EDIT also facilitates studying the evolution and diffusion of investment agreements by placing a larger dataset at the disposal of researchers. It thus becomes easier to identify where a particular treaty language first appeared. This, in turn, can support capturing and isolating certain events that trigger design changes over time for historical institutionalist scholars (Poulsen, Reference Poulsen2019), or provide additional evidence as to the type of diffusion mechanisms (e.g. coercion, competition, learning, emulation) that could have been instrumental for the evolutionary path (Solingen, Reference Solingen2012). EDIT can also prove useful for estimating how treaty texts change during re-negotiations on investment treaties and what internal and external factors principally drive observed changes in treaty language (Haftel and Thompson, Reference Haftel and Thompson2018).Footnote 22 In particular, EDIT creates new opportunities to study how investment arbitration impacts treaty design and how treaty design changes affect arbitral outcomes.
The data could provide a new impetus to the political-economy literature on the impact of BITs which has predominantly focused on FDI flows (Bonnitcha et al., Reference Bonnitcha, Poulsen and Waibel2017; Pohl, Reference Pohl2018; Sauvant and Sachs, Reference Sauvant and Sachs2009; UNCTAD, 2014). Whereas the evidence on whether aggregate FDI flows have increased due to BITs remains inconclusive (Bellak, Reference Bellak2015; Büthe and Milner, Reference Büthe, Milner, Sauvant and Sachs2009; Hallward-Driemeier, Reference Hallward-Driemeier2003), and some work finds no clear link between treaty protections and investment (Yackee, Reference Yackee2008, Reference Yackee2010), research has over time increasingly focused on the conditions under which treaties influence investment flows positively. Starting with the work by Egger and Pfaffermayr (Reference Egger and Pfaffermayr2004), the argument has been made that the credibility of ratifying BITs and therefore the implementation stage matters. Tobin and Rose-Ackerman (Reference Tobin and Rose-Ackerman2005, Reference Tobin and Rose-Ackerman2011) further find that domestic institutions and the overall investment climate matter for FDI inflows for developing countries. Allee and Peinhardt (Reference Allee and Peinhardt2011) in their work provide evidence that BITs do increase FDI into countries that sign them, but only if those countries are not subsequently challenged before an arbitration tribunal (e.g. ICSID). They further observe notable losses of FDI for countries that are challenged in international arbitration. EDIT could allow us to study in more detail what type of commitments (BIT design) in conjunction with domestic factor constellations are causally related to economic effects. Following this line of research, Desbordes (Reference Desbordes2016) has found that BITs specifically granting access to ISDS and protecting foreign investors from discrimination have a large, positive, and statistically significant effect on FDI. EDIT could also prove useful for more recent work that departs from analysing aggregate FDI data and focuses more on heterogeneous effects across sectors by quantifying the effect of BITs on firm ownership (Egger et al., Reference Egger, Erhardt and Masllorens2019). More systematic attention to the type of sectors and the elements within global supply chains that will be most likely affected by FDI provisions can certainly complement existing case-study evidence and contribute to more knowledge about BITs economic effects.
Finally, on the impact side of investment agreements, the data will make it also easier to evaluate how commitments diffuse vertically, namely how legal texts affect the formulation of domestic Legislation (direct textual boilerplating). Also by combining investment agreement commitments in various international treaties, it can help disentangling for instance effects as part of trade agreements vs single-standing investment agreements on economic parameters ranging from investment and trade flows to wages, jobs, and technology transfers. In other words, studies can more accurately estimate the impact by types and forms of treaties and further explore where treaty commitments act as substitutes or complements.
5. Using EDIT to Explore the IIA Universe
In this section, we showcase possible usage of EDIT by exploring empirical questions on IIAs in law and international relations research with respect to (1) global and (2) national treaty practices as well as in relation to (3) specific treaty provisions.
5.1 Global Treaty Patterns
The global structures underlying the IIA regime are of interest to lawyers, political scientists, and economists. Here, we pick two research questions with interdisciplinary significance. First, is there a global treaty model underlying the IIA universe? Second, have IIAs become less mandatory and more hortatory as treaties become longer? Such questions matter for a better understanding of the evolution, diffusion of treaty design as well as the effect of investment treaties.
5.1.1 Diffusion of Treaty Design: Divergent Practices or Universal Norms?
A first question is whether a specific treaty design or model dominates the IIA regime. On a spectrum between treaty design uniformity and diversity, existing research suggests that the IIA regime sits somewhere in the middle: on the one hand, scholars found that BITs tend to be highly similar within national (mostly developed) country treaty networks, but generally differ between states (Allee and Peinhardt, Reference Allee and Peinhardt2014); on the other hand, researchers have also pointed to the fact that many of these agreements, while differing in language, share a set of common norms and principles (Salacuse, Reference Salacuse2010). By providing the most comprehensive database to date, EDIT allows to situate and trace the regime development more accurately between the extremes of national practices and universal norms and trends of convergence or divergence.
We investigate whether underlying the diverse language of national BIT programs lies a common treaty design denominator. Put differently, we want to assess to what extent a particular underlying treaty design diffused so successfully that, over time, it came to dominate the treaty network (or parts thereof). EDIT supports a myriad of different research strategies to investigate the question.
One could test existing explanations about how core features are diffusing over time and space. One could also look at imprints of model conventions and systematically test propositions, such as the one put forward by Stephan Schill, that the 1967 OECD model inspired later treaty language (Schill, Reference Schill2009). However, for the purpose of illustration we let the data speak and opt here for an inductive research approach. We calculated the textual similarity of EDIT BITs and use a hierarchical clustering algorithm to group agreements by their similarity in order to find a latent model that underlies the broadest possible practice of a diverse set of states.
Figure 1 shows the results as an ordered heat map. The axes symmetrically regroup 3054 BITs and display a color-coded version of the similarity matrix: high similarity between two treaties is indicated as red, whereas yellow signifies low similarity. The red quadrangles along the diagonal axis of the heat map indicate the internally coherent national treaty networks that closely follow a national model template, on which Alschner and Skougarevskiy reported (e.g. French and Dutch BIT networks are annotated in Figure 1) (Alschner and Skougarevskiy, Reference Alschner and Skougarevskiy2016b). In the lower right corner, however, there is a large grouping of similar worded agreements (indicated by the more reddish colouring). While the overall similarity of BITs in our dataset is 32%, the average similarity within that cluster is 40%, and 27% outside of it. The cluster of more similar BITs encompasses (parts of) the national BIT programs of several countries, including Great Britain, Israel, South Korea, India, Czechia, Hungary, Slovakia, Barbados, and others. The global/systemic analysis thus indicates that, in addition to country models, there is a latent similarity connection among a subset of BITs by a group of states.
Cluster of similar agreements in global treaty network
Note: The figure represents textual similarity in form of a heat map. High textual similarity is indicated as red, low textual similarity is indicated as yellow. The x and y-axis are symmetrical and regroup 3054 BITs in our dataset. The treaties are ordered through a hierarchical clustering algorithm to place similar texts together and dissimilar ones apart.

What have these more similar BITs in common apart from their treaty language? They seem to have been inspired by the BIT design of Great Britain. The earliest BIT within that group is the 1975 Egypt–UK BIT, which is the first investment treaty the UK signed. Moreover, within the cluster, the 104 British treaties stand out for their internal consistency displaying an average of 69% of similarity. Over time, other states seem to have been inspired by the UK's treaty design. As an illustration, we compare the expropriation clause of the South Korea–Sri Lanka BIT (1980) and of the Barbados–Venezuela BIT (1994) with the same clause in the 1975 Egypt–UK BIT. They are almost perfect copies (see Table 3).
Expropriation clauses across diverse treaty parties to use language originally found in UK BITs.

Why did the language of the UK BIT programme diffuse so successfully? Why did South Korea or Barbados not copy from German or U.S. agreements? We leave it to future researchers to find answers to these questions. We do want to highlight, however, that this analysis shows another strength of EDIT: the ability to trace diffusion even when the language of the original treaty differed. For example, in 1995 the two Baltic States Estonia and Lithuania concluded a BIT. Given their geographical proximity and close cultural and historical connection, one could have suspected that the BIT they signed was very unique in its design. Since the only publicly available version of that BIT is in Lithuanian, it would have been difficult to computationally relate its content to BITs in other languages to assess its uniqueness. EDIT's translation now enables us to situate such foreign language BITs into the wider treaty universe. It turns out that the 1995 Estonia–Lithuania BIT is not unique at all. It belongs to a family of similarly worded central eastern European agreements – to 75% it mirrors the language of the 1994 Albania–Czechia BIT and the 1994 Hungary–Mongolia BIT – which are, in turn situated in the cluster of more similar BITs inspired by the style and language of UK BITs.
5.1.2 Relative Normativity: Towards More Hortatory Language?
BITs have on average become longer over time. As can be seen in Figure 2, the main text of BITs excluding schedules or annexes has increased from an average of around 1500 words in the 1960s to around 4000 words in the 2010s. Canadian BITs signed over the last decade even increased to an average of around 8000 words. What does this increased length mean for the design and impact of investment treaty? Have treaties become more constraining over time including more obligations or have they become less constraining by devoting treaty space to exceptions or aspirational clauses? These are important questions that developed out of international law and international relations’ research programmes on rational design and legalization (Goldstein et al., Reference Goldstein, Kahler, Keohane and Slaughter2000; Koremenos et al., Reference Koremenos, Lipson and Snidal2001; Manger and Peinhardt, Reference Manger and Peinhardt2017).
Expansion of BITs and increase in treaty length

One way of answering this question of relative normativity (or degree of obligation or ‘bindingness’ of law) is by investigating whether longer BITs contain more mandatory commitments (‘shall’) or hortatory expectations (‘should’, ‘may’). If the share of mandatory clauses has remained constant or even increased, this would suggest that longer BITs impose more commitments on its signatories. If, however, the share of mandatory clauses has declined and, conversely, the share of more hortatory language has increased, this would suggest that BITs are increasingly used to set forth best-effort clauses and the average level of obligation decreases. EDIT enables researchers to track such relative normativity over time.Footnote 23
We have investigated mandatory language – must, shall, will – as well as hortatory language – may, should, endeavour, strive – in order to discern trends over time. In absolute terms, both mandatory and hortatory language in BITs has expanded over time. BITs signed in the 1960s contained on average 90 mandatory words and seven hortatory words per treaty; BITs signed in the 2010s contain 140 mandatory words and 19 hortatory words. However, this increase is in part due to the fact that treaties got longer.
When we look at mandatory and hortatory language in relative terms (occurrence per 100 words), then the impression changes, as can be seen in Figure 3. Mandatory words have decreased over time. That is, less treaty space is today devoted to setting hard, normative commitments. At the same time, the relative inclusion of hortatory language has barely increased. As a result, BITs today are less about setting forth strong commitments; yet, neither has the focus shifted to aspirational clauses.
Mandatory and hortatory language of IIAs over time

At the same time, in absolute terms, hortatory clauses have become more common. One frequent critique in this respect is that while traditional investment protection obligations are typically framed in mandatory language, more recent non-investment protection obligations, such as commitments to protect the environment, labour rights, or to promote corporate social responsibility, are consistently framed in hortatory term (Chi, Reference Chi2015; Morin et al., Reference Morin, Dür and Lechner2018). To test this empirically, we investigated all clauses on environmental protection, labour rights, and corporate social responsibility. We found that, indeed, all of them are couched, at least in part, in hortatory language. As calls for binding investor obligations to respect human rights intensify, future iterations of this analysis will show whether these hortatory clauses ‘harden’ into mandatory investor obligations in new IIAs (Choudhry, Reference Choudhury2020). Textual analysis based on signalling words can quickly identify the degree of normativity of different provisions and describe patterns over time.
5.2 National Treaty Patterns
We now turn from global to state-level patterns. Here, EDIT allows researchers, for the first time, (1) to systematically explore non-English treaty networks and (2) to investigate in more depth why some countries are rule-takers and others rule-makers.
5.2.1 Consistency and Innovation: Exploring Non-English Treaty Networks
Alschner and Skougarevskiy used a dataset of predominantly English agreements to track consistency and innovation in national BIT networks (Alschner and Skougarevskiy, Reference Alschner and Skougarevskiy2016b). EDIT now allows us to explore countries with predominantly non-English treaty networks. Only seven English-language BITs of France, for example, are included in the original mapping by Alschner and Skougarevskiy; EDIT now contains the text of 113 French BITs. Similarly, only 23 Russian BITs up to 2006 were previously available; EDIT now includes 82 BITs up to 2016. Figure 4 visualizes the textual similarity of the respective treaty networks. France's BITs concluded between 1963 and 2007 display an average similarity of 55%, placing it among the most consistent treaty networks in the world. Russia, in contrast, has a comparatively inconsistent treaty network. Its BITs concluded between 1989 and 2016 on average have only 45% of their text in common. However, Russia's heat map illustrates an evolution in the country's BIT policy. In 1998, Russia seems to have adopted a new BIT model, which it followed particularly consistently in the 2000s and 2010s resulting in a new generation of more similar agreements with an average similarity of 64% color-coded in red in the lower right corner.
Consistency (textual similarity) heat maps of the BIT networks of France (left) and Russia (right). Axes list BITs chronologically by date of signature (earliest treaties upper left, latest treaties lower right).

By allowing us to track the changes to national investment treaty programs, EDIT can monitor responses to current criticism against the IIA regime. While some countries have denounced BITs or stopped concluding new ones, other states have drastically departed from their earlier investment agreements to conclude new BITs that remedy shortcomings in earlier BITs.
One of these countries is Colombia. Up to 2006, Colombia concluded relatively short and simple agreements that focused exclusively on the protection of investors. Although the country had not faced any investment claims, it decided to revise its IIA practice in 2006 in response to global developments in ISDS. As a Colombian representative explained, its revised 2006 model reflected ‘careful consideration of [investor-state arbitration] awards from that time, interpretations from the NAFTA Free Trade Commission and consultations with UNCTAD officials’.Footnote 24 The IIAs Colombia began to sign from 2007 onwards then embodied a more detailed and elaborate balance of investment protection commitments, on the one hand, and public policy flexibilities, on the other hand.Footnote 25 Figure 5 neatly traces this shift. While its newest generation of BITs does not display heightened consistency, the white and yellow tiles in the upper right and lower left corner indicate that the new generation of agreements is extremely dissimilar from earlier Colombian treaties. As more states revise their approach to BITs, EDIT can track how radical the departure from the past has been.
Consistency (textual similarity) heat map of the BIT network of Colombia. Axes list BITs chronologically by date of signature (earliest treaties upper left, latest treaties lower right). Framed quadrangles help highlight the shift in Colombia's BIT practice between distinct periods when treaties were more internally consistent.

5.2.2 Negotiation Success: Rule-Takers versus Rule-Makers?
Previous research suggests that the consistency of BIT language over time is a proxy for the negotiation success of states (Allee and Peinhardt, Reference Allee and Peinhardt2014; Alschner and Skougarevskiy, Reference Alschner and Skougarevskiy2016b; Berge and Stiansen, Reference Berge and Stiansen2016). Powerful capital exporting countries tend to reproduce highly coherent treaty language across their negotiated agreements. They thus tend to be the system's rule-makers. Vice versa, less developed capital importing countries tend to have a patchwork of relatively inconsistent agreements. They appear to sign on to the language proposed by their more powerful negotiation partners and are thus the system's rule-takers. In short, a country's consistency of BIT language correlates strongly with economic power expressed as GDP per capita (Alschner and Skougarevskiy, Reference Alschner and Skougarevskiy2016b).
However, some similarly situated countries drastically differ in their treaty consistency. Hence, factors other than economic power need to be considered in the literature to understand negotiation success. Besides economic power, past experience in negotiations, geographic location, or shared historical trajectories might predict design similarity.
Let us consider the Czech Republic and Poland for example. Poland has four times the population of the Czech Republic and therefore a larger GDP (although its GDP per capita is lower). This would suggest that Poland has a bit more negotiation leverage. Moreover, Poland was one of the first countries east of the Iron Curtain to conclude investment treaties starting in the late 1980s and acquired considerable negotiation expertise even prior to the end of the Warsaw Pact. The Czech Republic only entered the BIT universe in the 1990s and started to seriously pursue BITs only after the break-up of Czechoslovakia in 1992.Footnote 26 Hence, based on economic power and experience, one could expect that Poland and not the Czech Republic developed a more consistent BIT regime. Yet the opposite is true. While the Czech Republic achieved a highly consistent treaty network (51% average similarity), Poland was less successful (43% average similarity). This difference in consistency persists (and increases) when we control for selected treaty partner characteristics. Below, we illustrate examples where the counterparts were economically less powerful. In other words, even in instances of potential rule-making power, Poland followed a much less consistent approach than the Czech Republic. Table 4 lists the relative similarity of the BITs the two states signed with Latvia, Romania, Vietnam, and Mongolia in the 1990s. While in the case of the Czech Republic these four BITs resulted in overall highly similar agreements, the BITs Poland signed differed significantly overall (Table 3).
Textual overlap of BITs signed by the Czech Republic and Poland in the 1990s with selected countries (percentage indicates textual overlap)

What can be observed is a puzzle. Why do some states negotiate highly consistent agreements while others fail to do so in similar situations? Whereas relative economic power is an important indicator, in this case it cannot explain the variation we observe. Future research can help us better understand when states opt for more or less consistency, through which agreements they might change their trajectory and how and when they might transform from rule-takers to rule-makers.
5.3 Specific Clauses: Changing Preambles
Computational analysis can also be conducted on the level of specific clauses, given that EDIT provides for structured treaty texts that can be broken down into clauses. Here we exemplify this procedure by assessing the preambles in all BITs.
Preambles do not create rights or obligations, but they set out the goals, aspirations, and reasons for entering into BITs, which is important to understand the regime's shared principles (Ruggie, Reference Ruggie1982). A textual analysis of preambles offers a window into how these contracting objectives have evolved. Preambles have become more extensive over time from an average length of 35 words in the 1960s to an average of 55 words in 2015. Moreover, new considerations have been added to preambles. Figure 6 tracks the share of annually concluded BITs over time that contain a set of key words that illustrate the gradual inclusion of new considerations into BITs.
Share of annually concluded BITs with key terms in Preamble (3-year moving average)

First, although BITs were substantively concerned with the protection of foreign investment rather than its promotion or liberalization, BITs included the stimulation of capital flows as the principal purpose of the treaties from the outset. Already the 1959 Germany–Pakistan BIT expressed the hope that the treaty would ‘promote investment’. The majority of subsequent BITs similarly indicated their expectation that the BIT would ‘stimulate’ investment flows. In recent BITs, however, the reference is found less frequently. This speaks to the work of Lauge Poulsen who claims governments in developing countries typically overestimated the economic benefits of investment treaties and practically ignored their risks (Poulsen, Reference Poulsen2015).
Second, in the late 1960s, BIT preambles began to frequently refer to the objective of ‘reciprocal protection.’ Such preambular wording seems to have originated on the behest of developing countries, starting consistently with Indonesia through the 1969 Indonesia–Norway and the 1970 Indonesia–Belgium BITs.Footnote 27 Although BITs were formally reciprocal from the outset, earlier preambles, following the 1962 OECD Draft Convention on Foreign Property, gave the impression of a one way-street: ‘promoting the flow of capital [from the North] for economic activity and development [in the South].’Footnote 28 While, de facto, investment flows in the 1960s and 1970s were likely more uni- rather than bidirectional, the emphasis on reciprocity by developing states underscored sovereign equality at a time of decolonization.
Third, references to ‘environmental protection’ began to appear in BITs in 1994. This development was spearheaded by the United States, which, starting with the 1994 USA–Georgia BIT, began to assert that investment protection and promotion could be achieved without relaxing health, safety, or environmental protection. Around 2000, this wording was picked up in a series of BITs by Finland (starting with its 2000 BIT with Bosnia and Herzegovina) and by Trinidad and Tobago (starting with its 1999 BIT with Cuba) before becoming more widely adopted in the 2000s and 2010s.
Fourth, explicit mentions of ‘sustainable development’ as one of the BIT goals appeared first in the 2006 BIT between Canada and Peru. A year earlier the International Institute for Sustainable Development (IISD) had published its model agreements on investment for sustainable development. In 2016, two thirds of BITs concluded that year contain references to sustainable development.
With the above snap-shot on the preambles, EDIT allows us to detect shared principles and norms that are introduced in the treaty network and also locate first movers (or rule-makers) in the context of competing norms that have shaped the investment regime.
6. Conclusions
The recent politicization of investment treaties has increased public and scholarly interest into the investment regime while the demand for more systematic data has gone up as well. This article has introduced EDIT and has provided some illustrations for its potential use and empirical application in fields of international law and international relations. Further, we think that the data can be of significant interest to researchers applying network analysis and econometric modelling for studying evolution and impact of the investment regime. Using advanced methods across disciplines, EDIT can contribute to answering questions with promising interdisciplinary applications. In conclusion, EDIT aims to stimulate the application of emerging new methods of text retrieval and analyses and provide an empirical backbone for engaging with both older and emerging debates in international relations, international law, and international economics about the role of investment treaties.
Acknowledgments
We acknowledge funding from the Swiss Network for International Studies (SNIS), the World Bank and the John R. Evans Leaders Grant, Canada Foundation for Innovation. We wish to thank Dmitriy Skougarevskiy, the SNIS project team and research assistants based at University of Ottawa and the World Trade Institute for their contribution to the database.







