Narratives of International Law
Analyzing international law as narrative is a commonsensical yet curiously overlooked contribution of the social sciences to the field. Despite the rich vein of hermeneutics in the study of law, and, further, despite the inherent nature of international law as lacking a worldwide legislative authority, the study of international law as discourse over rule remains underdeveloped, especially its methodology.Footnote 1 Inspired by the interpretive tradition of anthropology, this brief Article argues that the time is ripe for studying international law as narrative. In the contemporary moment, “talk” about international law is everywhere—as argumentation, legal reasoning, and political discourse—and yet, at the same time, the willful violation of international law has never been so apparent. Rather than diagnose the reasons for states’ non-compliance, this Article’s aim is more modest in pointing to a method that can unpack narratives of international law, narratives that states deploy to fill in the gap between aspirations to international law (whether utopian or dystopian) and actual state practice.Footnote 2
I. An Example
“China will stay committed to the path of multilateralism and always be a defender of the international order.”Footnote 3 General Secretary of the Chinese Communist Party and President of the People’s Republic of China (PRC) Xi Jinping made this statement at the United Nations (UN) at the fiftieth anniversary of Resolution 2758 which switched the sole representative of China in the UN from Taiwan to the PRC. In 2021, the year of the anniversary, Joe Biden had just assumed the presidency of the United States after a raucous four years of the Trump administration, and in so doing, appeared to assure the world that the U.S.-led “international order” was still strong. China’s position in that order was anomalous; it was a latter-day invitee, passive beneficiary, and sometimes revisionist challenger. The idea that China is the defender of the international order seemed, in 2021, laughable. Four years later, no one is laughing. Xi’s statement, as self-serving as it may be, has gained precedential weight: the narrative has superseded reality.
Long before the 2021 fiftieth anniversary, Chinese leaders have sought to create a narrative of China’s “peaceful rise,” one that champions international law, and that the United States, in contrast, is hypocritical and war-hungry.Footnote 4 This narrative is relational: it juxtaposes the re-emergence of China as leader of the developing world and one which safeguards sovereignty against the United States, a hegemony that weaponizes international law tools against others. The United States has, in turn, generated its own narrative about China’s integration into international law: China circumvents trade laws, violates human rights norms, and causes environmental harms.Footnote 5 Historically, the U.S. narrative about its own position in international law has been (implicitly, at least) one of general alignment even if the United States has been a chronic violator.Footnote 6 This narrative has been discarded under Trump 2.0 in favor of a more muscular embrace of utilitarianism: international law serves U.S. interests or it is out.Footnote 7
One current focal point is the question of extraterritorial law. Whereas the PRC government has lambasted what it conceives of as U.S. “unlawful” use of its law extraterritorially, China is in the process of building out shewai fazhi (foreign-related “rule of law”) in part to mirror U.S. approaches to extraterritoriality.Footnote 8 While the messaging has changed to degrees under different leaders on different sides of the Pacific, there has been continuity in these duelling Janus-faced narratives.
Narratives are reproduced through a salvo of different genres: official statements from foreign services, propaganda of political parties, reports issued by think tanks, and the writings of academics. The infusion of social media, including AI-generated social media, into governmental communications, and the complex relationship between tech companies and public bodies in both states has amplified the capacity of such narratives to animate thinking.Footnote 9 As Robert Cover opined, “We inhabit a nomos—a normative universe.”Footnote 10 This assertion applies as much as to international law as it does to domestic law and it may be in “comparative international law”—with its inherently relational nature—that nomos finds its fullest expression.
The problem with such narratives is that, as social constructions, they can powerfully orient (and disorient) populations in their attitudes toward their state’s engagement with international law, other states’ engagement, and the nature and validity of international law itself. In the context of “post-truth” and the return to primordial attachments, narratives of international law can mediate individuals’ perception of and participation within international law. In its more extreme versions, this may lead to psychological hijacking, emotional contagion, and gaslighting.Footnote 11
II. Anthropological Tools to Decode Narratives
International lawyers need to be able to analyze these narratives to understand where they come from, how they circulate, and what their effects are on state practice and popular support. One strain of anthropology, derived from the Weberian sociology of verstehen (“interpretive understanding”) emphasizes subjective meanings that people attribute to social actions.Footnote 12 Clifford Geertz is perhaps the most well-known popularizer of Weber in calling for “a hermeneutic tacking between the two fields [of law and anthropology].”Footnote 13 While Geertz’s hermeneutics have, in turn, been critiqued,Footnote 14 comparative international law may occasion a return (sufficiently recalibrated) to this line of interpretive anthropology. Specifically, one that balances meaning with power, discourse with practice, and the specificity of culture with comparative doctrinal work can resuscitate some of hermeneutics’ strengths without reproducing shortcomings.
As applied to the case of the U.S.-China conflict over interpreting international law—a conflict that plays across a wide array of legal domains—a focus on narratives can explain some of the intractability of these disputes and their moorings in history, national consciousness, and identitarian politics. Under the comparative international law framing, attention to narratives shows both the stories we tell about others’ approaches to international law, the stories we tell about our own approaches, and the interaction between the two. Interpretive methods point to both the producers of narratives (e.g., state functionaries, private industries, tech platforms, etc.) and their audiences (i.e., domestic versus international). These methods allow analysts to trace out narratives from genesis to reception, identifying their various channels, and, arguably most importantly, the processes of interpretation of those messages, the discursive referents and cultural milieu through which narratives gain their persuasive force.
To return to the example of Chinese critiques of U.S.-style extraterritoriality, different actors in the Chinese information ecosystem, including foreign affairs officials and academics, have made various arguments about the “illegality” of U.S. long-arm statutes under customary international law. These arguments are sometimes connected to Qing histories of Western imperialism, thereby re-igniting national memories of the “century of humiliation.” Predominantly, they operate through Chinese state media, including social media that is filtered through censors. Such arguments are also lodged in English language via official governmental channels and are demonstrative of the Chinese state’s “standing up” to American aggressors.Footnote 15
At home and abroad, both in English and in Chinese, these arguments are equally about the excesses of American empire. The pull-back by the Trump administration of the United States’ obligations under international law coupled with its preposterous claims over Greenland, Gaza, Ukraine, and Panama all suggest a rhetorical return to empire which plays to the Chinese narrative. Rhetoric is not just konghua (empty speech), but can incite legislation. Nationalism in both countries generates domestic laws that are protectivist and, at their worst, xenophobic. MAGA rhetoric about Chinese invasion in the United States and Chinese nationalist discourse about American empire have produced laws which prohibit (in some instances, unconstitutionally so), the movement of peoples, capital, and even ideas from one side of the Pacific to the other.Footnote 16 These laws, in turn, are made sense of within broader narratives of national shame and pride, decline and revival, and self-sufficiency and exceptionalism. Only by coming to terms with the narratives that fuel lawfare can analysts make sense of their internal logics and resonance with their supporters.
Among the branches of international law scholarship that borrow tools from other disciplines, computational legal analysis is perhaps the most traditional, but also the most innovative.Footnote 1 It is traditional in the sense that computational legal scholars tend to focus on studying the law itself rather than its effect, social perception or underlying narrative. Indeed, the field often has a decidedly doctrinal bent. It seeks to track precedent, parse treaties, or unpack judicial reasoning.Footnote 2 Computational legal scholars tend to pursue research questions that are descriptive, rather than inferential.
Computational legal analysis is therefore less innovative in the questions it asks than in the methods it uses to answer them. Words, paragraphs, citations and their legal meaning are still the object of analysis, but computational legal scholars look at them through very different methodological glasses than traditional lawyers. Rather than using the millennia-old technology of exploring legal text through reading and interpretation, computational legal scholars turn to state-of-the-art tools developed by computer and data scientists to process legal text as data.Footnote 3
Processing text as data allows researchers to spot high-level patterns or minute details that traditional legal analysis would easily miss. Because algorithms do not grow tired or lose concentration, computational legal research methods are powerful and scalable. Researchers can reveal a bird’s-eye view that captures decades of lawmaking across thousands of treaties. On the other extreme, these tools can also be employed to sift through mountains of data to find the needle in the haystack that long eluded researchers.
In short, computational legal analysis offers a different vantage point for studying age-old legal questions. As Sadl and Olsen put it, “methods, such as corpus linguistics and citation network analysis, ensure the reproducibility, generalizability, and empirical validity of doctrinal studies. They add to the transparency of legal methodology … [but] without compromising its normative core.”Footnote 4 Computational legal analysis thus augments the ability of legal scholars to understand law at scale and in detail. It is not about producing obscure numbers or unintelligible statistics but about acting as powerful complement to traditional legal methods and social science tools to study international law more comprehensively.
I. Scaling Legal Research Without Compromising on Nuance
Examples from my own research can help illustrate the impressive capabilities of computational legal analysis. When I started my PhD, I knew that I wanted to investigate the evolution of over 3,000 bilateral investment treaties, but I also knew that I did not want to read 3,000 treaty texts.Footnote 5 Anyone who has studied this field knows that these treaties tend to be broadly similar but also differ in subtle but highly systematic ways. Isolating signal from noise can be tedious, though.
The traditional legal method, based on reading and textual interpretation, excels at extracting meaning from a small set of texts. It is meant for capturing nuance and detail and thrives in a small data environment. While it thus works on comparing a handful or, at most, a dozen treaties, it breaks down once the task involves a systematic comparison of hundreds or thousands of treaties.
Since traditional legal methods would not get me very far, I looked for analytical alternatives. Political scientists often work with large datasets and leverage one of two simplifications.
First, political scientists have a long tradition of creating representative samples that allow them to investigate patterns in larger populations. They may, say, draw a representative sample of 100 treaties and then extrapolate the insights for the entire treaty population using statistical methods. That technique, however, appears unsatisfying to many lawyers interested in understanding treaty variation. It is typically impossible to know beforehand which treaties can be considered representative. Moreover, it is often the outlier treaty that may be most interesting on doctrinal grounds. So, I discarded that path of using samples leveraged through statistics.
A second technique used by political scientists looks at the entire population but reduces the analysis to a manageable set of core variables. One such example for the analysis of international legal text is the legalization framework.Footnote 6 Developed in the early 2000s, this very popular analysis framework looks at three dimensions of legal documents: precision, obligation, and delegation. Others have classified investment treaties along these dimensions.Footnote 7 While the exercise still involves significant manual work, it is doable. At the same time, the technique has less appeal for lawyers who cherish detail. Legal scholars are interested in the precise terms of delegation to tribunals to assess what and how adjudicatory and interpretive power is conferred. Or they are interested in identifying what protections bind states and how they vary in scope from one treaty to another. Legalization is too coarse a measure to provide these insights, but a more detailed coding of treaty variation would be prohibitively time-consuming for a PhD-level scholar.
I came to the view that the best way to tackle my research ambition—looking at all the treaties but conducting a granular content analysis—required a different toolbox. I was attracted to computer and data science because these disciplines have spent decades thinking about how to investigate texts at scale and detail. They conceive of text as a form of data and use advances in natural language processing and machine learning to mine texts as datasets.
There are many ways to process text as data and there is no “right” way to do it. As Grimmer and Stewart put it, “[a]ll quantitative models of language are wrong—but some are useful.”Footnote 8 How text is processed as data thus ultimately depends on which method best matches research objectives. Relatively crude corpus linguistic methods that focus on key words or word frequencies are often highly effective in spotting patterns and trends in the data. At the same time, more sophisticated models are needed when tasks involve an understanding of semantics to distinguish between differences in text and differences in meaning.
The method that I have ultimately been most attracted to involves similarity algorithms.Footnote 9 These algorithms can reveal surprising insights both on a high and on a granular level. They can also be integrated into a wealth of research designs, including to track legal processes, such as the progressive consensus-building in treaty negotiation, to evaluating influence, for example, by comparing party submissions to final judgments, just to name a few. While I employed a range of computational techniques in my scholarship, similarity algorithms proved key to understanding the patterns underlying the universe of 3,000 investment treaties.
II. Rule-Takers and Rule-Makers: A Bird’s-Eye View of Thousands of Investment Treaties
A striking feature of the universe of investment treaties is how visible patterns of power asymmetry become once you look at treaties through the lens of data science. Together with my colleague Dmitriy Skougarevskiy, I compared the textual similarity of thousands of investment treaties.Footnote 10 It had been well-known anecdotally that both developed and developing countries used model agreements to prepare negotiations. However, our analysis showed that developed countries have been significantly more successful in consistently shaping treaties based on their models and preferences.
Our approach was relatively simple. We used a similarity metric, known as Jaccard distance, to essentially calculate how many words two treaties have in common. That is done for every single possible treaty pair, resulting in a large matrix of pair-wise treaty similarity scores. To then make sense of the results, we displayed the large matrix of pair-wise similarities as a heatmap. In that heatmap treaties that have very many words in common are color-coded as deeply red; if they have very few words in common, they are color coded as yellow.Footnote 11
We first ordered the matrix alphabetically by the name of the economically more powerful signatory to a bilateral investment treaty. A treaty between the United Kingdom and Angola would thus be placed with other United Kingdom treaties. What emerged from this bird’s-eye view was a map of the internal consistency of developed country treaty networks. Large red quadrangles indicated that agreements of rich industrialized countries, like those of the UK or the Netherlands, closely resembled each other. Developed states were revealed as the system’s rule-makers because they achieved the same treaty design in negotiation after negotiation.
We then repeated the analysis but ordered agreements by the less wealthy party, e.g., placing all treaties signed by Angola together. The consistency patterns disappeared. Developing countries were revealed as the system’s rule-takers. They signed onto very different agreements, essentially accepting the treaty design proposed by their wealthy country counterpart, rather than succeeding in making their own model text shape successive negotiations.
We had thus used data science to verifiably and reproducibly show that deep power asymmetries characterized the investment treaty universe. What struck me most was how a simple, but scalable method could produce insights in seconds that would have been impossible to generate using more traditional methods.
III. Southern Innovation: Finding the Needle in the Haystack
It would, however, be misleading to reduce computational legal approaches to the study of large-scale patterns or trends. True, these methods are often most impressive when they generate visually appealing heatmaps or help to draw colorful networks that depict the linkages produced by thousands of judicial citations. But these methods can also be used to yield highly granular insights. One such endeavor flowed directly from the above-described finding that the investment treaty universe was characterized by rule-takers versus rule-makers.
What if developing countries use their more limited bargaining power to affect changes on specific treaties? Our hypothesis was that while these states may accept to contract on the richer party’s template, they may still succeed when it comes to extracting concessions on individual clauses. Together with Rodrigo Polanco and Manfred Elsig, I created the Electronic Database of Investment Treaties, which provides open access to machine-readable and pre-structured investment treaty texts.Footnote 12 Using that dataset enabled us to look deeper and compare treaty texts on the clause level. Across several different projects we were thus able to reveal instances of “Southern Innovation”: clauses that developing states insisted upon across successive treaty negotiations.
Two examples stood out. First, starting with the 1985 China-Singapore bilateral investment treaty, a new exception clause began to appear in the treaty networks of mostly developed states.Footnote 13 Under the heading of “Prohibitions and Restrictions” these provisions reserved the right for a contracting state to “apply prohibitions or restrictions of any kind or take any other action which is directed to the protection of its essential security interests, or to the protection of public health or the prevention of diseases and pests in animals or plants.” The language diffused to other developing states through mostly South-South investment treaties, but occasionally also made its way into North-South agreements, like those signed by India in 1995 with Germany, the Netherlands, and Italy.
A second example involved Bangladesh. In 2009, the country pioneered the use of investor obligations in its treaty with Denmark stipulating that the host country must be “accorded adequate and effective compensation as per its laws and regulations, and if necessary as per International law, by the investor of the other Contracting Party” for the “loss, destruction or damages with regard to its public health or life or the environment, including natural resources by the investor.”Footnote 14 The same clause was subsequently inserted into three other Bangladeshi treaties.
By using data science and similarity algorithms, we could thus spot language that was unique to developing countries. We could also trace how these developing country norm entrepreneurs used their leverage to insert their preferred language into successive negotiations. In a universe otherwise dominated by developed country rule setters, finding such Southern innovation was like identifying a needle in the haystack. And we could not have done it systematically without having recourse to computational legal methods.
IV. Do I Need to Know How to Code?
Notwithstanding all its promise, many international lawyers will feel daunted by the task to implement these data science methods themselves. There are three reasons why that may be easier than many think.
First, data and not the skills or technology is typically the greatest bottleneck in computational research projects. Most of my time working in the space has been devoted to creating new datasets, because existing data was not available, restricted or not amenable to computational work. The good news is that datasets are becoming more abundant. Open government data has helped improve access to international law texts; legal researchers have created bespoke datasets on major areas of international law and advances in technology have made it much easier to convert legacy data, such as scanned pdfs, into structured datasets. In short, data is becoming less of an obstacle.
Second, increasingly lawyers can deploy data science methods in a no or low code environment. Generative artificial intelligence (AI) can do many of the things that researchers like myself long used computer programming for. And not only is generative AI better at traditional machine learning, it is also accessible through a natural language environment. For some applications, like large-scale similarity analysis, traditional coding is still the better route, but even there, generative AI can help significantly by writing sample code based on natural language prompts which can then be run in programming environments. Future AI agents may simplify that task further.
Third, growing resources exist to support lawyers to leverage computer and data science. Websites and courses are available that introduce the field.Footnote 15 Scholarship in the area is growing, too, providing aspiring computational scholars of international law with templates and examples on how to apply new tools to old legal questions.Footnote 16 As many fields of international law remain woefully underexplored, low hanging fruit may be reaped by transposing methods from one field, such as the study of investment treaties, to another.
So, should international lawyers interested in deploying these methods know how to code? The answer is a “yes, but.” Legal researchers will be far more effective and impactful in their work, including in collaboration with computer and data scientists, if they have a rudimentary understanding of coding, computer science and data science. Basic familiarity with the language of the other discipline helps avoid misunderstandings, shapes expectations and guides the research conversation. But international lawyers do not need to become expert coders or pursue a PhD in computer science to engage in impactful work.
One of my most memorable classroom encounters occurred when I first began teaching international law to political science undergraduates at Duke University. I was teaching from an international law case book, largely because there were few options at that time for teaching students of politics and policy about international law. As I was lecturing on the laws of war, a young man in a ROTC uniform in the front row raised his hand and asked, “Professor Simmons, what difference does any of this make?” (Long pause.) Eventually I managed something about how it must be important since states had invested too much time and effort over the decades to develop these rules. It was a lame rookie answer to one of the most important questions we face in legal scholarship and practice.
That question, asked in my very first year of teaching, has inspired my research ever since. In my work on human rights and international criminal law, it is central. It is clear why this skeptical question is so compelling. Even under the best of circumstances, the world is complex, and developing, disseminating, and internalizing legal norms takes a lot of time. We cannot usually “see” legal effects very clearly. The formal international legal response to violations is typically weak—no police, few court cases, rare accountability in formal settings. On the other hand, law violation is not uncommon. And when it occurs, it is spectacular, often tragic, and reported prominently in the news. Images of such violations are often visceral. No wonder people want 100 percent compliance, but of course no area of law is expected to achieve such a record.
To answer whether any of this international law stuff matters, we have to ask: compared to what? Social science helps: it requires that we articulate (and to the extent possible, test) outcomes against a very clearly articulated counterfactual.Footnote 1 For me, the most compelling comparison is the difference between what would have happened without the law (or a legal intervention, or a legal commitment, or a legal decision) compared to what happened with the legal intervention. This reflects my view that the goal of international law is to make the world better. To assume 100 percent compliance is, quite simply, to make the perfect the enemy of the good. If international law has been a “failure,” let us at least be clear about what we are expecting it to do.
Opacity regarding counterfactuals is common in areas I have researched: human rights and international criminal law. Skeptics of the effects of international human rights law are often “disappointed” that it has achieved so little.Footnote 2 Disappointment stems from an expectation, which is just another way to express a hoped-for counterfactual. Disappointment is rife in the human rights field because it is natural not to want anyone to suffer a rights abuse. Any rights violation is too much violation. And you need look no further than the headlines of your newsfeed to hear of violation after violation. Studies show that human rights coverage tends to be negative.Footnote 3 Compliance with international human rights or criminal law just does not hit the news in the same way an egregious violation does—or at all. So, to read the news is to risk a common form of bias, availability bias, whereby the most available information, like the news media, unduly influence perceptions. If the most available information emphasizes violations of human rights, then international law cannot be working, right? Unfortunately, availability bias means our informal estimates of what law actually delivers is likely to be systematically biased downward.
A similar point can be made about the effects of the International Criminal Court (ICC). The news has reported primarily on its weakness and shortcomings—but in comparison to what? Impunity?Footnote 4 In early years, comparisons were made to domestic justice systems—with their full panoply of police, investigative authority, and resources. Such comparisons inevitably made the Court look weak. Others fastened on how few prosecutions and convictions the Court had achieved, how it might interfere with peace negotiations, or that it was impotent in the face of politics (e.g., the Kenya cases). And close media studies note that the ICC has faced the quandary posed by doing its job: ICC actions were in fact associated with more opposition because media coverage amplified voices of the court’s critics even further.Footnote 5 Maybe it is not in a “crisis,” but the Court has been said to be facing a “fiasco”—an inability to achieve its own mission.Footnote 6
What would it mean to conclude that the ICC was in fact achieving its mission? It would require a clear statement of its mission and an explicit counterfactual. Recall that the primary purpose of the ICC is not to indict. Nor should it be too harshly criticized if it sometimes is overwhelmed by a lack of political cooperation. Others may disagree but in my view, one important purpose of the ICC is achieved in great measure if it is credible enough to deter (to quote the preamble) “atrocities that deeply shock the conscience of humanity.”Footnote 7 This is the basis of my preferred counterfactual: Would there have been more civilian killing in the absence of an ICC intervention than there in fact was with intervention? People have the right to choose any counterfactual they want to, but they do need to be explicit about comparison, especially if they want to make credible claims about legal effects.Footnote 8
Using counterfactuals effectively also requires a defensible model of what drives outcomes, other than the legal intervention itself. I do not claim this is easy; the world is complex and contestable. I plead no contest to the claim that causality is extremely difficult to parse, even with great observational data, which is hard to come by. But assuming these caveats are true whether one uses social science or a Ouija board, we will still be better off if we are clear and intentional about what we are comparing—about our counterfactuals.
Social science methods have been helpful in testing various propositions in relation to explicit counterfactuals. Difference-in-difference analysis, randomized selection of subjects, propensity score matching and instrumental variable analysis are all possibilities.Footnote 9 Even the careful selection of qualitative casesFootnote 10 in which the only plausible condition that differs between them is a legal condition can be useful for counterfactual analysis, if one is willing to make the (often heroic) assumption that one case plausibly serves as a counterfactual for the other. Methods need not be unpacked here. I am more interested in the first-order issue of how valuable counterfactual analysis can be at all.
Counterfactuals are crucial for understanding international law’s influence on outcomes we should all care about. The ICC provides a useful example. First, it is important to be clear about what we are expecting the International Criminal Court to do. As an institution to address atrocity “that deeply shock the conscience of humanity,”Footnote 11 one clear purpose of the ICC is to reduce intentional killing of civilians, which qualify as a war crime (during wartime), an egregious human rights violation, and likely a crime against humanity—all within the jurisdiction of the Court. Not prosecutions, not peace, not an end to the politics of ethnic cleansing. We want to compare the atrocity patterns without some kind of ICC intervention versus the atrocity patterns with an intervention, controlling for or matching on other factors that could be driving such atrocities. An important justice intervention may never even reach a conviction or even a prosecution. Even a signal from the court that they are watching, that they may be willing to use their authority, may have the normative focal power potentially to raise the risks of prosecution. The point is to detect important, observable behavioral changes with and without the ICC’s justice intervention. When one has a time series, this often means that the best counterfactual compares behavior before and after the Court’s intervention, ceteris paribus.
This figure helps to illustrate. Despite its busy look, it is extremely simple. It shows the monthly count of the Lord’s Resistance Army’s attacks on civilians in Uganda for about a decade. Given the availability of data over time, the comparison is explicitly between intentional killing before the ICC’s investigation into the Ugandan “situation” and after.
The relevant counterfactual is to imagine what would happen to the relationship between military conflict, military attacks, and attacks on civilians in the absence of the ICC’s investigation. Before this ICC investigation begins, the number of attacks on civilians mirrors very closely the pattern of military attacks generally. Intentionally killing civilians appears, essentially, to be a “normal” part of the LRA’s military operations. There is a notable shift once the ICC begins to investigate. Military attacks do not end (the ICC is not “peacekeeping”) but the relationship begins to pull apart as military operations involve fewer attacks on civilians. Social scientists can alter the controls in their models according to their theories of other plausible influences on civilian killing. For our purposes, however, rather than defend a specific model, the point is to emphasize clarity. Now it is possible to have an informed debate about data (how reliable?), interpretation (what does this mean?), modeling (are the controls sensible?), scope conditions (can we expect relationships from 1997 to 2007 to hold up under current conditions?), mechanisms (what links investigation to behavioral shifts?), and about the meaning and message of international criminal law generally. But one thing we will not have to do is to talk completely past one another.
Thinking about counterfactuals—a core strategy in the social sciences—is one of the most useful insights of the scientific method. Clear, explicit counterfactuals are a start in answering that basic question that even undergraduates think to pose: “what difference does international law make?” Comparing two possible realities—where law is leveraged and where it is absent—is a great start. Utopia is a worthy goal, but as Elliott reminds his friend in E.T., “This is reality, Greg!” And for those who feel unease about social science “thought experiments,”Footnote 12 let us keep in mind that everyone engages in such experiments all the time to draw conclusions. Being clear even if ultimately we are disappointed is better than being disappointed without being clear.
To conclude, I am not advocating that we all do statistical models or even use quantitative data. That is not the point. As mentioned above, carefully chosen case studies are useful material for counterfactual analysis as well. But if you do want to take a quantitative turn, I recommend that you develop partnerships. I think about it like this: a while back, my husband wanted to get a boat. And I said, “Sweetheart, you don’t need a boat. You need a friend with a boat.” That could be an ideal way to cultivate complementary skills. Figure out what you bring to the table and find the friend with the boat. We all want to try to ask the right questions and try to make the answers as accurate and persuasive as possible. Using explicit, rigorous, and reasonable counterfactuals will spark meaningful conversations—and disagreements!—across disciplines and perspectives.
Legal scholarship has long grappled with the challenge of making complex analysis accessible and compelling. While quantitative methods dominate much contemporary social science research, there remains tremendous value in qualitative approaches that leverage lawyers’ natural analytical skills and institutional access. Rather than requiring mastery of advanced technological tools, these methods build upon traditional legal reasoning while incorporating social science rigor to produce scholarship that resonates with both specialists and broader academic audiences.Footnote 1
I. The Power of Most Difficult Case Selection
One of the most effective qualitative techniques involves identifying and analyzing the most difficult or revealing case rather than attempting comprehensive coverage.Footnote 2 This approach mirrors constitutional law pedagogy, where foundational claims emerge from challenging the impact of landmark cases like Roe and Brown. The method’s power lies in its ability to capture attention and provoke engagement—when scholars challenge conventional wisdom about pivotal cases, they compel others to respond and debate.
This strategy proves particularly valuable when studying institutional behavior and judicial independence. Consider the German Constitutional Court, widely regarded as the guardian of EU treaties and a model for constitutional courts worldwide. When such an institution—this bastion of independence—reverses itself within two years following pressure from the legal academy and public criticism, it provides crucial insights into how elite and public pressure can influence even the most conservative judicial environments.Footnote 3 Such dramatic institutional shifts offer more analytical leverage than studying hundreds of routine decisions.
II. Visual Communication and Accessibility
Effective legal scholarship increasingly requires making complex institutional relationships and power dynamics accessible to diverse audiences. Traditional visualization techniques, while seemingly simple, can powerfully illustrate these dynamics. A single political cartoon showing the German Constitutional Court president attempting to halt the European Central Bank’s monetary policy captures the institutional imbalance between European financial and legal powers more effectively than lengthy textual descriptions.
This approach reflects a broader principle: sophisticated analysis need not require sophisticated technology. While interactive websites with advanced mapping and data visualization capabilities certainly have their place,Footnote 4 much can be accomplished through careful selection and presentation of simple visual elements. The key lies not in technological complexity but in choosing illustrations that crystallize the essential tensions and relationships under examination.
III. Systematic Media Analysis
Beyond case selection and visualization, qualitative methods can incorporate systematic but straightforward analytical techniques. Media coverage analysis provides one such approach, requiring no statistical sophistication yet yielding valuable insights into institutional legitimacy and public reception of judicial decisions.
Courts typically receive different media treatment than other government branches. While executive and legislative actions routinely generate balanced coverage with competing perspectives, judicial decisions often receive more deferential treatment—“the Supreme Court ruled X, period.”Footnote 5 This coverage pattern contributes to courts’ legitimacy cycles. However, when courts face unusually critical coverage across the political spectrum, it signals that judges have stepped outside conventional institutional roles, potentially undermining their traditional authority.
Simple counting exercises—measuring volume and tone of coverage across left-wing and right-wing publications—can reveal these patterns without requiring complex statistical analysis. The unusual nature of balanced critical coverage itself becomes the finding, demonstrating how courts’ institutional position depends partly on maintaining public and elite confidence in their conventional behavior.
IV. Leveraging Professional Networks Through Interview Methods
Legal scholars possess a distinctive advantage in their extensive professional networks and institutional access. Lawyers routinely contact colleagues in various agencies to test interpretations, understand decision-making processes, and identify relevant documentation. This informal consultation process can be systematized into rigorous interview-based research methods.Footnote 6
One challenge lies in knowing when sufficient interviews have been conducted. Social scientists employ the concept of “saturation” to determine research completion. Unlike quantitative studies, where power calculations determine necessary sample sizes in advance, qualitative interview research achieves completeness through two complementary measures.
First, researchers must ensure comprehensive perspective coverage. If studying regulatory processes, interviews should include both industry representatives and civil society advocates. If examining litigation, both plaintiff and defense attorneys should participate. Balance also involves noting access patterns—whether some participants spoke on the record while others required confidentiality, or whether certain doors remained permanently closed despite persistent efforts.
Second, researchers reach saturation when interviews begin yielding repetitive information. When multiple participants independently identify the same expert (“talk to Jane about this—she’s the authority”), and Jane was interviewed rounds earlier, this repetition signals comprehensive coverage. Similarly, when participants consistently identify the same institutional cleavages and power dynamics, researchers can be confident in their findings’ completeness.
V. Traditional Textual Analysis Enhanced
Even conventional legal textual analysis benefits from qualitative social science methods. Close reading of judicial language, enhanced by systematic attention to linguistic patterns, can reveal institutional attitudes and strategic behavior that might otherwise remain hidden.
Constitutional courts’ language choices often signal their confidence levels and institutional positioning. When courts resort to double negatives (“it does not appear completely implausible”), qualified assessments (“under exceptional circumstances”), or tentative conclusions (“it appears doubtful … ultimately, however, it cannot be clearly ruled out”), they reveal institutional uncertainty rather than confident constitutional interpretation.
By highlighting such language patterns visually—through selective emphasis and systematic presentation—scholars can demonstrate judicial ambivalence or strategic positioning more effectively than through conventional legal argumentation alone. This approach combines traditional legal interpretive skills with social science attention to systematic patterns and institutional behavior.
VI. The Arbitrage Opportunity
These qualitative methods represent a significant arbitrage opportunity for legal scholars. Political scientists have extensively developed and debated qualitative methodologies, producing sophisticated theoretical frameworks and practical guidance. However, this scholarship remains largely inaccessible to legal academics, creating opportunities for translation and application.
Legal scholars can leverage this disconnect by adapting established social science techniques for legal questions, often producing work that appears highly original to legal audiences while building directly on well-developed methodological foundations. This approach has proven successful across multiple venues—the same methodological insights can be presented in international law journals, general law reviews, and specialized legal handbooks, each time reaching new audiences unfamiliar with the underlying social science literature.Footnote 7
VII. Conclusion
Qualitative social science methods offer legal scholars powerful tools for producing compelling, rigorous scholarship without requiring extensive technological expertise or quantitative training. By combining strategic case selection, accessible visualization, systematic but simple analytical techniques, professional network leveraging, and enhanced textual analysis, legal researchers can generate insights that resonate with both specialist and general academic audiences.
The key insight is that sophisticated analysis need not require sophisticated tools. Instead, success depends on applying systematic thinking to traditional legal skills, enhanced by social science methodological awareness. This approach not only produces more compelling scholarship but also creates opportunities for cross-disciplinary dialogue and influence, ultimately strengthening both legal and social science understanding of institutional behavior and public law dynamics.
