This symposium is dedicated to the memory of Ran Hirschl.
The demand for rule of law measurement is widespread and consequential. International organizations rank countries by their adherence to the rule of law,Footnote 1 investors and firms price political risk around it,Footnote 2 development agencies condition funding on it,Footnote 3 and the European Union has tied billions of euros in disbursements to it.Footnote 4 At the same time, data-manipulation scandals and political contestation have intensified longstanding concerns about how these measures are constructed and used. In 2021, following allegations of corruptionFootnote 5 and reviews documenting data irregularities,Footnote 6 the World Bank discontinued its Doing Business report.Footnote 7 The essays in this symposium examine other threats to rule of law measurement: conceptual slippage, directional bias, legal multiplicity, strategic responses by measured governments, and the omission of legally consequential practices that are difficult to observe. With rule of law decline widely documented across Western democracies,Footnote 8 the need to examine, improve, and develop alternatives to standard measures has become urgent.
This symposium brings together scholars who have been thinking about measurement, existing rule of law indices, and their policy implications. The essays combine conceptual analysis, empirical critique, legal ethnography, and systematic interviewing. They connect measurement choices to development policy, investment decisions, treaty compliance, executive power, judicial independence, and the European Union’s response to democratic backsliding among its member states.
Three questions organize the symposium. The first is conceptual: what is the rule of law that we are trying to measure? Tom Ginsburg maps the familiar spectrum from procedural accounts associated with Lon Fuller to thicker conceptions that include judicial independence and fundamental rights.Footnote 9 Adam Chilton and Mila Versteeg complicate the thicker end of that spectrum: their account of Ecuador’s “Rule of Rights” shows how proliferating and conflicting rights can expand executive discretion and undermine nonarbitrary government.Footnote 10 Janka Deli, Katerina Linos, and Mark Pollack, in turn, show that judges often discuss the rule of law alongside liberal democracy and fundamental rights, while warning that combining related concepts can inflate apparent agreement.Footnote 11 Other contributors deliberately narrow the object of measurement. Diego A. Zambrano focuses on decisional judicial independence,Footnote 12 Elena Chachko on legal constraint within the executive.Footnote 13 Kevin E. Davis, by contrast, widens the frame, showing how domestic, foreign, supranational, and private law jointly shape legal outcomes.Footnote 14
Another cross-cutting choice is the kind of evidence measures use. We use “inputs” as shorthand for formal rules, institutional designs, and official practices thought to support the rule of law, and “outcomes” for observable institutional behavior and consequences—whether courts constrain officials, judgments are obeyed, rights are protected, or contracts are enforced. Inputs are easier to observe and compare, but input-based measures can generate false reassurance both when institutions are being built and when they are being dismantled. New rules and institutions may exist on paper without changing practice. Conversely, under what Kim Lane Scheppele calls “autocratic legalism”—the use of formally legal measures to hollow out democratic checks—institutions may retain familiar forms while their operation is transformed.Footnote 15 Both are variants of the gap between law on the books and law in action. Outcome measures come closer to observed behavior, but Zambrano and Davis show their potential pitfalls: case outcomes reflect selection and litigation incentives,Footnote 16 while regulatory outcomes may be shaped by foreign, international, subnational, and private law as well as domestic institutions.Footnote 17
The second question is evaluative: are existing measures valid and reliable? Ginsburg argues that leading indices conflate concepts and may measure impartial administration rather than the rule of law.Footnote 18 Deli identifies directional bias: indices lag and understate deterioration but often prematurely record and overstate recovery.Footnote 19 Davis shows that legal multiplicity weakens the inference from national legal institutions to outcomes.Footnote 20 Scheppele demonstrates that governments can evade or game measures that miss how backsliding works on the ground,Footnote 21 and Chachko identifies an executive-branch blind spot in measures oriented toward externally visible legality.Footnote 22 These critiques also expose a tension between scientific and policy uses. Measures designed for comparison and causal inference may overlook country-specific mechanisms, while measures used to allocate funds, guide investment, or sanction backsliding governments may invite strategic compliance by the actors they assess.
The third question looks forward: how should rule of law measurement change in an era of new data and artificial intelligence? Zambrano proposes disaggregating to individual cases and government win rates.Footnote 23 Chachko calls for evidence about internal executive practices.Footnote 24 Deli, Linos, and Pollack develop systematic interviewing of judges and other legal elites.Footnote 25 Ginsburg sees promise in using large language models to analyze legislation and judicial decisions, but warns that models trained on the existing, highly correlated indices may simply reify their weaknesses, conveying more confidence than the underlying data warrant.Footnote 26
The Essays
The first essays diagnose conceptual, structural, and strategic limits of prevailing approaches to rule of law measurement—and point toward the need for new ones. Tom Ginsburg, Professor of International Law and Political Science at the University of Chicago, surveys the state of rule of law measurement fifteen years after his influential critique.Footnote 27 Ginsburg argues that, while the rule of law remains an essentially contested concept, concrete measures have not advanced much. Existing indices conflate distinct concepts, correlate so highly that they may capture impartial administration rather than the rule of law itself,Footnote 28 and risk being reified rather than improved by large language models.Footnote 29
Kevin E. Davis, Professor of Business Law at New York University, argues that “legal multiplicity”—the overlapping operation of supranational, subnational, foreign, and private law within a single territory, often driven by the extraterritorial reach of U.S. and EU law—attenuates the connection between national legal institutions and outcomes. This attenuation threatens the validity of metrics that presume one can be inferred from the other.Footnote 30
Janka Deli shows that the indices used to price political risk share the structural pathology of pre-2008 credit ratings: they lag and understate rule of law decline and prematurely record and overstate recovery. Deli argues that this asymmetry systematically misprices political risk and misallocates capital, contributing to the economic and political sustainability of institutional erosion.Footnote 31
Adam Chilton, Professor of Law at the University of Chicago, and Mila Versteeg, Professor of Law at the University of Virginia, examine a conceptual concern through Ecuador’s constitutional embrace of a “Rule of Rights.” They argue that constitutional rights can proliferate to the point that conflicts among them enlarge executive discretion and license arbitrary government—the very opposite of the rule of law.Footnote 32
Kim Lane Scheppele, Professor of Sociology and International Affairs at Princeton University, examines how rule of law measures behave in the world—and how the world reacts to being measured. Building on decades of legal ethnography in post-communist Europe, Scheppele shows how the EU Justice Scoreboard failed to capture judicial erosion in Hungary and Poland and how governments can manipulate the indicators it does track. Because those being measured react to measurement, she argues, indicators that ignore how institutional backsliding and capture materialize on the ground will measure the wrong things, fall prey to manipulation, and thus lose their information value.Footnote 33
A final group of essays proposes new approaches to measurement. Diego A. Zambrano, Professor of Law at Stanford University, argues that tracking how often governments lose important, politically sensitive cases over time offers a direct, longitudinal proxy for decisional judicial independence that existing de jure and perception-based measures miss.Footnote 34
Elena Chachko, Assistant Professor of Law at the University of California, Berkeley, argues that leading indices focus overwhelmingly on externally observable legality while rule of law erosion increasingly originates inside the executive branch. Chachko proposes indicators of “executive legalism” that the measurement enterprise should incorporate, from the bindingness and transparency of internal legal advice to the institutional standing of government lawyers.Footnote 35
Finally, Janka Deli, Katerina Linos, and Mark Pollack, Professor of Political Science and Law at Temple University, argue that systematic interviewing of judges and other legal elites belongs in the empirical toolkit of rule of law scholarship. Using twenty-four on-the-record interviews with members of the Court of Justice of the European Union, they illustrate best practices in selection, design, and reporting, and show that the inside view reveals what indices capture only at a distance.Footnote 36
Conclusion
The essays do not converge on a single definition of the rule of law, and that is part of the point. Read together, they suggest that the choice among thin, thick, and purpose-specific conceptions is not a preliminary hurdle but a substantive decision with consequences for what gets counted, who does the counting, and how the resulting numbers are used and abused. They also share a sensibility: skepticism toward composite indices that promise comparability across radically different contexts, and optimism about a new generation of targeted, transparent, and methodologically self-aware measures.
The stakes extend beyond methodology. Rule of law measures shape where capital flows, which governments face sanctions or receive funds, and how democratic backsliding is diagnosed and contested. If those measures lag deterioration, reward gaming, or mistake legality for the rule of law, they will fail the institutions they are meant to protect. We hope this symposium moves the field toward measures that are conceptually honest about what they capture, attentive to the reactions of those they measure, and useful to the scholars, courts, policymakers, investors, and firms who must act on them.
Open access
