Introduction
Defining the rule of law is not easy, let alone measuring it.Footnote 1 Decades of scholarship have hashed and rehashed debates over its definition, and most essays in this vein begin with a canned statement about how the concept is itself “essentially contested.”Footnote 2 (Truth be told, that statement is overused.) Despite lack of agreement over definitions, scholars and policymakers regularly make claims that hinge on the rule of law as a quantifiable concept. International organizations rank countries by it. Investors price risk around it. Development agencies condition funding on it. The demand for rule of law measures will continue to increase and so the quality of those measures will continue to matter. Yet most existing tools cannot fully capture what the rule of law seems to require at its core.
This essay argues that a useful complement to existing rule-of-law measures is to track how often governments lose in court. Regardless of endless debates about the rule of law, everyone agrees that judicial independence is a core value. And, in turn, if judicial independence means anything, it must mean that courts can and do rule against the government when officials appear before them. A judiciary in which the government almost never loses—regardless of the quality of its cases—is one whose independence deserves scrutiny. Governments that rarely lose probably face weaker courts.
To be sure, government win rates are not at all a complete measure of judicial independence, let alone the rule of law. Win rates are shaped by litigation incentives, selection effects, substantive law, and the quality of legal representation. But as a proxy, and a measure that we can track longitudinally, they may reveal something that formal measures miss: whether judicial power is meaningfully exercised against the state. Based on that, scholars could also use the data to study whether changes to judicial decisional independence correlate with certain kinds of judicial reforms.
This essay sketches a vision of how the idea of tracking government litigation win rates could be operationalized into a measure of judicial independence.
De Jure, De Facto, and Decisional Judicial Independence
We can sidestep most rule of law definitional problems by focusing specifically on judicial independence. Scholars broadly agree that the judiciary’s ability to act independently from the political branches is key to a rule of law state. To the extent that rule of law indices have grown in the past decades, they all incorporate some metric to evaluate the independence of the country’s judiciary.Footnote 3
But hold on a second—the move toward judicial independence runs into another definitional problem: judicial independence is itself a contested subject of debate. True, it is nowhere near as contested as the rule of law.Footnote 4 At a minimum, the literature distinguishes between de jure judicial independence and de facto judicial independence. De jure independence refers to formal legal arrangements like fixed tenure, salary protections, budgetary autonomy, tenure protections, and similar institutional devices. De facto independence, by contrast, refers to actual practices—whether judges actually decide cases free from pressure, threats, inducements, or just obey political actors.Footnote 5
Much of the difference between de facto and de jure judicial independence can be attributed to informal norms. Courts are institutions embedded in a profession and networks of connections between political actors and lawyers. In recent work with Ludmilla Martins da Silva, we argued that informal norms can determine how judicial selections work in practice.Footnote 6 Even where constitutions and statutes appear to create neutral selection systems, informal expectations can narrow the field and shape downstream judicial behavior. Any serious effort to measure judicial independence, then, must assess the role of informal practices.
De facto judicial independence is therefore best understood as a cluster of related ideas—autonomy, impartiality, norms, and insulation from external control.Footnote 7 Within the broad category of de facto judicial independence, Maria Popova has usefully narrowed the inquiry still further by focusing on what she calls decisional judicial independence. Popova shifts attention away from generalized descriptions of judicial behavior, toward a more specific question: to what extent do non-judicial actors succeed in shaping outcomes in particular cases? On Popova’s account, judicial independence turns on “[t]he number and the identity of the actors who do not exercise power over the adjudication process, as well as in terms of the magnitude of the power of the actors who do manage to impose their preferences.”Footnote 8 This means that judicial independence is low when external pressure is routine, when multiple actors can influence outcomes, or when one powerful actor can consistently secure preferred results.Footnote 9 It is high when such influence is rare or ineffective.Footnote 10
This essay builds on that narrower concept of decisional judicial independence. The claim, more fully developed in the section below, is that one promising way to observe decisional independence is to examine government win rates in cases in which state actors have meaningful interests at stake.
Existing Measures and Their Problems
Scholars have long struggled to measure judicial independence, especially in comparative work. Existing measures generally fall into a few categories: formal legal protections, expert surveys, reputational assessments, and observable threats to courts.Footnote 11 While each captures an important component of judicial independence, they also focus on the conditions thought to support independence, or the signals surrounding it, rather than judicial independence itself.
Some measures focus on the superficial rules and laws that, at least formally, guarantee independence, such as tenure protections, salary rules, removal procedures, or budgetary autonomy.Footnote 12 These indicators can be useful, but they do not address whether judges behave independently in practice.Footnote 13
As discussed above, other measures build on this by focusing on de facto judicial independence.Footnote 14 These generally rely on expert surveys, reputational impressions, or judges’ own reported sense of independence.Footnote 15 But even these measures rely on indirect proxies rather than on judicial outputs themselves. As a result, they may still miss whether judges rule against the state when the state has interests at stake.
A final category of measures focuses on observable attacks on courts: judicial purges, court-packing efforts, and congressional hostility.Footnote 16 These signals, however, also face the problem that attacks on courts may actually reflect an independent judiciary resisting pressure, while their absence may indicate an already captured court.
None of these measures fully get at whether judges are truly making decisions without undue influence.
Proposed New Measure
One direct way to observe decisional independence is to track how often governments lose cases.Footnote 17 The idea is not that government win rates provide a complete measure of judicial independence, much less of the rule of law as a whole. Rather, the narrower point is that in politically sensitive cases, repeated patterns of government success or failure are a proxy for whether courts can constrain public power. Here are a few preliminary guidelines:
First, any effort to build a large database of judicial decisions would have to focus on intra-country changes over time, not cross-country comparisons.Footnote 18 Comparing countries is difficult because too many variables impact whether governments win cases, including differences in substantive law, procedural rules, litigation incentives, access to counsel, and broader political context. That means such a measure is likely useful only for longitudinal comparisons, rather than purely cross-sectional. That is, changes in government win rates within a country over time can tell us more about its judiciary than cross-country comparisons. To be sure, a regional project covering countries like Chile, Colombia, Mexico, Brazil, Uruguay, and Argentina might still be useful, especially because these democracies have experienced significant institutional changes in recent decades. But even then, the most important payoff would be to track variation over time.
With these limits in mind, a scholar could implement a modest and realistic version of the project. A scholar might, for example, study a single country over a defined democratic period—say, Chile in the decades following democratic transition—and measure whether government win rates in a narrow set of politically sensitive cases changed over time. Following the suggestions above, such a study can isolate a subset of comparable cases in which judicial pressure is plausible. If there are changes over time, the researcher might scrutinize whether shifts in state success rates corresponded with institutional reforms, changes in court structure, or moments of political contestation. Of course, any such analysis needs to account for confounders (discussed below). But if background conditions are held sufficiently constant, changes in government win rates over time may tell us whether courts have become more or less willing to check government power.
Still, government win rates are subject to multiple forms of endogeneity.Footnote 19 A government might win frequently because it generally acts lawfully and brings strong cases. Or it might win because the costs of suing the state are too high, because potential plaintiffs face many other barriers in challenging public authority, or because government attorneys are talented. Win rates may also be shaped by strategic behavior: litigants may choose whether to bring claims based on likelihood of success, and judges’ own ideology may influence outcomes independently of actual political pressure. A society that is especially litigious could generate a very different case pool from a society in which few are filed. And governments can even sue themselves, adding an additional confounder to the data.Footnote 20 These problems do not make the project impossible. But they show that any effort would have to be done carefully.
Second, then, to reduce the problems discussed above, the focus should be on cases in which government stakes are substantial, judicial pressure is plausible, and the state’s preferences are likely to be salient. Several variables can be proxies for high-stakes cases: whether senior officials are named parties, who else joined (amici, intervenors, or politically connected parties), the amount in controversy, and the level of media and political attention. At least three legal areas stand out: defamation, land expropriation, and police misconduct. These disputes recur across legal systems, carry obvious political stakes, and are typically resolved in courts.Footnote 21
Start with defamation suits brought by officials against media outlets. Obviously, those suits do more than protect reputation. They can become tools for suppressing dissent. For instance, Diosdado Cabello, an important politician from Venezuela’s ruling authoritarian party, sued the country’s largest newspaper for simply republishing an article originally published by a foreign media outlet. The article alleged that Cabello was under investigation in the United States for suspected ties to drug trafficking. In response, the regime-captured Supreme Court found El Nacional liable for USD ＄13.3 million in damages.Footnote 22 Cabello then filed an identical defamation claim against the Wall Street Journal in U.S. courts, but, facing an actually independent judiciary, the claim was dismissed.Footnote 23 If politically connected plaintiffs systematically win more often, or if they receive unusually high damages awards compared with non-governmental plaintiffs, that may suggest a judiciary responsive to political influence. Maria Popova’s work on Russia and Ukraine shows how judicial outcomes in defamation litigation can show whether political affiliation predicts success in court and whether compensation patterns themselves reflect bias.Footnote 24
Land expropriation disputes are another potential area of study because they place individual rights and state power in conflict. Eminent domain implicates property, compensation, and the scope of public purpose. In illiberal contexts, governments may stretch those concepts in order to expand discretionary control over land. Again, how the courts react can demonstrate whether courts actually review executive assertions of public necessity or merely ratify them. Here too, outcomes alone may not tell the whole story. Compensation levels matter. A nominal plaintiff victory accompanied by inadequate compensation is the same as a government win. Sultan Mehmood’s work suggests that this field is especially useful for studying how judicial institutions affect outcomes in politically salient disputes.Footnote 25
Police cases also offer a good third area of study because policing is closely related to state coercion. When individuals challenge police misconduct, courts confront the most obvious exercise of official power. Governments often have strong incentives to shield police officers, whether to preserve institutional loyalty, avoid embarrassment, or protect broader state interests. A judiciary willing to invalidate police decisions or hold officers accountable is evidence of decisional independence. Yutian An’s work on police cases in China points in this direction, showing how judicial treatment of these claims may correlate with judicial independence.Footnote 26
The basic methodology should follow and extend Popova’s multi-stage win-rate approach but it should also incorporate mixed-methods.Footnote 27 Scholars would begin with one of the categories of cases (defamation, land expropriation, and police misconduct) and compare government success rates to broader litigant baselines. They should adjust where possible for selection effects through close attention to remedies, including the size of damages or compensation awards. But quantitative analysis alone would not be enough. The next step is to incorporate qualitative evidence—interviews with judges and perhaps lawyers or other court actors—to better understand how external pressures, informal expectations, or professional norms shape outcomes in practice.Footnote 28 That mixed-methods design would offer a better picture of decisional judicial independence. None of this would yield a perfect measure. Still, as a first step, tracking government win rates in politically sensitive cases could provide valuable insight into shifts in judicial behavior over time.
Conclusion
Beyond measuring judicial independence at a moment in time, the approach proposed in this essay could help evaluate whether judicial reforms actually work. A longitudinal dataset would allow researchers to ask whether shifts in judicial behavior correlate with identifiable institutional changes—judicial selection reforms, constitutional amendments, changes in court composition, transitions between governments, and so on. If government loss rates rise following a reform that strengthens judicial tenure protections, that correlation might (with the usual causal identification requirements in place) offer support on a question that has long resisted answer: do formal institutional reforms actually produce behavioral change? By contrast, if win rates remain flat despite formal reforms, that would be evidence that the reforms were superficial. (Again, this assumes careful work that controls for many other variables.
In conclusion, no measure of the rule of law will ever be complete, and government win rates are no exception. But the goal here is just improvement. If existing indices tell us too little about whether courts actually constrain public power, then asking how often governments lose in court is a step in the right direction.
Open access
