Economist Charles Goodhart was considering how to measure the money supply when he said something that has passed into lore through the words of anthropologist Marilyn Strathern: “When a measure becomes a target, it ceases to be a measure.”Footnote ¹ Goodhart’s law identifies a certain persistent problem in designing indicators. When a particular measure is known to those who are being measured and it is within their power to alter its value, they often will try to do so, thus ruining the usefulness of the measure.

We often think about measuring the rule of law as if those whose conduct is being observed were not watching us measuring them. But measuring social phenomena requires considering that people and institutions can and do react to being measured. As we think about building rule of law indicators, then, we should consider whether what we measure encourages good behavior or rewards deception. How we convert a messy on-the-ground assessment of rule of law into numbers should encourage rule of law to flourish rather than encouraging cheating.

I practice legal ethnography,Footnote ² seeing legal systems whole rather than in disconnected variables, ripped out of context, so measuring discrete elements of a system across a large number of countries is not how I work. Ethnography is a close cousin of in-depth interviews, such as the ones explained by Janka Deli, Katerina Linos, and Mark Pollack in this symposium, teasing out how actors in specific times and places understand what they are doing.Footnote ³ That said, quantitative measurement is essential if one wants a quick overview of how countries differ in their commitment to the rule of law, something everyone from policymakers to academics may legitimately want to know.

First-rate quantification requires precise conceptualization, in addition to attention to ensuring validity, reliability, and the lack of bias.Footnote ⁴ Some of those issues can be fixed with care, but other issues cannot. When one “variabilizes” the world by cutting it up into discrete items to measure, one assumes that measuring A in one context is roughly equivalent to measuring the closest thing to A in another. But a prosecutor in Hungary does not look much like a prosecutor in the United States, so counting them in the same category already disregards key differences. In addition, the way A and B are connected in one case may not run through the same mechanisms as the way that A and B are connected in another case. A prosecutor may be established as a fourth branch institution in one country but part of the judiciary (or the justice ministry) in another.

So how can the knowledge we have gained from ethnographic research help to improve the way we count things? Among the many things that ethnographic knowledge can add to this picture, we need to consider that people (and countries) will try to game the measures. Rather than proceed in the abstract, I will use my experience in post-communist Europe to illustrate the problem.

As first Hungary (starting in 2010) and then Poland (starting in 2015) began systematically destroying the independence of their judiciaries, the European Commission wanted some objective way to measure how far they had fallen and how they were different from the other member states. The EU Justice ScoreboardFootnote ⁵ was an early Commission tool to measure compliance with the rule of law, focusing on justice-sector performance.Footnote ⁶

Up close, what was happening in those two countries was no mystery. In Poland, the government took a sledgehammer to the courts, packing the Constitutional Tribunal with unlawful appointments,Footnote ⁷ changing the system for appointing new judges from one that walled the judiciary off from politics to one that rewarded party loyalty.Footnote ⁸ (More than 3,000 judges were appointed in a politically compromised process over eight years.)Footnote ⁹ Polish courts routinely violated European Union and European human rights law under political tutelage.Footnote ¹⁰ The pro-autocratic government installed disciplinary procedures to punish judges who did not toe the party line.Footnote ¹¹

In Hungary, the process was more legal but just as destructive. The government expanded the number of judges on the Constitutional Court and changed the procedure for appointing them so that the governing party soon had a loyal majority.Footnote ¹² The government created a National Judicial Office, directed by a party affiliate, to vet all new judges and to control the careers and salaries of judges already in the system.Footnote ¹³ The government changed the qualifications for judges currently on the Supreme Court to weed out those who might cause trouble and they changed the qualifications again when they had someone in mind to occupy the court presidency who did not meet the new qualifications.Footnote ¹⁴ Across the system, politically loyal court presidents were given the power to create panels of judges to whom they assigned specific cases so that the government could count on disputes of political interest always being assigned to friendly judges.Footnote ¹⁵

The European Commission is very careful about treating all member states with equal respect, so it is reluctant to bring legal enforcement actions without hard evidence that the state is an outlier. In its first foray into coping with the “rule of law problem,” the Commission developed the EU Justice Scoreboard to compare all countries across the EU, imagining that this would illuminate precisely where the independence of the judiciary had been compromised. Every year since 2013, then, every EU member state has dutifully filled out the long questionnaire to score themselves on various measures; other agencies around the EU have dutifully carried out requested surveys and aggregate data collection; and the European Commission has dutifully published the results in its annual Justice Scoreboard.Footnote ¹⁶ Producing the Scoreboard occupies many people for much of the year, each year.

How did the European Commission measure the rule of law in its Justice Scoreboard? Largely through tracking features of the legal system that were easy to count. They included measures of the efficiency of the judicial system: How many cases are there each year (and how many in each of a predefined set of legal categories)? How long does it take from beginning to end of a case, on average? How much of a backlog of cases is left at the end of each year?

Then, they included measures of the quality of the judiciary: What is the budget for the judiciary? How does the average judicial salary compare to the average national salary? What is the budget for legal aid services? What is the percentage of judges who are women? Have courts digitized their records? Are there offices in charge of enforcing judgments so that the courts have backup muscle?

Then, they included measures of the integrity and independence of the legal system: Does the general public perceive the courts as independent? Do companies? Are judges subject to sanctions if they fail to recuse in cases where they have conflicts of interest? How are prosecutors appointed? And who has the authority to fire them? Are disciplinary measures against lawyers subject to judicial review?

As it turned out, neither Hungary nor Poland scored badly on these measures. That is because the way that the judiciaries were captured tracked the stories I have told above, which intersected hardly at all with what was measured.

For example, the length of proceedings is not usually related to whether the judiciary has been compromised. A rule of law advocate should frown when it takes forever to resolve cases, as the Commission envisioned. But a rule of law advocate should also frown when a case takes almost no time at all. A trial with no evidence, affirmed by a judiciary that rubber stamps the decisions of lower courts can be especially speedy, which should constitute an even greater danger signal that judicial independence has been compromised. So, then, which end of this continuum gives you the “better” score rule of law? For what it is worth, in its 2026 Justice Scoreboard report, Hungary came in second among EU member states in having the speediest resolution of cases across all courts.Footnote ¹⁷ Poland was precisely in the middle. As Diego Zambrano notes in his contribution to this symposium, it is tragically easy to game many courtroom statistics like, for example, win rates in different kinds of cases.Footnote ¹⁸

The Commission used budgets as a key indicator of the quality of the judiciary. Since poorer countries always do worse, standardizing the measure against some benchmark makes sense. I might have opted for the fraction of the national budget that goes to courts, but the Commission instead measured the expenditure on courts as a percentage of GDP. The top performers were Bulgaria, Latvia, and Malta with the worst scores going to Denmark, Cyprus, and Ireland.Footnote ¹⁹ Hungary and Poland, the rule-of-law-challenged countries, were squarely in the middle. When the European Network of Councils of the Judiciary interviewed judges across the EU to get their impressions of judicial independence, another potential measure of quality, the order was very nearly reversed.Footnote ²⁰

Or take another measure that tracks something different than what the Commission sought to discover, which was progress toward gender equality on the bench. The history of women judges is different in Eastern and Western Europe. In communist “Eastern Europe,” virtually all of the judges were women because it was a low-status field, precisely because it was politically controlled.Footnote ²¹ When the EU asks now what proportion of the bench are women, it presupposes that the bench was male-dominated until EU gender equality rules kicked in so that a higher percentage of women is “good” because “progress.” But according to the 2026 Justice Scoreboard, four of the top five countries on that measure are East-Central European countries, countries not otherwise known for their commitments to gender equality.Footnote ²²

I could criticize many of the other measures that the EU Justice Scoreboard uses—for example, keeping track of whether a procedure exists for disciplining judges who fail to recuse when they should—rather than asking whether judges who should have recused but did not actually faced consequences. Same with asking whether lawyers who were disciplined have recourse to courts rather than asking whether the procedure for disciplining lawyers is free of political control in the first place. (And, how would you gather data on this? Would you trust a government filling out the questionnaire to answer that question accurately?)

Because the Commission wanted to address outliers for the purposes of assistance and enforcement, it needed measures that could be used in all member states. That meant that countries doing one-off terrible things simply did not show up in the measures. The Commission did not ask, “Have judges been unlawfully appointed to the Constitutional Court?” or “Have new disciplinary procedures, staffed by politically appointed judges, been created?” or “Are judges disciplined for sending reference cases to the European Court of Justice?” Only Poland and Hungary would have answered yes, which was precisely why the questions were not asked.

In short, if you knew how judicial systems are corrupted these days from on-the-ground research, you would know that most of the measures that the Scoreboard uses are irrelevant at best. They simply do not capture how politicians compromise their judiciaries these days. It is as if someone who wanted to know the causes of crime measured the average height of buildings in the vicinity.

The Scoreboard goes beyond irrelevant to tragic, however, when Goodhart’s law kicks in. Take the questions about legal aid, which were designed to measure access to justice. True, Hungary is the worst country in the EU on this measure in the 2026 Scoreboard Report,Footnote ²³ but it would have been catastrophic on this measure if the Hungarian government did not pad the numbers. The Legal Aid Service in Hungary (Jogi Segítségnyújtó Szolgálat) does not just provide free legal counsel for court cases, but it also distributes identity documents, building permits, and more.Footnote ²⁴ The entire budget of this agency was reported as “legal aid,” even though it covers every public-facing legal service provided by government-employed lawyers.Footnote ²⁵ Reporting “legal aid” this way disguises the pitiful provision of counsel for litigants, which is presumably what the Commission wanted to know. As the 2023 country report based on interviews for the International Legal Aid Group noted about Hungary’s legal aid system: “Unless there is willingness to invest in the system, it is likely to become completely dysfunctional.”Footnote ²⁶ The Justice Scoreboard does show that legal aid is a problem in Hungary but the magnitude of the problem is disguised. Failures of measurement like this may be one reason why Janka Deli concludes in her contribution that rule of law declines can happen much more quickly than statistics track because the statistics can mask serious problems for some time before a catastrophic collapse makes the underlying weaknesses visible.Footnote ²⁷

Take another measure that the Scoreboard deploys: digitalization of court records. Here, Hungary has one of the best scores in the EU. But the story behind the scenes is much more complicated. In Hungary, the government controls the courts by choosing court presidents who assign political cases to friendly judges.Footnote ²⁸ In 2022, the European Commission withheld EU money that would otherwise have gone to Hungary but for its compromises of judicial independenceFootnote ²⁹ and demanded, among other things, that the Hungarian Supreme Court move to a system of randomized case assignment. The court obediently set up a system through which every case upon arrival was assigned a number in an electronic database that randomized the identifiers—before the case left the digitized system in order for the president of the Court to assign the case to friendly judges, at which point it was entered back into the digitized system.Footnote ³⁰ The Court digitized the records (check!) but the corruption of the system remained.

I could go on, but you see the point. Things that are easy to measure are not necessarily what one needs to know to assess whether the rule of law is present or absent—and what can be measured that might in fact be useful can also often be gamed. Clever legalistic autocratsFootnote ³¹ have become masters of the one-off who-would-have-thought-of-it workarounds to avoid having indicators catch up with them.

Because legalistic autocrats invent new strategies for undermining the rule of law by constantly innovating, measures that rely on many countries being able to answer the same questions over time will miss most of what autocrats now do. So for example, when Viktor Orbán lowered the judicial retirement age in Hungary to remove many of the court presidents, the Justice Scoreboard would not have caught it because no one thought to measure stability of the retirement age as a factor in judicial independence.Footnote ³² Similarly when Viktor Orbán instituted a new qualification for a Supreme Court judge—that s/he had to have been a judge in the Hungarian legal system for five years before becoming eligible to sit on the Supreme Court—it looked like a sensible requirement, until one realized that it disqualified precisely one person: the Court president who had only sat on the Hungarian Supreme Court for three years, after he had been a judge at the European Court of Human Rights for seventeen years before that (which did not count).Footnote ³³ Five years later, when the government wanted to appoint someone else as Supreme Court president, they amended the law to drop this requirement in order to parachute their favorite candidate into that position.Footnote ³⁴ None of that made it into the Justice Scoreboard. And, in fairness to those who invented the indicators, who would have thought to measure things like these when they happened in precisely one country?

Knowing how legal systems work on the ground and in the field helps in understanding where measures of the rule of law can go wrong and how they may be improved. But between governments gaming the measures and engaging in behavior that evades capture, quantitative measures of the rule of law are always going to be a bit like the drunk who searches under the streetlight for the keys he dropped down the block. He may look where the light is, but it will not be where the keys are.