II. The Origins of Judicial Disclosure

Today’s courts devote far more energy to judicial transparency than was the case a century ago. Worldwide, many courts selectively publish opinions, disclose court statistics, hold press conferences, offer status updates on cases, maintain social media accounts, and allow television coverage of court proceedings. An emerging strand of the comparative literature on courts examines why courts go public and what effect transparency has on public trust and legitimacy. So far, however, this literature has focused on democracies, particularly in Latin America (Staton Reference Staton2010; Ingram Reference Ingram, Davies and Taras2017) and Europe (Grendstad, Shaffer, and Waltenburg Reference Grendstad, Shaffer, Waltenburg, Davies and Taras2017). Certainly, government transparency is associated with democracies because sharing information seems intuitively linked to responsive, accountable government. Yet the trend toward judicial transparency is not limited to democracies, and one contribution of this article is to document how it extends even to authoritarian states such as China.Footnote ⁵

When it comes to making court decisions available online, China is a trendsetter in the authoritarian world and is unusual even among other civil law jurisdictions. Most court judgments in China have technically long been public documents, meaning that a Chinese citizen has the right to view a case at the courthouse.Footnote ⁶ In practice, however, until recently court decisions were typically available only to the people directly involved in the case. This meant that collecting even a small set of court opinions required personal contacts. Today, China’s Supreme People’s Court claims that the tens of millions of cases available online make its website the largest collection of public cases globally (Sina Court Channel 2016).Footnote ⁷ Vietnam, a Communist Party–led one-party state where political and legal reforms are often inspired by Chinese practice, has already followed suit. Starting on July 1, 2017, all Vietnamese courts became obligated to make their judgments public, with decisions centrally collected on a website managed by the Vietnamese Supreme People’s Court (Nguyen Reference Nguyen2019).

Why did China’s judicial leadership embrace the practice of making court judgments public? One angle on this question is historical. As sudden as the 2013 decision to release the vast majority of court decisions might have seemed, it followed a history of experimentation and intellectual debate. The SPC recognized the value of making some court decisions publicly available to educate both judges and litigants early in the reform era. In 1985, the SPC began publishing an official gazette (最高人民法院公报), which included a small number of cases. Although not formally recognized as precedent, published cases were meant to guide lower courts on how to handle particular points of law and to improve the uniformity and quality of court decisions across China. Collections of representative cases (典型案例), some curated by the SPC and others by academics or local courts, also became standard fare for legal publishers and a source of practical guidance for their readers (Liebman and Wu Reference Liebman and Wu2007, 289). Likewise, in the late 1990s and early 2000s, individual courts began posting selected representative cases online. How much material was available varied, from a handful of decisions to hundreds of cases, with a few standouts attempting to cultivate a reputation for innovation by pursuing transparency. Fee-based databases also started to sprout up, as new actors saw a commercial opportunity to build the Chinese version of America’s LexisNexis and Westlaw, although coverage remained spotty. A few law firms also started posting cases handled by their lawyers to project professionalism and attract clients.

In the early 2000s, liberal scholars began to call for courts to place all opinions online as a way to fight corruption and restore public confidence in the courts (He Reference He2003). Somewhat ironically, it was not until Wang Shengjun—an official widely perceived as ideologically conservative—became president of the SPC in 2008 that the SPC itself made a push to place large numbers of court decisions online. The SPC endorsed judicial transparency as a goal in its third 5-year plan for legal reform in 2009 and also encouraged lower courts’ efforts to compile and publish judicial decisions (SPC 2009a, 2009b). These central government cues prompted local initiatives, including in Henan, where a mid-2009 Henan High People’s Court order mandated that all courts in the province place the vast majority of decisions online. This new requirement followed a wave of high-profile wrongful convictions in Henan and was one of a number of populist measures adopted by Henan High People’s Court President Zhang Liyong (Liebman Reference Liebman2015, 161–62). Other provinces followed Henan’s lead. Building on these local experiments, the SPC under the leadership of SPC President Zhou Qiang called on courts nationwide to begin posting most cases online in 2013 (SPC 2013).Footnote ⁸ The new rules created a centralized website, called China Court Judgments (中国裁判文书网), which launched on July 1, 2013. By the middle of 2015, the website included documents from across the country (Sina Court Channel 2016).

What the SPC has embraced is a form of “controlled transparency” (Liebman Reference Liebman2011, 847). Although the trend is toward releasing a greater number and variety of court documents, certain types of cases are exempt, and local courts were granted discretion to hold back individual decisions. For example, the first set of SPC rules governing the public release of court opinions, issued in 2013, provided exemptions for cases involving state secrets or personal privacy, juvenile criminal cases, disputes concluded through mediation, and other documents deemed “inappropriate” (不宜) to publicize (SPC 2013). Revisions to the rules in 2016 expanded the range of publicly available court documents and also began requiring courts to release the case number of any decision deemed unsuitable for posting online, along with an explanation of why the judgment was held back (SPC 2016a).Footnote ⁹ At the same time, the 2016 rules reiterate the principle that local courts have discretion not to post decisions that the courts deem inappropriate for online posting, and they expand the list of case types shielded from public view.Footnote ¹⁰ In addition, decisions may not be posted online for release until after the appeal process is exhausted, an area that was ambiguous under the 2013 rules.Footnote ¹¹ To be sure, these carve-outs restrict our view of significant areas of everyday adjudication, such as family law, as well as of how courts resolve difficult or sensitive disputes. And yet even this modest form of controlled transparency signifies a sea change for China’s courts.Footnote ¹²

Why did the court leadership do it? Another angle on this question is political, and the large-scale release of court documents may be viewed first and foremost as a way to serve Party goals by curbing wrongdoing in the courts. Court officials in Henan made this line of argument explicit: judges are more likely to follow the law and less likely to engage in malfeasance when they know their work will be made public. An SPC white paper endorsed this logic in 2017, noting that placing cases online fits with President Xi Jinping’s calls for judicial openness and increased public supervision (SPC 2017). Viewed from this perspective, disclosure of court documents was a way for the courts to participate in Xi’s signature anticorruption campaign and also join a broader Party-state move toward transparency, a word that entered the Chinese lexicon of governance over the past decade. Freedom-of-information regulations passed in 2007 coexist with a media spotlight glaring enough to expose at least some malfeasance. Officials are also required to release certain types of data, including selected environmental statistics and now court decisions. Somewhat unexpectedly, given the association between democracy and transparency, the contemporary Chinese Communist Party has emerged as a champion of Justice Brandeis’s well-known idea that publicity can act as “a remedy for social and industrial diseases” and that “sunshine is … the best disinfectant” (Reference Brandeis1913, 10).

Greater judicial disclosure also appealed to strategic thinkers in the court bureaucracy who thought it could improve the standing of courts with the public or at least help them more effectively monitor and run a far-flung court system. Chinese courts have long been regarded as weak actors, and some thought greater transparency could help improve trust in courts and make it easier for them to resist external pressure.Footnote ¹³ Placing decisions online was also plainly a way to expose the activities of individual judges to their superiors in the court hierarchy. Databases of legal decisions can be a powerful tool of court administration insofar as they alert higher-ups to unusual patterns of decision making, perhaps even identifying individual judges who work exceptionally slowly or routinely hand down outlier decisions.

Technophiles inside China’s political-legal system have further argued that algorithms derived from mass digitization of court opinions can advance the efficiency and standardization of China’s courts. Some court leaders are on the record discussing hopes that artificial intelligence can ensure consistent decision making (同案同判) and reduce judges’ workload by drafting parts of opinions or deciding easy cases. A number of experiments are underway to use technology to ensure standardization. SPC President Zhou Qiang, in particular, is associated with the idea of smart courts (智慧法院) and has talked about how computer-assisted judging could improve litigant satisfaction by ensuring consistent, fair, and transparent dispute resolution (Jie Reference Jie2016). In contrast to past concerns about catching up with other countries, there is a possibility that Chinese courts could leapfrog past the rest of the world into the futuristic world of computerized judging. China’s first forays with computer-assisted judging have been small-scale, and some Chinese judges express skepticism about computerized adjudication. Nevertheless, these experiments place new importance on court opinions as the source of data that programmers are now using to construct algorithms used to standardize outcomes or decide cases.

III. Data and Method: The Henan Data Set

The Henan data set is a collection of 1,058,986 court documents from the Henan High People’s Court website (table 1). We chose to focus on cases from a single province, Henan, for three reasons. First, Henan started putting cases online earlier than most other provinces and posted hundreds of thousands of cases before the launch of the SPC website. Studying Henan allows us to look further back in time than would be possible in other provinces. Second, Henan ranks in the bottom third of Chinese provinces in GDP per capita. Examining court practice in Henan is a useful corrective to scholars’ tendency to focus on courts in rich areas, where researchers have often enjoyed better access. Our study also reveals significant differences even within Henan—perhaps not surprising given that the province is home to nearly 100 million people and 184 courts. Third, a provincial focus makes finer-grained analysis possible. It is feasible to collect information about individual courts and explore differences among them.

Table 1.

Documents in the Henan Data Set

Creating this collection of court documents took 18 months of effort by a team of research assistants with a background in computer science. They developed software to scrape more than 1 million court documents and wrote code known as a “parsing script” to help the computer differentiate between parts of court documents. Owing to the number of documents, a parsing script is necessary for such basic tasks as counting document types or determining how many documents each court uploaded. In terms of format, court opinions follow guidelines set by the SPC, with some local variation.Footnote ¹⁴ They open with a header that includes the court name, case type, and case number. Next comes a list of parties to the case, including information about the lawyers, legal workers, or other persons acting as legal representatives involved in the proceedings. The substantial middle of the opinion follows, usually with a summary of the claims and arguments presented by each party, the facts and evidence reviewed by the court, and the court’s legal reasoning and decision. The final paragraph of the decision apportions legal fees to the parties, before closing with the names of the court personnel who heard the case and the date of the judgment.

The ability to parse cases yielded table 1, a detailed look at the types of documents contained in the Henan data set.Footnote ¹⁵ The 1,058,986 court documents span the years between 1996 and 2015, with the great majority dating from 2008–15. Major categories include 693,747 decisions in civil disputes, 255,255 decisions in criminal cases, and 31,710 decisions in administrative cases. There are also a substantial number of enforcement actions (54,192).

IV. What is Missing? Assessing Bias

For all that is available, the Henan data set is also clearly incomplete, and documenting what is missing is an urgent task for researchers. In what follows, we document what we call the “missingness problem”: variation in court compliance with the national mandate for disclosure. We find wide variation in disclosure rates across courts that does not fully fit with the conventional explanations, such as mediation rates or GDP per capita.Footnote ¹⁶ In contrast, our analysis suggests that variation in transparency across courts is most likely due to variation in how courts have interpreted the national rules surrounding disclosure. Some courts are more diligent than others in disclosing decisions, especially as court leaders likely vary in how much emphasis they place on transparency.

In 2014, internal court statistics report that Henan courts completed 685,890 cases, compared to 286,101 documents in our collection.Footnote ¹⁷ On average, this means that Henan courts placed just over 41% of their docket online, a proportion in line with recent national estimates that slightly less than half of 2014 and 2015 cases appear on the SPC website (Ma, Yu, and He Reference Ma, Yu and He2016). However, this average disguises tremendous variation. The highest-ranking court in our sample released enough documents to plausibly cover 83% of completed cases, compared to just 14% for the least compliant court.Footnote ¹⁸ This variation is particularly striking for a province that had released rules concerning what must go online by 2014 and had already begun ranking courts based on their compliance with provincial transparency policy.

The variation in disclosure rates across courts has not disappeared. During our research trip in May 2017, many Henan judges and scholars argued that 2014 was early in the adoption of mandatory disclosure and that disclosure rates had dramatically improved and become more uniform since then.Footnote ¹⁹ There are now clearer SPC rules about what should be disclosed, and disclosure is now the default. In addition, the Henan courts introduced software in mid-2016 that automatically uploads court documents to the SPC server unless the presiding judge indicates a reason the case should not be made public and a higher-ranking judge in the court approves a request not to disclose. Yet our analysis of 55 courts with publicly available work reports for 2016 shows an average disclosure rate of 54%—an improvement over 2014’s average disclosure rate of 41% but hardly the end of the missingness problem (fig. 1).Footnote ²⁰ Variation across courts also persists. The highest-ranking court in our 2016 sample has a disclosure rate of 93%, compared to just 26% for the least compliant court.

Figure 1.

Proportion of cases online by court in the Henan data set, 2014 and 2016

What might explain the variation in disclosure rates across courts? One obvious explanation is that the national disclosure rules may affect courts differently. For example, mediation agreements are not required to be posted online, and courts with lower disclosure rates could rely more heavily on mediation. However, further analysis of the 2014 data shows that accounting for mediation does not explain differences across courts. In Henan, internal court statistics count 110,134 mediated cases in 2014, and accounting for mediation boosts the average disclosure rate to 52%.Footnote ²¹ Adjusting for mediation also results in even more variation across courts. Although the least compliant court released only enough documents to plausibly cover 20% of completed cases, adjusting for mediation yields four courts that released enough documents to potentially cover 100% of decided cases.Footnote ²²

Another obvious place to look for explanations for variation in transparency is court resources. Resource bias suggests that variation in court transparency stems from underlying resource constraints, particularly the availability of personnel to collect judicial decisions, black out personal information, and place them online (Grimmer, Roberts, and Stewart Reference Grimmer, Roberts and Stewart2019). Nationwide, Ma et al. (Reference Ma, Yu and He2016, 208–9) show a statistically significant correlation between GDP per capita and the 2014 interprovincial disclosure rate, which provides some support for the idea that resource bias might matter. However, our parallel efforts to investigate the relationship among GDP per capita, population, and the 2014 disclosure rates for individual courts within Henan Province show no statistically significant relationship among the variables (app. B; apps. A–C are available online; apps. A–C are available online).Footnote ²³

Rather, we find evidence within the Henan data that how courts interpret the rules explains at least some of the variation in transparency. In other words, diligence bias—or the degree to which courts scrupulously adhere to national guidelines—helps explain missingness. Divorce cases provide a good example of variation in how closely courts adhered to national rules on disclosure.Footnote ²⁴ From January 1, 2014, through September 20, 2016, the SPC rules were that divorce decisions should be made public only if the names of the parties were redacted. Then, in July 2016, the SPC changed the rules to protect personal privacy more fully and prohibited courts from disclosing divorce decisions.Footnote ²⁵ In our data set, however, we find that unredacted divorce cases are common before 2016 and that some courts continued posting divorce cases even after the prohibition on releasing them took effect. Our data set contains 29,982 divorce decisions in 2014, an average of 5% of the cases posted in each court, and this average falls to 3.5% of the cases posted in each court only in 2016 (for a total of 38,585 cases posted). As shown in figure 2, in both 2014 and 2016, the proportion of divorce cases posted online is positively correlated with the proportion of the total docket online. This suggests that courts with high transparency rates might be releasing more documents than SPC rules suggest and that courts with low transparency rates are more strictly following the SPC rules.

Figure 2.

Relationship between transparency and proportion of divorce cases, 2014 and 2016. Both years show a strong correlation between transparency and divorce cases, indicating that court policy on releasing divorce cases may explain variation in transparency.

Diligence bias could also help explain differences in disclosure rates between basic and intermediate courts. Under SPC rules, first-instance decisions may be released only after any subsequent appeal concludes. This rule creates a time lag between first-instance decisions and the posting of those cases online, which may lead to an unintended disclosure gap between basic and intermediate courts. Even though almost all 2014 cases should have been resolved by the time we collected the data, the average intermediate court released enough documents to cover 56% of cases decided in 2014 compared to an average of 40% for basic-level courts, a statistically significant difference ( $p<.001$ ).Footnote ²⁶ In 2016, this trend held, with intermediate courts disclosing 63% of documents compared to 51% for basic-level courts, still a statistically significant difference ( $p=.047$ ). Our interpretation is that some first-instance cases may not be posted online because some lower-court judges are unaware when appeals are resolved.Footnote ²⁷ Diligence bias also helps explain why there is no relationship between court transparency and GDP per capita. Although our initial expectation might be that resource-rich courts have the human bandwidth to place more documents online, they might also use that capacity to follow SPC standards surrounding nondisclosure strictly, such that fewer documents end up online.

Of course, it is impossible to determine exactly what is not in our data set, and it is also likely that more than one type of bias is at play. For their part, Henan legal professionals typically reached for explanations about the importance of court leadership when we asked about variation in disclosure rates during fieldwork there in 2017. Their explanation is that court presidents vary in the value they place on transparency, particularly how much they care about their court’s performance in annual province-wide disclosure rankings and the degree to which they prioritize disclosure rates in evaluating individual judges.Footnote ²⁸ There is also some evidence that this might be true and that court presidents respond to the incentives created by rankings and evaluation. Ma et al. (Reference Ma, Yu and He2016, 204) discovered through fieldwork that some courts face a “small exam” each quarter (季度小考), as well as a major year-end evaluation (年度大考), and that uploads of judicial decisions to the SPC website spike at the end of the quarter just ahead of court evaluations.

In contrast, American audiences typically speculate that records of politically sensitive cases are suppressed by the court leadership. Again, there is some evidence that this is true. In Henan in particular, lawyers routinely note that cases they have handled involving politically well-connected parties are frequently missing from online databases. Administrative censorship is hard to document, however, because courts are reluctant to discuss it. Technically, Chinese courts are supposed to publicly release a written reason for any judicial decision they choose not to disclose publicly. Even though courts rarely comply with this directive, our analysis of 2,472 such documents includes only three examples of courts willing to either cite the catchall “inappropriate to publish” provision of the rules or admit that they are suppressing collective cases (群体性案件). Of these courts, only a Shandong court disclosed enough detail to identify the type of case held back from public view, a string of collective labor disputes.

In sum, there are three credible explanations for what is missing: administrative censorship, incentive bias, and diligence bias. These three explanations collectively lay the groundwork for future research. What is needed is a sustained, cooperative effort to document the relative importance of different types of bias in China’s new online case databases and to trace changes over time.

What to do in the meantime, then, as missing cases will affect every analysis? Building on Grimmer and Stewart’s (Reference Grimmer and Stewart2013) four principles of automated content analysis, we offer three concrete suggestions about how to learn from a rich, incomplete data set. First, we can seek out pockets of excellent data. Although the results will not be generalizable to all courts, they will provide more certain answers. At the very least, some researchers may decide that a smaller-scale analysis of a more complete corpus is a worthwhile trade-off.Footnote ²⁹ Second, we can use what we know about missingness to bound our estimates of the prevalence of a phenomenon. Frequency estimates of any type of case can be bounded by assuming that all missing cases belong to the set (the upper bound) and then assuming that no missing cases belong to the set (the lower bound). And, third, computational text analysis can be combined with qualitative research. Computational text analysis is particularly valuable preceding fieldwork, when it can be used to uncover patterns and suggest questions that we would not have otherwise known to ask. We are especially optimistic about what we can learn by asking local legal professionals—especially experts or repeat players in a particular area of law—to comment on preliminary results from computational text analysis. There is no substitute for local knowledge, and given the extent of the missing data, checking whether findings are credible is an important reality check. Conversations with Chinese scholars, judges, lawyers, and even plaintiffs can also help suggest explanations for trends, as well as further research questions. We illustrate these three approaches in the next section, using administrative litigation as a case study to illustrate how to work with a large, incomplete data set.

V. Court Decisions as Data: Insights into Administrative Litigation

Administrative law is a natural starting point for students of Chinese politics because it is intrinsically political. This is the area of law that governs interactions between citizens and the state, in which citizens use law to challenge illegal state actions. Because of this, administrative litigation lawsuits are often framed as part of the basic repertoire of contention and a way for ordinary Chinese citizens to contest government decisions over sensitive issues such as land seizures, fines, and police detention (Mahboubi Reference Mahboubi, Xiao Zhou, Rigger and White2014; Cui Reference Cui2017). Both the Chinese- and English-language scholarship have closely tracked the total number of administrative lawsuits, a number that has been treated as a signal of citizen willingness to challenge the state, evidence that the populace is undergoing a “legal awakening” (Zhang and Ginsburg Reference Zhang and Ginsburg2019, 285), and an indicator that the Chinese state is increasingly constrained by law (Zhang and Ginsburg Reference Zhang and Ginsburg2019).Footnote ³⁰

Despite the fact that the raw number of administrative lawsuits is seen as an important data point for understanding trends in state-society relations, no one has yet looked behind the numbers at what kinds of disputes end up counted in official statistics. In large part, this is because a more granular look became possible only following the Chinese court system’s decision to release publicly most court decisions. In what follows, we use a topic model to explore the 25,921 decisions in administrative litigation cases in our data set, with the goal of better understanding the range of cases that collectively constitute “administrative litigation.”Footnote ³¹ To the best of our knowledge, no prior scholarship has analyzed such a large collection of Chinese administrative cases, and only a few scholars have used topic models to analyze judicial decisions in other countries.Footnote ³²

Methodologically, this section shows how topic modeling can lead to new discoveries and, in particular, complicate categories that were previously taken for granted.Footnote ³³ After sketching a portrait of the data set using topic modeling, close reading of cases revealed a large number of topics related to disputes involving third parties. In these cases, the litigant is using administrative litigation to try to obtain assistance from the state in a dispute with another private individual or organization. We then go on to estimate that 47% of our administrative corpus consists of third-party cases, based on hand-coding a random sample of cases. This finding suggests that a large proportion of administrative lawsuits are private disputes in which litigants are trying to leverage the power and authority of state agencies, rather than efforts to challenge or constrain officials. Mindful of the possibility that missing data could affect our analysis, we then further examine courts with high transparency, bound our estimate, and engage in qualitative fieldwork. All these strategies confirm the prevalence of third-party disputes—a finding that ought to shift the field’s interpretation of what a growing administrative caseload means and temper conclusions that it signifies either popular dissatisfaction with the state or a step toward the expansion of judicial review.

Before walking through our research process and results, a brief introduction to topic modeling is helpful.Footnote ³⁴ Topic modeling, a tool that originated in computer science and is now used across the social sciences, helped us examine the broad categories of more than 20,000 administrative litigation cases without reading each one. We used the Structural Topic Model package in R (Roberts et al. 2014; Roberts, Stewart, and Airoldi Reference Roberts, Stewart and Airoldi2016; Roberts, Stewart, and Tingley Reference Roberts, Stewart and Tingley2019) to estimate topics, which are groups of words that are likely to appear together within documents. For example, the model suggests that the words “land,” “use permit,” “collective,” “issue,” “land used,” “construction,” and “use” frequently appear together, collectively forming a topic we labeled as “rural land-use permitting.”Footnote ³⁵ In addition, the model estimates topic proportions for each document, with each document containing a mixture of topics. Research assistants helped us review the highest-frequency words associated with each topic, as well as 20 cases associated with each topic, in order to assign each topic a topic label (e.g., “birth planning” or “withdrawals—individuals”).Footnote ³⁶ A list of all 82 topics and the highest-frequency words associated with them appears in appendix A.

Six broad categories of disputes emerged from the topic model—topics related to land and property; procedure; fines or punishment; benefits, labor rights, or compensation; general words; and permits and registration. We assigned each topic to one or two of these categories, based on the most important themes in the 20 cases we reviewed (table 2). To describe the range of topics and categories, figure 3 is a correlation plot coded along these six themes, in which the node size is proportional to the amount the topic is discussed within the corpus of text.Footnote ³⁷ In other words, larger nodes indicate more common topics. Edges, or the lines connecting nodes, indicate that there is a correlation between topics greater than 0.01 or that the topics are more likely to appear within the same document. In addition, groups of topics that are likely to appear together within documents also come into focus. For example, a group of topics related to the benefits, labor, and worker compensation theme cluster in the right side of the plot. These documents are likely to contain similar language related to monetary demands, particularly related to health and retirement benefits. Land and property claims largely congregate toward the bottom left of the plot. Language related to court procedure and fines appears in many types of cases, and such topics are scattered throughout the correlation plot.

Table 2.

Categories of Administrative Litigation in the Henan Data Set

Note.—We put each topic into one or two broad categories; i.e., some topics were assigned to one broad category and some to multiple categories. We then estimated the proportion of words in each category by estimating the expected topic proportions across documents assigned to that category.

* These are general litigation-related words, such as “respondent” and “rights,” that persist even after removing words that appear in more than half of the court cases.

Figure 3.

Correlation plot of administrative cases in the Henan data set. Topics 54, 66, 78, and 80 were removed because they were not connected to other topics.

Using the topic model output, we can estimate the proportion of the documents related to each of the broad categories of topics. Table 2 shows the sum of the topic proportions for topics related to each of the broad categories in the topic model. Overall, we see that the topic model estimates that 30% of the corpus is related to land and property disputes. In addition, 11% of the cases are related to fines and punishment.

By closely reading the cases most representative of each topic, we discovered that 37 of 82 topics mark an attempt to draw the state into private disputes, typically by challenging an administrative decision in favor of another individual, group, or entity. Sometimes, the fact that the underlying dispute is between private parties is obvious because a designated third party (第三方) appears in the list of parties that opens all Chinese court decisions. In a 2013 case, for example, a concrete factory sued the Dengfeng Human Resources and Social Security Bureau over a decision to award work-related injury compensation to Wencan Liu. The factory had been privatized 4 years before the lawsuit and argued (unsuccessfully) that they should not bear liability for a case of tuberculosis caused by a 2-decade work history that largely fell into the period when it was a state-owned enterprise.Footnote ³⁸ Another example of this type of third-party dispute is a 2014 case from Pingdingshan City involving a complicated property dispute—and attempt to kick out several alleged squatters with a competing claim—in which the Real Estate Administration Bureau was dragged to court to (successfully) defend its property registration decision.Footnote ³⁹ When the third party was not clearly identified at the start of the decision, it took more attentive reading to surface the underlying civil dispute. In topic 44, for example, there were a number of cases challenging business registration with the State Administration of Industry and Commerce. The text of these decisions, however, makes clear that these disputes are in fact about the underlying ownership of the company in question.

Having uncovered the importance of third-party disputes through topic modeling, we sought to estimate the frequency of third-party disputes by hand-coding a random sample of 500 cases. This was also an important validation step, along the lines recommended by Grimmer and Stewart (Reference Grimmer and Stewart2013), to check findings from an unsupervised topic model. Research assistants with a background in Chinese law decided whether each case represented a dispute between two private parties or between a private party and the government.Footnote ⁴⁰ Overall, 47% of cases ( $n=234$ ) were third-party disputes, whereas 40% ( $n=201$ ) of cases reflected disputes with the government. (In 13% of cases, we were not able to make a determination based on the text of the decision.) At first glance, these numbers are striking: third-party disputes are approaching a majority of administrative cases in our sample. This is a finding that would surprise even close observers of the Chinese legal system. Although there is a parallel trend in criminal law in which litigants sometimes use criminal sanctions to seek compensation in private disputes owing to difficulty enforcing civil judgments (Liebman Reference Liebman2015), existing work on Chinese administrative law has not been attentive to how administrative litigation can get piggybacked onto civil disputes.

VI. Accounting for Missing Data

How do missing data affect our analysis? On one hand, perhaps our data set is missing cases brought against the state owing to their political sensitivity. On the other hand, perhaps our data set contains only a partial view of third-party disputes because courts hold varying interpretations of what they are required to disclose or simply devote insufficient resources to ensuring cases are placed online. In order to buttress our finding and also illustrate how to work with incomplete data, we use three approaches to make sure third-party disputes remain a substantial portion of the docket even accounting for missing data.

As a first step, we look only at courts with well above average transparency to see whether the analysis changes. Within our random sample of 500 cases, 63 were from courts with higher than 65% transparency. In these high-transparency courts, our coding suggests that 38% of cases were third-party cases, 46% of cases represent an underlying dispute with the government, and 16% could not be determined from the text of the decision. Over one-third of administrative lawsuits are still third-party disputes even in high-transparency courts for which missing data are less of a problem.

Second, we draw bounds around our estimate of the prevalence of third-party disputes. From the analysis we described in the previous section, the corpus contains about 41% of all types of cases in 2014. The province-wide disclosure rate for administrative cases was higher than for the corpus as a whole: our data set includes 9,067 administrative cases from 2014, out of a total of 15,453 administrative cases that Henan courts reported resolving in 2014—a disclosure rate of 59% (Henan High People’s Court 2015). We therefore use this 59% transparency rate in administrative cases to bound our estimate. Assuming our estimate that 47% of all cases are third-party cases is correct, we can calculate the lower-bound proportion by assuming that all cases that are missing are not third-party cases. This would leave us with 28% of cases being third-party cases, still a substantial number. But if all missing cases were third-party cases, then a full 69% of all cases would be third-party cases. In either case, we believe that such a substantial portion of cases deserves more attention in the literature.

Finally, one of the authors traveled to China in 2018 to present our findings about the prevalence of third-party disputes to audiences that included Chinese judges and administrative law professors.Footnote ⁴¹ In cooperation with local universities, we held interactive forums in Beijing, Shanghai, Chengdu, and Changchun that closed with a specific request for feedback about whether listeners had noticed that administrative litigation often masks a civil dispute. Across all four cities, audience participants noted a close fit between our findings and their experiences. In particular, judges noted that third-party disputes are common. “We have a sense of administrativization of civil cases,” said a Beijing judge in a typical reaction to our slide deck. Although it is possible that audience members were disposed to politely agree with our findings, we take solace in the fact that at least some participants felt comfortable publicly expressing disagreement. In particular, a few participants pushed back against our interpretation of our findings and disputed whether third-party disputes are common or, as one judge in Beijing put it, only “a sprout.” But the majority of commentators agreed that the courts are often caught in the middle between private disputants and that judges have no option but to become involved even if they would prefer not to hear or resolve the case. One judge, expressing his frustration at being drawn into such cases, drew a link to the wider phenomenon of what Chinese judges call “abusive litigation” (滥用诉讼). One scholar noted that courts are taking steps to address the frequency of private litigation being transformed into administrative litigation and predicted that over time our data would begin to show a decline in the percentage of cases involving third-party disputes.

Regardless of the exact number of third-party cases, their prevalence in our corpus highlights the limits of conceptualizing administrative litigation primarily as a tool for angry people to petition a repressive or unresponsive state. To be sure, some cases certainly fit this mold. The topic model includes topics challenging reeducation-through-labor decisions (topic 1), appeals relating to petitioning (topic 2), claims by individuals detained for petitioning to Beijing (topic 28), and challenges to birth-planning fines (topic 50). However, the “citizens versus the state” frame flattens away a huge amount of complexity. Looking closely at our corpus makes plain that the aggregate number of administrative cases is a poor measure of citizen willingness to sue the state, even though observers of Chinese administrative law often treat it this way. In addition to third-party disputes, the prevalence of case strings shows how easy it is to count the same dispute numerous times. The huge number of documents related to court procedure—which are generally counted in official court tallies of administrative litigation—also suggests that the aggregate number of administrative lawsuits includes many mundane procedural decisions, such as remands for new trials (topics 17 and 18) and decisions related to statutes of limitations (topic 53) and jurisdiction (topic 52).

All of this shows how topic modeling can be a useful tool of discovery to surface underappreciated motifs in administrative litigation, even when working with incomplete data. The prevalence of third-party disputes, in particular, is not a theme that would necessarily emerge from examining national statistics, tracking the news, or reading scholarship. Rather, it reflects the vantage point of the data: the perspective of the lower courts responsible for processing the diverse claims that collectively constitute administrative law. This wide-angle view offers a concrete sense of how Chinese administrative judges actually spend their time and reminds us how much of administrative law is about leveraging state authority rather than challenging it.

VII. Conclusion

A new wave of digital scholarship on law and the courts is already underway, as researchers start to bring the tools of computational text analysis to bear on the texts core to legal scholarship. For many of us, this digital turn is exciting because of the ways growing digital archives can contribute to knowledge across disciplines. As collections of court decisions grow, for example, legal scholars will be able to trace evolving interpretations of concepts such as fault, causation, or damages or to examine patterns in how courts handle specific types of cases or parties. Political scientists can also deploy topic models to visualize the rise and fall of certain topics in court dockets and investigate the relationship to political priorities or legislative changes. At the same time, however, the millions of Chinese court decisions now online will not be the only corpus that is tantalizingly large and frustratingly incomplete. Digitizing documents offers an opportunity for censorship, and there is evidence that this happens. Selected legal academic articles from the 1950s, for example, no longer appear in China’s two leading online databases (Tiffert Reference Tiffert2019). And even when censorship does not occur, commercial databases have incentives to oversell the completeness of their product rather than account for what is missing. Our general call is for legal scholars to take the missingness problem seriously and consider following one of the three strategies detailed above, rather than succumbing to the temptation to treat even a very large-n sample as an accurate reflection of reality.

At the same time, taking missing data seriously does not mean giving up. Our investigation of administrative litigation shows how topic modeling opens a wide-angle perspective on courts’ daily activity, exposing unappreciated trends and also surfacing new questions for research. Answering the deeper “why” and “how” questions, though, will likely continue to require the type of information about local context that typically emerges from time on the ground, especially as court judgments provide only one, often limited, view of actual practice. Thus, we hope that the migration toward treating text as data in the field of law will also mean a surge of multimethod work that combines digital tools with interviews, participant observation, archival work, and close reading. After all, “computers amplify human abilities,” not replace them (Grimmer and Stewart Reference Grimmer and Stewart2013, 4). This also means that the best work is done in teams that combine skills and insights from law, the social sciences, and computer science.

Theoretically, the sudden availability of so many data from China’s courts raises questions that our field will grapple with for years to come. Overall, is the availability of so much information about court judgments changing the practice of Chinese law? Does publication of court judgments encourage certain types of cases or legal arguments? Does the imperative to post court decisions affect court dockets by making courts more or less willing to accept certain types of cases? Is greater transparency resulting in greater standardization and fairness in a legal system that has often been criticized as arbitrary and vulnerable to corruption? In this new environment, the list of possible research questions is nearly infinite, and the primary challenge is no longer obtaining data but rather effectively using the public record and prioritizing among research questions.