Mixed Methods: Machine Learning + Experimental Text Analysis + Close Reading

Understanding the literary, intellectual, and cultural history of ancient and medieval Chinese literature presents the traditional scholar with imposing challenges. The authenticity, authorship, and dates of composition of texts are often either unknown or widely contested (Loewe Reference Loewe1993). Furthermore, since many early Chinese texts are compilations of texts composed by different authors at different times put together by later editors, just what qualifies as a single text is debatable (Boltz Reference Boltz and Kern2007). Except for recently excavated manuscripts, most extant early Chinese texts are the products of scribal copying, censorship, redaction, loss of books, and other forms of textual corruption. These documents rarely received study independent of traditional commentary. On top of these concerns, the sheer size and complexity of the ancient and medieval Chinese corpus prevents any one individual from mastering all its texts.

To situate our method, we will distinguish between three approaches to texts. The first is distant reading, increasingly popular across the humanities due to contexts in which the size and complexity of a corpus precludes its mastery. Coined by Franco Moretti (Reference Moretti2000), the portmanteau “distant reading” refers to a method using computational tools to analyze texts and overcome these challenges. This makes distant reading a form of machine learning that leverages the power of programming to address canonical research questions in the humanities. Distant-reading and machine-learning methods compute relationships between texts, terms, and topics via mathematical algorithms rather than expert judgments. Distant-reading methods differ from experimental text analysis. In experimental text analysis, scholars code terms, classify synonyms, track associations between texts, or examine the contexts of keywords. These procedures occur in the context of the scientific method, but without the help of machine-learning algorithms that find patterns in texts. Formalizing interpretive procedures and testing specific hypotheses means that experimental text analysis takes a huge step toward the scientific study of literature. For example, Slingerland and Chudek (Reference Slingerland and Chudek2011) used expert coders to track changes to the meaning of xin 心 or “heart-mind” in pre-Qin texts; Clark and Winslett (Reference Clark and Winslett2011) used one expert coder to determine whether terms for high gods and deities co-occurred with terms for morality. Experimental text analysis formalizes scholars' interpretations of parts of texts, which separates this method from a third, traditional close reading. Close readings of texts by experts produce unparalleled insights into the meaning, subtlety, beauty, and power of historical texts in a way that neither of the other methods can hope to replicate.

Each method has its challenges. But suppose that our goal is to infer the meaning of a text from what it says? Here a mixed method combining elements of all three approaches stands head and shoulders above the individual methods as the most promising way forward. Experimental text analytics exclusively uses human coders to determine meanings from words and sentences. This may allow flexibility, but studies that rely on coders are subject to human error and bias. Traditional close reading faces a number of challenges in determining the meanings of texts from their sentences. These include in-group biases, fallacies, self-deception, cognitive limits (when corpora are large), and social pressures.Footnote ¹ Researchers have argued that close-reading methods also make very little cumulative progress in the understanding of a text, given that continual interpretive disagreement is a feature of the humanities (Dietrich Reference Dietrich2011). Yet machine learning and distant reading may provide a means of side-stepping some forms of human error and bias. They are not free of bias (Goldstone and Underwood Reference Goldstone and Underwood2014, 364); they cannot infer meaning from words without the help of area expertise earned through years of close reading, and the form of results in a topic model often means the data are difficult to interpret. Yet no method is better able to identify patterns that are often hidden from the view of scholars not because of scholarly bias but because these patterns only appear at scale, or involve word usage that does not typically catch the human eye.

In the present case, we have designed our study as taking the best from all three approaches. We start with a robust distant-reading and machine-learning method. This provides us data to work with. How do we interpret the data? Three of us are experts in ancient Chinese thought, so we interpret the topic models in light of many close readings of relevant texts. How then do we control for our own biases and foibles? Since we did not trust ourselves to deliver error-free interpretations, we enlisted about sixty other experts in ancient Chinese thought to independently interpret our topic models. This experimental text analysis work provides a validation check on our interpretations of the data.

We feature the machine-learning component of our method because no one has used it on a corpus as we have. This method converts words into data and uses algorithms to find patterns among those words and their relationships. At the root of this process is computation. To get a sense for what computation involves, consider the following example. If we are given an unordered, random list of whole numbers, we might map the variable larger than onto integers in order to compute the largest number in the list. This mapping is algorithmic. An algorithm is a bit-by-bit recipe for implementing a computation. Familiar processes like tying one's shoes and baking a cake are processes that can be described algorithmically. In these cases, the user of the algorithm identifies the data to which the algorithm will apply (the ingredients), writes a set of instructions that structure the iteration of a step-by-step process (the recipe), and has a specific outcome in mind (the cake). In other cases, algorithms are exploratory and used without this sort of supervision. For example, we might have no prior idea about how many prime numbers there are between 2,576 and 6,509,322. Despite not knowing the outcome in advance, we can still write an algorithm to give us this information.

This leads to three takeaway points for what follows. First, at the most basic level, our modeling activity represents a simple algorithmic mapping of the distance between character frequencies across sentences, chapters, and texts within our corpus. Second, just like the prime number example, we undertake this modeling activity without knowing what relationships between characters we will uncover. This is often described as an “unsupervised” analysis. Third, algorithms operate on diverse types of data, and the outcomes of computations are purely mathematical constructions. In other words, the meanings of the units of data—physical movements of an assembly robot, changes in velocity in an orbital reentry, Chinese characters—are irrelevant to the computation. Understanding the meanings of our data is left for experts in the area of inquiry.

Topic modeling has supplemented and invigorated a number of other humanities research areas, including history, philosophy, journalism, and literary studies. Literary and historical studies have benefited the most from topic modeling, as is apparent in the work and influence of Matthew Jockers and his remarkable study of nineteenth-century novels in the United Kingdom, Ireland, and America (Jockers Reference Jockers2013). He explores major themes and, having created sub-corpora at the level of national literature, often contrasts emergent themes in national corpora. For example, landlord-tenant relations become a significant topic in Irish novels while race becomes a significant topic in American novels. In history, Robert Nelson topic modeled the archives of the Richmond Daily Dispatch newspaper from November 1860 to December 1865 during the American Civil War. Nelson tracked changes in relationships between words about the Confederate military draft, fatalities, and patriotism by using the algorithm to compute a mapping of words to words and words to dates. Combining knowledge of dates of movements of the Union army, Nelson found that ads for fugitive slaves spiked on the two occasions when the Union army came closest to Richmond. These results work in harmony with research by historians by providing correlational evidence for a theory: a minority of civil war historians have argued for greater appreciation of the role of the Union army in the destabilization of slavery in the Confederate south, independent of the Emancipation Proclamation (Nelson Reference Nelson2015).

Asian studies has not yet caught up with other humanities areas in the use of topic modeling, though this may be changing (Chen et al. Reference Chen, Borovsky, Kawano and Chen2014; Hou and Frank Reference Hou and Frank2015). We hope that these results—and others' to follow—can inspire Asian studies researchers with concrete questions, or even testable hypotheses motivated by secondary literature. For example, using metadata about dates of texts, one might test a hypothesis that in the later Han dynasty topics associated with trade and commerce peak; or one might predict that the Yìjīng 易經 has had much more influence on Daoism than on Confucianism, and test that hypothesis by examining the relative weights of topics loading heaviest in Yìjīng with those loading heaviest in the set of Daoist or Confucian texts; or using metadata about dates, one might explore (rather than test) whether opinions in secondary literature about the relative dates of chapters of Shangshu can be confirmed on the basis of their linguistic similarity.Footnote ²

Here we apply a topic-modeling algorithm to a corpus of 5.74 million characters across ninety-six ancient and medieval Chinese works, including many of the most important texts in the tradition.Footnote ³ We selected this corpus because of the scope of the texts it includes, its accessibility, its familiarity, and its temporal breadth. The corpus spans several eras of historical Chinese literature. It includes the pre–Warring States Book of Poetry (Shījīng 詩經), the Warring States Dào Dé Jīng 道德經, the short treatise on philosophy of language Gōngsūnlóngzi 公孫龍子, the lengthy history text Hàn Shū 漢書, Han medical texts like Huángdì Nèijīng 黃帝內經, and pre-Qin encyclopedic texts like Lǚ Shì Chūnqiū 呂氏春秋. (See appendix 1, “Texts, Genres, and Dates,” for the complete list of texts and table 1 for era classifications of the corpus.)

Table 1.

Corpus composition by era.

Topic Modeling

We do not duplicate the comprehensive and friendly introductions to topic modeling for humanists already written (see Blei Reference Blei2012b; Mohr and Bogdanov Reference Mohr and Bogdanov2013; Underwood Reference Underwood2012a; Weingart Reference Weingart2012). Yet we see broad benefits in directly providing researchers across subfields of Asian studies with hands-on knowledge of the topic-modeling process, since many scholars of texts of any kind will soon benefit from—or need to acquire—the ability to interpret topic models in their research area.

Topic modeling was developed for search and retrieval in large collections of text-heavy data, but topic models efficiently sum, visualize, and explore the semantics of any kind of text corpus. Words are assigned to topics based on their tendency to co-occur in texts with other terms found in the topic.Footnote ⁴ For topic modeling we use an algorithm that maps, that is, computes probabilistic values for, the relations amongst all the terms in the corpus to all the other terms in the corpus. Through this process, the model extracts topics in large collections of documents. The model is probabilistic because the topics consist of words that have a high probability of occurring together in documents (Blei, Ng, and Jordan Reference Blei, Ng and Jordan2003). The model is generative because topics are formulated from latent relationships amongst terms in documents. Unlike an algorithm for tying one's shoes, but like an algorithm for discovering a set of unknown prime numbers, our model works without supervision. This means that the algorithm discovers the topics without its being fed prior knowledge about genre or date or any other information about the texts. The upshot is that before seeing the results, we do not know what the topics will be or which topic will have the biggest representation in the corpus.

Topic models produce several different types of data, including word weights, corpus weights, and text weights. In practice, many digital humanities papers using topic modeling neglect much of these data in preference for focusing on the resulting topics. Since we attempt to exploit the full range of these data to address our research question about the relationships between Analects, Mencius, and Xunzi, we now introduce these types of data. The number of times the topic-modeling algorithm has assigned a given term to the topic determines its word weight. “Weight” in this context refers to the relative size of the contribution that a word makes in a topic. The centrality of the word (or character or term or glyph) to the topic can be determined by rank-ordering the word weights. Topics are customarily split into short lists of the top-ranked words, which we refer to as the topic's keywords. Keywords serve as a metonym for the long list of words in the entire topic. The plot for Topic 29, included in figure 1, shows the character tiān 天 (heaven or God) as having the largest weight of any keyword in that topic. In this context, the large word weight of tiān results from its nearly 12,000 occurrences in Topic 29.

Figure 1.

Keyword loading in highest weighted ten topics in our corpus. Individual plots in this figure represent the distribution of heaviest keywords within the target topic. Characters along the horizontal axis represent central characters in the target topic, with the most central character nearest the vertical axis. The numbers along the vertical axis represent the number of occurrences of each character. Typically keyword weights approximate a discrete power law distribution, with the weights being inversely proportionate to the keyword's rank for any given topic. This describes Topic 21 because mín 民 (people) has nearly four times the word weight as 21's second-ranked character jūn 君 (prince). Contrast Topic 23, which is almost linearly distributed.

Next is corpus weight. Table 2 and figure 2 illustrate a topic's weight in the corpus, or corpus weight. A topic's corpus weight is the ratio of the sum of words in a topic over the total number of words in the corpus. Corpus weights are not standardized (ours sum to 14.9) because they are based on the total occurrence of words within each topic. Instead of representing the weights of individual characters, table 2 depicts keywords (in the order of their word weight within the topic) along with the corpus weight of the topic. Table 2 represents our topic model's findings as to the ten most weighty themes in ancient and medieval Chinese writing. Figure 2 visualizes the corpus weights of all 100 topics in our model.

Figure 2.

Corpus weights for Topics 0–99.Footnote ⁵

Table 2.

Highest weighted ten topics in the corpus.

The third and final type of data produced by a topic model is the text weight. This term refers to the proportion of a text's vocabulary that is assigned to a given topic, which represents how saturated a text is by a topic. Text weights are normalized and sum to 1. In each text in the corpus, some of the 100 total topics will have greater representation than others. For example, in Xunzi experts would expect that topics having to do with ritual matters will have bigger representation, and so larger text weights, as compared to Mencius.

Let us illustrate text weight, word weight, and corpus weight. Consider the topic that has the heaviest corpus weight in Xunzi, Topic 34, which we call “Ethical Rulership.” First, Topic 34 has a text weight of 0.256 in Xunzi. In contrast, its text weight in Analects is only 0.043 and half of that in Mencius at 0.023. This alone represents a discovery in terms of our research question, since the distribution of Topic 34 into Xunzi is six times greater than its distribution in Analects and eleven times greater than in Mencius. This warrants a practical inference for scholars of ancient Chinese documents, namely, Topic 34 sets Xunzi apart from Mencius.

To understand the significance of this discovery, we turn to look at the characters in Topic 34 and information about them. (See table 3 for the keywords and word weights of Topic 34.) To avoid misapprehending topic model results, it is important to understand information about characters that make up topics. Person (rén 人) has 219 occurrences in Analects, 611 in Mencius, and 1,241 in Xunzi. Frequencies of terms are often relevant to answer research questions, but for purposes of comparison the use of frequencies neglects a couple of issues. Mencius is 2.3 times the size of Analects, and Xunzi is 5.3 times the size of Analects, facts that hamper one's ability to interpret semantic importance from character frequencies alone. Zipf's law has the same effect (Zipf Reference Zipf1949). Zipf's law states that in any given text in a natural language, a word's frequency is inversely proportional to its rank in the corpus. This means that, in a given text, the most frequent word is typically twice as frequent as the second most frequent word, three times as frequent as the third most frequent word, and so forth. A better way of understanding the importance of a character in a set of texts is to examine its rank within and across the texts, and to look at its rate of occurrence per 1,000 characters. Raw frequencies do not disclose that, once common stopwords are removed (see below), rén is the most frequent character in each of Analects, Mencius, and Xunzi, and has a rate of occurrence per 1,000 characters of 28.7, 34.6, and 30.6, respectively. To understand the importance of a character in a topic rather than in a text, however, we must consult its word weight (see table 3, column 3). By doing so, for example, we see that with a word weight of 0.037, nobleman (jūn 君, occurring for example in jūnzǐ 君子) is three times as important to Topic 34 as is peace (ān 安). The algorithmic mapping at the heart of topic modeling allows us to go beyond information about simple frequencies to discover much more robust and reliable relationships between terms and texts.

Table 3.

Word weight & character-level data for Topic 34.

Corpus weight is not a helpful statistic unless a topic's corpus weight is put in comparison with others. The corpus weight of Topic 34 is 0.375. Of 100 topics in our model, this is a very large corpus weight, ranking Topic 34's corpus weight eighth of 100 topics (see table 3). This fact justifies the inference that, despite the fact that it was not nearly as representative in Mencius, Xunzi’s particular focus on ethical rulership is well-distributed in the corpus.Footnote ⁶

While topic models are a form of unsupervised machine learning, human decisions play some role in what topics are generated. At many junctures, we pooled our expertise in programming, in preprocessing, and in classical Chinese language and thought to make decisions that influence the quality of the topics generated by the algorithm. The effects of the subset of decisions that are made prior to the application of a topic-modeling algorithm to a corpus is referred to as preprocessing. Due to the nature of our texts in Chinese, we removed punctuation, tokenized, and applied a stopword list. Classical Chinese manuscripts do not include much punctuation at all, but the Chinese Text Project (CTP) texts include punctuation. Therefore we removed all but sentence-ending punctuation from the corpus. Tokenization refers to the management of word boundaries. We used a procedure that rendered each character separated by spaces before and after from every other character. This allowed us to treat each character as a unit of semantic meaning.

In a second preprocessing step, we used experts' knowledge to generate a stopword list. A stopword is a high-frequency word that tends to be highly ranked in topics but that also tends to make the topics less valuable for interpretation. Stopwords typically consist of common function words. Applying a stopword list means removing those common characters from the corpus prior to analysis. Examples of terms on our stopword list are zhī 之, a grammatical term used as a pronoun and subordination particle, and yě 也, a grammatical particle used to indicate noun predication (among other things). These and other stopwords were removed because during a series of pilot studies those words tended to blur the semantic coherence of topics. Applying a stopword list is standard procedure in topic modeling. We provide a full list of stopwords used in this study in appendix 2, “Stopwords.”

In a third preprocessing step, we encountered problems with the software to implement our topic-modeling algorithm because that software was not designed to handle all the Chinese characters in our corpus. We scripted a method of encoding our input and decoding our output that allowed us to work around that problem.Footnote ⁷ Following common practice using LDA on texts, we did not chunk or split the texts in our corpus for analysis.

Moving from preprocessing to processing, the most important decision is the number of topics chosen to model. Too few topics may combine semantically unrelated material into so-called chimera topics; too many may cause related material to split into separate topics, redundancy between topics, or accumulation of irrelevant “junk” topics (Schmidt Reference Schmidt2012). Topic quality is typically determined by semantic coherence of the keywords in the topic. Although significant strides have been taken in algorithmic determination of the ideal number of topics (Marshall Reference Marshall2013), the assessment of topic coherence is typically a product of the scholar's interpretation. There is ongoing discussion in digital humanities scholarship over the interpretive significance of topics—whether they constitute subjects, themes, or discourses—and topic models do not always produce topics that appear semantically coherent to the scholar (Underwood Reference Underwood2012b). To the extent that text corpora are composed of figurative language, such as that found in poetry (our corpus contains poetry), topic models produce higher rates of apparently incoherent topics (Rhody Reference Rhody2012). After experimenting with a number of models in several pilot studies using different numbers of topics, we settled on 100 topics, which, after the removal of stopwords, seemed to offer a good balance of scope and granularity while yielding few junk or chimera topics.

After the topics are generated, researchers are faced with interpreting them and their relations to texts in the corpus. Some scholars in the field of ancient Chinese thought have argued that contemporary interpretations of ancient Chinese documents, especially philosophical, political, and religious documents, fall victim to debilitating biases and errors, for example, either Orientalizing or Westernizing the texts (Ames Reference Ames and Hua2001). Since the texts were canonized long ago, a commentarial tradition two millennia long continues to structure the (presumed) central themes of the early Chinese source texts. But this tradition makes assumptions that are open to reexamination. Topic modeling has the potential to reveal the unexpected and even challenge canonical claims about themes and contents of these texts, opening up new avenues for our understanding of ancient and medieval Chinese thought.

At the same time that our results may challenge leading interpretations of certain texts, we are well aware that our interpretations of the topics may be subject to biases of which we ourselves are unaware. Since three of the six of us publish actively in early Chinese thought, we aimed to minimize scholarly biases of our own that, unbeknownst to us, might influence our interpretations of our topics. For this reason, we decided that interpretation of our topics should be informed by independent expert knowledge in historical Chinese thought and language. So we enlisted the help of over sixty experts in the field recruited from the Warp, Weft, and Way blog (http://warpweftandway.com) to independently code topics. We refer to these results frequently in what follows to demonstrate a partial validation of our interpretations. This process worked as follows.

Expert coders were presented with word clouds showing a target topic's keywords. First they were given an open-ended question reading, “Suppose you had to guess what is the theme of this word cloud. What are one to three English words you would use to describe this theme?” Second, experts were asked how confident they were about their judgment in the open-ended question. Third, experts received a forced-choice question with answers enabling us to probe their opinions about the contents of these topics. In response, they could inform us that the topic was about the military, politics, philosophy, the mind, etc. Due to the likelihood of chimera or junk topics, and limitations among our experts, we included an option of “uncategorizable” as well. Fourth and finally, if an expert coder responded to a top-level multiple-choice question by saying that, in his or her opinion, the topic was about military affairs, he or she would receive a supplemental forced-choice question inquiring whether the topic represented issues including weaponry, peace, the state, war, violence, order, and/or government. Experts were always able to select multiple answers. These three levels of answers allowed us to use the expertise of generous volunteers knowledgeable about ancient and medieval Chinese thought to partially confirm or contest our interpretations of specific topics. (See appendix 3, “Survey Given Independent Coders,” for the survey text and an example word cloud.)

To take an example from our own corpus, consider Topic 27 in table 4. Traditional scholars skeptical of our methods may think that a topic as incoherent as 27 is evidence that our method is of little assistance in answering research questions about early Chinese thought. However, to experts of Warring States philosophical discourse on logic and language associated with Later Mohists and the School of Names, this topic makes perfect sense. These logicians focused on problems of reference (zhǐ 指) and how words are related to “things” (wù 物). They wanted to know whether a “white horse” (báimǎ 白馬) is a “horse,” a famous example, and how attributes such as “hard and white” (jiān bái 堅白) relate to substances. Such was our initial interpretation, but to minimize our own bias and error, we took additional steps. We partially confirmed this interpretation of Topic 27 by reviewing its text weights in specific texts to determine in which documents the weight of Topic 27 is heaviest. The fact that its heaviest topic weight is in the School of Names text Gōngsūnlóngzi provides further justification of our interpretation. We then examined responses from our independent expert coders to determine whether their interpretations were supportive of our “Logic and Language” interpretation. One of three experts assigned Topic 27, presumably not as knowledgeable about philosophical materials in our corpus as about other materials, did not understand this topic. This was revealed in his or her answer to the top-level forced-choice question, which was “uncategorizable.” The other two coders agreed that it was a coherent topic. Furthermore, these two knew precisely what this topic was about. In open-ended questions, they reported that Topic 27 concerned “logicians, philosophy,” “disputation,” and “appearance, language.” This too provides further justification of our interpretation.

The foregoing discussion about how we arrived at our interpretation of Topic 27 provides a self-contained illustration of the mixed methods we champion in this article: our close-reading knowledge of the Later Mohists and School of Names prompted our initial understanding of the topic; our machine-learning outputs revealed that Topic 27 was heavily represented in just the texts that we would hypothesize it to be; and the experimental text analysis that enlisted our experts' opinions in the process further confirmed the interpretation.

Table 4.

Topic 27 keywords and weights.

What Topics Make Analects, Mencius, and Xunzi Each Unique?

Longstanding debate surrounds the relationship between Confucius of Analects and his two declared successors, Mencius and Xunzi (Lau [1970] Reference Lau2005; Van Norden Reference Van Norden1992). The notion that it is Mencius, rather than Xunzi, which is the true inheritor of the teachings of Confucius contained in Analects has deep roots in the late imperial Chinese commentarial tradition. (For the distribution of all topics in our model across these three texts, see figure 3.)

Figure 3.

Text weights in Analects, Mencius, and Xunzi across the corpus.

Tang dynasty scholar Han Yu (768–824) first asserted that authentic transmission of the teachings of Confucius ended with Mencius. The point was reiterated by Song dynasty (960–1279) neo-Confucians and canonized by Zhu Xi's (1130–1200) inclusion of Mencius in the collection of Confucian texts referred to as Four Books—along with Analects, Great Learning (Dà xué 大學), and Doctrine of the Mean (Zhōng yōng 中庸). The Four Books were a central part of the core curriculum memorized by students and examination candidates from the early 1300s to the abolition of the examination system in 1905. Xú Fùguān 徐復觀 (1904–82) reinforced the traditional idea that when Confucius speaks of “human nature” he is expressing the same idea that Mencius later formulated, namely, that human nature is good (Xú Reference Fuguan1969, 89; see also Zhang Reference Zhang2012, 197). Following in Xú’s footsteps, influential contemporary scholar Fù Pèiróng 傅佩榮 (1950–) argues that Mencius's theory of the potential for goodness latent in human nature “is an excellent expression of Confucius's thought” (Fù Reference Pèiróng2011, 872).

Others argue that this traditional interpretation is problematic and that analysis of the language of self-cultivation, including craft metaphors, indicates closer affinities between Analects and Xunzi. If human nature is like a raw piece of jade, values have to be carved into it by an outside force (Analects 1.15, tr. adapted from Slingerland Reference Slingerland2003, 6–7; Xunzi 27.514–523, tr. Hutton Reference Hutton2014, 309). Knowledge of normative values is not innately present in human nature; it has to come from an external source. In contrast, Mencius contains an internalist theory that assumes normative values to be innate. Human nature is a seed with the potential to grow into a fully developed plant (Ivanhoe [1993] Reference Ivanhoe2000, Reference Ivanhoe, Littlejohn and Chandler2008; Slingerland Reference Slingerland2003).

Our guiding research question in this section is “How do our topic models describe conceptual and linguistic differences that set each of these texts apart from the other two?”Footnote ⁸ So, which topics do we select for analysis in order to initiate a new conversation about this canonical issue? We look at the top ten topics in each document. We begin by looking at unique topics, those that show up in one text's top ten topics, but not in the other texts' sets of top ten topics (see table 5). In other words, in this section we discuss only topics that render each of these texts unique and different from one another. Given our results, this means we discuss Topics 5 and 82 in Analects; 10, 99, and 86 in Mencius; and 23, 71, and 17 in Xunzi. In the following section, we discuss those topics that our texts share in common with one another.

Table 5.

Highest weighted ten topics in each of Analects, Mencius, and Xunzi.

Analects

We focus first on Analects, which is a collection of sayings attributed to Confucius (551–479 BCE) and his followers and contains material likely dating predominantly to the early Warring States.Footnote ⁹ Two topics in the top ten differentiate Analects from other texts, including other texts within Confucianism. These are Topic 5, with a text weight in Analects of 0.038, which we label “Sacrifice, Ritual, Etiquette,” and 82, with a text weight of 0.029, which we call “Rulers, Virtue and Governing the People.” Since the text weight of Topic 5 in Analects is 0.038, this means that 3.8 percent of Analects is composed of the clustered terms representing Topic 5 (see table 6).

Table 6.

Topics differentiating Analects, Mencius, and Xunzi from one another.

Keyword characters in Topic 5 include great (dà 大), sacrifice (jì 祭), feed or eat (shí 食), gate or school (mén 門), wife (fù 婦), first or before (xiān 先), enter (rù 入), submit or ritual garb (fú 服), exit or go out (chū 出), drink (yǐn 飲), assist or assist someone (xiàng/xiāng 相), and weep or cry (kū 哭). These terms describe a semantic space revolving around important rituals and sacrifices, particularly those involved in ancestor worship and mourning. Independent coders reported that this topic concerned ritual and religion. Topic 5 has heavy text weight in only a handful of the texts in the corpus, including in The Classic of Rites (Lǐjì 禮記, 0.175, and Yílǐ 儀禮, 0.170) and The Rites of Zhou (Zhōulǐ 周禮, 0.052), which contains the core of the Book of Changes or Yìjīng (see table 7). Together these three texts form a unit known in the Chinese commentarial tradition as the Three Ritual Texts (Sānlǐ 三禮). The bulk of each of these works consists of long lists of ritual prescriptions specifying the correct way of executing various rites and sacrifices, for example, specifying which clothes to wear and which color of accouterments to use. In form and content, parts of the Analects, especially Chapter 10, are strikingly similar to the Three Ritual Texts. This is a distinctive feature of Analects in comparison to Mencius and Xunzi. The fact that unsupervised topic modeling is able to pinpoint this scholarly insight powerfully demonstrates the value of this new research tool.

The results of Topic 5 appear to have important implications in the adjudication of an ongoing debate about the role of sacrifices and spirits in Analects. Consider the opinion of a key voice in Chinese intellectual history about Confucius, spirits, and sacrifices. Feng Youlan 馮友蘭 (Reference Youlan and Yu-Lan1952–53, 1:58) uses a close-reading method to conclude that Confucius “displayed a rationalist attitude [toward spirits], making it probable that there were other superstitions of his time in which he did not believe.” In contrast, Thomas Wilson uses a close-reading method to emphasize Analects' advocacy of ritual, sacrificial rituals to ancestors and deities in particular. Wilson (Reference Wilson2014, 185) reasons that “contrary to modern accounts, imperial-era commentaries on the Analects 論語 disclose the figure of Confucius as committed to pious sacrifices to gods and spirits.” Unlike Xunzi, who explicitly reports his intentions to endorse the use of sacrifice for social-functional reasons (chap. 19, “Discourse on Ritual”; see Campany Reference Campany, Reynolds and Tracy1992), the text of the Analects leaves readers uncertain with regard to Confucius's intentions about sacrifice. For this reason, debate about Confucius's relation to sacrifice will not be easily settled by topic modeling or by close reading alone. Feng Youlan cites Analects 7.20 to argue for Confucius's pragmatic epistemology, and Wilson cites Analects 3.6 to demonstrate Confucius's concern with Mount Tai's sacredness and ritual importance; Feng Youlan cites 6.22 showing that Confucius keeps ghosts and spirits at a distance and prioritizes social harmony, not metaphysics, and Wilson cites 3.12 to argue for Confucius's earnestness during sacrifices to the spirits. Perhaps the process of tit-for-tat close-reading commentary will continue ad infinitum.

But machine-learning results from topic modeling provide two reasons to think Wilson is likely correct. First, numbers of scholars argue that belief in gods in early China had prudential, not rational, origins. Prudential concerns arose through divination and knowing the future (Overmyer et al. Reference Overmyer, Keightley, Shaughnessy, Cook and Harper1995), ancestor reverence and seeking ancestors' blessings (Eno Reference Eno1990a, Reference Eno1990b), and avoiding curses through shamanism (Ching Reference Ching1997). To this, however, advocates of the alternative view will, as we have seen, return to the discussion with additional texts and interpretations, and the two sides will continue to trade texts in support of two mutually incompatible interpretations of Analects into the indefinite future. This brings us to what our model can contribute to consilience. Second, our interpretation of Topic 5 offers evidence in favor of the unique importance of practices associated with these sources of religion, especially religious ritual and sacrifice (jì 祭), for the compilers of the received Analects. If the compilers of Analects were as rationalist as, say, Xunzi, we would not expect to see Topic 5 so prominently, and uniquely, featured in this text. Were Feng Youlan correct, Topic 5 would probably not differentiate Analects from the other two texts.

Topic 82, “Rulers, Virtue and Governing the People,” is a reflection of the fact that numerous passages in Analects discuss the importance of the charismatic virtue (dé 德) of rulers, dukes (gōng 公), and kings (wáng 王) as they govern (zhèng 政) the people (mín 民). Rulers are advised to employ officials with virtue (dé 德) and ability (néng 能) to serve (shì 事) them by bringing affairs (shì 事) to completion (chéng 成). Independent coders reported in open-ended questions that this topic concerns “history, statecraft, philosophy” and “civil-affairs, reports, officialdom.” Turning to the word ranks of its keywords across the three target texts, we see governance or order (zhèng 政) is much more important in Analects (forty-three occurrences, thirty-first in rank, 5.6/1000 characters) than to authors of Mencius (fifty-four occurrences, fifty-seventh in rank, 3.0/1000) and Xunzi (ninety-five occurrences, eighty-fifth in rank, 2.3/1000). Topic 82 is the heaviest weighted topic in Guoyu 國語, which is a collection of historiographical and fictional anecdotes set in the pre-Qin period. Many of these anecdotes feature professional persuaders or diplomats who use their command of language to persuade rulers to “do the right thing.”

Table 7.

Topic 5's text weights across texts in the corpus.

Topics 5 and 82 represent themes that are unique to Analects and not shared with Mencius or Xunzi. The prevalence of Topic 5, “Sacrifice, Ritual, Etiquette,” and Topic 82, “Rulers, Virtue and Governing the People,” confirms the scholarly consensus that Confucius (as he is portrayed in Analects) projected social leadership emphasizing the importance of ritual and sacrifice as elements in individual self-cultivation practice and in achieving social order. The social-functional values enshrined in this tradition could be transmitted to disciples and followers through teaching, observation, and emulation. In many reported conversations with rulers, these same values were transmitted through advice on how to govern a state through virtue rather than laws and military force. These topics focus on institutionalized ritual practice and do not reveal much concern with cognition or emotion or other internal states of mind.

Intersecting Topics in Analects, Mencius, and Xunzi

We are driven to understand the semantic content and relationships between these three texts. The previous section was intended to provide an understanding of what makes these texts different from one another. Those discussions combine with the discussion in this section, which is about what makes these texts similar to one another, to preliminarily address our guiding research question about whether the contents of Mencius or Xunzi most resemble the contents of Analects. Using the same list of the ten most weighty topics in each of our three texts, we calculated the topic intersections between documents as a shared set of topics.Footnote ¹¹ Now we focus on topics that load into pairwise unions of these three texts in an effort to explore their similarities. Before discussing those topics, however, we briefly report on the four topics that lie at the union of all three texts (see table 8 and figure 4).

Figure 4.

Topic intersections in Analects, Mencius, and Xunzi. Topic intersections of the ten most central topics for each document. Circles represent the presence of a topic within the ten most central topics. Circle size is proportional to the document's centrality (topic weight). Links (vertical lines) indicate an intersection.

Table 8.

Formal interpretation matrix of the intersections of Analects, Mencius, and Xunzi with topic keywords (∩ = intersection of sets).

At the intersection of Analects, Mencius, and Xunzi are Topics 29, “Heaven, Earth, Man and the Way”; 76, “Rulers, Ability, Knowledge”; 21, “Political and Social Order”; and 63, “Ritual, Family, and Governance.” These topics possess some of the largest corpus weights of all 100 topics in our model. Topic 29 is ranked number one, Topic 76 is ranked number three, Topic 21 is ranked number four, and Topic 63 is ranked number twenty-nine. Our expert coders report that Topic 29 is concerned with “cosmology, virtue” and “cosmology, time, philosophy.” They report that Topic 76 is concerned with “lordly leadership” and “politics and leadership.” Topic 21 is concerned with “governance, kingdom, people” and “people, masses,” and Topic 63 with “ritual, masters of ritual (rú),” and “ritual, rites, ceremonies.”

This information supports two major inferences about shared semantic content in these texts. First, these heavy corpus weights provide strong evidence that topics at the heart of early Confucianism are exceptionally well seeded throughout ancient and medieval Chinese literature. When we compare topics weighty in these texts with topics weighty in what are traditionally referred to as “Daoist,” “Legalist,” and “Mohist” texts, we find that those loading into Confucian texts are much, much more likely to be represented with heavy corpus weights.Footnote ¹² Second, consider that Mencius and Xunzi both self-identified as masters of ritual (rú 儒), who followed in Confucius's rú footsteps. The fact that Analects, Mencius, and Xunzi have a shared interest in Topics 29 (“Heaven, Earth, Man and the Way”), 76 (“Rulers, Ability, Knowledge”), 21 (“Political and Social Order”), and 63 (“Ritual, Family and Governance”) indicates that the pre-Qin concept of rú referred to a set of philosophies characterized by a high degree of internal coherence. Earlier we noted that Topic 86 separated Mencius from other texts by virtue of its internalist perspective about moral motivation and normativity. We contrasted internal moral motivation with external motivation, which we associated with ritual and law. Here, however, we see that Mencius (0.027) and Xunzi (0.025) share in Topic 63, “Ritual, Family and Governance,” to the same degree. This topic is fronted by ritual or rites (lǐ 禮, word weight = 0.042), which is why it is strong in Analects (0.069). This apparent conflict in our interpretation can perhaps be resolved by keeping in mind that Mencius's internalist stance naturally made him less interested in discussing the external tradition embodied in the rituals and rites, but this does not mean that he dismissed them altogether.

To appreciate how results at the intersection of all three texts might inform current debate, let us continue examination of Topic 63 in light of some secondary literature. Topic 63 does not prominently feature moral terms. The difference between Topic 34 and Topic 63 allows us to grasp a subtle but important difference in the scope of the shared social ideals across the three books. Topic 34 sees the virtue of duty or right action (yì 義, word weight = 0.031) traveling together with terms connoting high social status like lord or nobleman (jūn 君, word weight = 0.037) and rituals (lǐ 禮, word weight = 0.022). This informs our understanding of what Brindley (Reference Brindley2009) has called the “sociology of the junzi.” Specifically, considerations about how the distribution of Topics 63 and 34 differs between the three books can add considerable subtlety to this scholarly debate. In contrast to Mencius’s appeal to internal states (see discussion of Topic 86 above), Xunzi appears to link rites that accord high social status and certain moral virtues. Yì 義, with its connotations with animal sacrifices to gods, not benevolence, has a particularly heavy word weight in Topic 34. Topic 34 is represented in Xunzi at five times the level and Analects at twice the level it is represented in Mencius.

This suggests an interpretive hypothesis worth exploring through traditional scholarship. Some scholars, including Ivanhoe (Reference Ivanhoe, Littlejohn and Chandler2008, 5), argue that the Confucian “ethical ideal” is “something anyone can achieve and a way of being human that can be manifested in a wide range of social roles.” Others concur (Hsu Reference Hsu [Xu]1977, 162; Wills Reference Wills2012, 25). However, Brindley (Reference Brindley2009), echoing Hall and Ames (Reference Hall and Ames1987, 188), argues with some force that achievement of the status of gentleman or nobleman (jūnzi 君子) is restricted to high-status males, or males who are entitled to perform certain rites.

While we concur with Brindley on this matter, our topic-modeling results suggest value in further research on two pleasingly concrete questions: First, is the jūnzi ideal preferentially associated with ritual and the virtue of duty or righteousness, rather than other virtues like benevolence (rén 仁)? An affirmative answer is suggested by an analysis of Topic 63 and the word weights of its keywords. Second, consider that Brindley (Reference Brindley2009) states in the title of her paper that her thesis is restricted to Analects. When that restriction is accompanied by claims about “Confucian” morality, it can sow confusion. This leads to another research question: Might there be significant cross-textual variety in early Confucian texts' association of the jūnzi ideal with high social status? Topic 63's text weights in Analects, Mencius, and Xunzi place it in each text's top ten, but its distribution in Analects is twice that of its distribution in Mencius and Xunzi (see table 5). This raises the probability that Brindley and Ivanhoe are talking past one another, which is easy to do using exclusively close-reading methods. Could it be that Brindley emphasizes what is true but only of Analects, while Ivanhoe emphasizes what is true of Mencius and Xunzi but not Analects? On the strength of our results, we surmise this is likely to be true. More important, our interpretation of the results suggests value in pursuing a concrete research question with close-reading methods to narrow in on an answer.

We now move from our brief review of topics at the intersection of all three texts to discussion of those topics that intersect only in pairwise fashion. The topics at the intersection of Analects and Xunzi include Topics 34, “Ethical Rulership,” and 78, “Learning and Governance.” The corpus weights of Topics 34 and 78 rank number eight and number nine of 100 total topics, suggesting their wide influence. Independent coders report that Topic 34 is concerned with “subject-ruler relations” and “virtue, politics,” and that Topic 78 is concerned with “governance, learning, talent” and “human, knowledge, culture.”

Topic 34 has a heavy text weight in Xunzi (0.256) and a moderate weight in Analects (0.043) but a very low weight in Mencius (0.023). Why might Topic 34, on “Ethical Rulership,” load lightly in Mencius? Moral leadership is a common theme in that text, after all. Investigating heavyweight characters in Topic 34, Analects and Xunzi represent rites (lǐ 禮) at very similar rates (9.8/1000 and 8.5/1000 respectively). But the rate of lǐ in Mencius falls far below this (3.8/1000). Since in Analects and Xunzi human nature (xìng 性) is initially ignorant of normative values, these texts recommend use of the rituals and rites (lǐ) to build morally refined gentlemen (jūnzǐ 君子) who possess traits such as right action (yì 義), worthiness (xián 賢), and goodness (shàn 善). With rites and rituals, the Confucianism of Analects and Xunzi says that the nobleman orders himself and leads a state that is neither chaotic (luàn 亂) nor characterized by widespread badness (è 惡), but rather at peace (ān 安). While not unimportant, rites play a much less significant role in the philosophy of Mencius, since the Mencian strategy of self-cultivation is directed at motivating normative behavior through appeal to and training of the innate sprouts of virtue (Ivanhoe [1993] Reference Ivanhoe2000). Topic 34's emphasis on rites in Analects and Xunzi is what we expect to observe given the “internalist” morality represented in Topic 86, which had a heavy topic weight in Mencius.

Topic 78, “Learning and Governance,” contains a few concepts, including wén 文 (pattern; patterned civility, high culture), emphasized in Analects and Xunzi but not Mencius. Learning (xué 學) and knowledge (zhī 知) have higher saturation in Analects and Xunzi. Their presence in Topic 78 suggests that keys to rulership involve cognitive preparation of the mind for rule or management (zhì 治). This contrasts with the model of rulership discussed in Mencius Books 1 and 2, in which kings are challenged to deeper levels of empathy and emotion. The contents of Topic 78 raise the probability that Analects and Xunzi are semantically linked by virtue of their advocacy of a set of normative values deriving from learning (xué) and an external, refined (wén) tradition. The sizeable text weights of Topic 78 in Analects (0.074) and Xunzi (0.084) provide counter-evidence to the claim that differences between Mencius and Xunzi on the contents of human nature are merely a matter of emphasis rather than the result of different views of the moral resources located within the individual.Footnote ¹³

In terms of differentiating the influence of Analects on Mencius and on Xunzi, evidence weighs in favor of greater discursive overlap between Analects and Xunzi. This appears to reduce the probability that the traditional theory about Mencius’s closer relation to Analects is correct. But one might suspect that consideration of topics at the intersection of Analects and Mencius will increase the justification of a closer relation between Analects and Mencius. At this intersection we have Topics 61, “Analects Stylistics,” and 33, “Knowledge, Rulership and Heaven.” Both Topics 61 and 33 occur in the top ten largest loading topics in Analects (0.307 and 0.026) and Mencius (0.121 and 0.122). Yet note they also both fall within the top thirteen topics of Xunzi (0.020 and 0.021). From this we draw the important inference that no heavy loading topics falling at the intersection of Analects and Mencius successfully differentiate their semantic contents from Xunzi’s contents. This alone increases the probability that Xunzi tracks the semantic contents of Analects more closely than does Mencius, all things considered.

Ranked sixty-fifth, with a corpus weight of 0.077, Topic 33 is not commonly represented in the corpus. We call this topic “Knowledge, Rulership and Heaven.” Independent coders report that Topic 33 is concerned with “heaven, knowledge,” “ruling,” and “kingship.” Three of three independent coders in forced-choice questions report that Topic 33 concerns leadership, though kingship and statecraft were also regarded as important. The dominant keywords in this topic are people (rén 人), big or great (dà 大), heaven (tiān 天), know (zhī 知), rulership (wáng 王), thinking (xīn 心), and right action (yì 義). Examining character frequencies, we find that terms from this topic often appear at similar rates in Xunzi and Mencius, and dissimilar rates in Analects. For example, 心 heart-mind is the fourteenth most frequent term in Mencius (126 occurrences, 7.1/1000 characters) and thirty-second in Xunzi (168 occurrences, 4.1/1000) but only the 255th in Analects (six occurrences, 0.78/1000). This is so despite the fact that Topic 33 sits at the intersection of Mencius and Analects but not all three texts. Right action (義) is the thirteenth most frequent term in Xunzi (315 total, 7.8/1000 characters) and the twenty-fifth in Mencius (107 occurrences, 6.0/1000) but sixty-third in Analects (twenty-four occurrences, 3.1/1000). The effect of these character level data is to raise doubts about Topic 33's ability to pull Analects and Mencius together and away from Xunzi.

Topic 61 ranks twenty-ninth in the corpus with a corpus weight of 0.188. Topic 61, “Analects Stylistics,” contains keywords including Confucius's name (kǒng 孔), as well as three other characters used in names of followers of Confucius (zhòng 仲, lù 路, and gòng 貢). We infer that Topic 61 represents linguistic features of Analects' and Mencius’s literary style, particularly dialogic prose. This explains why it is prominent neither in Xunzi nor in the corpus as a whole. While Analects and Mencius consist mostly of reported dialogues, Xunzi contains lengthy essays. In sum, although Topics 33 and 61 feature among the top ten topics in Analects and Mencius (Topic 61 is number thirteen and Topic 33 is number twelve in Xunzi), Xunzi also contains high word frequencies of keywords found in Topics 33 and 61. Examination of topics at the intersection of our three texts, and at the pairwise intersection of two of three of our texts, appears to serve as evidence to shift the burden of proof onto those traditionalists who argue that Mencius is the inheritor of Confucius's mantle.