Introduction
In the face of evolving threats and increasingly complex global dynamics, policymakers and scholars alike are seeking more effective and nuanced understandings of how states respond to the instruments of national power. Existing research has largely focused on the military dimension of deterrence; consequently, the roles of diplomacy, information, and economic statecraft in deterrence and broader policy dynamics remain under-theorised.
We address this imbalance through a new methodology that measures a state’s public response to these other instruments of national power. Our proposed methodology offers two main contributions: 1. the ability to analyse the effects of non-military tools; 2. how these effects (if any) can contribute to deterrence strategies.
For our analyses we use DIME, a framework relatively well known across the scholarly and policy-making communities.Footnote 1 Per its initials, DIME disaggregates the tools of statecraft into four distinct categories: diplomacy, information, military, and economic. We incorporate DIME into our methodology because of its portability across the scholarly and policy communities and, as yet underutilised, utility for assessing the non-military tools of statecraft. By exploring the conditions under which different tools are most/least likely to be (in)effective, our methodology provides a more nuanced understanding of the ability of DIME tools to achieve deterrence outcomes, contributing to the development of more effective and sustainable deterrence strategies.
We assess the impacts of DIME tools by measuring how states react in their propaganda, official statements, and media (we term these POSM), to a range of adversary state actions. Typically, a target state B would rely on its own diplomatic, information, military, and/or economic instruments to respond to adversary state A’s use of a DIME tool against B. For instance, B can respond to an economic sanction from A by using a combination of diplomatic, information, military, and/or economic tools of its own. We assess B’s reaction through its information (POSM) instrument, reflecting Schelling’s conjecture that states can ‘create a bargaining position by public statements, statements that are calculated to arouse a public opinion that permits no concessions to be made’.Footnote 2 Importantly, the degree of factual, propagandistic, or dis/misinformation included in such statements is of less concern because their public nature signals threat(s), promise(s), and commitment. The data used in our methodology allows our focus on DIME to capture deterrence activities across domains and POSM to capture (information) reactions from the target state.
We test our methodology using a case study of the Chinese government’s POSM reactions to United States (US), select non-governmental organisations (NGOs), and United Nations (UN) DIME tools. The case study includes two elements: an overview highlighting key findings across DIME tools and a statistical analysis testing our initial findings from April 2009 to December 2022.
Among our key findings is that Beijing’s POSM (information) response is more sensitive to information tools like naming and shaming – criticisms of its human rights and press freedom policies – than to military exercises. This includes iterations of Taiwanese exercises that involve the US military, a surprising finding with clear relevance for policymakers and scholars; we further explore this finding, including possible explanations, in our discussion at the paper’s close. This sensitivity to information activities, even compared to military exercises by potential adversaries, demonstrates the importance of new methodologies for deciphering the effects of non-military policy tools. We explore our methodology, case study, and findings below, before concluding with a discussion of how our work contributes more broadly to deterrence theory and practice.
Deterrence
Four generations of deterrence theory
Deterrence theory has advanced considerably since its initial focus on nuclear threats during the Cold War.Footnote 3 As the threat environment evolved, so too did deterrence scholarship.Footnote 4 Nuclear deterrence remained the conceptual bedrock in the second generation,Footnote 5 but the analytical approaches increasingly relied on formal modelling and game theory to account for the influence of credibility, commitment, and communication on deterrence strategies, outcomes, and potential failures. The third generation remained focused on nuclear threats but expanded theorising to account for a state’s objectives and political and socio-demographic factorsFootnote 6 and emphasised empirical evaluation of testable hypotheses.Footnote 7 The end of the Cold War forced a recalibration of the threat environment and led to a fourth generation of deterrence theory focused on emergent threats and adversaries, including aggressive behaviour, short of nuclear capabilities, by ‘rogue states’ and violent non-state actorsFootnote 8 and how a wider array of foreign policy and force projection tools, like information operations, could be deployed against them.
More recent developments in fourth generation deterrence theory address hybrid interference,Footnote 9 which includes authoritarian regimes’ deployment of non-military practices designed ‘to penetrate democratic society’ and manipulate ‘other states’ strategic interests’.Footnote 10 While the fourth generation incorporates information tools, often placing them in a central position for power projection and influence,Footnote 11 we continue to lack a sufficient understanding of how targeted countries react to these non-military deterrence tools.
Before proceeding to our proposed methodology and applying it to a case study of China, we first offer a brief overview of how China conceptualises deterrence, including differences with traditional, often Western, nuclear-focused concepts. Then we proceed to examine how US views have evolved, landing on the new US government strategic concept of integrated deterrence. This is important because ‘Chinese decision-makers assess successful deterrence differently from their American counterparts. […] For Chinese decision-makers, successful deterrence is ultimately a form of political activity and psychological warfare, whereby an adversary is constrained in their actions, allowing China to achieve its goals.’Footnote 12 We then describe the POSM dataset. Finally, we conduct a China case study demonstrating the methodology’s applicability and added value for analysing effects across the spectrum of conflict, under differing deterrence concepts, and across a range of DIME tools.
Chinese deterrence
The Chinese approach to deterrence is more in line with coercion.Footnote 13 ‘The Chinese term that is most often equated with deterrence is weishe (威慑). […] But the attendant meanings and implications underlying the term are very different. For the Chinese, the term weishe embodies both dissuasion and compellence.’Footnote 14 Note the dissimilarity below with Schelling’s emphasis on distinguishing deterrence from compellence. Discussions of deterrence in Chinese policy and military circles include the use of conventional and space force and coercion, DIME tools, mobilisation activities and planning, political influence, and scientific and technological tools to exert national power, influence, and preferences.Footnote 15 ‘The act of deterrence is to help achieve a particular goal; deterrence is not the goal itself.’Footnote 16
It is important to note the expansiveness of China’s deterrence concepts in terms of the instruments of state power, including DIME tools, and the emphasis on information deterrence. Per Chinese analysts, ‘we now live in the Information Age, with information being the primary currency of international power […] The growing role of information and associated technologies has led to “information deterrence” becoming a new aspect of weishe […] information itself has become an instrument of conflict, with the ability to establish “information dominance” a central focus in future wars.’Footnote 17
Under information deterrence, as in deterrence’s other forms, a state requires both offensive and defensive capabilities. In China’s view, the ability to conduct offensive information operations is the most important means for information deterrence, but the ‘demonstrated ability to defend and safeguard one’s information resources and systems’ is also vital to deter an adversary.Footnote 18 If an adversary is unable to withstand China’s offensive information operations, then China has achieved information dominance and can deploy ‘information deterrence’. The centrality of information means this dominance will affect other domains, constraining an adversary across multiple domains of conflict and furthering China’s ability to achieve overall deterrence or compellence.Footnote 19
Chinese deterrence concepts clearly include what are traditionally viewed as non-military tools. This conceptualisation goes well beyond the nuclear-focused approach that was at the centre of the first three generations of deterrence theory and appears well aligned to the fourth generation discussed above. Interestingly, China’s explicit cross-domain approach and use of information appears to be affecting US deterrence concepts, which we turn to now.
US deterrence
US deterrence concepts have evolved from traditional practices of ‘deterrence by punishment’ and ‘deterrence by denial’Footnote 20 to newer practices of ‘extended deterrence’,Footnote 21 ‘deterrence by resilience’,Footnote 22 ‘deterrence management’,Footnote 23 and ‘integrated deterrence’.Footnote 24
During the Cold War, the American approach largely focused on preventing a nuclear warFootnote 25 and deterring by ‘turn[ing] aside or discourage[ing] through fear; hence, to prevent from action by fear of consequence’.Footnote 26 Deterrence and compellence were seen as different methods (and objectives) because deterrence relied on creating a menu of suboptimal choices for an adversary rather than forcing one specific behaviour. Deterrence is ‘setting the stage – by announcement, by rigging the trip-wire, by incurring the obligation – and waiting [… it is] nonintrusive, nonhostile, nonprovocative’.Footnote 27 It is indefinite in its timing – the threatened act to deter the adversary lies in wait for the initiating action from the adversary, ‘we can wait – preferably forever; that’s our purpose’.Footnote 28
Mearsheimer shifted the discussion by explicitly focusing on conventional deterrence rather than nuclear deterrence and defined deterrence in its broadest sense as ‘persuading an opponent not to initiate a specific action because the perceived benefits do not justify the estimated costs and risks’.Footnote 29 He added a reminder that leaders need to consider non-military deterrence tools and how adversaries and allies may react but omitted these tools from his conceptualisation of deterrence because it becomes too difficult to account for so many possibilities and choices in developing a coherent theory of deterrence.Footnote 30
Recent strategic documents released by the United States appear to signal a broadening of the American approach and an effort to incorporate non-military tools. The 2022 National Security Strategy (NSS) devotes a full page to a new concept called ‘integrated deterrence’, which is defined as ‘the seamless combination of capabilities to convince potential adversaries that the costs of their hostile activities outweigh their benefits’.Footnote 31 Note the close similarity of the NSS’s cost versus benefit description with Mearsheimer’s views on conventional deterrence, but a clear difference emerges: the specific inclusion of non-military approaches.
The NSS portrays military and non-military approaches, labelled combinations of capabilities, as entailing five integrations: 1. across regions, 2. US government entities, 3. the spectrum of conflict, 4. domains (including the land, air, maritime, cyber, and space domains, plus the economic, technological, and information domains), and 5. with allies and partners.Footnote 32 The explicit mention of multiple non-military domains and a spectrum of conflict that goes beyond nuclear and conventional war to include tensions ‘below the threshold of armed conflict’ highlight a broadening of US deterrence concepts.Footnote 33
Integrated deterrence builds from the 2022 US National Defense Strategy (NDS), which devotes an entire chapter to the idea.Footnote 34 Both of these documents appear (a lack of citations in the NSS and NDS can make it difficult to establish provenance) to have grown from the Department of Defense’s process of creating a Joint Concept for Competing, which makes explicit the need for the United States to engage in cross-domain competition with adversaries at a level below armed conflict.Footnote 35 The NDS acknowledges different deterrence approaches – ‘our competitors, particularly the PRC, are pursuing holistic strategies that employ varied forms of coercion, malign behaviors, and aggression to achieve their objectives and weaken the foundations of a stable and open international system’Footnote 36 – and introduces tailored deterrence approaches to respond to these ‘varied forms’. Deterring adversaries’ use of nuclear weapons (labelled Deterring Strategic Attacks) becomes just one of several approaches; other challenges meriting a tailored approach include: deterring attacks against the homeland and attacks from China, Russia, North Korea, and Iran – each country receives a uniquely tailored deterrence approach.
Self-described as an ‘intellectual paradigm shift’, the new concept is meant to help the United States improve its ability to engage in strategic competition, defined as ‘a persistent and long-term struggle that occurs between two or more adversaries seeking to pursue incompatible interests without necessarily engaging in armed conflict with each other’.Footnote 37 By adding the new concept, the United States appears to acknowledge it requires a better answer to the question of what to do when an adversary initiates harmful actions, including those of a non-military nature, at thresholds below what would meet traditional criteria of jus ad bellum.
This cross-domain broadening of US deterrence concepts has long been incubating in practitioner and academic communities. But just as it took nuclear weapons to give rise to deterrence theory out of age-old deterrence concepts, it appears to have taken the evolving threat landscape and socio-technical innovations of the twenty-first century for the United States to crystallise deterrence concepts that include non-military tools and emphasise the need to deter strategic competitors at levels below armed conflictFootnote 38 and in grey-zone conflicts.Footnote 39
Deterrence, POSM, and DIME
As shown throughout this section, China, the United States, and presumably other countries now conceive of deterrence in broad terms, to include the use of military and non-military tools at levels below armed conflict. While this conceptual broadening fits the fourth wave of deterrence theory, the empirical investigation of the impacts of cross-domain deterrence on targeted states can be challenging – scholars must now determine how to measure the impacts of this broader range of tools.
To address these challenges, we make two arguments. First, the concept of ‘integrated deterrence’Footnote 40 suggests that the United States (and other powers) may draw on tools from multiple (military and non-military) domains to deter their adversaries. We believe that the DIME framework can guide researchers in measuring these cross-domain deterrence activities. Second, the concept of ‘information deterrence’Footnote 41 suggests that China (and possibly other powers) places great emphasis on information as a tool that can be used to either deter or respond to deterrence activities from adversaries. To capture states’ offensive and defensive use of information, we argue that the POSM framework can be of great value to researchers. In the remainder of the paper, we empirically apply the POSM and DIME frameworks to investigate China’s use of information in reaction to cross-domain deterrence activities.
The central goal of this study is to assess how a deterring state’s cross-domain activities affect a target state’s information response. This provides an opportunity to investigate and test fourth generation deterrence theory that highlights information as a tool used by states for deterrence and counter-deterrence purposes.
A methodology for measuring state reactions through public statements
The goal of this study is to assess how states respond when targeted by an adversary state’s cross-domain activities. To accomplish this, our proposed methodology relies on the POSM framework to capture target states’ responses and the DIME framework to capture cross-domain activities. The methodology is tested using a China case study and can be replicated for other countries. The methodology includes:
1. A framework for capturing various instruments of national power (DIME);
2. A framework for capturing states’ use of information (POSM) in response to an adversary’s statecraft;
3. Country-level data with which to conduct our analysis (we use state media and foreign ministry data);
4. A method for analysing the data (we use text and sentiment analysis along with the statistical analysis described below).
If effective, our methodology identifies:
1. A country’s baseline sentiment;
2. POSM reactions to the use of instruments of national power, including statistically significant variations from the baseline.
Measuring DIME
States may initiate deterrence activities against an adversary by drawing on a variety of instruments of national power. The DIME framework captures the four most critical cross-domain instruments – diplomatic, information, military, and economic – available to states to accomplish their strategic goals.Footnote 42 Governments can use the four elements independently or in combination. Measuring DIME deterrence activities, instruments of national power, or tools of statecraft is challenging because there is no set standard for what activities represent which DIME tools. We use the following conceptualisations:
Diplomatic. During the period surveyed, there are four primary diplomatic themes targeting China – human rights abuses, South China Sea activities, foreign aid, and the Covid-19 outbreak. Previous research shows autocrats are sensitive to naming and shaming campaigns and human rights abuse allegations.Footnote 43 One of the key events was the United Nations (UN) Human Rights Special Rapporteur’s report on Xinjiang.
Information. This proved the most difficult for selecting representative activities. Authoritarian states regularly react strongly to press and internet freedom reports from the NGOs Freedom House, Reporters Without Borders, and Human Rights Watch.Footnote 44 But such reports are not strictly or necessarily government supported or reflective of state agendas. While the US State Department produces annual Country Reports on Human Rights Practices,Footnote 45 the mix of diplomatic and information tools inherent to State Department reports reduces their clarity (and thus their usefulness) in terms of our research.Footnote 46
While governments have public broadcasting information tools like Voice of America (VOA) and Radio Free Asia (RFA), these prove less utile for our methodology because they also risk mixing diplomatic and information tools. Targeted publics are also aware of the potential propaganda objectives, which reduces the potential for deterrence objectives, and the targeted states can intentionally boost reports that are critical or negative of the broadcasting state (e.g., the United States). For example, Chinese outlets typically mention VOA or RFA when they provide positive coverage of China or negative coverage of an issue in the United States that China wishes to highlight (racism, Covid deaths, crime, etc.). Neither condition provides much insight into the outlets’ value as a tool of statecraft. When Chinese outlets respond to VOA or RFA criticisms of China, including of censorship and information controls, the counter-narratives frequently reflect a range of factors (censorship, sanctions, comments perceived as racist or anti-China by business or political leaders, etc.). These muddy waters make it difficult to discern whether any negativity is in response to VOA/RFA, specific criticisms of China’s information controls, or a combination of these and other activities. Further, the recent decision by the Trump administration to shutter many of America’s information capabilities (including the VOA and RFA), a decision widely praised in Beijing, Moscow, and elsewhere, adds to the challenges in selecting government activities specific to the information domain.Footnote 47
For the sake of clarity, we focus on Beijing’s reactions to NGO reporting as a proxy for information activities targeting China.Footnote 48 Governments wishing to avail themselves of information’s capabilities may wish to study the processes and output of the various NGOs.
Military. Despite China not being at war during the period surveyed, there was an increasing emphasis on great power competition and continued US military exercises in the region. The military exercises, often bi- and multi-lateral and including sea, air, and land activities, definitively have deterrence objectives (among other goals). Military exercises are the primary measurement for this category, but arms sales, military aid, and new security pacts like AUKUS (Australia, United Kingdom, United States) provide additional context.
Economic. This is also relatively straightforward, with economic sanctions as our primary measures. Economic tools can be short of sanctions and focus on securitising against threats from foreign companies, like the 2024 discussions in the United States about TikTok or previous discussions (available during our time period) about Huawei (a Chinese information and technology company).
Measuring POSM reactions
Assessing the success/failure of deterrence requires leaders to include the reactions of adversaries.Footnote 49 When targeted by activities from an adversary, a state may react in a variety of ways, including through military and/or non-military responses. In this study, we focus on the use of information as one of many possible responses; we leave it to other studies to examine states’ military and other non-military (e.g., economic) responses.Footnote 50 Bargaining positions are created by public statementsFootnote 51 – and this research analyses such statements in response to an adversary’s military exercises, economic sanctions, and other DIME activities.
To capture states’ use of information in response to deterrence activities, we examine states’ records of propaganda, official statements, and public news media (POSM). POSM data can be obtained from official government websites and publications by state media outlets. For our case study, we use foreign ministry articles/statements from the Chinese Ministry of Foreign Affairs (MOFA), and news articles from two state media outlets: Global Times and People’s Daily.Footnote 52 The data are from the authors’ FOCUSdata Project,Footnote 53 which contains all English-language articles available from the three sources at the time of data collection. FOCUSdata uses English-language articles because English is the international language of diplomacy, public information, and signalling.Footnote 54 Our data was collected in early 2023 and encompasses the following:
1. MOFA data consists of 2,607 articles/statements published on the ministry’s website from 15 November 2000 to 31 December 2022.
2. Global Times data consists of 677,532 articles published from 9 April 2009 to 31 December 2022.
3. People’s Daily data includes 544,940 articles published from 12 May 2007 to 31 December 2022.
We rely on computational text analysis and the search terms in Table 1 to identify relevant articles in the three datasets and then apply sentiment analysis to quantify Chinese POSM and analyse Beijing’s reactions to DIME tools. The results provide insight into how states perceive and respond to deterrent signals – especially in environments where strategic messaging is tightly controlled. Our POSM analysis provides a conceptual approach that allows assessing state behaviour in response to external pressure.
DIME search terms.
Table 1 Long description
Two columns list DIME Tools and their associated Search Terms. Under Diplomatic, the terms are, UN Human Rights Special Rapporteur; UN Human Rights; Uyghur Human Rights Policy; South China Sea; Covid Investigation5555We specified the term Covid Investigation to exclude the thousands of articles discussing the virus and pandemic in general.; and Foreign Aid. Under Information, the terms are, Freedom House; Reporters Without Borders; Human Rights Watch; and Propaganda. Under Military10, the terms are, Ulchi; RIMPAC; Keen Sword; Han Kuang; Balikatan; AUKUS; Arms Sales; Military Aid. Under Economic, the terms are, Sanctions and Huawei.
Method
We use a replicable five-step mixed methods approach to analyse our data:
Step 1: We begin by analysing each dataset using sentiment analysis (at the document level), a ‘computational treatment of opinion, sentiment, and subjectivity in text’.Footnote 56 Applying sentiment analysis to government reports and state media articles, or a government’s every mention of a subject, allows quantification of official state opinion.Footnote 57 This is a critical tool for studying public information, rhetoric, and polarity but is underutilised in international relations. The quantification can turn text data into a Likert-scale positive–neutral–negative numerical range. There are dozens (if not hundreds) of sentiment analysis tools, including those from Amazon, Google, IBM, and Microsoft, dozens of mid-tier and smaller companies, and the many open-source tools that pioneered quantitative sentiment analysis. For our research we used two of the open-source tools: AFINNFootnote 58 and Bing.Footnote 59 To avoid possible confusion, it is important to note that the open-source Bing sentiment analysis tool is unrelated to the Microsoft search engine of the same name. All sentiment analysis was conducted in R using R Studio; all articles were analysed by both tools. While the ranges of the output varied between the tools, the results are generally consistent, as explained in greater detail in the case study. Step 1 produces a baseline sentiment score per data source (see Figures 1–6).
MOFA baseline sentiment using AFINN.
MOFA baseline sentiment using Bing.
Global Times baseline sentiment using AFINN.
4 Global Times baseline sentiment using Bing.
People’s Daily baseline sentiment using AFINN.
People’s Daily baseline sentiment using Bing.
Step 2: We focus the sentiment analysis on reactions when targeted with specific DIME tools, which requires specific activities to analyse within each category (see Measuring DIME section above and Step 5, below). We welcome other researchers to test our methodology with additional activities and/or categories, using our data or their own. Table 1 showed our selected search terms.Footnote 60
Step 3: We then conduct sentiment analysis on the DIME and non-DIME articles to quantify China’s discussion and reactions to the DIME tools. This creates sentiment scores that can be compared across DIME tools and to the data source’s baseline. See Figures 7–25.
Step 4: We analyse sentiment variation between DIME and non-DIME articles and across DIME tools using a series of two-sample t-tests. Statistically significant patterns and differences are identified. See Tables 2–5.
Two sample t-test results (if only specific DIME tool is identified in article, comparing DIME & non-DIME articles).
Table 2 Long description
Two sample t test results compare mean values between All non-DIME and four groups, Diplomatic, Information, Military, and Economic, for Peoples Daily, Global Times, and MOFA, with N and Mean reported for each. For All non-DIME, Peoples Daily has N 408,025 and Mean 4.54; Global Times has N 644,276 and Mean 2.85; MOFA has N 1,910 and Mean 48.13. For Diplomatic, Peoples Daily has N 5,229 and Mean 6.76 with a difference of minus 2.22, t statistic minus 7.99 with three asterisks, and degrees of freedom 5304.17; Global Times has N 7,897 and Mean 5.32 with a difference of minus 2.47, t statistic minus 11.08 with three asterisks, and degrees of freedom 8018.79; MOFA has N 201 and Mean 67.66 with a difference of minus 19.53, t statistic minus 5.06 with three asterisks, and degrees of freedom 227.98. For Information, Peoples Daily has N 562 and Mean minus 6.69 with a difference of 11.23, t statistic 15.34 with three asterisks, and degrees of freedom 562.17; Global Times has N 1,201 and Mean minus 9.59 with a difference of 12.44, t statistic 20.97 with three asterisks, and degrees of freedom 1202.61; MOFA has N 1 and Mean 22, with the difference and t statistic shown as a dash and no degrees of freedom listed. For Military, Peoples Daily has N 283 and Mean 3.85 with a difference of 0.685, t statistic 0.697, and degrees of freedom 282.33; Global Times has N 596 and Mean 0.624 with a difference of 2.23, t statistic 3.38 with three asterisks, and degrees of freedom 596.05; MOFA has N 18 and Mean 41.28 with a difference of 6.85, t statistic 0.622, and degrees of freedom 17.27. For Economic, Peoples Daily has N 5,733 and Mean 0.196 with a difference of 4.34, t statistic 19.18 with three asterisks, and degrees of freedom 5858.53; Global Times has N 13,547 and Mean minus 1.24 with a difference of 4.09, t statistic 28.91 with three asterisks, and degrees of freedom 14080.4; MOFA has N 202 and Mean 39.46 with a difference of 8.67, t statistic 2.81 with two asterisks, and degrees of freedom 247.74.
*** p ≤ 0.001, ** p ≤ 0.01, * p ≤ 0.05 | Robust standard errors in parentheses
Summary statistics of standardised sample, 9 April 2009–31 December 2022.
Table 3 Long description
Afinn summary statistics are reported for Peoples Daily, Global Times, and Mofa, with rows for All, Us Related, and Non Us Related, and columns for N, Mean, Std. Dev., Min, and Max. Peoples Daily: All N 420,108, mean 4.48, standard deviation 15.21, min minus 255, max 274; Us Related N 79,439, mean 5.37, standard deviation 17.89, min minus 255, max 184; Non Us Related N 340,669, mean 4.28, standard deviation 14.51, min minus 101, max 274. Global Times: All N 668,224, mean 2.76, standard deviation 15.82, min minus 200, max 232; Us Related N 198,620, mean 3.88, standard deviation 17.85, min minus 200, max 194; Non Us Related N 469,604, mean 2.29, standard deviation 14.85, min minus 119, max 232. Mofa: All N 2,476, mean 47.88, standard deviation 44.69, min minus 97, max 229; Us Related N 997, mean 56.14, standard deviation 54.17, min minus 97, max 229; Non Us Related N 1,479, mean 42.32, standard deviation 35.91, min minus 33, max 201. Totals: All N 1,090,808, mean 3.53, standard deviation 15.88, min minus 255, max 274; Us Related N 279,056, mean 4.49, standard deviation 18.39, min minus 255, max 229; Non Us Related N 811,752, mean 3.20, standard deviation 14.90, min minus 119, max 274.
Two sample t-test results (if only specific DIME tool is identified in article, comparing US & non-US articles).
Table 4 Long description
Results are reported for three sources, Peoples Daily, Global Times, and MOFA, each with columns for N and Mean for US and non-US groups, plus a Difference, a t-statistic, and Satterthwaites degrees of freedom. For Diplomatic, Peoples Daily reports US N 1806 mean 4.42 and non-US N 3423 mean 7.99, with difference minus 3.56, t-statistic minus 5.72 with three asterisks, and degrees of freedom 3016.61; Global Times reports US N 4476 mean 5.39 and non-US N 3421 mean 5.24, with difference 0.147, t-statistic 0.332, and degrees of freedom 7682.21; MOFA reports US N 126 mean 66.63 and non-US N 75 mean 69.37, with difference minus 2.74, t-statistic minus 0.383, and degrees of freedom 191.62. For Information, Peoples Daily reports US N 45 mean minus 18.76 and non-US N 517 mean minus 5.64, with difference minus 13.12, t-statistic minus 3.88 with three asterisks, and degrees of freedom 48.33; Global Times reports US N 182 mean minus 14.99 and non-US N 1019 mean minus 8.62, with difference minus 6.36, t-statistic minus 4.06 with three asterisks, and degrees of freedom 260.77; MOFA reports US N 0 with mean shown as a dash and non-US N 1 mean 22, with difference shown as a dash and t-statistic shown as a dash. For Military, Peoples Daily reports US N 203 mean 3.54 and non-US N 80 mean 4.65, with difference minus 1.11, t-statistic minus 0.520, and degrees of freedom 151.09; Global Times reports US N 524 mean 0.985 and non-US N 72 mean minus 2.00, with difference 2.98, t-statistic 1.67, and degrees of freedom 100.00; MOFA reports US N 17 mean 34.35 and non-US N 1 mean 159, with difference minus 124.64 and t-statistic shown as a dash. For Economic, Peoples Daily reports US N 3477 mean minus 1.00 and non-US N 2256 mean 2.04, with difference minus 3.05, t-statistic minus 6.67 with three asterisks, and degrees of freedom 4882.53; Global Times reports US N 10551 mean minus 1.27 and non-US N 2996 mean minus 1.13, with difference minus 0.132, t-statistic minus 0.410, and degrees of freedom 5175.86; MOFA reports US N 158 mean 35.87 and non-US N 44 mean 52.34, with difference minus 16.47, t-statistic minus 2.43 with one asterisk, and degrees of freedom 72.22.
*** p ≤ 0.001, ** p ≤ 0.01, * p ≤ 0.05 | Robust standard errors in parentheses
Two sample t-test results (if only specific DIME tool is identified in article).
Table 5 Long description
Two sample t test results compare mean values between pairs of DIME tools for three sources, Peoples Daily, Global Times, and MOFA, reporting N, mean, the mean difference, t statistic, and Satterthwaite degrees of freedom. For Diplomatic versus Information, Peoples Daily has N 1806 mean 4.42 versus N 45 mean minus 18.76, difference 23.18, t statistic 6.929 with degrees of freedom 46.39; Global Times has N 4476 mean 5.39 versus N 182 mean minus 14.99, difference 20.37, t statistic 13.95 with degrees of freedom 198.21; MOFA lists Diplomatic N 126 mean 66.63 and Information N 0 with no mean, and no difference or test statistics. For Diplomatic versus Military, Peoples Daily has N 1806 mean 4.42 versus N 203 mean 3.54, difference 0.886, t statistic 0.684 with degrees of freedom 294.93; Global Times has N 4476 mean 5.39 versus N 524 mean 0.985, difference 4.40, t statistic 5.66 with degrees of freedom 733.06; MOFA has N 126 mean 66.63 versus N 17 mean 34.35, difference 32.28, t statistic 3.09 with degrees of freedom 28.03. For Diplomatic versus Economic, Peoples Daily has N 1806 mean 4.42 versus N 3477 mean minus 1.00, difference 5.43, t statistic 8.84 with degrees of freedom 2870.89; Global Times has N 4476 mean 5.39 versus N 10551 mean minus 1.27, difference 6.65, t statistic 19.13 with degrees of freedom 7044.34; MOFA has N 126 mean 66.63 versus N 158 mean 35.87, difference 30.76, t statistic 4.98 with degrees of freedom 217.74. For Information versus Military, Peoples Daily has N 45 mean minus 18.76 versus N 203 mean 3.54, difference minus 22.29, t statistic minus 6.36 with degrees of freedom 55.68; Global Times has N 182 mean minus 14.99 versus N 524 mean 0.985, difference minus 15.97, t statistic minus 10.01 with degrees of freedom 276.67; MOFA lists Information N 0 with no mean and provides no difference or test statistics. For Information versus Economic, Peoples Daily has N 45 mean minus 18.76 versus N 3477 mean minus 1.00, difference minus 17.75, t statistic minus 5.36 with degrees of freedom 44.68; Global Times has N 182 mean minus 14.99 versus N 10551 mean minus 1.27, difference minus 13.72, t statistic minus 9.55 with degrees of freedom 185.65; MOFA again lists Information N 0 with no mean and no test results. For Military versus Economic, Peoples Daily has N 203 mean 3.54 versus N 3477 mean minus 1.00, difference 4.54, t statistic 3.75 with degrees of freedom 227.29; Global Times has N 524 mean 0.985 versus N 10551 mean minus 1.27, difference 2.25, t statistic 3.08 with degrees of freedom 577.80; MOFA has N 17 mean 34.35 versus N 158 mean 35.87, difference minus 1.52, t statistic minus 0.158 with degrees of freedom 20.53.
*** p ≤ 0.001, **p ≤ 0.01, *p ≤ 0.05 | Robust standard errors in parentheses
Step 5: When selecting our list of terms we relied on area studies knowledge, key events (e.g., the release of the UN Human Rights Special Rapporteur’s report on Xinjiang), and lessons learned from previous research to identify specific DIME-related articles. We used qualitative narrative analyses to triangulate the results.Footnote 61 Sentiment analysis does not replace ‘the need for careful and close readings of texts or otherwise obviate the need for human analysis’.Footnote 62 This verifies the computational text and statistical analyses, deepens understanding of how sources discuss various terms, and can iteratively refine the search terms (specific military exercises, UN resolutions, etc.). This latter aspect is especially important when testing our methodology on additional countries – lacking an agreed-upon collection of what tools represent which DIME element, qualitative familiarity with a targeted country assists researchers in determining representative instruments of national power best suited for analysis by the quantitative tools.
Case study – China
The case study is divided into subsections. It begins with our key findings, then presents data visualisations examining baseline sentiment in the POSM sources and variation in sentiment across responses to DIME tools based on the search terms in Table 1 (and their derivatives; see Appendix Table A). The final subsection reports results from a series of two-sample t-tests analysing variation in average sentiment (i.e., reactions) between DIME and non-DIME articles and across DIME tools. We identify several statistically significant variations indicative of strategic reactions, decision-making, and therefore, opportunities for customised DIME-based deterrence strategies.
Key findings:
1. The methodology effectively produces baselines that allow us to discover and compare Chinese government reactions to DIME tools. The baselines and findings are consistent across two different sentiment analysis tools, plus related statistical analyses.
2. The information tool generates strongly negative reactions. ‘Naming and shaming’ Beijing’s human rights record and domestic information controls appear to offer powerful options for attempting to deter the regime.
3. When China discusses military exercises, including those involving the United States and Taiwan, the average sentiment is less negative in all three sources than for diplomatic and information tools.
4. China’s MOFA is significantly less negative and more positive (i.e., more ‘diplomatic’) than state media outlets; this includes both baseline sentiment and sentiment regarding all DIME search terms except one, as noted below (#5).
5. Starting in 2021, the Chinese MOFA turned very negative when discussing the South China Sea. In 2021 and 2022, MOFA is more negative on the topic than state media, a rare occurrence we have not found in previous research on Russia, Iran, or North Korea.Footnote 63
6. Beginning in 2020, MOFA dramatically increased the number of English-language articles released on its website, from dozens to hundreds of articles annually. We see this reflected in MOFA’s coverage of countries around the world, from the Solomon Islands to Mali. While additional research is required to understand the purpose of the increase, it could indicate a growing interest in public diplomacy to advance favoured Chinese narratives and/or counter undesired English-language narratives.
7. Prior to 2020, negative and very negative sentiments were rare in MOFA articles, with entire years passing with no negative article sentiment; since 2020, however, negative and very negative sentiment account for 6–15 per cent of articles annually.
8. 2020 also reflects a clear change in Global Times articles discussing the United States. From 2009 to 2019, Global Times averaged approximately 4,000 articles per year that mentioned the United States, with positive outweighing negative sentiment each year. From 2020 to 2022, the number of articles mentioning the United States dropped to under 3,000, under 1,800, and under 1,300, respectively, and negative sentiment outweighed positive sentiment. Further research is required to better understand these recent changes.
9. In 2021, the Chinese government suddenly and dramatically increased the use of Xizang (a Chinese term) instead of Tibet in English-language state media. This name-change has raised concerns among Tibetan human rights groups.Footnote 64 The finding, while outside our focus, demonstrates an unexpected capability of the methodology.
Baselines
Important to our goal of comparing reactions to the various instruments of national power are sentiment baselines for each source. Figures 1–6 show, respectively, the results of both AFINN and Bing sentiment analysis for MOFA (2,607 articles), Global Times (677,532 articles), and People’s Daily (544,940 articles). The sentiment analysis produces a continuous numerical value, which we examine in detail below, but for ease of visualisation we group the sentiment scores into a standard five-point Likert scale: very positive (shown in green), positive (light blue), neutral (grey), negative (orange), and very negative (red). Positive (light blue) is typically the most common sentiment.
From the baseline sentiment, we can see that AFINN and Bing return similar results across all three sources. This is consistent across all search terms, not just with the baselines, and helps establish the viability and replicability of the sentiment analysis portion of our methodology. It is also important to note, across both sentiment tools, the very different levels of negative sentiment (shown in orange in the visualisations above) between the MOFA and state media baselines. This is consistent with previous research – diplomatic organisations tend to be more positive, less negative (i.e., more diplomatic) than state media outlets.Footnote 65
By comparing against these baselines, we begin determining what (if any) reactions the various DIME tools have generated from Beijing. Global Times and People’s Daily most frequently discuss DIME tools in the following order, from most to least: economic, military, diplomatic, and information. MOFA, perhaps not unexpectedly, primarily addresses diplomatic activities, then economic and military; it rarely addresses information activities. We examine the DIME tools and key findings in D – I – M – E order and provide select visualisations (for space considerations) and descriptive summaries of our data to help explain the findings. Our data is available to other researchers.
There are similarities between the information and diplomacy tools (both exemplify non-coercive means of influence, involve strategic messaging, seek to reinforce the messages/impact from other tools, and aim to advance a positive image of the state), but there are notable differences that warrant looking at these tools as separate means of power. Information activities are not necessarily conducted by state officials, can be more indirect than diplomatic communication, can target both foreign governments and populations, are less formalised, and can include grey-zone activities.
Diplomatic
China is generally positive in terms of the United Nations, but there are several UN-related terms or activities that create a clear increase in negative sentiment. China is consistently critical of certain diplomatic efforts related to human rights when discussing them in POSM. For example, Figure 7 shows the consistently negative sentiment when discussing the United Nations and human rights in People’s Daily. The only year with positive sentiment outweighing negative is 2018, when China successfully passed a UN Human Rights Council resolution, ‘which set out to replace the idea of holding states accountable […] and which omitted a role for independent civil society in HRC [Human Rights Council] proceedings’.Footnote 66
Number and sentiment of People’s Daily articles referencing UN [United Nations] Human Rights, 2007–2022.
When focused specifically on the UN special rapporteur’s criticism of China’s human rights practices in Xinjiang, the reaction is much more negative than the baselines, as seen in Figures 8–10. Note the appearance of very negative (red) sentiment, compared to its relative absence in the baseline visualisations in Figures 1–6.
Number and sentiment of MOFA articles referencing UN special rapporteur, 2017–2022.
Number and sentiment of Global Times articles referencing UN special rapporteur, 2009–2022.
Figure 9 Long description
The horizontal axis is labeled, Number of Articles, ranging from 0 to 30. The vertical axis lists years from 2009 to 2022. The legend indicates sentiment categories: Very Negative, Negative, Neutral, Positive. Each bar is divided into segments labeled with percentages, representing the proportion of each sentiment. Key observations: - In 2013, the largest segment is Neutral at 64.29 percent. - In 2018, Positive sentiment dominates with 90.00 percent. - In 2020, the segments are 34.32 percent Neutral, 72.73 percent Negative and 22.73 percent Positive, which appears inconsistent. Overall, Neutral sentiment is prevalent in many years, while Positive sentiment peaks in 2018. The year 2019 has the highest number of articles at 22, with 72.73 percent Neutral sentiment. Notable years include: - 2009: 7 articles, 71.43 percent Neutral. - 2015: 10 articles, 80.00 percent Neutral. - 2022: 6 articles, 16.67 percent Positive. The graph highlights trends in sentiment over time, with Neutral sentiment generally being the most common.
Number and sentiment of People’s Daily articles referencing UN special rapporteur, 2007–2022.
Figure 10 Long description
Horizontal stacked bar graph with legend, Very negative, Negative, Neutral, Positive. X axis label, Number of Articles, with tick labels 0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28. Y axis label, Year of Article, with years 2007 through 2022. 2007: Neutral 47.06 percent, Positive 47.06 percent. 2008: Negative 15.00 percent, Neutral 10.00 percent, Positive 70.00 percent. 2009: Neutral 25.57 percent, Positive 74.43 percent. 2010: Neutral 5.56 percent, Positive 94.44 percent. 2011: Very negative 3.23 percent, Neutral 6.45 percent, Positive 90.32 percent. 2012: Neutral 10.00 percent, Positive 90.00 percent. 2013: Neutral 50.00 percent, Positive 50.00 percent. 2014: Positive 100.00 percent. 2015: Very negative 100.00 percent. 2016: Positive 100.00 percent. 2017: Positive 100.00 percent. 2018: Very negative 40.00 percent, Negative 40.00 percent. 2019: Very negative 38.29 percent, Neutral 52.94 percent, Positive 11.76 percent. 2020: Very negative 14.81 percent, Negative 62.96 percent, Neutral 3.70 percent, Positive 18.52 percent. 2021: Very negative 8.66 percent, Negative 15.38 percent, Neutral 15.38 percent, Positive 53.85 percent. 2022: Very negative 8.66 percent, Negative 15.38 percent, Neutral 15.38 percent, Positive 53.85 percent.
In addition to the UN special rapporteur’s investigation and report, both the UN and US Congress have over the years released reports (UN) and expressed criticisms (UN and US) of China’s human rights policies specifically regarding Uyghurs.Footnote 67 Beijing’s responses to criticisms of its ‘Uyghur human rights policy’ are consistently negative across all three sources.Footnote 68
Finally, we note a recent change: starting in 2021, Chinese MOFA sentiment regarding mentions of the South China Sea turned very negative. Of the 2,607 articles from 2000 to 2022 in our Chinese MOFA database, only forty-eight are classified as ‘very negative’ and thirteen (27 per cent) of them are from 2021 and 2022 and mention the South China Sea. This is a significant outlier from the baseline. Additional research is required to better understand this finding.
Information
China appears to be quite sensitive to information tools. In both the Global Times and People’s Daily, articles discussing international NGO reports naming and shaming China’s human rights practices generated the most negative average sentiment across DIME tools.Footnote 69 MOFA rarely discusses the information-related search terms (n = 2) and thus our discussion, and subsequent analyses, are restricted to Global Times and People’s Daily. Naming and shaming seems to sting. Especially in authoritarian regimes, government decisions to respond strongly to critiques articulated by NGOs likely indicates that the government sees NGOs as influential enough to challenge the government’s legitimacy or controlFootnote 70 and that such narratives can have mobilising appeal among the masses.Footnote 71
Figures 11 and 12 highlight China’s reactions to Reporters Without Borders, a non-governmental organisation (NGO) focusing on international press freedom. Its reports and rankings regularly irritate authoritarian states and, matching previous research by the authors on Russia and North Korea, we find the same negativity in Beijing.Footnote 72 Global Times and People’s Daily changes from the predominantly positive baseline; negativity jumps from baselines of 37.7 per cent and 34.6 per cent to 86.3 per cent and 86.1 per cent, respectively.
Number and sentiment of Global Times articles referencing Reporters Without Borders, 2009–2022.
Number and sentiment of People’s Daily articles referencing Reporters Without Borders, 2007–2022.
Freedom House, a similar NGO focused on freedom of expression and information access, also regularly criticises Beijing’s control of its domestic information environment.
China’s authoritarian regime has become increasingly repressive in recent years. The ruling Chinese Communist Party (CCP) continues to tighten control over all aspects of life and governance, including the state bureaucracy, the media, online speech, religious practice, universities, businesses, and civil society associations.Footnote 73
While the effect is not as strong as reactions to Reporters Without Borders, Freedom House still produces variation from the baseline. Global Times and People’s Daily negativity jumps from the 37.7 per cent and 34.6 per cent baselines to 62.3 per cent and 41.7 per cent negative, respectively. Reports from a similar NGO, Human Rights Watch, produce a similar negative effect, as shown in Figure 13.
Number and sentiment of People’s Daily articles referencing Human Rights Watch, 2007–2022.
Figure 13 Long description
The stacked horizontal bar graph has a legend titled, Affin left parenthesis group right parenthesis, with categories, Very negative, Negative, Neutral and Positive. The x axis label is, Number of Articles, with tick labels 0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20. The y axis lists years 2007 through 2022. 2007: Positive 1, Negative 13. 2008: Positive 1, Negative 19. 2009: Positive 3, Negative 15, Very negative 1. 2010: Positive 3, Negative 17. 2011: Positive 1, Negative 18. 2012: Positive 1. 2013: Negative 6. 2014: Positive 1, Negative 3. 2015: Positive 1, Negative 4. 2016: Negative 1. 2017: Negative 1. 2018: Negative 4. 2019: no numeric label visible on the bar. 2020: Positive 1, Negative 3. 2021: Positive 1, Negative 11, Very negative 1. 2022: Positive 3, Very negative 1.
Number and sentiment of Global Times articles referencing Ulchi, 2009–2022.
The term ‘US propaganda’, unsurprisingly, also produces a negative swing. In Global Times and People’s Daily (a small sample, N = 6), negativity jumps from the 37.7 per cent and 34.6 per cent baseline to 70.2 per cent and 66.7 per cent negative, respectively.
Military
Rather surprisingly, when China discusses competitor/adversary (US, Taiwanese) military exercises, the average sentiment is less negative in all three sources than for diplomatic and information tools. For authoritarian states including China, some military activities (e.g., border adjacent training exercises) conducted by adversaries may in fact benefit regime leadership by supporting state narratives of outside threats that require public sacrifice and increased support for the political leadership. This aligns with previous research by the authors on authoritarian states including North KoreaFootnote 74 and Russia.Footnote 75 Another potential explanation is that Chinese official discourse may be deliberately moderate in its tone towards military exercises to project an image of strategic composure and control – both at home and abroad. It may also reflect a view that such events are routine and are to be expected – as opposed to, for example, less predictable and more provocative critiques by non-state actors. Or Chinese officials may prioritise the use of non-information tools (e.g., military or economic) when responding to military deterrence activities. These interpretations suggest that deterrence through military signalling may produce more complex effects in authoritarian regimes than commonly assumed.
On average, China’s sentiment is positive when POSM references the military search terms in Table 1. The reactions to these exercises are more negative than to the other military terms but are overall less negative than diplomatic and information activities. The exceptions are US military exercises with South Korea and Taiwanese exercises that the US military occasionally, but not always, attends. We look at those two examples: US and South Korean exercises (variously named Ulchi Freedom Shield, Ulchi Freedom Guardian, and Ulchi Focus Lens), and Taiwan’s Han Kuang exercises. The Taiwanese exercises typically involve only domestic forces, but in 2020 US forces attended: ‘neither Washington nor Taipei having officially acknowledged the presence of U.S. troops in Taiwan for more than 40 years, military exchanges between the two were a regular occurrence before COVID-19 […] Observers from Washington were invited for Taiwan’s yearly Han Kuang military exercises.’Footnote 76
For the Ulchi exercises, see Figures 14 and 15; Global Times and People’s Daily negativity jumps from 37.7 per cent and 34.6 per cent baselines to 54.4 per cent and 51.1 per cent negative, respectively. Han Kuang exercises, Figures 16 and 17, result in Global Times and People’s Daily negativity jumping from 37.7 per cent and 34.6 per cent baselines to 62.9 per cent and 75.0 per cent negativity, respectively. Note the attenuated differences compared to the information tools examined above; Beijing’s negativity increases were larger when reacting to information tools.
Number and sentiment of People’s Daily articles referencing Ulchi, 2007–2022.
Number and sentiment of Global Times articles referencing Han Kuang, 2009–2022.
Number and sentiment of People’s Daily articles referencing Han Kuang, 2007–2022.
Beijing’s public messaging reactions are more negative when targeted by criticisms of its human rights and press freedom policies than when faced with Taiwanese military exercises – specifically including iterations of those exercises that involve the US military. This is an important finding for both policymakers and deterrence theorists. China’s overall lukewarm POSM response to military exercises is not atypical. Though it requires further research, our initial interpretation is that authoritarian states, through their POSM, respond less negatively than expected to nearby military exercises held by competitors because they actually align with regime narratives of external threats and help produce a ‘rally around the flag/leadership’ effect. The exercises, in essence, may contribute to preferred regime threat narratives; we discuss this further at the conclusion of the paper.
Economic
As seen in Figures 18–21, there is a significant increase in the frequency Huawei is discussed starting in 2018, as the United States first considers, then imposes, sanctions. Following the sanctions, Huawei appears more frequently in MOFA’s lexicon, which bucked precedent by using negative and very negative sentiment at a much higher rate (20.3 per cent) than in the baseline data (7.2 per cent). China clearly countered discussions regarding Huawei with its own messaging. This provides potential lessons for policy discussions regarding TikTok and other Chinese entities facing US, European, or other sanctions or restrictions.
Number and sentiment of Global Times articles referencing Huawei, 2009–2022.
Figure 18 Long description
The stacked horizontal bar graph shows yearly totals of articles, with each bar split into sentiment categories. The vertical axis label is, Year, with categories 2009 through 2022. The horizontal axis label is, Number of Articles, ranging from 0 to 1500. A legend titled, Affirm, lists: Very Negative, Negative, Neutral, Positive, Very Positive. Several bar segments include percentage labels. The visible percentage labels are: 74.14 percent, 71.49 percent, 66.92 percent, 54.88 percent, 42.34 percent, 30.23 percent, 25.00 percent, 16.54 percent, 14.20 percent, 10.12 percent, 9.58 percent, 9.07 percent, 8.05 percent, 7.97 percent, 7.79 percent, 7.41 percent, 7.14 percent, 6.74 percent, 6.67 percent. The bars increase in total length from earlier years to later years, with the longest total bars appearing in the years 2019, 2020 and 2021 and shorter total bars in the years 2009 through 2017. The year 2022 bar is shorter than 2019 through 2021.
Number and sentiment of People’s Daily articles referencing Huawei, 2007–2022.
Figure 19 Long description
Huawei article mentions and sentiment trends from 2007 to 2022 in Peoples Daily. Bars are horizontal and stacked by sentiment: very negative, negative, neutral, positive and very positive. The x axis represents the number of articles, ranging from 0 to 500. The y axis lists years from 2007 to 2022. In 2012, the highest number of articles, approximately 500, were published, with 90.52 percent being neutral. In 2019, around 300 articles were published, with 71.47 percent neutral and 25.07 percent positive. In 2020, 62.99 percent of articles were neutral, with a notable 33.86 percent being positive. The graph highlights a significant increase in neutral sentiment in 2012 and 2019. The legend categorizes sentiments without the unclear ‘Affirm, group 1’.
Number and sentiment of MOFA articles referencing Huawei, 2014–2022.
Number and sentiment of Global Times articles referencing section 889 (Huawei Sanction).
Searching on ‘Huawei’ returned many articles, so we narrowed the search to include ‘section 889’, the provision in the 2019 US National Defense Authorization Act sanctioning Huawei.Footnote 77 This greatly narrows the scope of articles. Global Times is the only source that mentions ‘section 889’ (see Figure 21), and the sentiment is more commonly negative than when searching on ‘Huawei’ alone.
We also examined the general term ‘economic sanctions’, including during periods containing specific new applications of US sanctions on China. Overall, sentiment regarding economic sanctions was more negative than positive. In Figures 22 and 23, Global Times and People’s Daily negativity increased from 37.7 per cent and 34.6 per cent baselines to 52.7 per cent and 54.4 per cent negativity, respectively. In Figure 24, MOFA negative and very negative both increase to 13.0 per cent from 5.4 per cent and 1.8 per cent baselines, respectively, and positive sentiment decreases from an 87.2 per cent baseline to 69.6 per cent. Despite these changes, economic-focused terms induced less negative variation from the baselines than the other DIME tools.
Number and sentiment of Global Times articles referencing economic sanctions, 2009–2022.
Number and sentiment of People’s Daily articles referencing economic sanctions, 2007–2022.
Number and sentiment of MOFA articles referencing economic sanctions, 2014–2022.
Global Times was the only source that used the term ‘treasury sanctions’ (which contains a more specific US focus than other terms), with three articles in 2016 and one article in 2018. All were negative, but the small sample size obviates any claims to a finding.
Statistical analyses
We conduct a series of two-sample t-tests to analyse whether there are statistically significant variations in China’s POSM reactions to, between, and across DIME tools. All tests use an unequal variance specification to account for variations or imbalances in the number of observations for different DIME search terms within each data source. We report analyses and results using the AFINN sentiment scores because of space limitations (see Appendix D for replication using Bing sentiment scores). Analysing average sentiment provides a nuanced and sensitive measurement of China’s reactions. For example, placing documents into positive and negative buckets requires strict and inflexible boundaries distinguishing the two categories, but differences in sentiment between these two buckets could be (theoretically) as small as 0.1 or as large as 75.0. Descriptive statistics are provided in Appendix Table B.
First, to standardise interpretation across the three POSM sources, we restrict the temporal domain to the shared time period, 9 April 2009–31 December 2022. We split the sample based on whether an article includes DIME tool search terms and then by the specific DIME tool. Some articles include multiple search terms or DIME tools. This could influence the analysis because it risks double-counting or including an article as both diplomatic and information. To avoid this, we only compare articles that include one DIME tool. For example, the article only mentions diplomatic search terms. As a result, the sample shrinks to 1,090,808 POSM articles across the three sources. We conduct analyses using this sample.Footnote 78
As shown in Table 2, we find statistically significant differences in average sentiment when comparing articles that discuss DIME tools to ones that do not. In People’s Daily, Global Times, and MOFA, compared to non-DIME sentiment, China’s average diplomatic sentiment is 48.9 per cent, 86.7 per cent, and 40.6 per cent higher (p ≤ 0.001) and economic sentiment is 95.6 per cent, 143.5 per cent, and 18.0 per cent lower (p ≤ 0.001 and p ≤ 0.01), respectively. And, in all three sources, military sentiment is consistently lower than non-DIME sentiment, but the difference (78.1 per cent) is only statistically significant (p ≤ 0.001) in Global Times. Information sentiment is 247.4 per cent and 436.5 per cent lower (p ≤ 0.001) than non-DIME sentiment in People’s Daily and Global Times, respectively.Footnote 79 Even though several of the diplomatic search terms are associated with confrontational or negative Chinese policies, when China discusses these topics, it uses, on average, a more positive sentiment then when discussing other issues.
Next, we split the sample into US-related articles and non-US-related articles (Table 3).Footnote 80 Two patterns emerge. First, China’s average sentiment in the US-related articles is consistently higher (more positive) than in the non-US-related articles. Second, variation in China’s sentiment is larger when discussing the United States as seen in the larger standard deviations and negative scores in the minimum column. It appears China is ‘nicer’ when talking about the United States compared to a global sample, but China is also willing to be far more negative.
To isolate China’s reaction to US DIME tools we compare average sentiment between articles that mention the United States and a DIME tool and all non-DIME articles. The results are similar to the original analysis and are found in Appendix Table C1; we summarise the findings here. Diplomatic sentiment in Global Times and MOFA US-related articles is 89.1 per cent and 38.4 per cent more positive (p ≤ 0.001), respectively, compared to non-DIME articles. Information sentiment in People’s Daily and Global Times is 513.2 per cent and 625.7 per cent lower (p ≤ 0.001), respectively, in US articles compared to non-DIME articles. In People’s Daily, Global Times, and MOFA, economic sentiment in US-related articles is 122.0 per cent, 144.6 per cent, and 25.5 per cent lower (p ≤ 0.001) than sentiment in non-DIME articles. And again, military sentiment in US-related articles is slightly lower than in non-DIME articles but is only statistically significant (p ≤ 0.05) in Global Times (65.4 per cent). China is no longer consistently more positive toward the United States than the rest of the world when DIME tools are incorporated into the analysis.
We further restrict the analysis and compare sentiment between US DIME articles and US non-DIME articles. Statistically significant results are summarised here; complete results are in Appendix Table C2. People’s Daily, Global Times, and MOFA information (426.3 per cent, 457.8 per cent, respectively, p ≤ 0.001), military (38.4 per cent, 76.5 per cent, 48.9 per cent, p ≤ 0.05–0.001), and economic (117.4 per cent, 130.3 per cent, 46.6 per cent, p ≤ 0.001) sentiment is lower compared to US non-DIME articles. In this US subset, diplomatic sentiment is now 23.1 per cent (p ≤ 0.01) lower in People’s Daily and is 28.6 per cent higher in Global Times compared to US non-DIME articles.
Next, we compare average sentiment in US DIME and non-US DIME articles. The results in Table 4 show China tends to be less positive, and sometimes uses negative sentiment, when discussing US DIME tools. People’s Daily non-US diplomatic, information, and economic sentiment is 80.8 per cent, 69.9 per cent, and 304.0 per cent higher (p ≤ 0.001), respectively, compared to when discussing the United States. In Global Times only non-US information sentiment is 42.5 per cent higher (p ≤ 0.001) compared to US DIME sentiment. And in MOFA only non-US economic sentiment is 45.9 per cent higher (p ≤ 0.05). There are no statistically significant difference comparing US and non-US military sentiment, which suggests that China may maintain a consistent interpretation and reaction to any country, or adversary’s, military posturing and policies.
We also analyse sentiment scores across DIME tools within the US article set. Table 5 reports the results and shows several statistically and substantively significant differences in China’s reactions to US DIME tools. China reacts negatively to information and economic tools but not military tools (remains on the positive side of the sentiment scale). In People’s Daily and Global Times, average information sentiment is lower (p ≤ 0.01) than diplomatic (524.4 per cent, 378.1 per cent, respectively), military (629.9 per cent, 106.6 per cent), and economic (94.7 per cent, 91.5 per cent) sentiment. US economic tools also generate a lower average sentiment than diplomatic and military tools in People’s Daily (122.6 per cent, 128.2 per cent, p ≤ 0.001) and Global Times (123.6 per cent, 228.9 per cent, p ≤ 0.01–0.001). MOFA economic sentiment is 46.2 per cent lower (p ≤ 0.001) than diplomatic sentiment. Across all three sources, diplomatic tools consistently generate the highest (most positive) sentiment. Perhaps most interesting, when discussing traditional force projection and deterrence military tools, sentiment remains positive in all three sources. While military sentiment is lower (p < 0.01–0.001) than diplomatic sentiment in Global Times (81.7 per cent) and MOFA (48.4 per cent) the analysis suggests that the routineness of traditional force projection does not trigger extreme variation in sentiment.
We expanded the sample to compare articles that contain combinations of codes. This accounts for multiple DIME tools being wielded simultaneously. For example, we compare documents with only diplomatic, diplomatic and information, diplomatic and military, and diplomatic, information, and military to documents with only economic, economic and information, economic and military, and economic, information, and military. Considering combinations of DIME tools may provide stronger and more realistic policy recommendations and assessment of force projection and foreign policy. Investigating DIME combinations highlights the robustness of our preceding methodology because in our all-articles, restricted shared time period, and US-related article sets, 97.1 per cent, 96.9 per cent, and 95.7 per cent, respectively, are single DIME tool articles – meaning they only include diplomatic, information, military, or economic search terms. When multiple DIME search terms are included, the most frequent combination is diplomatic–economic (see Appendix Table C4). In the final series of analyses, we consider all combinations of DIME tools and compare China’s sentiment in articles discussing, for example, diplomatic–information and diplomatic–military–economic. Though we risk double-counting, the results support and nearly mirror the more restrictive primary analysis (see Appendix Table C5).
Information and military or economic search terms do not appear in combinations together but do appear in combination with diplomatic search terms. We find some statistically significant difference in China’s sentiment, but we hesitate to draw strong conclusions from the two-sample t-test analyses because the variation in the number of observations is quite large. We consider the statistically significant results as support for the preceding primary analyses. We continue to find that information tools (information search terms only or when present with other search terms) consistently, and often statistically significantly, elicit a more negative reaction, on average, than diplomatic, military, and/or economic tools (and combinations of these tools). We also find that the combination of diplomatic and economic tools results in lower average sentiment than when either term appears independently; but this is not always a statistically significant relationship.
Discussion
When examining Chinese POSM, we find that Beijing reacts more negatively to information tools (NGO reports on censorship or restrictions on internet access) than to diplomatic, military, or economic tools and the difference in average sentiment is consistently statistically significant. Explanations for why China reacts so negatively to the ‘I’ in DIME are under-theorised. One possible answer is that China responds to information activities in the information space, that is, in the same domain, and thus does not prioritise the use of information when targeted by adversarial economic or military actions. Beijing might prioritise economic/trade responses to adversarial use of economic statecraft (sanctions, tariffs, etc.) or military responses to military exercises by adversaries.Footnote 81 While this explanation is plausible, we cannot draw definitive conclusions in the absence of empirical studies that comparatively examine Chinese cross-domain reactions to a specific instrument used by an adversary. We draw on recent deterrence and Russia-focused scholarship to offer possible alternative explanations.
First, China recognises that in the contemporary international system, information tools are increasingly important for achieving objectives. In a world characterised by a complex media ecology, ‘victory may sometimes depend not on whose army wins, but on whose story wins’.Footnote 82 As noted at the paper’s outset, China regards information as the primary currency of international power in the information age; this includes deterrence concepts that see information as an instrument of conflict, and information dominance as a central focus in future wars.Footnote 83 To succeed – to achieve information dominance – a nation must be capable of both successfully conducting offensive information operations and demonstrating an ability to defend one’s own information resources.Footnote 84 Our findings of heightened sensitivity to information challenges, like the NGO reports criticising censorship, may reflect Beijing’s concerns about defending its own information resources and serve as a public, English-language warning to NGOs and others attempting to affect that control. Put differently, China’s strong reaction to information tools may serve as indicators, warnings, and confirmation of Beijing’s own deterrence strategy and views on the power of information – a country practicing what it is preaching.
Second, authoritarian regimes show a greater sensitivity to information activities as they may threaten regime legitimacy and survival. While economic and military threats present more obvious external material challenges, information threats often target the regime’s institutions and belief systems, including narratives about what is legitimate and illegitimate, the building blocks of history and national identity, and may seek to spur collective action to seek change.Footnote 85 For example, Russia portrays Western anti-Russian information operations as ‘western political warfare […] more of a menace for the Kremlin than nuclear or conventional threats’.Footnote 86 Adamsky’s research may be generalisable to other authoritarian regimes due to the perceived threat that information operations can pose to socio-political systems that rely on information control to ‘protect’ the populace and preserve the status of those in power. Our findings may be illustrating authoritarian concerns about outside threats to information control promoting ‘colour revolutions’ capable of regime change.
Furthermore, information control can influence the ‘rally around the flag’ effect. Adversaries conducting military exercises near a state’s border, imposing sanctions, or exerting diplomatic pressure all offer fertile soil for authoritarians hoping to harness outside threats to increase domestic support – to rally the people around their leader. Again, recent Russia-focused scholarship offers insights, highlighting Putin’s promotion of external threats (while presenting himself as a moderate) to help ensure public support.Footnote 87 The loss, or threatened loss, of censorship and information control do not offer the same opportunities to generate rallies. Instead, the loss of information control may undermine previously successful efforts to rally citizens based on threats from other tools of statecraft. Seen through this lens, information tools, specifically those that reduce a state’s ability to control information, appear to offer capabilities unavailable with other tools. Our findings of Chinese sensitivity to outside information may be serving as an indicator of information’s ability to both obviate the rally around the flag effect and undermine any previous/existing effects created by other tools; Beijing’s English-language negativity here can be seen as an illustration of this ability.
Fourth, being targeted by information tools may generate greater threat perception because this tool can be more indirect, amorphous, and less predictable than overt D-M-E tools. Information tools, and resulting threats, are harder to counter than traditional diplomatic or economic tools that can be countered in a reciprocal manner. Information operations may be more capable of targeting both the government and the population and operating in the grey area between diplomacy and covert warfare. Authoritarian states like China, with government-controlled domestic information environments, may also see themselves as comparatively vulnerable to foreign information operations, especially those that decrease such control. In this strategically ambiguous space, actors can deploy narrative and technological means to shape the information environment of the targeted state without crossing the thresholds that typically define coercive diplomacy, economic sanctions, or acts of war. Authoritarian regimes in particular aim to project an image of strategic composure and control – which may lead them to perceive a heightened threat from less predictable and more provocative critiques by state and non-state actors via information tools.Footnote 88
All, some, or none of these alternate explanations for our findings may be present in Beijing’s strongly negative reactions to information tools. In overall deterrence terms, whichever explanation(s) are salient, it suggests that China’s responses could be an intentional signalling mechanism aimed to deter future use of information tools.
Perhaps as surprising as the power of information tools was the lack of sentiment change when discussing military tools. The POSM analyses suggest that the routineness of traditional force projection does not trigger intense reactions or changes in sentiment. For example, signalling that the activities are exercises and not a prelude to invasion, along with their routineness and lack of key indicators,Footnote 89 appears to be effective at mitigating a rise in tensions. These findings require further research but provide contributions to the study of signalling, ‘cheap talk’, hand tying, and related concepts.
Conclusion
As concepts of deterrence and warfare have evolved to include a broader range of tools and a wider spectrum of conflict, analytical frameworks and assessment tools have not kept pace. Our methodology addresses this gap by providing the ability to use POSM to analyse non-military deterrence tools, and the ability to conduct this analysis across the spectrum of conflict. These contributions are particularly applicable for scholars working on the fourth generation of deterrence theory. While the DIME framework is applicable for our methodology, among our findings was the power of a combination of tools, including information and diplomacy. This highlights the importance of measurement tools flexible enough to conduct assessments across a range of domains and spectrums of conflict.
For policymakers attempting to assess the effectiveness of the tools of statecraft, our findings and POSM methodology offer a path to testable insights into which tools are likely to be most (in)effective in generating public negativity when targeting a particular state, including during periods of competition rather than conflict. This insight applies whether policymakers are attempting to measure possible effects under Chinese, US (integrated deterrence), or other concepts and frameworks.
Our process, using computational text analysis, algorithms, and data visualisation to analyse POSM’s large datasets, offers technologists a pathway for creating an AI capable of rapidly conducting analysis relevant to deterrence and international security.
The case study offers support to our goals of producing a methodology capable of determining a country’s baseline sentiment and then measuring for statistically significant variations from that baseline tied to a state’s reaction to the instruments of national power (even when wielded by NGOs). We look forward to other scholars testing our findings and methodology, using our data or their own. We also note the focus on authoritarian states; the utility of the methodology for democratic states remains unexplored.
If our finding is unique to authoritarian states, as we suspect, it offers promise for democratic state complaints regarding authoritarian interference in their elections and information environments – democratic states could harness their own information tools or work with NGOs of the type examined here to deter or punish such authoritarian behaviour. However, the Trump administration’s March 2025 decision to give up many US information capabilities may signal a lower likelihood of this occurring in the near term, or a decision to securitise information tools for use through primarily military and/or intelligence channels. Separately, as satellite internet becomes more widely available worldwide, people’s ability to circumvent authoritarians’ information controls (which largely operate through domestic internet service providers) are likely to grow.Footnote 90 This technological change should increase the saliency of our findings.
Finally, we acknowledge there may be elements of deception, or mis-/disinformation within the data. However, the breadth and temporal scope of the data, plus the ongoing, iterative quality checking and triangulating qualitative analyses, help mitigate against a loss of explanatory power. In our previous research, we found a coordinated campaign across Russian POSM relevant to these findings.Footnote 91 Initially, Russian outlets responded similarly to their Chinese counterparts when criticised over censorship or restrictions on internet access; however, in the late 2000s the Russian response changed – negativity levels decreased. A qualitative examination revealed a pivot in Russian discussions of the information-related criticisms. Rather than discussing Russian performance on related NGO reports, Moscow highlighted first Georgia’s poor performance, then poor US performance in the early 2010s, then poor Ukrainian performance from 2014. Thus, while elements of deception or mis-/disinformation activities are likely to appear in the data, careful analysis, in addition to the breadth and temporal scope of the data, can be employed to uncover and mitigate harm to the overall findings.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/eis.2025.10021.
Acknowledgements
The authors would like to thank Nicholas Donaldson (Leiden University, Institute for Security & Global Affairs) for support in data cleaning and coding.
Scott Fisher is an associate professor in the Security Studies Department at New Jersey City University. Prior to his PhD studies at Rutgers University, he received an MA in Security Studies from Georgetown University and a MA in Korean and International Studies from Seoul National University in South Korea. His research focus is information warfare, US security challenges in Asia, and open-source intelligence. His research has appeared in publications including Foreign Policy Analysis, RAND, International Journal of Korean Unification Studies, Journal of Information Warfare, and West Point’s Modern War Institute.
Graig R. Klein is an assistant professor (permanent contract, equivalent of tenure) in the Institute of Security and Global Affairs (ISGA) at Leiden University. His research focuses on the strategic use of political violence by non-state actors (NSAs) and governments, primarily focusing on terrorism, counterterrorism, protests, and repression. It is published in leading peer-review journals including International Organization, Terrorism and Political Violence, Studies in Conflict & Terrorism, Foreign Policy Analysis, and Conflict Management and Peace Science. He earned his PhD in Political Science from Binghamton University.
Juris Pupcenoks is Professor of Political Science at Marist University. He has held various leadership positions with the International Studies Association (ISA), including serving as Chair of the Programs Committee of the Ethnicity, Nationalism, and Migration Studies Section. His research explores diplomatic communication on conflict, strategic narratives, disinformation, threat perception, and securitisation, as well as Russian-speakers abroad. His publications have appeared in journals including International Interactions, The International Journal of Press/Politics, and Nationalities Papers.
Juste Codjo is Associate Professor of Security Studies and Director of the Doctor of Science in Civil Security at the New Jersey City University. His research focuses on national and international security, political violence, peace operations, governance, and stability in the developing world, with a regional focus on Africa and US–Africa strategic relations. His work has been published in academic journals, including African Studies Reviews, Foreign Policy Analysis, and International Studies Review. He recently contributed a chapter to a handbook on African intelligence cultures published at Rowman & Littlefield (2023). Prior to joining academia, Dr Codjo served as an army officer for more than twenty years in the Benin Army (West Africa). He is a graduate of the German Offizierschule des Heeres and of the US Army Command and General Staff College.
Open access
