Hostname: page-component-7b7567c8c-2rsb6 Total loading time: 0 Render date: 2026-11-12T09:16:25.292Z Has data issue: false hasContentIssue false
  • English
  • Français

Measuring Ideas More Effectively: An Analysis of Bush andKerry's National Security Speeches

Published online by Cambridge University Press:  06 October 2005

Cheryl Schonhardt-Bailey
Cheryl Schonhardt-Bailey
Affiliation:
London School of Economics and Political Science
Rights & Permissions [Opens in a new window]

Extract

On Sunday, October 10th, 2004, the New York TimesMagazine featured an article with the cover title,“Really, What Does He Think? John Kerry and thePost-9/11 World” (Bai 2004). On the coverof the magazine was a serious-looking photo of Senator Kerry,superimposed with keywords such as “Terrorism,” “Iraq,” “Al Qaeda,”“Multilateralism,” “Nuclear proliferation,” and so on. While thearticle itself was intriguing, even more intriguing was themagazine's attempt to capture Kerry's core ideas on Americannational security with the use of keyword graphics—namely, thekeywords on the cover, placed in what appeared to be a random orderaround the photo of Kerry, and the underlining of “John Kerry,”“terrorism,” and “Americans” in the inside title. Catchy graphics,but hardly an accurate depiction of the keywords that might actuallyrepresent Kerry's thinking on American national security. And, forall the comparison made in the article itself with President Bush'sstance on national security, where were the graphics for George W.?(They did not emerge in the next New York TimesMagazine.) The magazine was, nonetheless, making animportant point: that words (and the ideas they represent) areemotive—particularly in the highly charged climate of the 2004presidential campaign.I am gratefulfor comments and suggestions from Andrew Bailey, Diane Maurice,David Mayhew, and Frances Rosenbluth. I am also grateful to theGeorg Walter Leitner Program in International and ComparativePolitical Economy (Yale Center for International and AreaStudies) for funding that initiated this article, and to MinaMoshkeri (LSE Design Unit) for her assistance in preparing thegraphs.

Information

Type
Features
Information
PS: Political Science & Politics , Volume 38 , Issue 4 , October 2005, pp. 701 - 711
Copyright
© 2005 The American Political Science Association

On Sunday, October 10th, 2004, the New York Times Magazine featured an article with the cover title, “Really, What Does He Think? John Kerry and the Post-9/11 World” (Bai 2004). On the cover of the magazine was a serious-looking photo of Senator Kerry, superimposed with keywords such as “Terrorism,” “Iraq,” “Al Qaeda,” “Multilateralism,” “Nuclear proliferation,” and so on. While the article itself was intriguing, even more intriguing was the magazine's attempt to capture Kerry's core ideas on American national security with the use of keyword graphics—namely, the keywords on the cover, placed in what appeared to be a random order around the photo of Kerry, and the underlining of “John Kerry,” “terrorism,” and “Americans” in the inside title. Catchy graphics, but hardly an accurate depiction of the keywords that might actually represent Kerry's thinking on American national security.1

For interested readers, I have posted a picture of this cover on my web site: http://personal.lse.ac.uk/schonhar/.

And, for all the comparison made in the article itself with President Bush's stance on national security, where were the graphics for George W.? (They did not emerge in the next New York Times Magazine.) The magazine was, nonetheless, making an important point: that words (and the ideas they represent) are emotive—particularly in the highly charged climate of the 2004 presidential campaign. As most commentators have concluded, ideas mattered in this election, but as yet, we are unsure as to which ideas mattered and why. If political scientists are to take ideas seriously, then we should explore more effective tools with which to measure ideas, and, ideally, subject them to rigorous empirical analysis. However, no one yet has provided an accepted toolkit with which researchers might be able to measure confidently and represent spatially (certainly better than New York Times Magazine graphics) the ideas at play in the 2004 presidential election.

In the months preceding the election, one of the key issues was the War on Terrorism, and, in particular, Bush's ability to manage the crisis in Iraq. Many commentators expressed the view that war and global terrorism would be the key defining factor in the election. They were taken aback to find that 22% of exiting voters told pollsters that “moral values” was the most important issue of the campaign (ignoring, incidentally, that 78% ranked other issues more important, and that survey attitudes on abortion and gay marriage had remained unchanged from 2000) (Anonymous 2004a; Elving 2004). One NPR pundit noted that:

we came down heavily on the Moral Values Story because it seemed to explain how a nation dyspeptic about the economy and the war in Iraq could vote for an incumbent president responsible for both. We latched onto the ‘moral values’ explanation because it came readily to hand, and it was simpler than the panoply of alternative explanations. (Elving 2004)

The Economist, in contrast, theorized that “moral values” might not be “just a matter of social conservatism but also code for trust in the candidate, or respect for a man's willingness to take a stand—where Mr Bush won easily. Mr Kerry never quite managed to persuade voters of his leadership qualities” (Anonymous 2004a). And in a later analysis, The Economist noted that Bush's “moral majority” spread well beyond evangelicals to include mainstream Catholics and Protestants, thus forming a “traditionalist” constituency who maintain “that the president should have strong religious beliefs” (Anonymous 2004b).

So what was Bush's message? What were the words or themes that seemed to appeal to the American electorate? And how were these different from those used by Kerry? We are not short of speeches and statements (not to mention, independent analyses) of their respective messages, but what we do lack is a systematic and empirical textual analysis of these. In this article, I use computer-assisted text analysis to measure statistically and map spatially the key themes espoused by Bush and Kerry on the issues of national security and homeland security. I use a unique software, called Alceste, which essentially integrates text analysis with some statistical analysis. The result is an automatic (that is, computer-generated) classification of the thematic classes from Bush's and Kerry's speeches on national and homeland security. What is important to note, however, is that it is possible to begin the analysis with no pre-set hypotheses about the apparent appeal of Bush's message; in this sense, the method is very different from traditional programs that follow the instrumental approach but require the researcher to specify the coding and categories in advance. (For an overview of these programs, see www.textanalysis.info.)2

I am grateful to Roel Popping for helping to clarify some of the differences between Alceste and other text analysis programs. For more detailed analysis of these other programs, see Popping 2000. A detailed review (Garson 2003) of two somewhat similar programs, TextGrab and TextQuest, uses as an example the entire web sites of two ideologically distinct U.S. Senators—Jesse Helms and Hillary Clinton. These packages allow verbal images to be portrayed in property space, but do not create the more extensive spatial analysis available in Alceste. See also, Garson 2002.

Rather, we may begin with a simple notion that values mattered in the context of national security, but then explore more precisely how they mattered. To summarize in advance, the results of the analysis suggest that the 2004 election pitted emotive appeal against logic, and in the end, emotive appeal appears to have won.

The textual data consists of key speeches by Bush and Kerry on the issues of national and homeland security, as listed in Table 1.3

Speeches were selected on the basis of their prominence on campaign or White House web sites. “Plans” were not selected because the text was shorter and written in bullet-point fashion, and thus its format differed substantially from speeches and thus would likely have distorted the textual analysis. Also, no speeches by the two vice presidential candidates were included. Finally, while the transcripts of the presidential debates were initially included in the analysis, the rigid question and answer format was significantly distinct from the prose of speeches so as to create a distortion in the textual analysis.

These include Bush's critical National Security Strategy document of 2002 (a collection of speeches which encapsulates the Bush Doctrine) and his subsequent speeches on national and homeland security in 2004, along with Kerry's speeches on these issues during the 2004 campaign.

Key Bush and Kerry Speeches on National and Homeland Security

Methodology: Computer-Assisted Content Analysis

Computer-Assisted Content Analysis in Political Science

Many researchers in political science have used classical content analysis to describe textual data.4

This form of analysis provides a systematic and transparent way of managing large amounts of text. It also relies upon mainly “naturally occurring raw data”—e.g., newspapers, speeches, letters, public documents—which avoids problems of reactivity of the respondent that may occur in interviews. And, finally, content analysis provides researchers with well-documented procedures (Bauer 2000, 147). But content analysis can also suffer from problems of sampling and coding. The former raises familiar issues of representativeness, sample size and the unit of sampling, while the latter involves issues of the nature of the categories, the organization of the coding scheme, and the adequacy of the coding process and coders (136, 139). In particular, interpretation of the material must be reliable (that is, coding must be consistent between and among coders) and it must be valid (that is, the codes must relate to the words used in the text and the sample must represent the whole text) (143–44).

Computer-assisted content analysis—such as Alceste—offers a way to surmount the difficulties of traditional content analysis, while at the same time producing results that are entirely consistent with it (Allum 1998). Alceste stands in stark contrast to classical content analysis in four ways. First, it is an automatic procedure that is insensitive to meaning and context. This may result in missed nuances, but it also guards against researchers and coders infusing their own biases into the coding and analysis. Second, it can provide an impression of a voluminous data corpus within a very short space of time. Third, and following on from that, the issue of reliability which arises with human coding is no longer relevant. Fourth, because large amounts of text can be analyzed quickly—which means that sampling may not be required—problems of sampling may also disappear.5

Here, a text file of approximately 74,000 words is analysed by the program in about five minutes.

Computer-assisted analysis of political texts has, moreover, recently captured the attention and imagination of some political scientists (Gabel and Huber 2000; Laver and Garry 2000; Garson 2002; Laver and Benoit 2002; Laver, Benoit et al. 2002), and has received well-deserved praise: “The ability to analyze vast amounts of text quickly and cheaply has the potential to revolutionize the study of politics” (Laver, Benoit et al. 2002, 3).6

Recent work by social psychologists has also applied computer-assisted content analysis to address issues of political economy—see Oberlechner, Slunecko et al. 2004. Other applications include business administration (Kabanoff 1996; Kabanoff and Holt 1996) and market research (McDonald 1982).

Alceste

Alceste relies upon co-occurrence analysis, which is the statistical analysis of frequent word pairs in a text corpus. Alceste was developed by Max Reinert (1983; 1998) and was originally used in the humanities (1993), although its use has recently spread to the social sciences (Noel-Jorand, Reinert et al. 1995; Lahlou 1996; Allum 1998; Lahlou 1998; Wagner and Kronberger forthcoming) and to political science (Brugidou 1998; Brugidou 2000; Bailey and Schonhardt-Bailey 2005; Schonhardt-Bailey 2006). It has been described as a “methodology” insofar as it “integrates a multitude of highly sophisticated statistical methods into an organic whole that perfectly suits its aim of discourse analysis” (Kronberger and Wagner 2000, 306).7

“Taken together, the program realizes a complex descending hierarchical classification combining elements of different statistical methods like segmentation (Bertier and Bouroche 1975), hierarchical classification and dichotomization based on reciprocal averaging or correspondence analysis (Hayashi 1950; Benzecri 1981; Greenacre 1993) and the theory of dynamic clouds (Diday, Lemaire et al. 1982)” (Kronberger and Wagner 2000, 306).

More simply, it may be described as a marriage of textual and statistical analysis (Popping 2004).

Because Alceste is automatic (that is, the categories are generated by the program, not by the researcher), it is different from other qualitative software that supports manual content analysis—e.g., Atlas.ti or Nudist (Barry 1998). Similar to TextQuest, Alceste facilitates quantitative analysis, following in the tradition of Iker (1974; Iker and Klein 1974), and more recently, Miller (1997) and others (Hogenraad, Bestgen et al. 1995). In short, a variety of packages are on offer for computer-assisted content analysis. While other programs are useful for some purposes, Alceste is better suited for an analysis of political speeches for three reasons. First, following minimal editing, the text is ready for analysis. Second, Alceste proposes classes (or themes) based on word lists and characteristic phrases. Both key words and sentences are, moreover, ranked in terms of their statistical significance (of which more is said below), and both can be traced back to the original text so that we may evaluate their context. Third, the technique generates correspondence analysis so that the speaker may be mapped onto the same policy space as the identified classes or themes.8

Further tags could be added to test various hypotheses—e.g., the timing of the speech, to evaluate changes in positions over time.

The value of Alceste will be illustrated more clearly in the reporting of the results.

There are two preconditions for good results with Alceste: (1) the textual data must be coherent (that is, it must focus on one topic); and (2) the text must be large enough for the statistical output to be relevant (with a minimum of 10,000 words). The software is particularly adept at analyzing naturally occurring (or non-reactive) textual data (Kronberger 2004). The speeches from Table 1 fit these preconditions precisely: the speeches all relate to national security, the total word count is roughly 74,000 (with Kerry's speeches contributing about 3,000 more words than Bush's), and the textual data are non-reactive.

Alceste determines word distribution patterns within a text, with the objective being to obtain a primary statistical classification of simple statements (or “contextual units”)9

For Alceste, “statements” are defined as “contextual units.” The program automatically determines contextual units with reference to punctuation and the length of the statement up to a maximum of 250 characters. This description of Alceste follows Kronberger and Wagner 2000.

in order to reveal the most characteristic words, which in turn can be distinguished as word classes that represent different forms of discourse concerning the topic of the text.10

Through its dictionary, Alceste prepares the text by reducing different forms of the same word (in the form of plurals, suffixes, etc.) to the root form and transforms irregular verbs to the indicative, thereby producing a matrix of reduced forms. It also subdivides the corpus into “function words” (articles, prepositions, conjunctions, pronouns, and auxiliary verbs) and “content words” (nouns, verbs, adjectives, and adverbs). The content words are understood to carry the meaning of the discourse and the final analysis is based on these. (Content words are sometimes referred to as the “meaningful words.”) The program creates a data matrix (an “indicator matrix”) which allows an analysis of statistical similarities and dissimilarities of words in order to identify repetitive language patterns. This matrix relates relevant words in columns and contextual units in rows, so that if a given word is present, a 1 is entered in the cell; otherwise, the entry is 0. Then, using descending hierarchical classification analysis, the program identifies word classes. (The term “class” is used for descending hierarchical classification analysis while the term “cluster” is used for the more traditional ascending cluster analysis (Kronberger and Wagner 2000, 308).) The first class comprises the total set of contextual units in the initial indicator matrix. The program then attempts to partition that class into two further classes that contain different vocabulary and ideally do not contain any overlapping words. The methods used for this are optimal scaling and the adoption of a maximum chi-squared criterion for cutting the ordered set of words. Alceste compares the distribution of words in each of the two new classes with the average distribution of words. Different forms of discourse that use different vocabulary will result in an observed word distribution that deviates systematically from one where the words are independent of each other. The procedure searches for maximally separate patterns of co-occurrence between the word classes. The chi-squared criterion is thus used as a measure of the relationship that exists between words, rather than as a test.

Following an iterative process, the descending hierarchical classification method decomposes the classes until a predetermined number of iterations fails to result in further divisions. The result is a hierarchy of classes, which may be schematized as a tree diagram (e.g., Figure 1, discussed below).

Tree Graph of the Classes for Bush and Kerry on National and Homeland Security

Results of Analysis of National and Homeland Security Speeches

Identifying the Themes

Table 2 provides a summary of the basic statistics from Alceste. The total word count for the text file is 73,715 and of these, 34,883 are unique words that were analyzed by the program.11

Plurals and conjugation endings are reduced to a single form and nonce words are eliminated from the analysis. The leaves a smaller word count which is analyzed by the program.

The passive variables12

These are deemed “passive” as they do not contribute to either the calculation of the word classes or the factors in the correspondence analysis.

(also referred to as tagged indicators) are units of the text that may be identified according to certain characteristics, and here there were just two, “Bush” and “Kerry.” (Others could have been introduced—e.g., the date of the speech.)

Basic Statistics for Key Bush and Kerry Speeches on National and Homeland Security

The “Initial Context Unit,” or ICU, is essentially the sampling unit—i.e., a pre-existing division of the text specified by the user. Here, each “case” constitutes an ICU and so, Table 2 notes that we considered just two cases, namely Bush's speeches and Kerry's speeches.13

Alternatively, each speech could have been considered as a separate case, but as the goal of this analysis was to illustrate the methodology, the approach was kept as simple as possible.

The “Elementary Context Unit,” or ECU, is a “gauged sentence,” which the program automatically constructs based upon word length and punctuation in the text.14

Popping notes that the ECU is akin to the “recording unit” used in other programs, where it is usually defined by the researcher (Popping 2004).

Using the presence or absence of words in each ECU, the program calculates matrices on which to build the classification process.15

A simple analogy is given to understand the relationship between contextual units, ICUs, and ECUs: “a contextual unit is to an ICU what a paragraph is to a chapter, and to an ECU, what a paragraph is to a sentence” (Reinert 1998, 11).

(Table 3 provides examples of ECUs.) The program conducts two preliminary analyses, each using slightly different lengths for the contextual unit.16

A contextual unit is equivalent to one or more successive ECU(s). The two calculations are done with two different parameters for the selected number of words per contextual unit in order to check the reliability of the classes and the stability of the results (Reinert 1998, 14).

It then opts for the length that successfully classifies the greater proportion of ECUs relative to the total available. From Table 2 we can see that 1,253 ECUs were classified, equating to 69% of the ECUs retained for analysis.

Bush & Kerry Speeches on National & Homeland Security: Examples of Most Typical ECUs in each Class

Table 3: Continued

The final two rows in Table 2 indicate the number of classes identified and the size of each class (as measured by the percentage of the total ECUs classified within each). In total, seven classes are identified in Bush and Kerry's speeches. The labels for each class (e.g., Iraq War Critique, and so on) are not, however, automatically given by the program. The output provides the researcher with a number of different tools for conceptualizing the content of classes. Of the many tools, two are particularly useful. The first is a list of the most characteristic function words for each class, along with their χ2 statistical significance (with the minimum χ2 value for selection set at 2.13, below which the level of statistical significance fails to reach the 10% level, using the standard chi square table with 1 degree of freedom). The most characteristic words are those with high χ2 values. Words ending with “+” indicate that these are reduced forms (e.g., president+ may refer to president, presidential, or presidents). For Class 1, highly representative words include Iraq+, President+, dollar+, Bush+, and wrong (with χ2 values of 148, 115, 80, 69, and 60, respectively). For Class 2, they are thank+, honor+, famil+, service+, men, and brothers (with χ2 values of 221, 117, 97, 67, 65, and 64, respectively). For Class 3, they are relation+, human, interest+, dignity, democracy+, institut+, peace, and principle+ (with χ2 values of 79, 71, 62, 59, 55, 49, 48, and 48, respectively). For Class 4, they are applause17

The word “applause” refers to transcripts of speeches where the applause from the audience was recorded. For sake of transparency and completeness, the word was retained in the textual analysis and—for some research purposes—may serve a purpose. This was a distinct feature of some of Bush's speeches, while none of Kerry's speeches recorded applause from the audience. This finding in itself may be noteworthy, but is not explored further in this paper.

, kill+, September, see+, there's, guilt+, terror+, innocent+, and enemy+ (with χ2 values of 126, 82, 65, 58, 49, 43, 40, 38, and 37, respectively). For Class 5, they are force+, police, guard+, equipment+, milit+, firefighter+, and homeland (with χ2 values of 112, 110, 109, 70, 63, 63, and 56, respectively). For Class 6, they are nuclear, weapon+, material+, Iran+, Soviet, rogue, North, and Korea+ (with χ2 values of 361, 272, 201, 112, 103, 90, 84, and 76, respectively). Finally, for Class 7, they are develop+, growth, aid+, financ+, poor+, countries+, and econom+ (with χ2 values of 253, 193, 168, 155, 152, 150, and 131, respectively). While these word lists help the researcher identify the content of the classes, their contextual meaning is not obvious. For contextual meaning, we examine the most representative ECUs for each class in order to better identify their thematic content, which leads us to the second tool.

Table 3 provides examples of two of the top four most representative ECUs for each class, in which the context is given for the characteristic words, and where these characteristic words are indicated in bold.18

The standard report lists the top 20 ECUs for each class, ranked by chi square association. However, a separate file is produced that lists all the ECUs for each class, where the default cut-off for selection is zero.

From both the word lists and the ECUs, we can arrive at conceptual headings for the classes. For instance, in Table 3, the highest scoring ECU from Class 1 is:

He should use more Iraqi contractors and workers, instead of big corporations like Halliburton. He should stop paying companies under investigation for fraud or corruption. And he should fire the civilians in the Pentagon responsible for mismanaging the reconstruction effort. Fourth, the President must take immediate, urgent, essential steps to guarantee the promised elections can be held next year.

This ECU and the remaining 20 top ECUs for this class, together with its most characteristic words (Iraq+, President+, dollar+, Bush+, and wrong) strongly suggest that this class is a critique of Bush's war in Iraq. Moreover, as the program also provides the χ2 statistic for the most representative tag (recall that we have just two tags—“Bush” and “Kerry”), not surprisingly the Kerry tag is the only associated tag for this class, with a χ2 value of 162. (Since the cut-off value is 2.13, this means that the Bush tag scored below 2.13 for this class, or failed to obtain significance at the 10% level.)

In a similar fashion, we arrive at thematic headings for the remaining six classes, along with their associated tag. Class 2 consists of Kerry's attempt to identify himself with Fellow Veterans by making an emotive appeal to the families of servicemen and women for their support. For instance, from the third ranking ECU (Table 3), Kerry's appeal to service personnel is clear:

And to my fellow veterans, the band of brothers who have been with me for so long and to whom I owe so much, I pledge that unlike the time when we fought side by side, I will be a president who does what's right for our men and women in uniform.

Bush's appeal in Class 3 is quite different. Rather than invoking Kerry's image of shared wartime experiences and sacrifice (and thus encouraging an unfavorable comparison with Kerry's military service), he invoked the image of America as defender and proponent of Western, democratic values. Following the arguments of Theodore Roosevelt, Harry Truman, and Ronald Reagan, for whom American power and American ideals were intertwined in foreign policy (Durham 2004), he painted an image of America's pursuit of “freedom,” “democracy,” “political and economic liberty,” “human dignity,” and “peace” in world affairs. I thus label this class Democratic Institutions, Human Dignity, and Peace.

Class 4 consists of Bush's definition and exposition of the War on Terror, as the leading ECU clearly demonstrates:

There's nothing they can do to intimidate, to make us change our deepest belief. They're trying to kill to shake our will; we're too tough, too strong, too resolute, and too determined to ever have our will shaken by thugs and terrorists [applause].

This class is essentially Bush's rallying cry for peace through strength, akin to Ronald Reagan's appeal for a military build-up in the face of an “evil empire” (Reagan 1983) or Franklin Roosevelt's characterization of America as the “great arsenal of democracy” (Roosevelt 1940).

In Class 5, Kerry presents his vision of Homeland Security, in which he highlighted the need for more funding and more personnel, as opposed to Bush's “under-funded” provision. As illustrated in the top ranking ECU (Table 3), Kerry argued that:

The threats of terrorism and the conflicts of the future can only be met with more engineers, more military police, more psychological warfare personnel and civil affairs teams more special operations forces and more training for peace keeping missions.

Class 6 proposes the distinct theme of Nuclear Non-Proliferation. While Bush did not ignore this topic, Kerry gave it far more attention and clarity, proposing, for example, “a verifiable global ban on the production of materials for nuclear weapons” (Table 3).

And finally, in Class 7, Bush linked U.S. national security to Economic Growth in Less Developed Countries, arguing that by contributing to improvements in “education, health, rule of law and private sector development” (Table 3) the U.S. would boost prosperity and spread the values of freedom and democracy in the rest of the world.

One might, at this point, ask what we have gained from this analysis. The astute observer of the 2004 campaign may very well have compiled (from these same speeches or other statements) a reasonably accurate list of thematic topics associated with each candidate. Yet, our observer would have found the task of assigning quantities to the relative importance of the classes and empirically measuring their linkages far more difficult. Turning to these linkages, the real value of this analysis becomes apparent.

Linkages between the Themes

Identification of the word classes and thereby the themes enables us to describe Bush and Kerry's discourse on national security matters, but Figures 1 and 2 help us understand the relative importance of and the relationships between those themes.

Correspondence Analysis of Classes for Bush and Kerry on National and Homeland Security

Tree Graph

Figure 1 is a tree graph of the classes, schematized according to Alceste's descending hierarchical classification procedure (the percentage weight given to each class by the analysis is indicated in parentheses). Beginning at the “trunk” of the tree (from right to left—that is, from least related classes to most related classes), we see that the speeches contain two basic dimensions—one is U.S. Specific and the other focuses on issues relating to the Global Order. This means that U.S. national security (as expressed by Bush and Kerry) splits into matters pertaining specifically to the U.S., and matters of order and security in the global arena. Perhaps not surprisingly, the U.S. Specific dimension receives greater attention with around 58% of the total classification, while the Global Order dimension comprises 43%.

Following the U.S. Specific path, we observe two further clusters. Forces/Personnel consists of Kerry's dual critiques of the Iraq War and Homeland Security, in which Kerry focused extensively on numbers of personnel and equipment given to each effort. A second cluster is more emotive. Gratitude versus Fear highlights a clear difference in how the candidates appealed to the hearts and minds of the American electorate. Kerry, on the one hand, invoked the image of shared military experience and sacrifice (Fellow Veterans) while Bush painted the world as a fearful and evil place (War on Terror), but one in which terrorism can be overcome with American strength and courage. Thus, when focusing on national security as it pertains specifically to the U.S., Kerry tended to emphasize—and criticize—practical matters (Forces/Personnel), over emotive concerns (30% to 14%, respectively). This comes as no surprise as most observers were well aware of the on-going difficulties in Iraq and in administering Homeland Security (e.g., the controversial PATRIOT Act).

More surprising is the second, more emotive cluster related specifically to the U.S., where Kerry relied upon the appeal of shared military experience, bravery, and sacrifice while Bush invoked images of fear and courage. It is here that we see distinctly the different means to the same goal of winning the hearts and minds of Americans. In sum, Kerry appeared to dominate the public discourse on the U.S. specific aspects of national security (particularly the management of forces and personnel19

Notably, some of Bush's speeches from 2002 predated Kerry's critiques of Iraq and Homeland Security, but nonetheless, Bush's speeches from 2004 did not appear to offer much response to Kerry's attacks.

), leaving Bush to focus exclusively on the emotive theme of the War on Terror.

Turning to the Global Order path, a link can be seen between Bush's emphasis on Democratic Institutions, Human Dignity, and Peace, and his call for further Economic Growth and Development in poor countries. Both of these themes stress American liberal values and a free market economy as core to American foreign policy, one of the planks in the Bush Doctrine. Kerry's call to Global Order—i.e., his concern for Nuclear Non-Proliferation—is quite different in contrast. While both candidates sought to provide Americans with a rationale for international engagement in a post-9/11 world, Bush embedded his argument in the tradition of American “exceptionalism,” where “promoting liberal values abroad” is “seen as a principal U.S. value” (Lepgold and McKeown 1995, 372). Some describe the Bush Doctrine as “neo-conservative” in which America is perceived as the “benevolent hegemon … fighting terrorism and seeking to extend democracy” (Durham 2004, 265) while others maintain that it follows in the tradition of the Monroe and Truman Doctrines, seeking to “foster a world environment where the American system can survive and flourish” (quoting Paul Nitze's phrase from the famous NSC 68 memorandum (Donnelly 2003, 1)). It might be argued more simply that Bush seeks to create a vision of the international order that both resonates with the American people (i.e., squares well with U.S. national interests) and adheres to core liberal (and American) values. As Ruggie has noted, visions of international order as espoused by American presidents should not be dismissed as mere rhetoric, with “sophisticated leaders” being “forced to play on [Americans'] idealism to get anything done” (1997, 119). Rather, the interplay between ideas and interests is usually very complex. In capturing this interplay between American ideas and American interests, Bush won the “visionary prize.” As the aforementioned New York Times Magazine article noted, “Kerry seemed to offer no grand thematic equivalent.” Inasmuch as Kerry's vision of a global order (as captured in his speeches) appears to have rested upon the more narrow (but arguably equally important) goal of nuclear non-proliferation, it was both more limited in scope and had less emotive punch.

Correspondence Analysis

The results from Alceste's classification can also be represented graphically in correspondence space. The program cross-tabulates classes and words in their root form in order to create a matrix which can then be subjected to factor correspondence analysis (Greenacre 1993).20

While correspondence analysis is well-established in the French literature (see Benzecri 1973 and the journal Cahiers de l'Analyse des Donnees) its use has spread with the publication of English applications (Greenacre and Underhill 1982; Greenacre 1984; Weller and Romney 1990; Greenacre 1993). Correspondence analysis has only recently received attention by political scientists (Blasius and Thiessen 2001). Correspondence analysis using numerical data is available in several major statistical packages, including BMDP, SPSS, and SAS.

In this way, we obtain a spatial representation of the relations between the classes, where distance reflects the degree of association.21

For this, correspondence analysis uses the “chi-squared distance,” which resembles the Euclidean distance between points in physical space. However, in correspondence analysis, each squared difference between coordinates is divided by the corresponding element of the average profile (where the profile is a set of frequencies divided by their total). The justification for using the chi-squared concept is that it allows one to transform the frequencies by dividing the square roots of the expected frequencies, thereby equalizing the variances. This can be compared to factor analysis, where data on different scales are standardized. Greenacre provides further geometric reasons for using the chi-squared distance in correspondence analysis (Greenacre 1993, 36).

Correspondence analysis aims to account for a maximum amount of association22

Correspondence analysis usually refers to the “inertia” of a table, which can also be called “association” (Weller and Romney 1990). A corresponding chi-squared value can be obtained by multiplying the association value by the total n of the table.

along the first (horizontal) axis. The second (vertical) axis seeks to account for a maximum of the remaining association, and so on. Hence, the total association is divided into components along principal axes. The resulting map provides a means for transforming numerical information into pictorial form. It provides a framework for the user to formulate her own interpretations, rather than providing clear-cut conclusions.23

The association and chi-squared statistic may be interpreted geometrically as the degree of dispersion of the set of rows and columns (or, profile points) around their average, where the points are weighted.

Figure 2 presents a map of the correspondence analysis of the classes for Bush and Kerry. (When a large number of classes is identified, the program occasionally fails to locate the center point for the class with the fewest representative words, which in this case is Class 7. Hence, I have estimated the position of Class 7 from the correspondence analysis of the representative words, as illustrated in the Appendix, Figure 3.24

As with problems of degrees of freedom and multicollinearity in regression analysis, the way to resolve this difficulty is to obtain more information—e.g., add more speeches. In this case, however, more speeches (of equivalent and suitable nature) were not readily available.

) The two tags for “Bush” and “Kerry” are superimposed into the same correspondence space; and as stated above, distance between the classes and tags reflects the degree of association. To the side of the map, we can see that the first two factors together account for just 52% of the total association, with the first factor accounting for about 28%. The relatively moderate degree of association within a two-dimensional space, along with the large number of classes, suggests that Bush and Kerry's discourse on national security contains multiple cleavages.25

In total, five factors are identified in the correspondence analysis. Had the file size been larger, the program would have more information (equivalent to degrees of freedom) with which to plot Class 7, thus creating six factors. (Usually, the dimensionality of the system is one less than the number of classes in the profile (Greenacre 1993: 14).)

Nevertheless, two of these cleavages account for over half the variation and therefore should be considered prominent.

Correspondence Analysis of Classes for Bush and Kerry on National and Homeland Security, Representative Words

Figure 2 illustrates a primary dimension26

It should also be noted that the concept of dimensionality in textual analysis is distinct from that in the analysis of votes. For a discussion of the dimensionality of voting vis-à-vis the dimensionality of textual analysis, see Schonhardt-Bailey 2006.

along the horizontal axis, namely the same Global Order–U.S. Specific cleavage that we saw in the tree diagram. Thus, Kerry's dual critiques of Bush's failings in Iraq and the administration's failure to provide adequate Homeland Security (Classes 1 and 5), Kerry's call to fellow veterans (Class 2), and Bush's War on Terrorism message (Class 4) all fall into the right-hand quadrants, while Kerry's push for nuclear non-proliferation (Class 6) and Bush's dual themes of spreading democratic institutions and fostering economic growth (Classes 3 and 7) fall into the left-hand quadrants. This is simply another way of observing that the basic content of the seven classes divided into those relating mostly to the U.S. and those concerned with maintaining global order.

As we could have predicted from the tags associated with each of the classes, Kerry is identified relatively more closely with the U.S. specific issues while Bush is linked more with those relating to the global order. At first glance, this seems somewhat counterintuitive. If Bush appeared to be relatively more concerned with the global order and relatively less with U.S. specific issues, then why did he prevail among the American voters (and, for that matter, become the demon of electorates elsewhere in the world)? This analysis does not, of course, gauge the responses to the candidates' speeches, but if we turn to the vertical axis, we can at least begin to appreciate Bush's appeal to (some of) the American electorate. The vertical axis can be interpreted as separating themes that were expressed in more emotive terms (in the top quadrants) from those expressed using more practical language (in the bottom quadrants). Classes 1, 5, and 6 all fall in the bottom quadrants. All three of these classes were linked to Kerry and all had the common thread of money (billions of dollars wrongly spent in Iraq), personnel (more specialists required for specific Homeland Security tasks), and weaponry (securing nuclear weapons and equipment). In essence, John Kerry was proposing a new way to Manage the Military, one that he argued differed considerably from that of the Bush administration. Bush did not significantly engage in this discourse and thus the lower two quadrants of the spatial diagram belong to Kerry alone. (A simpler way to think of this is that Kerry trumped the Managing the Military verbal clash between the candidates.)

Meanwhile, Bush was fighting another battle—the battle for the Hearts and Minds of the American electorate. In the upper quadrants we find that the common thread between Classes 2, 4, 3, and 7 is an emotive appeal to shared values. In Class 2, Kerry invoked his Vietnam wartime experience to capture the hearts of American veterans and their families, promising that, “As president, I will always remember that America's security begins and ends with the soldier, sailor, airman and marine—with every man and every woman in our armed forces standing at a post somewhere in the world” and that “patriotism isn't just about saying you love your country, it's about living it every single day.”27

Quotes from the 6th and 7th ranked ECUs from this class.

This was an appeal to American patriotism. In contrast, Bush tugged at American heartstrings with a different emotive appeal, that of American exceptionalism. As noted above, by linking U.S. national security with the spread of democratic values and economic growth, and by pitting good against evil in the War on Terrorism, Bush's rhetoric evoked powerful and appealing images in the minds of the electorate. While hindsight is a wonderful thing, the simple conclusion of this full-text analysis is that Kerry's emotive message struck a fairly shallow chord.

A final observation from Figure 2 is the position of the “Bush” and “Kerry” tags in the two-dimensional space. On the horizontal axis, Kerry is positioned in the U.S. Specific quadrants, while Bush falls in the Global Order half. This reinforces the finding that, with regard to national security, Bush devoted relatively greater attention to arguing the case for America's role in the world arena. More striking, however, is the vertical distance between the two tags, with Bush positioned firmly at the end of the “emotive” half of the diagram and Kerry at the end of the “practical concerns” half. It appears that Kerry fought and won the battle of logic, but Bush fought and won the battle of emotions.

By way of conclusion, it is useful to compare the findings of this analysis with that of Matt Bai in his New York Times Magazine article. Bai, who relied for his analysis on an exclusive interview with John Kerry, quoted Kerry as espousing a surprisingly benign worldview in which America should seek to reduce terrorism to the point of being a “nuisance,” akin to “prostitution,” “illegal gambling,” and “organized crime,” where “it isn't threatening people's lives every day, and fundamentally, it's something that you continue to fight, but it's not threatening the fabric of your life” (2004, 45). Bai characterized Kerry's “multinational, law-enforcement-like approach” to terrorism as “discordant” with a frightened American electorate, concluding that Kerry's “less lofty vision might have seemed more satisfying … in a world where the twin towers still stood” (70). So what has computer-assisted content analysis added to Bai's assessment?

This textual analysis of the well-publicized arguments of Bush and Kerry on U.S. national security provides first, an objective, easy-to-use, and transparent method for measuring ideas in political texts (devoid of the researcher's subjective bias, since the coding and classification are entirely program-generated). Thus, other researchers may replicate these findings and/or challenge them by introducing new textual data or by subjecting them to other, similar software programs.28

See my web site (http://personal.lse.ac.uk/schonhar/) for the data for this article.

Second, this analysis provides a way to measure ideas statistically and gauge their relationships spatially. The use of the chi-squared statistic for both representative words and phrases (ECUs) allows us not only to rank order the keywords and key phrases by importance, but also to gauge the intervals within these orderings. The tree diagram and correspondence analysis provide us a way to visualize the primary dimensions (global versus U.S. specific; emotive versus practical) that divided the candidates' positions on national security. In short, this article has lent a specificity and clarity to the role of ideas in political speeches, using a method that can be easily replicated and challenged by future researchers. (And, in response to my initial criticism of New York Times Magazine graphics, in this article, the position of words in an illustration have substantive meaning.) And finally, the analysis provides empirical support for the contention that the relevant battle in the 2004 presidential election may well have been one of emotions rather than logic, and it is here that Bush trumped Kerry.

Author Bio

Cheryl Schonhardt-Bailey is senior lecturer in political science at the London School of Economics and Political Science. Her areas of research include: the interplay between interests, ideas, and institutions (and how they affect political behavior), legislative politics, including roll call voting and textual analysis of legislative debates, new methodologies for measuring ideas, and political economy (particularly relating to contemporary U.S. policy and 19th-century Britain).

Appendix

Figure 3 provides a more detailed correspondence graph of the representative words in the analysis. This graph allows us to visualize the spread and overlap of the representative words from the analysis. It is provided here for illustration only.

References

Allum, Nicholas C. 1998. A Social Representations Approach to the Comparison of Three Textual Corpora Using Alceste. M.Sc. Dissertation, London School of Economics and Political Science.Google Scholar
Anonymous. 2004a. “George Bush Wins: Back to Basics.” The Economist, November 6, 23–27.Google Scholar
Anonymous. 2004b. “The Triumph of the Religious Right.” The Economist, November 13, 27–29.Google Scholar
Bai, Matt. 2004. “Kerry's Undeclared War: John Kerry Has a Thoughtful, Forward-looking Theory about Terrorism and How to Fight It. But Can It Resonate with Americans in the Post-9/11 World?” New York Times Magazine, October 10, 38–45, 52, 68, 70.Google Scholar
Bailey, Andrew, and Cheryl Schonhardt-Bailey. 2005. “Central Bankers and Big Ideas: Independence, Credibility, Uncertainty and Measurement in FOMC Transcripts.” Paper presented at European Public Choice Society Annual Meeting, at Durham (UK).Google Scholar
Barry, Christine A. 1998. “Choosing Qualitative Data Analysis Software: Atlas/ti and Nudist Compared.” Sociological Research Online 3 (3): 1–16. (www.socresonline.org.uk/socresonline/3/3/4.html).CrossRefGoogle Scholar
Bauer, Martin. 2000. “Classical Content Analysis: A Review.” In Qualitative Researching with Text, Image and Sound: A Practical Handbook, eds. Martin W. Bauer and George Gaskell. London: Sage Publications.CrossRefGoogle Scholar
Benzecri, J. P. 1981. Pratique de l'analyse des donnees: linguistique et lexicologie. Paris: Dunod.Google Scholar
Benzecri, J. P. 1973. L'Analyse des Donnees. Tome1: La Taxinomie. Tome 2: L'Analyse des Correspondances. Paris: Dunod.Google Scholar
Bertier, P., and J. M. Bouroche. 1975. Analyse des donnees multidimensionnelles. Paris: Presses Universitaires de France.Google Scholar
Blasius, Jorg, and Victor Thiessen. 2001. “Methodological Artifacts in Measures of Political Efficacy and Trust: A Multiple Correspondence Analysis.” Political Analysis 9 (1): 1–20.CrossRefGoogle Scholar
Brugidou, Mathieu. 1998. “Epitaphs. Francois Mitterrand's Image: An Analysis of an Open Question Asked on His Death.” Revue Francaise de Science Politique 48 (1).CrossRefGoogle Scholar
Brugidou, Mathieu. 2000. “The Discourse of Demands and Action in [French] Trade Union Press Editorials (1996–1998).” Revue Francaise de Science Politique 50 (6).CrossRefGoogle Scholar
Diday, E., J. Lemaire, J. Pouget, and F. Testu. 1982. Elements d'analyse des donnees. Paris: Dunod.Google Scholar
Donnelly, Thomas. 2003. The Underpinnings of the Bush Doctrine. American Enterprise Institute for Public Policy Research. (www.aei.org/publications/pubID.15845,filter./pub_detail.asp). (24 Nov. 2004).Google Scholar
Durham, Martin. 2004. “The American Right and the Iraq War.” Political Quarterly 75 (3): 257–265.Google Scholar
Elving, Ron. 2004. “Moral Values and the Next ‘Big Story’.” National Public Radio, November 8, 2004. www.npr.org/templates/story/story.php?storyId=4158311.Google Scholar
Finkel, Steven E., and John G. Geer. 1998. “A Spot Check: Casting Doubt on the Demobilizing Effect of Attack Advertising.” American Journal of Political Science 42 (2): 573–595.CrossRefGoogle Scholar
Gabel, Matthew J., and John D. Huber. 2000. “Putting Parties in Their Place: Inferring Party Left-Right Ideological Positions from Party Manifestos Data.” American Journal of Political Science 44 (1): 94–103.CrossRefGoogle Scholar
Garson, G. David. 2002. “Researching and Teaching Political Culture through Web-Based Content Profile Analysis.” Paper presented at the Annual Meeting of the American Political Science Association, Boston.Google Scholar
Garson, G. David. 2003. “Doing Web-Based Content Profile Analysis.” Social Science Computer Review 21 (2): 250–260.CrossRefGoogle Scholar
Greenacre, Michael J. 1984. Theory and Applications of Correspondence Analysis. London: Academic Press.Google Scholar
Greenacre, Michael J. 1993. Correspondence Analysis in Practice. London: Academic Press.Google Scholar
Greenacre, Michael J., and L. G. Underhill. 1982. “Scaling a Data matrix in Low-dimensional Euclidean Space.” In Topics in Applied Multivariate Analysis, ed. D. M. Hawkins. Cambridge: Cambridge University Press.Google Scholar
Hayashi, C. 1950. “On the Quantification of Qualitative Data from the Mathematics-statistical Point of View.” In Annals of the Institute of Statistical Mathematics, II.CrossRefGoogle Scholar
Hill, Kim Quaile, Stephen Hanna, and Sahar Shafqat. 1997. “The Liberal-Conservative Ideology of U.S. Senators: A New Measure.” American Journal of Political Science 41 (4): 1395–1413.CrossRefGoogle Scholar
Hogenraad, Robert, Yves Bestgen, and Jean Louis. 1995. “Terrorist Rhetoric: Texture and Architexture.” In From Information to Knowledge: Conceptual and Content Analysis by Computer, eds. E. Nissan and K. Schmidt. Oxford: Intellect.Google Scholar
Iker, Howard P. 1974. “An Historical Note on the Use of Word-Frequency Contingencies in Content Analysis.” Computers and the Humanities 8 (1): 93–98.CrossRefGoogle Scholar
Iker, Howard P., and Robert H. Klein. 1974. “WORDS: A Computer System for the Analysis of Content.” Behavior Research Methods & Instrumentation 6 (4): 430–438.CrossRefGoogle Scholar
Jenkins-Smith, Hank C., Gilbert K. St. Clair, and Brian Woods. 1991. “Explaining Change in Policy Subsystems: Analysis of Coalition Stability and Defection over Time.” American Journal of Political Science 35 (4): 851–880.CrossRefGoogle Scholar
Kabanoff, Boris. 1996. “Computers Can Read as Well as Count: How Computer-aided Text Analysis Can Benefit Organisational Research.” In Trends in Organizational Behavior, eds. C. L. Cooper and D. M. Rousseau. Chichester: Wiley.Google Scholar
Kabanoff, Boris, and John Holt. 1996. “Changes in the Espoused Values of Australian Organizations 1986–1990.” Journal of Organizational Behavior 17 (3):200–219.Google Scholar
Kahn, Kim Fridkin. 1992. “Does Being Male Help? An Investigation of the Effects of Candidate Gender and Campaign Coverage on Evaluation of U.S. Senate Candidates.” The Journal of Politics 54 (2): 497–517.CrossRefGoogle Scholar
Kronberger, Nicole. 2004. Personal communication with the author, 24 September.Google Scholar
Kronberger, Nicole, and Wolfgang Wagner. 2000. “Keywords in Context: Statistical Analysis of Text Features.” In Qualitative Researching with Text, Image and Sound: A Practical Handbook, eds. Martin W. Bauer and George Gaskell. London: Sage Publications.CrossRefGoogle Scholar
Lahlou, L. 1996. “A Method to Extract Social Representations from Linguistic Corpora.” Japanese Journal of Experimental Social Psychology 36: 278–291.CrossRefGoogle Scholar
Lahlou, S. 1998. Penser manger. Paris: Presses Universitaires de France.CrossRefGoogle Scholar
Laver, Michael, and Kenneth Benoit. 2002. “Locating TDs in Policy Spaces: Wordscoring Dail Speeches.” Trinity College: Dublin.Google Scholar
Laver, Michael, Kenneth Benoit, and John Garry. 2002. “Placing Political Parties in Policy Spaces.” Trinity College: Dublin.Google Scholar
Laver, Michael, and John Garry. 2000. “Estimating Policy Positions from Political Texts.” American Journal of Political Science 44 (3): 619–634.Google Scholar
Lepgold, Joseph, and Timothy McKeown. 1995. “Is American Foreign Policy Exceptional? An Empirical Analysis.” Political Science Quarterly 110 (3): 369–384.CrossRefGoogle Scholar
McDonald, Colin. 1982. “Coding Open-Ended Answers with the Help of a Computer.” Journal of the Market Research Society 24 (1): 9–27.Google Scholar
Miller, M. Mark. 1997. “Frame Mapping and Analysis of News Coverage of Contentious Issues.” Social Science Computer Review 15 (4): 367–378.CrossRefGoogle Scholar
Noel-Jorand, M. C., M. Reinert, M. Bonnon, and P. Therme. 1995. “Discourse Analysis and Psychological Adaptation to High Altitude Hypoxia.” Stress Medicine 11: 27–39.Google Scholar
Oberlechner, Thomas, Thomas Slunecko, and Nicole Kronberger. 2004. “Surfing the Money Tides: Understanding the Foreign Exchange Market through Metaphors.” British Journal of Social Psychology 43: 133–156.CrossRefGoogle Scholar
Popping, Roel. 2000. Computer-assisted Text Analysis. London: Sage.CrossRefGoogle Scholar
Popping, Roel. 2004. email correspondence with author (October 29).Google Scholar
Reagan, Ronald. 1983. Remarks at the Annual Convention of the National Association of Evangelicals. www.americanrhetoric.com/speeches/ronaldreaganevilempire.htmGoogle Scholar
Reinert, Max. 1983. “Une methode de classification descendante hierarchique: application a l'analyse lexicale par contexte.” Les Cahiers de l'Analyse des Donnees 8 (2): 187–198.Google Scholar
Reinert, Max. 1993. “Les ‘mondes lexicaux’ et leur ‘logique’ a travers l'analyse statistique d'un corpus de recits de cauchemars.” Langage et societe 66: 5–39.CrossRefGoogle Scholar
Reinert, Max. 1998. ALCESTE users' manual (English version) 4.5 Pro. Image, Toulouse.Google Scholar
Roosevelt, Franklin Delano. 1940. The Great Arsenal of Democracy. www.americanrhetoric.com/speeches/fdrarsenalofdemocracy.htmlGoogle Scholar
Ruggie, John Gerald. 1997. “The Past as Prologue? Interests, Identity, and American Foreign Policy.” International Security 21 (4): 89–125.Google Scholar
Schonhardt-Bailey, Cheryl. 2006. From the Corn Laws to Free Trade: Interests, Ideas and Institutions in Historical Perspective. Cambridge: MIT Press.Google Scholar
Wagner, Wolfgang, and Nicole Kronberger. Forthcoming. “Quantitative and Qualitative Cross-Cultural Comparison: Cultural Metrics.” In Cultural and Cross-Cultural Psychology, ed. J. Straub.Google Scholar
Weller, Susan C., and A. Kimball Romney. 1990. Metric Scaling: Correspondence Analysis. London: Sage Publications.CrossRefGoogle Scholar
Figure 0

Tree Graph of the Classes for Bush and Kerry on National and Homeland Security

Figure 1

Basic Statistics for Key Bush and Kerry Speeches on National and Homeland Security

Figure 2

Bush & Kerry Speeches on National & Homeland Security: Examples of Most Typical ECUs in each Class

Figure 3

Table 3: Continued

Figure 4

Correspondence Analysis of Classes for Bush and Kerry on National and Homeland Security

Figure 5

Correspondence Analysis of Classes for Bush and Kerry on National and Homeland Security, Representative Words