Analytical Strategy and Method

To determine readability levels, we used the Flesch-Kincaid Grade Level Index. This index has been utilized in a variety of research fields, and a large number of studies have confirmed its validity (Tefki 1987). The index is based on the average number of syllables per word and the average number of words per sentence.⁴

The score provides a grade-school level. For example, a score of 8.0 suggests that a median student in the eighth grade would be able to understand the text. The index stretches from below grade level four, which is considered very easy, to very difficult reading at the post-graduate level.

Finding and scanning old textbooks is a time-consuming process: we limit ourselves to a preliminary case study of the exceptionally durable and wellknown introductory American government text by James MacGregor Burns and Jack Walter Peltason, Government by the People. For approximately 50 years, this text has been among the best selling in the country. Government by the People was ideal for our purposes because we could determine whether the same text has become easier or harder to read across a significant span of time. We chose an American government text, because those are the most frequently used texts in political science and therefore the most likely to have multiple exemplars of multiple editions.⁵

We sampled the texts in a fashion consistent with Berland et al. (2001), who analyzed reading grade levels of medical information web sites. Three passages of text, exactly two pages each, from the beginning, middle, and end of relevant chapters were sampled, then scanned and converted to Microsoft Word documents. The Flesch-Kincaid Index is available in the “Spelling and Grammar” options of Microsoft Word.

We determined readability levels for sample passages from the 1957, 1966, 1969, 1972, 1990, and 1995 editions (editions 3 through 16).⁶

We took samples from three chapters across each of the editions, first selecting the introductory chapter. Though not devoted to specific sub-field content, the function of this chapter is to familiarize students with the various topics of political science; the reading accessibility of its content is clearly important. In addition, given its placement in the book and on most syllabi, the introduction is arguably the chapter most widely read by undergraduates.

The other two chapters we sampled concerned civil liberties and public opinion. These two chapters offered needed variance on the important dimension of sub-field development. We expected that as a sub-field matures, its language is likely to become increasingly complex. In the 1950s, public opinion was in its infancy, whereas the study of constitutional rights and liberties was well established within the discipline. To the extent that readability mirrors sub-field development, we would anticipate reading levels for civil liberties and public opinion to differ initially, with civil liberties generating more diffi-cult content, and then to converge as the field of public opinion matured.

Findings

Figure 1 shows the average readability level across editions. The trend is clear, striking, and apparently contrary to the common charge of declining standards. Since the publication of the third edition in the late 1950s, the reading level of Government by the People has become increasingly higher. Difficulty has increased by approximately a grade level, rising from just below an 11th grade to nearly a 12th grade level. The magnitude of this change is not trivial. The readability of chapter two in Harry Potter and the Sorcerer's Stone is at a high 10th grade level and that of specific articles in The New York Times is at a 12th grade level (Biskupic 2001; Neilan 2001). Thus, the change in readability of Government by the People over the past several decades is approximately the same as the difference in readability between The New York Times and the initial volume of the Harry Potter series.

Figure 2 presents the data disaggregated by chapter. Across all editions, the readability of the civil liberties chapter has remained at grade level 12. By contrast, the readability of the introductory and public opinion chapters exhibits significant change. As expected, the readability of the public opinion chapter starts substantially below that of the civil liberties chapter but ascends rapidly during the 1970s to approximately 12th grade level by the mid 1990s. The introductory chapter takes a similar path, although its ascent is much steeper, moving to a 12th grade level by the 7th edition in 1969.

To this point, the data appear to undermine the familiar and admittedly attractive critique of declining texts and permissive pedagogy. Rather than “dumbing down,” Government by the People has increased in reading diffi-culty. We cannot, however, dismiss the charges of the critics quite so easily. Though reading difficulty has increased, communication extends beyond words and sentences. Perhaps an alternative form of communication gives rise to the now pervasive “dumbing down” charge. Pictures, after all, are worth a thousand words, and inspection of our introductory texts suggests publishers have accepted the wisdom of this old adage. Modern texts are filled with splashy photos, graphics, tables of all sorts, and boxed inserts. This type of presentation, more than the written word, may in fact trigger the critics' lament of market-driven (i.e., dumbeddown) texts that are attempting to appeal to wider, and hence less academically seasoned, audiences.

In Figure 3, we present documented changes in the number of graphics across the chapters. Anything not part of the main text was included in the count, namely figures, tables, photos, and boxes. Clearly, the early 1970s represented a significant change in the form of communication. While previous editions had comparatively small number of graphics, by the mid 1980s virtually every page contained at least a picture, table, or figure. Studies are mixed on whether these changes are merely cosmetic or provide pedagogical substance (Jordan and Sanchez 1994); the trend is evident.

Discussion

To our knowledge, this case study is the first attempt to examine the readability levels of an introductory textbook in political science. In this case, we found that the level of readability has increased by roughly a grade level from high 10th grade to nearly 12th grade. The change occurred over several decades and varied by sub-field. The introductory and public opinion chapters showed the most changes while civil liberties remained at grade level 12 for the entire period. This evidence suggests that readability levels are tied in part to sub-field development, but the extent of this linkage remains an open question for future analyses.

We also saw a dramatic increase in graphics and pictures in the introductory text. Before the 1970s, a picture or graphic appeared approximately once every five pages; our current text offers at least one graphic per page, sometimes more. If this change is typical, it may be an important cause for the general criticisms of current textbooks compared to those of the fabled past, but combined with more difficult reading levels, the claim of “dumbing down” may be hollow.

This case study has some important limitations, even as it raises some important questions. First, the introductory textbook market is broad, and we have sampled only one text. Changes in readability may be different for other texts and conceivably different for introductory texts in the comparative, international relations, or political theory fields. While we hesitate to draw more general conclusions, our selection of one of the more popular and enduring introductory texts in American politics leads us to expect that our results reflect a more general trend in the field. Second, we studied readability only, utilizing one measure. Alternative measures may alter our findings. Third, our research has not explored the causes or consequences of changes in readability. Our primary goal was simply to document changes in textbooks, and calculating readability levels represented a step toward realizing this end. Finally, there are potential variations in texts that this case study explicitly did not seek to explore. One might ask how changes in the discipline affect readability and complexity in our textbooks, and there may also be non-disciplinary causes for variation having to do with marketing, editorial processes, pedagogical culture, and the like, that the present study does not address.

Several further research possibilities exist. For example, researchers have examined introductory texts in psychology to determine whether a set of core concepts define a common language in the field (Zechmeister and Zechmeister 2000). A similar study of how political science texts treat key concepts such as power, class, representation, and con-flict and whether they have changed over time and across introductory texts would make a substantial addition to the topic. Comparisons could also be made between brief and full editions, analyzing, pedagogical aids, and topic coverage (Griggs and Koenig 2001). More generally, one could explore relative changes in text length, number of chapters, and coverage of traditional versus non-traditional topics (Griggs, Jackson, Christopher, and Marek 1999). The questions are many and the topic is important.

If readability has, in fact, changed in the direction this study indicates, researchers may wish to examine whether students are more likely to process and absorb the newer textbook content. This question has been examined in other disciplines (Santa and Burstyn 1977). Although ours is a preliminary study, its results should raise awareness of the need for studying political science textbooks generally, at a minimum encouraging a more critical eye in choosing our textbooks. The means by which knowledge is communicated to students through our introductory texts remains a key factor toward effective teaching, and the surprisingly sparse consideration it garners in our discipline deserves expansion.

It should be mentioned that the Flesch-Kincaid Grade Level Index for this note is 12.0 (high school level).