Limitations of Traditional Data

Event coding has become an essential part of data collection in various sub-fields of political science (Woolley 2000). Coding events, such as military conflicts,¹

often across countries and over a long period of time, has given scholars opportunities to conduct meaningful comparative analyses, thereby allowing them to avoid the potential bias associated with small-n studies (King, Keohane, and Verba 1994). Protest events, such as strikes or riots, are also coded and have become widely available. Some of the most commonly used protest event datasets include Cross-National Time Series, 1815–1999 (Banks 2001) and World Handbook of Political and Social Indicators (Taylor and Jodice 1983), the contributions of which are indispensable to the study of protests. However, in spite of such contributions, these datasets have practical and theoretical limitations which have inhibited further development of protest research. The problems are associated with the generalization of categories of events, loss of information, and lack of adequate sources.

First, categories of events in the traditional datasets can obstruct efforts to discern their specific nature because their categorizations are often too general, arbitrary, and inaccurate. In these datasets, protest events are typically categorized into no more than a dozen types, according to multiple factors. These factors usually include the basic nature of an event (political/social), such as the level of violence, participants' characteristics and numbers, target, purpose, scope, and level of spontaneity. The end product typically gives the frequency of certain types of events for a given time period. This categorization provides a straightforward data structure. However, this practice produces several problems: 1) categories of protests are confusing because of coding inconsistencies; 2) an event may fall into multiple categories or no category at all; 3) the practice discards potentially valuable details; and 4) it limits the possibility for an end-user to sort and recode events based on his or her need.

For example, as Table 1 illustrates, the World Handbook has six categories for political protest—protest demonstration, regime-support demonstration, political strike, riot, armed attack, and political assassination (Taylor and Jodice 1983, 19–47). “Riot” is clearly an event distinguished from “demonstration” because of its violent nature, “destruction of property.” However, unlike “demonstration” which is divided into anti-and pro-regime, the targets of riots are left unspecified, leading to the incorrect assumption that political riots occur only as protests against regimes.²

Actually, only “demonstration” has been sub-categorized into anti and pro government. In addition, a riot, to be categorized as such, needs people in “hundreds, if not thousands” (29). Is a protest with all the attributes of “riot,” but with only 90 participants, not a riot? If not, what is it? No answer is given. Again, this aspect of “riot” introduces the question of the number of participants in the other types of protest. How many participants are required for an event to be called a “demonstration”? Is this a meaningful criterion for other types of protest? If not, why is it the case that only “riot” has a minimum number of participants? Unfortunately, these questions are also left unanswered. Moreover, according to the data, there were two riots in South Korea in 1990. However, the available information about these riots goes no further than this. It remains unclear what the riots were about, who the targets were, or how violent the events were. This lack of specific information about events, in turn, deprives users of the ability to contemplate their attributes and recode events according to their interests. And it also precludes the users' ability to verify the reliability of data simply because one will not be able to find vital information pertaining to the coded event.

The second problem is that only yearly-summed numbers of events are available, eliminating the possibility of quarterly or monthly, let alone weekly or daily, analyses. For example, one finds from Banks' dataset (2001) that in South Korea there were six protest events—two riots and four anti-government protests—in 1990. However, no information is available on exactly when in 1990 these events occurred, where in South Korea, and what these events specifically entailed. Therefore, this annually aggregated dataset is insufficient if a researcher wanted to analyze protest events within a year³

or if the main interest was the variation of protests on a daily basis over time.

Third, and most important, the traditional protest datasets have often relied on a limited number of sources. It is common for only one or two sources to be sampled. The Prodat project sampled German protest events from two German newspapers while Kriesi and his colleagues (1995) used only one national newspaper for each country.⁴

To make matters worse, they both only coded events from Monday issues.⁵ The ratio of sources to countries (Prodat project, 2:1; Kriesi et al., 1:1) significantly decreases in Banks' data (1991), Domestic Conflict Behavior, 1919–1966, which has only one source, the New York Times, for a total of 114 countries (1:114). It is hard to imagine, despite its fame for international coverage, that the Times is capable of covering all protests of a single country, let alone “virtually all nations independent” (Banks 1972, 41). In terms of space, the Times has about a dozen sections. Sections like business, education, health, obituaries, science, technology, or weather are unlikely to report protests. So only a few sections remain. Among the remnants—front page, international, New York region, national, and Washington—only two sections, front page and international, are likely to report protests overseas. The space to report protests of the world is simply too small for full coverage. Therefore, when protests are sampled from one or two sources, especially non-local sources, the data is very unlikely to contain as many events as they intended or claimed to. Moreover, if potential coding bias—due to divergent trends of coverage among different media (Woolley 2000, 160–161) or changing indexing practices (162)—is considered, the risks involved in relying on a small number of sources significantly increase.

A New Perspective: Protest and Coercion Data

The cited problems can be overcome by 1) coding events with minimal generalization; 2) providing needed information about events and sources; and 3) increasing the number of sources. Protest and Coercion Data (PCD)⁶

methods adopt all of these solutions at once. Hence, PCD provide users with data containing much more information about protest events in greater volume and quality. First, the problem of inconsistency of categorizations is resolved by focusing on an event itself rather than on a category. Each event is sampled as a distinctive incident regardless of its duration, number of participants, purpose, target, or property damage. Detailed information about each event is filed to form data. Thus, users have details about each event rather than getting summarized information about a collection of events. Recorded information of an event includes the event date, identity of protesters,⁷ their target, the issue, and a brief description of the event. Activities of protest and coercion are informed by the number of protesters, the number injured, the number arrested, the number of deaths, state actors present (i.e., police), and the number of state actors injured and killed. Finally, the data include source information, such as the name of newspapers and publication date.

Such information is gathered and coded in PCD, creating detailed information about each event. Such an abundance of information allows the end-user to sort events into whichever categories he or she wishes. For instance, from the French PCD, we know that on Monday April 25, 1983, about 30 “farmers” stopped and emptied two trucks carrying Italian wines in Sete, France in protest against EU imports. In the course of the event, property-damage occurred, but no one was injured or arrested. According to the World Handbook, this event must be categorized somehow, but it is not clear in which category this event would belong. It could have been coded as “riot” because of the property damage, but there were only about 30 participants, which would disqualify this event as a riot according to the same codebook. Instead, it could be coded as an “armed attack” because this category does not specify the minimum number of participants and it does cover events with property damage. However, it is difficult to determine if the action was planned—which is an important criterion for an event to be “armed attack.” The more serious problem is that once this event is coded, much of the information about the event disappears; instead it increases the count of a category by one. On the other hand, from the perspective of PCD users, this event does not need to be categorized, and various information of this event can be used in multiple ways. Or, if needed, this event can be categorized in more than one category. Since the information pertaining to the event is included in the event description, users can always go back, review the detailed information, and change the categorization to fit their coding. The farmers' behavior can be categorized as a riot, armed attack, demonstration, “foreign policy” (by topic), or “under 100” (by the number of protesters) depending on the needs of different researchers.

Second, to overcome the yearly aggregated data problem, the PCD project codes sub-daily events. As mentioned, PCD contain every and all reported protests and repressive events in a country. All events in a given day are separately coded, and an event is distinctly coded if it evolves into a different one. For example, assume that an event occurs where students protest on a campus. Later that day, the students try to march outside of the campus, and a riot breaks out when the police try to stop them. In terms of students' behavior, there are two distinct events. The protest on campus does not involve any confrontation, but it turns into a violent event against state actors. Here two events, rally and riot, are coded as separate events on the same day, and the numbers of participants in each event (some protesters do not take part in the riot) are also separately coded. This practice of coding allows researchers to study protests on a sub-annual basis. One can trace, for example, weekly or daily trends of protests if necessary, which gives a more detailed and informed knowledge about protests than studies based on yearly data. Thus, PCD allow researchers to find patterns that are almost impossible to study using annually aggregated data.

The third requirement, data from a small number of sources, is solved by simply employing many relevant sources. PCD have multiple sources. For example, the European PCD used approximately 500 newspapers, trade publications, and wire services available on Lexis-Nexis as their main sources. The magnitude of sources emphasized by PCD provides a critical improvement: events sampled in PCD outnumber other datasets by huge margins (Francisco 2006). For example, from 1980 through 1982, in France, 975 events were coded in PCD, while only 202 were coded in the World Handbook and none in Banks' data (1991). Even in comparison with data focused on a single country, PCD do better. German data by Kriesi et al. (1995) contain 676 events for the period, whereas the PCD have 1,177 events. The magnitude of the sources used by the European PCD is even more impressive in regards to the ratio of number of sources to countries. On average, in the European PCD one country is covered by about 16 news media (16:1) while in Banks' data (1991), the ratio drops to 0.006 sources per country.

Besides the increased number of events, the inclusion of more sources in PCD has improved the data on two other critically important dimensions. First, PCD are based on news sources with a wide range of interests. The specialty of sources ranges from general to very specialized interests. For example, news services such as Agence France Press, Associated Press, and Reuters provide general news, while news media such as Energy Economist focuses on a theme, and thus offers detailed reporting on their specific issue. The ideological mixture also increases the diversity of the PCD. There are conservative news media sources such as Die Welt of Germany as well as progressive media sources such as Greenpeace Business. By using such variety of sources, PCD extend the breadth and depth of sources to draw samples. And, this makes PCD more useful and less vulnerable to bias because of individual sources.

Second, PCD include many local sources. Local sources such as Belfast Telegraph, Cypriot Broadcasting Corporation, Neue Zürcher Zeitung (Daily Zurich newspaper), and Radio Telefis Eireann (Radio and Television of Ireland) were rarely, if ever, used in coding protests in any of the traditional datasets. Using local sources is important, especially for protest data, because protests tend to be about local issues and last only for short periods of time. If an event is serious enough—usually tragic and deadly and/or about an internationally appealing issue—the event would be covered not only by local news media, but also by the general news media. However, such events only occur sporadically because protests are usually about mundane issues like wage increases,⁸

and they often end with no injuries or arrests. Such events are not so interesting to an international audience—they usually have similar issues of their own—and therefore are not likely to be reported by general news sources. In other words, local sources will report events that global sources will not.