ICT-DERIVED DATA: SOURCES AND METHODS

What kind of ICT-derived data exists, and how can political scientists use these different types of data? The articles in the pages that follow deal with a range of ICT-generated data, drawn from sources that will interest readers working across subfields and topics. Half of the contributions to this symposium—those by Granka, Koehler-Derrick, and Mellon—focus on Internet search data. The others go beyond search data: Hassanpour discusses Google's Ngram tool, a huge corpus of digitized books, Goldstein explores “Social fMRI” data generated by smartphones, and Greitens examines states' ICT policies themselves as a source of data.

This list is far from exhaustive. Other research has examined sources including Facebook posts (Bond et al. Reference Bond, Fariss, Jones, Kramer, Marlow, Settle and Fowler2012); Twitter feeds (Mostak Reference Mostak2012); cell phone coverage (Shapiro and Weidmann Reference Shapiro and Weidmann2012); and non-Western blogs, microblogs, and website content (Bamman, O'Connor, and Smith Reference Bamman, O'Connor and Smith2012; King, Pan, and Roberts Reference King, Pan and Roberts2012; Nielsen Reference Nielsen2012). We are confident that this symposium is only the first step in finding creative ways of using ICT technology and other new forms of electronic data to answer challenging research questions.

The contributions included here, however, do showcase some of the major advantages of that data: its consistency of format, the speed and ease with which it can be collected, and its high volume. The authors demonstrate that information and communication technologies can provide political scientists with vast quantities of microempirical data to examine changes in behavior over time, geographic relationships, and process questions about time and scale.

The articles that follow also use a range of analytical methods, some innovative and some more traditional, to leverage these data and gain new insights into political phenomena of interest. Koehler-Derrick uses Internet search data to analyze framing battles between Egyptian protestors and the military during a period of tense confrontations in late 2011, whereas Hassanpour relies on the NGram tool to analyze trends in political discourse. Granka uses search data to build and test a model for forecasting the American presidential elections and Republican primaries, while Mellon pairs search results with traditional survey data to discuss the challenges of validation. Goldstein explores the potential for Social fMRI data to be used in experimental projects on improving governance and development outcomes, and Greitens demonstrates how, through careful contextual study, ICT data can shed light on political developments inside even the most restrictive of authoritarian regimes.

ICT-DERIVED DATA: THEMES AND QUESTIONS

As political scientists begin to incorporate ICT-derived data into their work, what issues and potential pitfalls should they keep in mind? As we compared the symposium contributors' data sources and analytical approaches, several themes emerged. These emphasized both the potential that technological developments hold for political science research, and their current limitations. Throughout this symposium, therefore, we have sought to be clear about the strengths and weaknesses of particular data sources for different types of research inquiry, and to candidly discuss the best analytical frameworks for bringing that data to bear on questions of interest to political scientists.

The first theme that emerged was the need, when selecting methods of analysis and drawing conclusions, to pay attention to the political context within which ICT data are generated. In some of the symposium's articles, such as the contributions by Goldstein and Greitens, political factors clearly condition the adoption of technology and therefore may restrict or bias the generation of data from that country. Other contributions, such as the one by Mellon, stress the need to take linguistic and cultural differences into account.

The range of sources and methods used here naturally leads to a second major theme: variation. Differences in political context are one type of variation; as discussed earlier, political differences in countries' approach to ICTs play a critical role in the articles by Goldstein and Greitens, both because the policies themselves vary and because they create patterns of variation in ICT-derived data. The contributions included here, however, also deal more broadly with patterns of variation within ICT data, and raise the question of how best to use those variations to gain analytical leverage on important questions in political science. Much of the recent public attention to technology's importance in politics has been driven by the supposition that technology has a uniform and prodemocratic impact. Actually, however, it is characterizing and explaining variations in the role of technology in political life that will sustain a robust research agenda. We hope, with this symposium, to start a substantive discussion about how to think about the origins of variation in ICT data, its significance for research design, and implications for researching a range of political behavior and outcomes.

A third theme that recurred across contributors was the challenge of validating Internet or ICT data and comparing it to offline behavior. This issue is the central focus of Mellon's piece on how to draw accurate inferences from search data in countries other than the United States. Granka also deals with validation questions in testing the power of search data as compared to more conventional models that predict American election results. Two additional articles, by Koehler-Derrick and Goldstein, focus on using online data to explore offline behavior, but do so in situations where the degree of similarity between the two is more difficult to assess (either because Internet penetration levels are more limited or because official data that could be used as a baseline comparison is unreliable or nonexistent). Validation questions require careful research design, and often some initial testing, to determine what the online data actually reflect. As the contributions here show, the results of those tests can be surprising, and researchers should be careful about the degree to which they assume online data reflect offline behavior. As the field develops, validation looks likely to remain a significant methodological challenge.

The final theme raised by the symposium contributions was that of scope conditions. We mean that in two senses. First, the articles motivate us to consider the scope conditions of technology as an independent variable. What kind of outcomes do information and communication technologies influence, and what outcomes do they have little effect on? Are there scope conditions that affect which outcomes technology influences, and in what direction?

Second, though, the articles also challenge us to analyze the scope conditions and limitations of ICT-generated data relative to other types of data that could be employed. What is the utility of Internet-derived data in different contexts, and what are its limitations? As political scientists design future research projects, how does Internet- or ICT-derived data compare to, or complement, data obtained through more traditional methods? Several of the contributions here, including those by Granka and Mellon, suggest that search data may have a predictive function, but under conditions that are, as yet, still being explored and which will only be clarified by combining search data with other, more traditional sources. The debate over the potential predictive capacity of ICT-derived data is one example of how the impact and use of ICT data will have to be disaggregated and tested carefully in future work.