Modern everyday life is threaded with countless interactions with massive technological systems that support our communication, our transport, our retail activities, and much more. Through these interactions, we are generating increasing volumes of “big data,” documenting our collective behavior at an unprecedented scale.
Bentley et al. provide a timely account of the role of big data in the study of collective behavior. They offer a comprehensive analysis of what our interactions on the Internet, in particular using social network sites such as Facebook and Twitter, can tell us about how information flows throughout the large and complex network of human society. While we agree that this insight into the structure of social connections is important, we emphasize that big data do not only come from online social networks. We note a number of recent studies providing evidence that big data can tell us much more about real-world collective decision making than has been acknowledged in Bentley et al.’s account, and can even allow us to better anticipate collective actions taken in the real world.
For example, human decision making often involves gathering information to determine the consequences of possible actions (Simon Reference Simon1955). Increasingly, we turn to the Internet, and search engines such as Google in particular, to provide information to support our everyday decisions. Can massive records of our search engine usage therefore offer insight into the previously hidden information-gathering processes which precede real-world decisions taken around the globe? Recent results suggest that they can. A series of studies have shown that search engine query data “predict the present,” providing a measurement of real-world behavior often before official data are released (Choi & Varian Reference Choi and Varian2012). Correlations between search engine query data and real-world actions have been demonstrated across a range of areas such as motor vehicle sales, incoming tourist numbers, unemployment rates, reports of flu and other diseases, and trading volumes in the U.S. stock markets (Askitas & Zimmerman Reference Askitas and Zimmermann2009; Brownstein et al. Reference Brownstein, Freifeld and Madoff2009; Choi & Varian Reference Choi and Varian2012; Ettredge et al. Reference Ettredge, Gerdes and Karuga2005; Ginsberg et al. Reference Ginsberg, Mohebbi, Patel, Brammer, Smolinski and Brilliant2009; Preis et al. Reference Preis, Reith and Stanley2010).
Further studies have illustrated that data on online information gathering can also anticipate future collective behavior. Goel et al. (Reference Goel, Hofman, Lahaie, Pennock and Watts2010) demonstrated that search query volume predicts the opening weekend box-office revenue for films, first-month sales of video games, and chart rankings of songs. Our own investigations have suggested that changes in the number of searches for financially related terms on Google (Preis et al. Reference Preis, Moat and Stanley2013) and views of financially related pages on Wikipedia (Moat et al. Reference Moat, Curme, Avakian, Kenett, Stanley and Preis2013) may have contained early warning signs of stock market moves.
In a recent study, we exploited the global breadth of Google data to compare information-gathering behavior around the world. Our analysis uncovered evidence that Internet users from countries with a higher per capita gross domestic product (GDP) tend to search for more information about the future rather than the past (Preis et al. Reference Preis, Moat, Stanley and Bishop2012). For 45 countries in 2010, we calculated the ratio of the volume of Google searches for the upcoming year (“2011”) to the volume of searches for the previous year (“2009”), a quantity we called the “future orientation index.” We found that this index was strongly correlated with per capita GDP. In ongoing work, we seek to better understand whether these results reflect international differences in decision-making processes. Perhaps, for example, a focus on the future supports economic success.
Aside from search data, other research has provided evidence that the massive datasets generated by our everyday actions in the real world can also support better forecasting of future behavior (King Reference King2011; Lazer et al. Reference Lazer, Pentland, Adamic, Aral, Barabasi, Brewer, Christakis, Contractor, Fowler, Gutmann, Jebara, King, Macy, Roy and Van Alstyne2009; Mitchell Reference Mitchell2009; Vespignani Reference Vespignani2009). Large-scale datasets allow us to look for patterns in collective behavior which might recur in the future, similar to the way in which we as individuals rely on the statistical structure we have observed in the world when trying to forecast consequences of decisions (Giguère & Love Reference Giguère and Love2013; Olivola & Sagara Reference Olivola and Sagara2009; Stewart Reference Stewart2009; Stewart et al. Reference Stewart, Chater and Brown2006). For example, analysis of data collected through daily police activities has shown that the occurrence of a burglary results in a short-term increase in the probability that another burglary will occur on the same street, with implications for behavioral models of how these crimes are committed (Bowers et al. Reference Bowers, Johnson and Pease2004; Johnson & Bowers Reference Johnson and Bowers2004; Mohler et al. Reference Mohler, Short, Brantingham, Schoenberg and Tita2011). Such insights have been captured in predictive policing systems which aim to deploy police to areas before an offence occurs, with initial evaluations demonstrating a reduction in levels of crime (Johnson et al. Reference Johnson, Birks, McLaughlin, Bowers and Pease2007). Similarly, large-scale data on both long-distance travel by air and local commuting can improve predictions of human travel behavior and therefore the spread of epidemics, with clear consequences for the distribution of health resources such as vaccines (Balcan et al. Reference Balcan, Colizza, Gonçalves, Hu, Ramasco and Vespignani2009; Tizzoni et al. Reference Tizzoni, Bajardi, Poletto, Ramasco, Balcan, Gonçalves, Perra, Colizza and Vespignani2012).
When considered at greater breadth, we argue that, in contrast to Bentley et al.’s conjecture, big-data studies do far more than “allow us to see better how known behavioral patterns apply in novel contexts” (target article, sect. 4, para. 13). Online search data, for example, offer us insight into early information gathering stages of real-world decision-making processes that could not previously be observed, while large-scale records of real-world activity enable us to better forecast future actions by allowing us to identify new patterns in our collective behavior. Such predictive power is not only of theoretical importance for behavioral science, but also of great practical consequence, as it opens up possibilities to reallocate resources to better support the well-being of society. Our ability to extract maximum value from these datasets is, however, highly dependent on our ability to ask the right questions: a task for which experts in more “traditional behavioral science” (sect. 4, para. 13) are ideally placed.
Modern everyday life is threaded with countless interactions with massive technological systems that support our communication, our transport, our retail activities, and much more. Through these interactions, we are generating increasing volumes of “big data,” documenting our collective behavior at an unprecedented scale.
Bentley et al. provide a timely account of the role of big data in the study of collective behavior. They offer a comprehensive analysis of what our interactions on the Internet, in particular using social network sites such as Facebook and Twitter, can tell us about how information flows throughout the large and complex network of human society. While we agree that this insight into the structure of social connections is important, we emphasize that big data do not only come from online social networks. We note a number of recent studies providing evidence that big data can tell us much more about real-world collective decision making than has been acknowledged in Bentley et al.’s account, and can even allow us to better anticipate collective actions taken in the real world.
For example, human decision making often involves gathering information to determine the consequences of possible actions (Simon Reference Simon1955). Increasingly, we turn to the Internet, and search engines such as Google in particular, to provide information to support our everyday decisions. Can massive records of our search engine usage therefore offer insight into the previously hidden information-gathering processes which precede real-world decisions taken around the globe? Recent results suggest that they can. A series of studies have shown that search engine query data “predict the present,” providing a measurement of real-world behavior often before official data are released (Choi & Varian Reference Choi and Varian2012). Correlations between search engine query data and real-world actions have been demonstrated across a range of areas such as motor vehicle sales, incoming tourist numbers, unemployment rates, reports of flu and other diseases, and trading volumes in the U.S. stock markets (Askitas & Zimmerman Reference Askitas and Zimmermann2009; Brownstein et al. Reference Brownstein, Freifeld and Madoff2009; Choi & Varian Reference Choi and Varian2012; Ettredge et al. Reference Ettredge, Gerdes and Karuga2005; Ginsberg et al. Reference Ginsberg, Mohebbi, Patel, Brammer, Smolinski and Brilliant2009; Preis et al. Reference Preis, Reith and Stanley2010).
Further studies have illustrated that data on online information gathering can also anticipate future collective behavior. Goel et al. (Reference Goel, Hofman, Lahaie, Pennock and Watts2010) demonstrated that search query volume predicts the opening weekend box-office revenue for films, first-month sales of video games, and chart rankings of songs. Our own investigations have suggested that changes in the number of searches for financially related terms on Google (Preis et al. Reference Preis, Moat and Stanley2013) and views of financially related pages on Wikipedia (Moat et al. Reference Moat, Curme, Avakian, Kenett, Stanley and Preis2013) may have contained early warning signs of stock market moves.
In a recent study, we exploited the global breadth of Google data to compare information-gathering behavior around the world. Our analysis uncovered evidence that Internet users from countries with a higher per capita gross domestic product (GDP) tend to search for more information about the future rather than the past (Preis et al. Reference Preis, Moat, Stanley and Bishop2012). For 45 countries in 2010, we calculated the ratio of the volume of Google searches for the upcoming year (“2011”) to the volume of searches for the previous year (“2009”), a quantity we called the “future orientation index.” We found that this index was strongly correlated with per capita GDP. In ongoing work, we seek to better understand whether these results reflect international differences in decision-making processes. Perhaps, for example, a focus on the future supports economic success.
Aside from search data, other research has provided evidence that the massive datasets generated by our everyday actions in the real world can also support better forecasting of future behavior (King Reference King2011; Lazer et al. Reference Lazer, Pentland, Adamic, Aral, Barabasi, Brewer, Christakis, Contractor, Fowler, Gutmann, Jebara, King, Macy, Roy and Van Alstyne2009; Mitchell Reference Mitchell2009; Vespignani Reference Vespignani2009). Large-scale datasets allow us to look for patterns in collective behavior which might recur in the future, similar to the way in which we as individuals rely on the statistical structure we have observed in the world when trying to forecast consequences of decisions (Giguère & Love Reference Giguère and Love2013; Olivola & Sagara Reference Olivola and Sagara2009; Stewart Reference Stewart2009; Stewart et al. Reference Stewart, Chater and Brown2006). For example, analysis of data collected through daily police activities has shown that the occurrence of a burglary results in a short-term increase in the probability that another burglary will occur on the same street, with implications for behavioral models of how these crimes are committed (Bowers et al. Reference Bowers, Johnson and Pease2004; Johnson & Bowers Reference Johnson and Bowers2004; Mohler et al. Reference Mohler, Short, Brantingham, Schoenberg and Tita2011). Such insights have been captured in predictive policing systems which aim to deploy police to areas before an offence occurs, with initial evaluations demonstrating a reduction in levels of crime (Johnson et al. Reference Johnson, Birks, McLaughlin, Bowers and Pease2007). Similarly, large-scale data on both long-distance travel by air and local commuting can improve predictions of human travel behavior and therefore the spread of epidemics, with clear consequences for the distribution of health resources such as vaccines (Balcan et al. Reference Balcan, Colizza, Gonçalves, Hu, Ramasco and Vespignani2009; Tizzoni et al. Reference Tizzoni, Bajardi, Poletto, Ramasco, Balcan, Gonçalves, Perra, Colizza and Vespignani2012).
When considered at greater breadth, we argue that, in contrast to Bentley et al.’s conjecture, big-data studies do far more than “allow us to see better how known behavioral patterns apply in novel contexts” (target article, sect. 4, para. 13). Online search data, for example, offer us insight into early information gathering stages of real-world decision-making processes that could not previously be observed, while large-scale records of real-world activity enable us to better forecast future actions by allowing us to identify new patterns in our collective behavior. Such predictive power is not only of theoretical importance for behavioral science, but also of great practical consequence, as it opens up possibilities to reallocate resources to better support the well-being of society. Our ability to extract maximum value from these datasets is, however, highly dependent on our ability to ask the right questions: a task for which experts in more “traditional behavioral science” (sect. 4, para. 13) are ideally placed.
ACKNOWLEDGMENTS
Helen Susannah Moat, Tobias Preis, and Nick Chater acknowledge the support of the Research Councils U.K. Grant EP/K039830/1, and Moat further acknowledges support from EPSRC grant EP/J004197/1. In addition, Moat and Preis were supported by the Intelligence Advance Research Projects Activity (IARPA) via Department of Interior National Business Center (DoI/NBC) contract number D12PC00285.