Embracing Prediction

Breiman (Reference Breiman2001), in his seminal article, describes two approaches to analyzing data. The first of these approaches, called the data modeling culture, emphasizes a deductive approach where a model is hypothesized and then validated based on goodness-of-fit. This is the dominant paradigm in our discipline where psychologists explore causal relationships between constructs to explain human behavior using controlled experimental research designs and statistical techniques such as ANOVA, regression, and structural equation modeling. Though psychologists’ infatuation with the data modeling culture has certainly been useful, their sole reliance on it has likely stifled scientific progress. Many of the models psychologists test using this paradigm are likely oversimplified and underdeveloped, and often, competing models, that may fit the data just as well, are never tested nor evaluated. Thus, they may be placing their faith in models that fit the data, but nevertheless are not true.

Breiman’s (Reference Breiman2001) alternative to the data modeling culture is the algorithmic modeling culture, which encapsulates the approach of big data methods and focuses on maximizing predictive accuracy. Instead of applying what we think we know about the data, algorithmic models get to learn the qualities of the data. Such an approach confers a number of critical advantages like the ability to incorporate a wider variety of variables and more complex relationships between variables, even without knowing a priori what exactly the form of those relationship should be. By emphasizing predictive accuracy and capturing more elements of the relationship between variables, one ultimately learns more about the problem under investigation. For research to progress, psychology needs more flexible, complex models and the algorithmic culture enables this to happen. Thus, we argue that psychology as a science needs to embrace the algorithmic culture of big data methods and by doing so, we have the opportunity to transform the psychological sciences.

Transforming the Psychological Sciences

Tempering Algorithmic Concerns

An acceptance of the algorithmic modeling culture in psychology is not without limitations. To start, psychologists lack training in and exposure to many predictive methods. Additionally, the psychological sciences stress a theory-driven approach to modeling and view algorithmic approaches as atheoretical, uninterpretable, and biased. Though each of these concerns contains a kernel of truth, these concerns are often exaggerated and may in fact be more myth than reality.

Anderson (Reference Anderson2008) declared that the end of theory was at hand because big data makes the scientific method obsolete. Unfortunately, hyperbolic proclamations such as these create the mistaken impression that big data science is atheoretical, which is patently false. It is true that big data science is less theory driven than our traditional hypothetico-deductive approach, but big data approaches are certainly theory-laden and perhaps can be best described as theory-informed (Haig, Reference Haig, Woo, Tay and Proctor2020). The acceptance and implementation of algorithms in the psychological sciences does not inherently mean that psychologists would disregard the use or importance of theory. In fact, theory plays a critical role in the creation and interpretation of predictive models. A main consideration in big data and algorithmic modeling is the choice of predictors and which features to engineer. Theory is critical in guiding the feature engineering process, and the inclusion of theoretically relevant variables in the model is a necessity to have highly interpretable and meaningful results (Urban & Gates, Reference Urban and Gates2019).

Algorithmic approaches are commonly viewed as black boxes. The vast number of predictors utilized in big data projects and the complexity of algorithmic models (e.g., artificial neural networks) make big data approaches substantially harder to interpret than simpler analytical approaches (e.g., ordinary-least-squares regression). Although there is some truth to this criticism, it neglects several important facts. The first is that accuracy should be of paramount importance for any scientific model. A simple, easily explainable model that produces inaccurate predictions is not useful. One would be better served having an accurate model and trying to interpret it in all of its complexities. A model of our universe where Earth is at the center is certainly simpler than one that has our galaxy as but one of several hundred billion; nevertheless, we are better served by the more complex and harder to explain model. A second shortcoming of this black box criticism is that it neglects an ever growing field of research on explainable artificial intelligence (AI). When big data methods were in their infancy, lamenting about the interpretability of these models was perhaps well-founded. Now, however, tools exist that allow researcher to understand what features of a complex algorithm are driving the predictions being made (see Local Interpretable Model-Agnostic Explanations [LIME], Ribeiro, et al., Reference Ribeiro, Singh and Guestrin2016 as an example). In fact, one can even point to instances where algorithms can be even more revealing than traditional methods. For example, when humans make decisions or provide ratings, they may lack insight into what features are responsible for the response they are providing. In contrast, one can use one of these explainable AI tools to investigate why the algorithm is behaving the way that it is. In some ways, an algorithm that is trained on human labeled data can allow researchers to peer into the black box of the human mind to begin to infer why the human rater behaved the way they did (Thompson, et al., Reference Thompson, Koenig, Mracek and Tonidandel2020).

There is also a pervasive belief that algorithmic approaches are inherently biased, and numerous high profile examples exist to support this perception (e.g., Dastin, Reference Dastin2018; O’Neil, Reference O’Neil2016). Despite these spectacular failures, this criticism is not really one that should be leveled at the algorithms but rather at the underlying data. Existing societal injustices are baked into the data that the algorithms train on. The algorithms just learn and replicate the bias that already exists. Thus, these approaches are no more biased than the current non-algorithmic approaches. More importantly however, algorithmic researchers can inspect the algorithms for bias and can make efforts to train that bias out of predictions. It may even be easier to train bias out of algorithmic predictions than to train human decision makers to ignore their own subconscious biases.

Psychologists’ Key Role in the Revolution

One does not need to be a data scientist or a computer scientist to play an integral role in the big data science revolution. Even with the above transformations to psychological research, psychologists can advance big data methodology itself due to their extensive training and knowledge of measurement. Additionally, psychologists frequently face ethical considerations regarding the collection and security of human-generated data, which makes them uniquely suited to handle datasets that capture human behaviors and processes.

More important than the utilization of big data is the appropriate measurement and collection of big data, and psychologists’ extensive training in measurement can greatly improve the current practices in big data research (Woo, et al., Reference Woo, Tay and Proctor2020). Though data scientists are extremely capable at applying the latest and greatest algorithmic tools to the data that is made available to them, their lack of training in psychometrics can sometimes lead to situations where big data projects are criticized as “garbage in, garbage out”. Measurement is essential to maximizing the returns from big data projects, and psychologists, with decades of scientific literature and years of focused graduate training on how to accurately measure constructs of interest, are ideally positioned to contribute. Although data scientists have access to large quantities of data and the amount of data generated by humans every day continues to grow, the amassment of large datasets does not automatically result in more accurate or nuanced results. Jacobucci and Grimm’s (Reference Jacobucci and Grimm2020) study demonstrates how impactful measurement is to the big data literature. Jacobucci and Grimm (Reference Jacobucci and Grimm2020) tested the influence of measurement error on model estimation in a simulation study and demonstrated that algorithmic models perform really well when the data is exceptionally clean and measured without error. Nevertheless, when error was introduced to the data, 𝑅² was reduced by 76% in big data models. With the introduction of the slightest bit of measurement error, predictive accuracy falls. Thompson, et al. (Reference Thompson, Koenig, Mracek and Tonidandel2020) provide additional evidence of the meaningful gains that can be made when measurement is a central component of a big data study.

Psychologists also have a key roles to play in ensuring the ethical use of big data. Psychologists have advanced training and familiarity with rules governing the protection of human subjects stemming from the Belmont Report (National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, 1979). The central principles of Respect for Persons, Beneficence, and Justice, continue to be essential for ensuring ethical use of big data sources (Paxton, Reference Paxton, Woo, Tay and Proctor2020). These principles are imperative for the current use of big data, but also for unforeseen circumstances. As the volume of big data sources rises, new challenges concerning the ethical and fair use of that data will also increase and require the creation and enforcement of new privacy and security measures (Guzzo, et al., Reference Guzzo, Fink, King, Tonidandel and Landis2015). Psychologists, by virtue of their history and training, can help to ensure that the rights and dignity of participants are protected.

Although we strongly espouse that the field of psychology embrace these big data methods, we do not want to be accused of big data hubris (Lazer, et al., Reference Lazer, Kennedy, King and Vespignani2014), and we recognize that limitations exist for these methods as they do with any method. Nevertheless, we believe that big data, associated algorithmic methods, and a culture that embraces prediction affords an opportunity to transform the psychological sciences and propel the field forward.