What Is Big Data?

Three characteristics are frequently used to describe attributes of big data: volume, variety, and velocity (McAfee & Brynjolfsson, Reference McAfee and Brynjolfsson2012). Volume refers to the amount of data, the number of observations and of variables, for example. New technologies have greatly expanded the amount of data now accessible to I-O psychologists. For example, a substantial amount of relevant data is captured and stored through processes and databases typically found in organizations, such as human resource information systems, which serve as repositories of information about employees and the workplace events and conditions they experience over time. Other present-day technologies relevant to the research and practice of I-O psychologists can generate big data, such as the use of wearable devices to capture, simultaneously and rapidly, enormous quantities of data about physiological responses, movement, attributes of vocalization, and aspects of social interaction at work, all of which can be usefully supplemented by psychological measures of individual differences, attitudes, and perceptions. I-O psychologists also will find useful large, publicly available datasets such as those made available by government agencies as well as by social media and Internet service providers.

The characteristic of variety is intimately related to that of volume, and the capacity to merge multiple, discrete datasets can create truly mega-datasets. Joining data from quite different sources in I-O psychology is well-illustrated by “linkage analysis,” which historically combined survey data with business performance data related to outcomes such as customer service and profitability. The era of big data exponentially increases the opportunities for linking datasets to analyze connections and interrelationships among different domains to reveal new insights for research and practice.

Velocity is concerned with the speed at which data points accrue. Websites, for example, can stream data at a very high speed, as when clicks are tracked. Sensors can capture data about people and places and events at high rates. Technologies for capturing behavior in controlled settings, such as research laboratories, also can be deployed to generate data at extremely high rates.

Big data is defined by more than just the volume, variety, and velocity of electronic records, however. It also encompasses new sets of tools and techniques for statistical analysis. Indeed, the world of big data brings not only new types of data valuable to I-O psychologists but also alternative statistical ways of summarizing and drawing inferences from data. Big data also is the great equalizer in terms of interdisciplinary research and practice. The same streaming and archived data of value to I-O psychology also are often of value to other disciplines, thus creating occasions for the comingling of perspectives, theories, and analytics techniques.

Examples of Big Data in Use by I-O Psychologists

Before getting to recommendations, it may be useful to give a taste of recent and ongoing research in I-O psychology that reflects one or more of the volume–variety–velocity aspects of big data. What follows is not an exhaustive representation but merely illustrative of the variety of issues and settings of I-O psychology's big data research.

On the research front, big data analytics has emerged as an important tool for examining numerous issues that are core research domains for I-O psychologists. Some recent examples include, but are not limited to, the use of wearable sensors to track team interactions and communication patterns to better understand team dynamics and effectiveness (Kozlowski, Chao, Chang, & Fernandez, Reference Kozlowski, Chao, Chang, Fernandez, Tonidandel, King and Cortina's2015), the analysis of Twitter feeds to explore variability in job satisfaction in different regions of the country (Hernandez, Newman, & Jeon, Reference Hernandez, Newman, Jeon, Tonidandel, King and Cortina's2015), the application of text mining to job advertisements to supplement traditional job analytic methods (Kobayashi, Mol, Berkers, Kismihók, & Den Hartog, Reference Kobayashi, Mol, Berkers, Kismihók and Den Hartog2015), and the integration of advanced analytics to improve the scoring and subsequent predictive capabilities of biodata inventories (Putka, Beatty, & Reeder, Reference Putka, Beatty and Reeder2015). Other big data work includes the investigation of microbehaviors—such as changes in pupils—during test taking in the pursuit of finding better ways of developing culture-fair tests (Hanges & Park, Reference Hanges and Park2015). Studying microbehaviors that precede test responses generates very large per-person data files (e.g., 122 MB) in very short times. Another illustration of the study of microbehaviors as well as simultaneous cognitive and affective processes inherent in learning complex material obtains streaming data captured by measures of galvanic skin response, eye tracking and pupillary changes, video of head and facial changes, verbalizations of thought processes, test responses, and more (Azevedo et al., Reference Azevedo, Harley, Trevors, Duffy, Feyzi-Behnagh, Bouchet, Landis, Azevedo and Aleven2013; Azevedo et al., Reference Azevedo, Landis, Yeasin, Bouchet, Harley, Burlinson, Rahman, Hossain, Feyzi-Behnagh, Trevors and Duffy2011; Harley et al., Reference Harley, Bouchet, Papaioannou, Carter, Trevors, Behnagh, Azevedo and Landis2014). Aligning and analyzing this “river of data” in the interest of disentangling causal processes exemplifies the sorts of unique challenges that reside in big data environments.

A bevy of big data work also is being carried out in applied settings. For example, research-based consulting that often involves analyses of over 200 million records of data about recent workforce dynamics in a single employer has been applied to long-standing concepts of organizational climate and culture (Guzzo, Nalbantian, & Parra, Reference Guzzo, Nalbantian, Parra, Schneider and Barbera2014). Big data analytics are being used as an effective complement to traditional job matching approaches to better place workers in certain roles (Fink, Reference Fink and Roberts2015). I-O psychologists in multiple companies are performing textual analytics of social media data and customer feedback forms to identify service problems and enhance customer experiences (Dullaghan, Biga, Legge, & Kaminsky, Reference Dullaghan, Biga, Legge, Kaminsky and Roberts2015; Hawthorne & Miller, Reference Hawthorne, Miller and Roberts2015). Big data analytics are commonly being employed for personnel selection (Caputo & Boyce, Reference Caputo and Boyce2015), to aid in the identification of high performers (Roberts, Sinnett, & Walzer, Reference Roberts, Sinnett and Walzer2015), and to enhance employee retention (Mitchell, Blair, & Speer, Reference Mitchell, Blair, Speer and Roberts2015). Moreover, we are witnessing I-O psychologists implement novel applications of big data analytics such as Taylor's (Reference Taylor2015) use of a machine learning algorithm to evaluate digital interviews.

This flurry of interest and activity in big data also is captured by an examination of the three most recent SIOP conference programs. In 2013, there were a total of four separate sessions with “Big Data” or “Analytics” in the title. That number nearly tripled in 2014, with 11 separate sessions, and jumped again to 15 separate sessions in 2015. The sessions range from presentations of specific applications of big data to best-practice recommendations for establishing a human resource analytics function to discussions of ethical dilemmas created by big data.

Recommendations for I-O Psychology

The recommendations offered here are about dealing with big-data-related challenges and hazards that I-O psychologists may encounter in three issue areas:

• • Privacy and personal information: The extent of accessible information about individuals is a hallmark of the big data world. This section focuses on the challenges that relate to the use of big data that identifies individuals or that makes them identifiable.

• • Informed consent: The voluntary and informed choice to be a participant in a research project is a principle that underlies the conduct of much research activity in I-O psychology. Big data introduces new complexities and challenges with regard to informed consent that are addressed in this section.

• • Integrity of data and data analysis: Big data is not always good data. Furthermore, new data analytic techniques inherent to working with big data often lack well-developed professional standards of use. These issues are the focus of this section.

The following sections highlight some of the distinctive features with regard to what is different about the big data world and how those differences create circumstances and complications that have not been fully anticipated. Existing, widely shared principles that influence I-O activity with regard to privacy, informed consent, and data integrity do not always fit the realities of the big data era, thus creating “guidance gaps” for the scientist–practitioner. The recommendations offered are intended to close such gaps, to help the profession adapt effectively to new realities and generally help I-O psychologists judiciously address what could be perils that can be encountered in professional activity involving big data.

Privacy and Personal Information

Large, complex, dynamic, multidimensional datasets create risks to personal data privacy. Further, the magnitude and complexity of datasets, combined with modern data management strategies (e.g., cloud storage, pervasive connectivity), create vulnerabilities for the data we collect. As such, we recommend that research and practice efforts that involve big data will benefit by building and adhering to a data privacy protection plan.

A data privacy plan will have several aspects to it. One concerns time. The risks of reidentification of previously anonymized data grow over time, for example, as new data becomes available that can be related to stored, anonymized data. Such a plan also can include specifying what protections will be implemented with regard to data by individuals or their devices. A data privacy plan may also specify which among many datasets may be joined up and which are prohibited from being integrated for particular types of analyses. Physical and information technology (IT) protections for the master dataset, including where it is stored, how it is stored, for how long it will be stored, and who will have access, also can be part of a plan. Further, researchers complying with institutional review boards may find it useful to explicitly connect their data privacy to requirements to protect research participants from harm.

Although data often knows no borders, data privacy laws do, and data privacy laws vary considerably around the globe. As global employers, for example, organizations may find themselves with data collected in one county, from citizens of a second country, stored in a third country, and analyzed in a fourth. Untangling the nuances of which privacy standards apply requires careful consideration of the relevant legal standards in each country. In some cases, it may be appropriate and necessary to pursue multiple data privacy standards within the bounds of a single project; for example, participants from Germany may require fully anonymized data at the point of collection and a shorter retention period. Safe Harbor (http://export.gov/safeharbor/eu/eg_main_018475.asp) outlines a set of strategies for handling data from multiple countries. Data storage can pose challenges, too, as when different segments of the same population in a pooled dataset are subject to different privacy standards. Strategies around notification and opt-in and opt-out may also need to be different for different research populations.

For many years, anonymization has been a cornerstone of protecting research participants’ privacy and insulating them from unanticipated harm emanating from their contributions to a research effort. Also for many years anonymization was considered sufficient protection. However, times have changed. As Ohm (Reference Ohm2010) writes, computer science has “undermined our faith in the privacy-protecting power of anonymization” (p. 1701) because reidentifying (or deanonymizing) individuals in anonymized datasets now can frequently be achieved with methods that are “easy, cheap, powerful” (p. 1706). It is estimated, for example, that 87% of the U.S. population can be identified by knowing a person's gender, zip code, and birth date and year (Sweeney, Reference Sweeney2000). Ohm (Reference Ohm2010) goes on to assert that, relative to years past, the power of reidentification amplifies the potential harms of failing to protect privacy because of the vast amount of other data that could, in principle, be joined with reidentified data.

A data privacy protection strategy is thus recommended as an essential step in protecting individuals from harm. An overall strategy is likely to have many components—or tactics—to achieve the desired result, depending on circumstances. What are such tactics? Table 1 outlines several such steps that can be implemented to protect personal information and help ensure privacy. Table 1 starts with the fully identified dataset and offers several actions, each with a different nature that can be taken to protect the identified individuals in that dataset.

Table 1.

Data Privacy Protection Strategies

All of the approaches in Table 1 require some process between data collection and data analysis. Typically transformation is required at either the individual “row” level, as in anonymizing or pseudonymizing, or processing for some form of aggregation or, in the case of sharding, disaggregation. Theory-driven inquiry may provide an advantage here; with clear hypotheses in mind, researchers can more effectively remove irrelevant data, thereby reducing processing demand and, in most cases, improving privacy protection. However, removing data clearly eliminates opportunities for discovery of unexpected yet important relationships that big data analytic methods are designed to detect.

Beyond data collection and analysis there is also the oft-neglected question of data storage, particularly where longitudinal analysis will be required. Essentially, there are four types of storage for data like this: (a) local, transportable storage (like thumb drives or external drives); (b) local computer or laptop hard drives; (c) cloud based storage; and (d) shared storage inside a firewall. Although each type has risks, local, firewalled, physically protected servers represent the least risky option.

At a very practical level, there are several strategies research teams can take to protect data. These include simple steps like practicing good data cleaning upon intake or upload, such that confidential data is expunged from datasets, and running scanners to detect any files on their drives that contain key information types such as U.S. Social Security numbers—a commonly used identifier—with a characteristic format. Periodically conducting a “confidential data audit” to identify such data can be very helpful. Teams with ongoing needs for integrated datasets can run their own servers for data storage and access and ensure physical and IT security for those. From that secure server, individual researchers can query only the necessary data fields (i.e., use sharding) to answer a specific question. Ensuring that even computers with access to that server have appropriate barriers to data theft, in the form of options like whole disk encryption, can add additional layers of protection.

Finally, in practice, it is often necessary to employ multiple strategies at multiple points along the privacy protection chain. For example, the full database may need to be retained but held under high physical and IT security with limited access, encryption, and other safeguards. Individual research projects may extract subsets of data from the full database in a sharding strategy and pseudonymize the data within that shard for a specific analysis. Equipment (i.e., laptops, or mobile devices) used to access the databases should have robust local security measures in place, such as access passwords. Connecting this suite of security measures together at the level of the individual device, the individual dataset, the master dataset, and the master storage often will provide the best possible security for research participants’ data.

Below is an example of a suite of data protections that work together across multiple levels to protect data privacy:

1. 1. The local device (e.g., laptop) has additional layers of security such as whole disk encryption that prevents it from being used or remotely accessed by others.

2. 2. The accessing device has specific, time bound privileges for accessing the master database or original data.

3. 3. The specific dataset being analyzed contains only the critical few variables for the analysis at hand (sharding); it does not contain identifiers or personal information.

4. 4. The dataset is analyzed on a secure device. The specific file is encrypted. The dataset is never shared by e-mail or by a device such as a USB drive.

5. 5. That dataset is destroyed once analyses are run; parameters are documented for replication, but the actual data file is destroyed to prevent reidentification. Any temporary files or copies are destroyed, and a scan is run to find and remove any additional copies.

6. 6. If original data must be retained, it is stored on a server with physical and IT security.

Most organizations—employers, research institutes—establish rules and protocols for protecting privacy. The recommendations offered here may in fact be embedded in those local rules. Absent such rules—or to supplement existing rules—the recommendations presented here, we hope, provide useful guidance in a world full of threats to individual privacy.

Informed Consent

The practice of informing participants about the purpose of research, its procedures, and their individual rights is rooted in the sometimes-ugly history of unethical conduct in social science. As a result of this history, many organizations (particularly those funded by the government) have established institutional review boards to ensure that research minimizes the risks of unwanted harm to its participants. Informed consent, formal signatures, and debriefing for research participants are long-standing, essential ingredients in the practice of minimizing harm to research participants. However, the application of these traditional procedures in the world of big data often is infeasible, and yet the importance of protecting the rights of human participants remains unquestioned.

I-O psychologists can look toward existing documents to form new ways of thinking about informed consent in the context of big data. The SIOP (2003) Principles contains a general statement that “all aspects of the research described in the Principles should be performed in compliance with the ethical standards of the American Psychological Association as endorsed by SIOP” (p. 35). In turn, the American Psychological Association's (APA’s) code of ethics provides useful recommendations for considering when informed consent is not necessary. Specifically, in section 8.05, the APA's (2010) code states that

A substantial portion of what I-O psychologists do by way of big data research and use likely falls into these categories. Indeed, sections (b) and (c) in particular likely cover many of the research-based and organization-based analyses of interest to I-O professionals who deal with information, observations, or anonymous survey responses collected on the Internet and stored electronically. However, the nature of big data poses key challenges not anticipated by APA's code. For example, as previously discussed, although data may be collected in an anonymous fashion it is often possible to join such data with other often publicly available data such that the new, combined dataset enables the reidentification of individuals in the previously anonymized data. For example, Sweeney (Reference Sweeney2000) joined anonymized health records with publicly available municipal voter roll data to identify individuals whose health records had been anonymized.

APA's code provides further guidance in section 8.03 with regard to informed consent and recorded voices and images. Such recordings, especially digital photographic and video images, are increasingly common in everyday life and thus in big datasets:

The latter consideration raises an important set of questions about whether informed consent is necessary when deception is involved. In this case, three questions pertaining to potential risks may be helpful to consider (see King, Hebl, Botsford Morgan, & Ahmad, Reference King, Hebl, Morgan and Ahmad2013). First, I-O psychologists should consider whether informing participants about their involvement in a study affects the responses they provide. For studies of behavior in organizations that involve sensitive information, such as counterproductive behavior or abusive supervision, the answer quite likely will be, yes, involvement will affect responses. Consequently, a second question becomes important to ask: Could the research objectives be adequately served through a milder or less extreme method? Minimizing risk through milder (less reactive, less intrusive) methods, particularly in those cases where informed consent is not provided, should be considered carefully, and such alternatives should be adopted whenever possible. A third and related issue is whether the research requires participants to engage in any activities that are outside of their regular daily behavior. Many organizational studies involve observing responses to mild treatments in ongoing “regular” activities (e.g., changing work spaces and monitoring productivity). In other words, though deception is involved, participants are not asked to experience or engage in anything outside of their typical job duties.

In those cases where informed consent is appropriate, such as when the potential harm is anticipated or confidentiality is not protected, the APA recommendations (8.02) specify that

1. (a) When obtaining informed consent as required by Standard 3.10, Informed Consent, psychologists inform participants about (1) the purpose of the research, expected duration, and procedures; (2) their right to decline to participate and to withdraw from the research once participation has begun; (3) the foreseeable consequences of declining or withdrawing; (4) reasonably foreseeable factors that may be expected to influence their willingness to participate such as potential risks, discomfort, or adverse effects; (5) any prospective research benefits; (6) limits of confidentiality; (7) incentives for participation; and (8) whom to contact about the research and research participants’ rights. They provide opportunity for the prospective participants to ask questions and receive answers.

2. (b) Psychologists conducting intervention research involving the use of experimental treatments should clarify to participants at the outset of the research (1) the experimental nature of the treatment; (2) the services that will or will not be available to the control group(s) if appropriate; (3) the means by which assignment to treatment and control groups will be made; (4) available treatment alternatives if an individual does not wish to participate in the research or wishes to withdraw once a study has begun; and (5) compensation for or monetary costs of participating including, if appropriate, whether reimbursement from the participant or a third-party payor will be sought.

In summary, APA's code of ethics provides explicit guidance pertaining to when informed consent is and is not required. Much of this guidance applies to big data contexts. But there are “gray areas” that arise in a world of big data that are not fully anticipated by codes of ethics. Often those gray areas arise when individual users of a business service (e.g., Facebook, Uber) at the time they are creating an account explicitly consent to the use of their data for purposes such as “analysis and research.” They could have opted out. Instead, they signed on and in the process provided their consent to have their data analyzed. But what, exactly, have they consented to by way of research? It is fair to say that neither the business nor the individual could reasonably anticipate all possible “research and analysis” topics to which their data might one day be subject. Facebook, for example, caught flak for allowing the affective content of news feeds to a sample of users to be intentionally skewed positively or negatively to observe the effects on the affect of users’ subsequent postings; Uber caught flak for using its data to analyze transportation patterns that were potentially indicative of sensitive behavioral patterns of account holders. The Facebook example also illustrates that what may be ethical for an industry-based researcher with regard to informed consent may not be so for a university-based collaborator because of different standards imposed by their employing organizations. Indeed, a divergence of standards regarding what is ethically permissible research appears to be a growing risk, fueled by the big data world as it generates ever-increasing types of new information that can be joined up, for research purposes, with archived data. Big data poses new intricacies and opportunities and so brings with it the need for renewed consideration of what is to be considered right and wrong, ethical and unethical with regard to standards of informed consent and the consequences of such standards for the advancement of the science and practice of I-O psychology.

Integrity of Data and Data Analysis

There is no loss of the importance of adhering to high-quality scientific methods just because data comes with volume, variety, and velocity. Core principles that have guided the execution of traditional (pre-big data) studies continue to apply and should continue to be recognized. Yet big data brings with it the need for more nuanced views and guidance with regard to some familiar yet critical issues regarding the appropriate use of big data in research and practice.

An Invitation to Respond

Concerns may arise when reading the recommendations presented here. Perhaps the conservative approaches we advocate may be regarded as overstated and thus, if enacted, place unwarranted constraints on the speed of adopting big data approaches in I-O psychology. In academic settings, dealing with institutional review board procedures, participant queries, and design concerns undoubtedly takes time, and complexities of big data research may prolong the review process. In applied settings, security requirements are increasingly implemented as core business processes—in part in response to the accelerating frequency and severity of data breaches and cyberattacks—and those security requirements, especially if supplemented by recommendations offered here, can pose obstacles to the successful conduct and uptake of research and data-driven practice. All things considered, it is our view that, yes, these approaches may limit how quickly we are able to achieve the promise of big data but that the trade-off of speed is worth it.

The concerns and recommendations raised here, although intended to be constructive, are by no means intended to be final or definitive. Indeed, we believe that it is in the vital interest of I-O psychology to generate extensive dialogue, deeper consideration, and greater insights into the best ways for I-O psychologists to navigate matters of privacy, informed consent, data integrity, and other challenges that the era of big data brings to the fore. We invite response to the ideas presented in this article—disagreements, improvements, elaborations, war stories—through the formal commentary response that is a defining characteristic of this journal. We also invite responses that call out important issues we have left unaddressed. Alternatively, anonymous feedback and observations can be posted at https://www.surveymonkey.com/s/SIOPbigdata.