Procedure

We considered a variety of data sources for this investigation, each ranging in academic rigor and timeliness. Peer-reviewed publications in I-O psychology journals represent the most rigorous, yet least timely, source. The peer-review process for journal articles, despite ensuring quality research in our field, can take years to unfold and represents a poor measure of when I-O psychologists actually study a given technology. The other extreme is social media discussions among I-O psychologists, which represent a timely, yet questionably rigorous, source. Many members of our field use platforms like Twitter to exchange ideas about hot topics in I-O psychology, but many of these discussions are speculative and lack evidence. In the middle of these extremes are conference presentations, which include a peer-review process but are much more timely than academic journal articles. Furthermore, SIOP submissions reflect a desire to share one’s knowledge with peer I-O psychologists, representing a signal that the topic of these submissions is important in the I-O community. In contrast, both journals and social media are intended to influence others at large, without a clearly defined audience for the most part. Thus, we chose to study the timeliness of I-O technology research in conference presentations at SIOP.

The SIOP annual conference represents the largest gathering of I-O psychologists, and the content of each year’s program represents the topics that are predominant in the field at a given point. As such, the year that a work-related technological innovation first appears on a SIOP conference program should fairly accurately represent when that innovation entered the I-O psychology discourse. Examining the difference between when technological innovations are created or popularized (i.e., “technological milestones”) and when they appear on a SIOP conference program can thus provide quantitative evidence of whether I-O psychologists are truly lagging in producing timely technology research.

We used archival data analysis techniques to investigate the timeliness of technology research in SIOP by first generating two separate timelines. The first, the History of Technology (HoT) timeline, represents a compilation of five separate timelines of technology: the Technology Timeline from datesandevents.org, the Technology Through the Years timeline from King University, the Timeline of Computer History from the Computer History Museum, the Selected History of DARPA Innovation timeline from the Defense Advanced Research Projects Agency, and the Zakon Reference Zakon(2018) Hobbes’ Internet Timeline 25. Additions to the HoT timeline were made on a case-by-case basis when workplace technologies were not mentioned in the five primary sources. Additional sources included MIT’s Technology Review, the Museum of Arts and Design’s 3D Printing Timeline, and product or service websites (e.g., Fitbit.com, Uber.com, Skype.com). For any remaining cases, we used peer-reviewed articles found on Google Scholar to identify the year for which a specific workplace technology was first introduced. Across all of these sources, most data points indicated when a technology was first invented (e.g., social media) or when a technology-related word was first coined (e.g., Internet of Things). Other sources provided data points that indicated when a technology was popularized or became commercially available (e.g., GPS). The final HoT timeline included over 1,500 data points related to technological milestones from 2400 BCE to present day.Footnote ¹

The second timeline, the SIOP timeline, includes a comprehensive list of technology-related submissions over the last 23 SIOP conferences. The researchers manually reviewed all public SIOP Annual Conference programs available on SIOP’s website, which included 23 programs between 1998 and 2020. Researchers reviewed each program for any posters, symposia, panel discussions, or other submissions that related to technology concepts. Symposia, practitioner forums, and other special events containing multiple papers about technology were further sorted; any non-technology-related submissions in multipaper events were omitted from the SIOP timeline.Footnote ²

Any submission whose technology relevance was under question was highlighted in the timeline and reviewed by two researchers; cases in which both reviewers agreed that a submission lacked relevance were removed. The researchers then determined which submissions were also identified in SIOP conference programs as being related to technology. Physical copies of conference programs were reviewed to ensure accuracy in SIOP’s online programs. For example, the submissions listed online for 2001 contained only one item tagged for technology, whereas the physical copy listed 76 items. Additionally, a select few items marked as technologically relevant in the SIOP programs were not included in the SIOP timeline because the researchers did not agree that these items were relevant.Footnote ³ All remaining submissions were reviewed for inclusion in the final SIOP timeline, which included 2,365 technology submissions from 1998 to 2020.

We identified 48 technology milestones since 1998 with work-related implications from the HoT timeline.Footnote ⁴ For milestones related to the invention or popularization of a broad family of technologies (e.g., social media; virtual reality headsets), we calculated the average of the years for which specific technologies were introduced (e.g., Facebook and Twitter; Occulus Rift and HTC Vive). Of these 48 milestones, 40 had been studied at least once at a SIOP conference. To identify the first SIOP submission for a given milestone, the researchers searched the SIOP timeline for keyword variations of the milestone. For example, for the popularization of wearable devices, we included keyword searches for “wearable,” “tracker,” “smartwatch,” “Fitbit,” and “Apple Watch” to find the earliest SIOP submission discussing the milestone.

To analyze the data, a difference was calculated for each milestone using its years on the HoT and SIOP timelines, respectively. This difference represents the lag in I-O psychology research on a given technology. To better interpret each lag, we also explored the format of submissions (e.g., posters, symposia, etc.) across all first mentions associated with the milestones.

Results

Our results show that on average, technological milestones are first discussed at the SIOP conference 6.10 years (SD = 4.47) after they occur. As a follow-up analysis, we investigated whether these mean differences grew smaller, larger, or remained the same over time. To do so, we divided technological milestones across three periods. Alarge proportion of the 40 milestones occurred during the first few years of the SIOP timeline, so we selected intervals based on equal numbers of observations rather than equal numbers of years. Each interval (i.e., 1998–2001, 2002–2005, and 2006–2020) contained either 13 or 14 observations, allowing for a more representative comparison of means. Results show that during 1998–2001, there was an average lag of 7.08 years (SD = 5.69). This average lag decreased to 6.31 years (SD = 4.09) during 2002–2005, and it decreased again to 5.00 years (SD = 3.51) during 2006–2020.

Across the 40 technological milestones, we found 54 SIOP submissions that represented a “first” investigation. Seven milestones (i.e., essay scoring, autonomous robots, social media, big data, deep learning, gig economy, and wearables) were mentioned in more than one submission in the year they first entered our SIOP timeline. For example, there were eight submissions on big data in 2014, the year it first appeared in our timeline. Of the 54 first submissions, 29.63% were posters, 22.22% were symposia, 12.96% were special events, 11.11% were panel discussions, 7.41% were practitioner forums, 3.70% were roundtable discussions, 3.70% were master tutorials, 1.85% were alternative sessions, 1.85% were education forums, and 1.85% were debates.

Discussion

We found that SIOP members discuss technological milestones an average of 6.10 years after they have occurred, and this lag appears to have decreased over time. On average, SIOP members were slower to study the first third of the technological milestones (i.e., those occurring during 1998–2001) than they were with more recent technological milestones (i.e., those occurring during 2002–2005 and 2006–2020). However, we also identified eight work-related technological milestones that have never been covered at SIOP conferences (e.g., “coworking,” 3D printing). Seven of these unstudied topics occurred during the final third interval of our data, meaning that the actual timeliness of I-O psychologists’ research for the 2006–2020 interval may be slower than 5 years. To illustrate, consider the hypothetical scenario in which I-O psychologists investigated all of the unstudied topics at the 2021 conference. If this were the case, the new average lag for technological milestones occurring between 2006 and 2021 would be 6.62 years, which is comparable to the 2002–2005 lag. Still, such a scenario is unlikely, and as a result, this study offers preliminary evidence that I-O psychologists are beginning to produce timelier technology research at SIOP.

There are several possible explanations for the decreased lag we observed. Perhaps the most obvious is the increasing size of the SIOP community since 1998. According to the SIOP website, there were around 2,500 conference attendees in 1998. This estimate more than doubled to 5,348 attendees in 2019. With more attendees, one would expect greater diversity among members and, in turn, more diverse interests in the community. Another explanation is that that the number of journal outlets and the interests of those journals has broadened, thus opening up more opportunities to engage in technology research. Indeed, some I-O related journals now explicitly list technology as a desired subject matter of their papers, and others have recently called for special issues on technology-related topics. Yet another explanation for the reduced lag concerns the general history and evolution of I-O psychology. As the field has grown, I-O psychologists have established a body of knowledge that is mature, and thus it is more difficult to make a transformational contribution in those areas. Although some have urged new researchers to study established I-O topics (Campion, Reference Campion2011), it is worth noting that technology topics tend to emerge and present a variety of new opportunities for researchers wishing to make a name for themselves. Collectively, these incentives have likely encouraged I-O psychologists to engage in more technology research in recent years.

The variation in formats across first submissions provides some indication of the kinds of research I-O psychologists have conducted on new technological milestones. Over half (51.85%) of first submissions were posters or symposia, and these formats typically represent empirical investigations of psychological phenomena. Thus, I-O psychologists seem to gather data a substantial amount of the time when studying new technologies. It is particularly interesting to note that symposia, which involve multiple researchers coming together to present their work on a similar topic, represented the first submission for several technological milestones. For example, the first submission for text analysis was a symposium titled Automated Text Analysis in I-O Psychology: Research to Practice, which included papers from researchers at four different institutions sharing their work on the topic. It appears that I-O psychologists had been conducting research on text analysis for some time before presenting it at SIOP. The rest of first submissions were less empirical, such as practitioner forums or roundtable discussions. Nonempirical formats may be particularly useful for introducing scholars to new research areas for which theory or a conceptual understanding is lacking. For example, one theme track session was described as a TEDstyle talk that presented information about new technologies to attendees in a “digestible, engaging manner” that would perhaps not be possible with other formats (Waber etal., Reference Waber, Impelman and Shockley2014). Collectively, the variability in first submission formats seems to suggest a balance in I-O psychologists’ empirical and nonempirical investigations of new technologies.

Although the decrease in lags over time is encouraging, we should note that a sizable lag exists across every interval of our timeline. Even the most recent lag estimate of 5 years, although smaller than the earlier estimates, demonstrates a general deficiency in I-O psychologists’ coverage of new technologies. To better contextualize what our average lag estimates mean for I-O psychologists, let us consider technologies individually. We will begin by describing findings regarding the three technologies discussed in the introduction (i.e., videoconferences, virtual worlds, and AR wearables), then turn to a discussion of other noteworthy technologies.

For videoconferences, I-O psychologists actually studied the technology before its popularization in organizations. Although videoconferencing was popularized in 2003 in part due to the launch of Skype, we identified a 1998 poster that investigated videoconferencing software in the employment interview (Chapman & Rowe, Reference Chapman and Rowe1998). The study investigated organizations using similar software available at the time, finding that videoconference interviews were less effective for recruiting than other media. In addition to this study, a 1999 symposium discussed the challenges and opportunities associated with videoconferencing for team collaboration (Offermann & Eller, Reference Offermann and Eller1999), a 1999 poster investigated gender differences in videoconference negotiations (Graetz etal., Reference Graetz, Barlow, Prouix, Odenweller, Weierman, Blankenship and Strazzo1999), and a 2002 poster explored applicant reactions to videoconference interviews (Quintero & Strickland, Reference Quintero and Strickland2002). The timeliness and abundance of this research is aligned with the generally favorable public reactions to videoconferences since 2003. Science and practice should ideally inform one another: in the case of videoconferences, I-O psychologists were prepared to inform organizations in their decision to use (or not use) the technology in the early 2000s.

With a lag of 7 years, virtual worlds received less timely research from the I-O community. This technology was popularized in part due to the creation of World of Warcraft in 2004, yet a 2011 symposium about its uses in recruitment, training, and teamwork was the earliest SIOP presentation we identified on the topic (Landers & Behrend, Reference Landers and Behrend2011). The following SIOP presentations on the topic came several years later, with a 2017 presentation exploring virtual world assessment (Short etal., Reference Short, Weidner and Sirabian2017) and a 2018 poster exploring virtual world training (Goh etal., Reference Goh, Truman and Barber2018). This apparent lack of research from the I-O community did little to inform the decisions of those who adopted virtual worlds for work in the late 2000s. Although I-O psychologists are not solely responsible for the short-lived use of virtual worlds in certain contexts (e.g., in meetings; Pearlman & Gates, Reference Pearlman and Gates2010), more research on the technology could have helped organizations use the tool more effectively and thus facilitated a broader adoption.

Augmented reality wearables have received little to no attention from I-O psychologists at SIOP conferences. These devices were popularized in 2012 in part due to Google Glass, yet we identified no SIOP presentations with the phrases “Google Glass” or “augmented reality wearable” in the title or abstract. We did identify four studies about AR more generally (Keebler, Reference Keebler2017; Keeney etal., Reference Keeney, Halverson, Oster and Kegley2020; Roberts, Reference Roberts2018; Sydell etal., Reference Sydell, Boyce, Lahti, Landers, Mecca and Ross2017), but the earliest of these was presented in 2017. Moreover, most of these were sessions in which I-O psychologists reviewed the history or theorized about the future of AR training without conducting new empirical investigations of the practice. Such theorizing can indeed be useful, particularly for guiding subsequent research on technologies that are new or challenging to study. Moving beyond conceptual sessions toward more empirical sessions may require collaborations and partnerships, which we discuss further in the Collaboration section below. Encouragingly, Keeney etal.’s (Reference Keeney, Halverson, Oster and Kegley2020) poster about estimating the value of virtual, augmented, and mixed reality tools in training seems to represent an early empirical investigation into AR at work. Still, several gaps still remain in understanding AR’s work-related psychological effects. In light of Google Glass’s resurgence in organizations, I-O psychologists may soon find themselves rushing to collect empirical data that can resolve any unforeseen (or foreseen, i.e., privacy invasion) issues the technology presents.

Across various technologies, it appears that I-O psychologists are often slow to study technologies that are created for nonwork purposes. For example, there were notable lags for some technologies marketed toward gamers (e.g., virtual worlds) and everyday consumers (e.g., social media). To I-O psychologists, the work-related implications of such technologies may not be immediately clear. Indeed, we found it somewhat challenging to select tools for inclusion in our unstudied technologies timeline because there are many tools that could have work-related implications, despite their absence from many white-collar workplaces (e.g., 3D printers, drones). As technological advancements continue to occur in various industries (e.g., gaming, transportation, engineering), I-O psychologists will need to look outside of the traditional organizational context if they wish to study work technologies in a timelier fashion.

Changes at the within-technology level of analysis are also worth addressing. Consider wearable devices, which had a lag of 5 years between their popularization and I-O psychologists’ first empirical investigation. It is unwise to assume that an employee’s wearable device in 2009 possessed the same capabilities as an employee’s wearable device in 2014. For example, the first Fitbit device allowed users to track their movement, sleep patterns, and calories burned but was not smartphone connected. By 2014, the device acquired new features such as continuous heart rate tracking and passive phone-based tracking. Employees using the 2014 Fitbit could have exhibited different privacy concerns than those using the original Fitbit, though hypotheses such as this remain untested. Functional changes to a tool over time can affect employees in different ways, and these changes necessitate timely research from I-O psychologists much like newly introduced tools.

In addition to discrete technologies, we also included several “buzzwords” and trending topics in our analysis. Buzzwords and trending topics are distinct from individual technologies in that they reflect a broader, socially driven interest in a given tool or practice. For example, “social media influencer” represents a new job that many individuals now find lucrative by virtue of new technologies (e.g., social media, livestreaming). Perhaps unsurprisingly, I-O psychologists waited 9 years to formally study this job after its coining in 2011 (Bricka etal., Reference Bricka, Schroeder and Hameed2020; Freberg etal., Reference Freberg, Graham, McGaughey and Freberg2011). Or, consider “big data” as another example. Although specific technological inventions supported the existence of big data (e.g., smartphones, wearables, data analysis software), the phrase was not coined until 2005 when researchers recognized just how large and powerful their data sets were becoming (Emani etal., Reference Emani, Cullot and Nicolle2015). In this way, it is interesting that it also took I-O psychologists 9 years to first explicitly cover the topic at the conference (e.g., Levine, Reference Levine2014). Members of our community had certainly been discussing social media influencers and big data indirectly leading up to those presentations. In particular, the I-O community clearly had an interest in incorporating big data in their own research projects and practices leading up to 2014, as made evident by the eight submissions in the year it finally entered the SIOP timeline. Even so, informal discussions about new technologies do little in comparison with explicitly tagged sessions when it comes to capturing the attention of professionals who can benefit most from our research. Without easily identifiable research, we make it challenging for people to make informed decisions regarding trending technology topics.

Encouragingly, I-O psychologists have produced timely research for some technologies over the past 2 decades. Our field’s involvement in developing the O*NET database, for example, helped ensure that job seekers and human resources professionals had reliable data to inform their work decisions when it was released to the general public in 1998 (e.g., Kubisiak etal., Reference Kubisiak, Hanson and Borman1998). Moreover, I-O psychologists studied virtual reality headsets and identified their uses and misuses in training long before they were widely commercialized (e.g., Fletcher etal., Reference Fletcher, Robinson, Nester and O’Leary2000). I-O psychologists also had wisdom to share about unproctored internet testing before the COVID-19 crisis when the vast majority of the world’s students shifted to online learning environments (e.g., Landers & Sackett, Reference Landers and Sackett2009; Scarborough etal., Reference Scarborough, Chambless and Weiner2006). For all of these technologies and several others, I-O psychologists studied a new tool before or around the same time as its invention or popularization, a favorable situation that allowed us to lead the way in advising organizations during adoption.

Limitations and future research

Several limitations of this archival analysis are worth noting. Although 23 years’ worth of research encompasses several major technological milestones, SIOP conferences date another 12 years before 1998. Our HoT timeline includes hundreds of data points related to technological milestones before 1998 (e.g., popularization of the computer and the internet; coining of digital practices like “affective computing” and “electronic performance monitoring”), but we were unable to estimate these lags without access to earlier programs. Additionally, we chose to overcome errors associated with automatic keyword searches by manually searching existing technology timelines and SIOP conference programs. To minimize the influence of human error and bias on our analyses, we involved multiple researchers at each stage while creating the HoT and SIOP timelines. Even so, the 2,365 identified SIOP presentations we identified might not represent a perfectly comprehensive set of relevant studies. Similarly, the 48 data points generated from existing technology timelines and empirical research may reflect the researchers’ biases. It remains possible that our analysis overlooked work-relevant technologies or included technologies with limited relevance. We encourage members of the I-O community to refine and expand upon this list with relevant technologies used in their own subdomains of study.

We generated a dictionary of keywords, and we recognize that some of these dictionaries may have been imperfect. However, we feel this conceptualization of “first instance” is appropriate given that papers that do not use the keywords in our dictionary would not be discoverable by other researchers and therefore would not be contributing to the knowledge base of this topic. Our online SIOP timeline provides a variety of data points, such as submission titles, abstracts, and format types, which allow for further investigations of technology research in the community. We encourage interested readers to browse this timeline to develop their own research questions related to I-O psychologists’ technology research.

1. Action

I-O psychologists should strive to study new technologies as they are created and to imagine possible workplace applications, not wait until they are widely adopted. In other words, we should choose action over cautiousness when it comes to studying new technologies. As we are all researchers with limited resources (e.g., time, funding), this recommendation may seem to come from a place of luxury in that this approach involves risk. There is indeed an inherent risk associated with studying a new tool that may later fade into obscurity, whereas studying a widespread tool ensures that our research is more relevant to a larger audience. However, we urge the field to push back against this way of thinking for two related reasons.

First, I-O psychologists can learn as much from new technologies that are rejected as we can from those that gain widespread adoption. We cannot predict with any certainty which new technologies will eventually gain mass adoption, but this should not stop us from studying them. Any technology that enters organizations—regardless of its long-term lifespan—will influence outcomes of interest to I-O psychologists. Virtual worlds were, by many accounts, a fad technology but one that still presented I-O psychologists with useful research insights (Pearlman & Gates, Reference Pearlman and Gates2010). Further opportunities existed to expand and identify boundaries of existing theories related to communication, teamwork, or workplace privacy. Some other technologies, like videoconferencing software, sustain moderate influence until some environmental shock stimulates sudden widespread adoption. In these scenarios, early research on a new technology enables us to be informed about its uses and misuses when high-demand or crisis situations arise. In either case (fad or mass adoption), we have much to gain from a learning perspective when studying new workplace technologies.

Additionally, I-O psychologists can use the introduction of a new technology as an opportunity to capture its unique effects as it enters the workplace. Such scenarios represent an ideal condition for conducting fundamental A-B intervention studies that would not be feasible for older, more established technologies. The effects of new technologies are undoubtedly dynamic; they will interact with novelty effects, social contexts and norms, and perhaps, the effects of previous work processes that the technology is supplementing or replacing. Waiting for the widespread adoption of a technology is failing to capture these initial interactive effects that are very likely to change over time.

2. Exploration

In order to produce timely technology research, I-O psychologists must be comfortable conducting exploratory, inductive, and abductive technology research (Woo etal., Reference Woo, O’Boyle and Spector2017). From hypothesis generation to data collection and analysis, the typical timeline of a hypothetico-deductive research study—wherein a proposed or existing theoretical model must precede observation—may span multiple years. In this case, I-O psychologists may very well study new technologies shortly after they are introduced or popularized, but they are unable to complete or disseminate their research in a timely manner. Our archival analysis offers some support for this explanation, with only 11 studies explicitly described as “inductive” or “exploratory” in the title or abstract (Bowen etal., Reference Bowen, Davidson and Stevenor2019; Brown etal., Reference Brown, Ironside, Linn and Johnson2001; Haidar etal., Reference Haidar, Yankov, Kumar and Zheng2020; Lim etal., Reference Lim, Uy and Lim2008; McManus & Ferguson, Reference McManus and Ferguson2002; Paik etal., Reference Paik, Li, Zhang, Zibell, Griffeth and Robinson2013; Rosopa etal., Reference Rosopa, Moore, Klinefelter, Xoxakos and Watson2020; Scarborough & Somers, Reference Scarborough and Somers2008; Vandenberg & Connelly, Reference Vandenberg and Connelly2014; Woo, Reference Woo2020; Wright & Gloss, Reference Wright and Gloss2015).

Given the pace with which technologies advance and the relative infancy of technology research, relevant psychological theory to succinctly explain the effects of many new technologies often does not yet exist. For instance, organizations are increasingly using high-fidelity virtual-reality and AR technologies to train (Gasparevic, Reference Gasparevic2018), assess (Nassauer & Cutter, Reference Nassauer and Cutter2019), and even socialize (Reuter, Reference Reuter2020) employees. Suitable theory to explain how varying levels of immersion and fidelity should influence variables of interest like learning and performance trajectory has yet to be formulated (Bowman & McMahan, Reference Bowman and McMahan2007; Cummings & Bailenson, Reference Cummings and Bailenson2016). Trying to root these kinds of research questions in tenuously connected existing psychological theory will not only frame research within misfit models, but it will also slow the formulation of new better fitting theory (Locke, Reference Locke2007). Furthermore, waiting to test questions for which we already have mature theory will leave critical research questions about technologies and work in the hands of those less equipped to answer them.

Importantly, conducting inductive technology research does not imply an absence of theory. As a constantly evolving object of study, technology lends itself to the creation of theoretical frameworks as opposed to theoretical models. Although frameworks generally describe or organize groups of objects, models mathematically assert the interrelationships of constructs. It is challenging to develop models of technology because doing so forces us to assume that technology is a construct with a unifying causal structure (Landers & Behrend, Reference Landers and Behrend2017). Such an assumption is tenuous given that so many families of technologies exist and technologies regularly change in ways that make their precise measurement difficult. Of greater interest to I-O psychologists is an understanding of the psychological effects of technologies, so technology is better conceptualized as a boundary condition or stimulus of psychological theories. Several technology-related frameworks already exist, each describing stimuli or boundary conditions of psychological processes in a flexible way that allows for continual updating that reflects changes in technology (e.g., Potosky’s, Reference Potosky2008, framework of assessment media or Ravid etal.’s, Reference Ravid, Tomczak, White and Behrend2020, framework of electronic performance monitoring). When we, as researchers, shift our focus from being correct (i.e., deductive research/model testing) to being inquisitive (i.e., inductive research/framework building), technological change becomes quite compatible with our duty to conduct relevant psychological research.

3. Collaboration

I-O psychologists can also produce timelier technology research by forming interdisciplinary partnerships. Researchers from the fields of I-O psychology and those from the fields of computer science, communication, political science, and economics are all jointly interested in the effects of technologies on individuals (e.g., Christensen & Garfias, Reference Christensen and Garfias2018; Dalal, & Randall, Reference Dalal and Randall2017; Ma & McGroarty, Reference Ma and McGroarty2017; Treem etal., Reference Treem, Dailey, Pierce and Leonardi2015), but they often run parallel lines of research without intertwining. Differences in the ways that various academic fields think about, speak about, and actually conduct scientific research introduce research opportunities rather than barriers. For instance, political science often emphasizes describing phenomena, whereas data science and psychology tend to emphasize predicting and explaining phenomena, respectively (König etal., Reference König, Demetriou, Glock, Hiemstra, Iliescu, Ionescu, Langer, Liem, Linnenbürger, Siegel and Vartholomaios2020). In accordance with these different emphases, individuals in each field tend to develop unique languages and research methods to answer their research questions. An I-O psychologist may focus on reliability and validity for theory building and construct validation, a data scientist may focus on web scraping and machine learning for compiling and organizing big data, and a political scientist may focus on public trends for constructing multilevel models. These different languages and research approaches are complementary; an I-O psychologist’s expertise in validation can make a data scientist’s web-scraping tool theoretically meaningful, a data scientist’s machine learning expertise can support a political scientist’s analysis of public records, and so on. Research on the potential for functional diversity (e.g., diversity of skills, experiences) to increase collaborative performance and creativity (Bantel, Reference Bantel1993; Bunderson, Reference Bunderson2003; Harrison & Klein, Reference Harrison and Klein2007) stems from our own field, with particularly prudent implications when it comes to technology research. In combination, researchers across various fields of study are able to answer research questions about new technologies much more quickly and more effectively than any one group could do alone.

I-O psychologists can take a number of steps to actualize these partnerships. Members of our community are actively engaged in conversations about how best to promote interdisciplinary collaboration for technology research (Gonzalez etal., Reference Gonzalez, Capman, Oswald, Theys and Tomczak2019; König etal., Reference König, Demetriou, Glock, Hiemstra, Iliescu, Ionescu, Langer, Liem, Linnenbürger, Siegel and Vartholomaios2020; Oswald etal., Reference Oswald, Behrend, Putka and Sinar2020). We view these conversations as comprising three major interrelated activities: learning, communicating, and planning.

Individuals more effectively work with one another when they are open to learning about each other’s perspectives. As I-O psychologists, we should first identify areas for which our understanding of technology is lacking. For instance, other scholars have more expertise regarding the technological characteristics of new tools (e.g., computer processing), whereas others know more about how these tools influence individuals at varying levels of analysis (e.g., inside the human brain, across school districts, or in national economies). To start learning about these differences, we urge readers to seek existing primers and cross-disciplinary proposals related to technology (see Murray & Antonakis, Reference Murray and Antonakis2019, for an introduction to organizational neuroscience or König etal., Reference König, Demetriou, Glock, Hiemstra, Iliescu, Ionescu, Langer, Liem, Linnenbürger, Siegel and Vartholomaios2020, for advice on working with computer scientists). Other learning opportunities exist regarding methodologies—some fields regularly use tools like functional magnetic resonance imaging (fMRI) for data collection or Python for data analysis. Fortunately, plenty of online tutorials are readily available to I-O psychologists seeking an introduction to new data collection and analysis methods (e.g., see cbmm.mit.edu and ocw.mit.edu for resources regarding the basics of fMRI and Python). We refer interested readers to Oswald etal. (Reference Oswald, Behrend, Putka and Sinar2020) for additional links to tutorials and other online content relevant to the study of technology in I-O psychology.

Learning about new content and methods makes us more effective cross-disciplinary communicators. We encourage I-O psychologists with a basic understanding of a new domain to seek low-stakes opportunities to actually engage with experts from other fields. Websites such as KDnuggets provide online forums and regular webinars that give users the opportunity to ask researchers from various domains questions about artificial intelligence, big data, and other data technologies. Such settings represent an excellent opportunity for I-O psychologists to identify and discuss any important technology problems shared across disciplines. At the same time, I-O psychologists should strive to create content that supports interdisciplinary communication. When sharing research with those from other disciplines, I-O psychologists should communicate their ideas clearly, simply, and concisely; a variety of visualization tools exist to assist I-O psychologists in achieving these communication goals (Oswald etal. Reference Oswald, Behrend, Putka and Sinar2020).

Planning can help I-O psychologists anticipate and overcome obstacles that could potentially hinder future collaborations. Learning and communicating across disciplines can be quite challenging in practice, where I-O psychologists are faced with a number of resource constraints. For instance, some I-O psychologists may struggle to find job opportunities that support true collaboration with technology experts. Recently, Gonzalez etal. (Reference Gonzalez, Capman, Oswald, Theys and Tomczak2019) encouraged I-O psychologists to seek employment in companies focusing on artificial intelligence and machine learning as a means of advancing human knowledge of these technologies. In other organizations, I-O psychologists may benefit from clearly articulating the benefits of technology collaboration to their colleagues. Importantly, some I-O psychologists, particularly graduate students, may simply lack an awareness or an appreciation for collaborative research opportunities regarding technology. The Guidelines for Training and Education in Industrial-Organizational Psychology (SIOP, 2017) encourage faculty members to incorporate major workplace technology trends in their coursework. We agree with this suggestion and expand on it to suggest that senior faculty should introduce their younger colleagues to non-I-O researchers in their network whenever appropriate. Doing so will ideally alleviate awareness constraints and help establish norms for cross-disciplinary collaboration among future generations of I-O psychologists.

4. Dissemination

Sharing our work in other venues and with other audiences further supports timelier technology research in the I-O psychology community. It is worth noting that an I-O researcher who completes a study in the winter would need to wait over a year to share these findings by adhering to SIOP’s “fall submission” and “spring presentation” schedule. Although SIOP’s peer-review process is quick in relation to that of most academic I-O journals, it remains possible that I-O psychologists lack an awareness of or an interest in other, timelier means of dissemination. Although our study did not explicitly analyze I-O technology research outside of SIOP, we still urge I-O researchers to look beyond our own community and seek other audiences who can benefit from our work.

Many opportunities for research dissemination exist in the form of conferences both within and outside of the I-O psychology community. From within, SIOP now hosts the Leading Edge Consortium to advance our knowledge of special topics that often relate to technology. Balancing out the typical spring schedule of the general SIOP conference, the Leading Edge Consortium generally occurs in the fall. The consortium brings together scientists and practitioners who share an interest in a hot topic, such as technology’s uses during assessment, talent analytics, and coaching, among others. We encourage interested I-O psychologists to attend events hosted by I-O psychologists and related fields (e.g., Society of Human Resource Management, Academy of Management, Society for Personality and Social Psychology) but recognize that we also need to share our voice in other fields, particularly those that may overlook the psychological effects of technology. In most areas of technology, influential conferences occur on a more regular basis with published proceedings to encourage timely research dissemination. The Association for Computing Machinery, for example, hosts over 170 conferences, workshops, and symposia each year, and has published over 230,000 refereed papers. Other relevant avenues for technology research dissemination include the Institute of Electrical and Electronics Engineers, in addition to more cross-disciplinary avenues such as the APA’s annual Technology, Mind, and Society conference. Together, these events give I-O psychologists an opportunity to disseminate their technology research to multiple audiences and more regularly than once per year.

Beyond conferences, we urge I-O psychologists to make use of recent pushes for increased transparency in the publication process to disseminate technology research in a timely fashion. Outlets such as the Journal of Business and Psychology now offer results-blind submission processes (Kreamer & Rogelberg, Reference Kreamer and Rogelberg2020), which supports technology research in that the review process is initiated before data collection is completed. Other peer-reviewed outlets such as PLOS ONE are open access and have a significantly shorter acceptance-to-publication time than traditional journals (Björk & Solomon, Reference Björk and Solomon2013). In addition, other avenues such the Center for Open Science’s PsyArXiv allows researchers to upload manuscripts prior to the peer review process. Collectively, these open science endeavors offer many opportunities beyond conferences for I-O psychologists to share technology research more quickly.

In addition to sharing our work with other researchers, we must also disseminate our findings to practitioners who interact with new technologies on a regular basis. In a recent discussion on talent identification, Chamorro-Premuzic etal. (Reference Chamorro-Premuzic, Winsborough, Sherman and Hogan2016) showed that human resource practitioners adopt technologies more quickly than academic I-O psychologists study them. Workers who interact with new technologies often experience unique challenges as a result of this lag. Consider trends toward artificial intelligence and big data, wherein practitioners can amass data on a wide variety of variables to predict desirable outcomes such as job performance. Absent from many of these predictions is a sound theory to explain why or how random variables predict an outcome. Thus, variables measured using new technologies often lack construct validity evidence (Hickman etal., Reference Hickman, Tay and Woo2019) and therefore produce unanticipated outcomes such as adverse impact (Gonzalez etal., Reference Gonzalez, Capman, Oswald, Theys and Tomczak2019). To prevent such outcomes, we urge I-O psychologists to collaborate with their practicing colleagues when possible, to engage in scientist–practitioner sessions at SIOP and related conferences, and to practice effective communication with business leaders (see the communication suggestions above). In doing so, we are much better suited to identify and study the new technologies organizations are actually adopting.

5. Creation

Our final suggestion for I-O psychologists to produce timelier technology research is to actually create new technologies. As experts in human thought and behavior, I-O psychologists should play an active, rather than reactive, role in developing the tools that individuals will interact with on a regular basis. In our review, we identified several examples of I-O psychologists who used our science to produce a useful piece of technology for workers. Perhaps the most widely acknowledged examples are the many Rpackages and related scripts that I-O psychologists have created to aid professionals and researchers with data analysis (e.g., Dahlke & Wiernik, Reference Dahlke and Wiernik2019; Goebl etal., Reference Goebl, Jones and Beatty2016). We also identified several examples of innovative technologies created to advance our knowledge of measurement and surveying. For example, I-O psychologists working within technology firms have used their voice to develop gamified situational judgment tests (e.g., AON’s chatAssess). Academic I-O psychologists are also involved in the creation of new technologies. One example is Tay (Reference Tay2020), who used research on survey design and implementation to create the experience sampling tool ExpiWell. Similarly, Waber (Reference Waber2011, Reference Waber2020) used existing work on technology, social networks, and organizational design to help develop sociometric badges that measure conversational dynamics in social groups. We strongly urge other I-O psychologists to think similarly about the implications of their work; technology should be not merely a future research direction in our papers but also an opportunity to use what we know to create tools that can directly benefit workers.