Information communications technology (ICT) has revolutionized workplaces, bringing novel challenges and opportunities to employees across occupational contexts. Perhaps that transformational effect can account for scholarly interest from a wide range of fields, including communications, information systems, psychology, sociology, management, and medicine, among others. Given the breadth of interest and volume of research, it is not hard to imagine disciplines being edged out of the conversation. Thus, Hu et al. (Reference Hu, Barber, Park and Day2021) raise an important point: For industrial-organizational (I-O) psychologists to contribute maximally to this field and create effective applied solutions, they must proceed with an interdisciplinary and critical approach to ICT research. We firmly agree that, for I-O psychology to maintain its voice in the ICT conversation, we need to “defrag” and “reboot” the domain. However, we propose a different approach for creating the coherence that Hu et al. hope to achieve in this young, multifaceted literature. In this comment, we first discuss limitations of Hu et al.’s approach and then propose an alternative approach to guiding future research and practice in ICT.
Evaluating the uniqueness of “ICT research”
One of Hu et al.’s (Reference Hu, Barber, Park and Day2021) intentions was to “determine whether a compelling case can be made for the uniqueness of ICT-related concepts in studying employee[s] and performance in I-O psychology” (p. 371). However, an overemphasis on evaluating the uniqueness of ICT research is too limiting for the current state of the literature. This approach fails to account for two of the most salient characteristics of ICT research: its multifaceted nature and its youth. First, “ICT research” is unique in the breadth of its scope, so it cannot be treated as a coherent behavioral domain, as is job performance. Consider several ICT research areas that are relevant to I-O psychologists: computer security negligence, cyberloafing, hacking, cyberbullying, social media engagement, remote onboarding, and virtual interviewing, among many others. A quick glance at these disparate content areas should dispel any notion that they can be evaluated as one cogent domain.
The second reason that it is impossible to determine whether ICT-related concepts are unique yet because of the nascent state of research into many of these constructs. At best, we may have gathered sufficient evidence to suggest novelty of only the most studied ICT behaviors. That limited research suggests that some are distinct in nature, whereas others may be novel manifestations of established behavioral domains. For example, online social media use, despite its relevance to in-person social behavior, demonstrates unique relationships with mental health symptoms (e.g., Hardy & Castonguay, Reference Hardy and Castonguay2018); similarly, cyberloafing, which is akin to production deviance, exhibits a pattern of relationships with common correlates that are distinct from traditional forms of deviance (Mercado et al., Reference Mercado, Giordano and Dilchert2017). Even more challenging, some ICT behaviors do not have clear analogs (e.g., information-security deviant behavior; Chu & Chau, Reference Chu and Chau2014).
A third and related reason for the difficulty in assessing the uniqueness of ICT is the open-ended nature of what it means for something to be “unique.” I-O psychology has a long and fruitful past of studying highly correlated phenomena as unique. Consider any hierarchically structured domain, such as cognitive ability. The correlations between specific cognitive abilities can be larger than the reliability indices of some psychological constructs. Nevertheless, an entire industry exists to create assessments for measuring various forms of cognitive abilities. We should not require new investigations to first argue novelty but rather to first investigate and subsequently demonstrate either novelty or redundance. Importantly, investigating the similarities and differences between ICT and non-ICT analogous concepts (if they exist) is a crucial step to advance science, but it should not be the first step. If we attempt to determine whether these constructs are indeed unique prior to their rigorous investigation, we may hinder progress unnecessarily. Look no further than what happened in the personality domain in the 1960s (Guion & Gottier, Reference Guion and Gottier1965) when research was halted for decades before consensus was reached on a coherent framework for defining and measuring personality (e.g., Barrick & Mount, Reference Barrick and Mount1991).
A behaviors–experiences framework is not prescriptive
Although the technology behaviors and experiences framework applied by Hu et al. (Reference Hu, Barber, Park and Day2021) may be useful for a backward glance at what has been published, it does not provide a prescriptive look to the future to fruitfully guide how I-O psychologists can optimally contribute to these research domains. It categorizes, rather than drives, research questions. More importantly, a great strength of I-O psychology is its recognition that behaviors and experiences are typically best understood as interdependent, reciprocal elements. These elements are connected through often complex, structural relations [nomological network], which a prescriptive framework would highlight rather than ignore. Separating behaviors and experiences not only suggests an artificial independence but also reinforces the same disciplinary divides against which Hu et al. have cautioned us. Instead, we advocate for applying a prescriptive framework to serve as a baseline structure to facilitate the integration of interdisciplinary investigations into the broader ICT picture. Organizing a domain into its proximal and distal determinants and outcomes has proven instrumental in understanding other research areas (e.g., personality neuroscience, DeYoung, Reference DeYoung2010; genetics and counterproductivity, Krueger et al., Reference Krueger, Hicks, Patrick, Carlson, Iacono and McGue2002; Stanek et al., Reference Stanek, Ones and McGue2017), and we expect a similar utility in the ICT domain as well.
An alternative approach to ICT research
Instead of the approach proposed by Hu et al. (Reference Hu, Barber, Park and Day2021), we propose the following framework inspired by Campbell’s causal model of performance (e.g., McCloy et al., Reference McCloy, Campbell and Cudeck1994). Decades of research investigating employee performance has demonstrated the utility of understanding indirect/distal antecedents, direct/proximal antecedents, the behavior itself, and its outcomes. Applying this simple approach to the study of behavioral domains that are facilitated by ICTs drives meaningful research questions and helps overcome many of the literature’s limitations that were highlighted by Hu et al.
First, what is the behavior?
The foundation of Campbell’s model of performance (McCloy et al., Reference McCloy, Campbell and Cudeck1994) is understanding the behavior (or behavioral domain) itself. Importantly, the ICT domain is not itself a single behavioral domain but instead comprises varied technologically mediated behaviors (e.g., cyberloafing, virtual communications). Developing and validating quality measures of these behavioral domains requires precise conceptualizations and definitions that are informed by multi- and interdisciplinary reviews and inquiries. Such conceptualizations will also highlight distinctions between the wide range of constructs subsumed by ICT research and demonstrate why they cannot be evaluated as one domain.
Understanding and measuring behaviors of interest will require a classic, multifaceted approach to validation. The field is quick to demand evidence of discriminant validity for ICT constructs, claiming—perhaps hoping—that these behaviors are merely novel manifestations of what is known. However, convergence between ICT constructs and more traditional analogs can also catalyze research. Consider, for example, the strong correlation between overall counterproductive work behavior (CWB) and cyberloafing (ρ = .38; Mercado et al., Reference Mercado, Giordano and Dilchert2017). Although they are distinct, cyberloafing is much more accessible to measure, demonstrating both higher base rates in self-report data and the potential for objective data collection, thereby yielding new avenues to investigate more established constructs. Moreover, given their similarities, the well-established nomological network of CWB (Berry et al., Reference Berry, Ones and Sackett2007) provides a useful starting place for investigations into the nomological network of cyberloafing. However, especially as we seek to provide actionable results for practitioners, we must consider not only whether construct scales reflect a latent general factor but also the extent to which they are deficient or contaminated assessments themselves. For example, consider email performance, a component of job performance that is almost never mentioned or studied. Yet, for most office workers, email comprises a substantial proportion of one’s job; according to McKinsey Global Institute (2012), workers who collaborate and coordinate with others spend an average of 28% of their work time managing email. Appraising performance without considering email (i.e., using a deficient measure) would preclude helpful feedback to an otherwise excellent employee who demonstrates severe email delinquency. To provide useful and actionable interventions, it is imperative to understand the entirety of behavioral domains.
As we begin to deeply understand these behaviors, we will be able to integrate appropriate bodies of literature in a way that is infeasible without separating ICT research into coherent behavioral domains. For example, email incivility—a form of interpersonal deviance—has been notably informed by politeness, group dynamics, and CWB (e.g., Lim & Teo, Reference Lim and Teo2009). In contrast, hacking investigations require scholarly foundations in computer skills and technology acceptance and would not be best served by a communications-based theory.
Next, what are the outcomes?
With solid conceptualizations and measures of ICT behaviors, we can proceed to examine potential outcomes. Often, critics ask about the novelty of ICT behaviors when they really want to understand their importance. Because it is one of the most studied ICT behaviors, we can yet again consider cyberloafing as an example. For years, scholars cynically focused on whether cyberloafing was unique, with statements such as “this is surely a difference of degree, not kind [of problem]” (Block, Reference Block2001, p. 226). Unfortunately, this focus on novelty ignored the more essential questions of whether and how cyberloafing influenced productivity, profits, and employee well-being, which are only beginning to be investigated nearly two decades later. Hu et al.’s (Reference Hu, Barber, Park and Day2021) concern that ICT research has not yet aided policy and intervention is well placed, but this is not merely a symptom of the scientist–practitioner gap. Instead, scholars have lagged practice; we cannot advise policy and intervention development until we have evidence on which to base those practices. Rather than calling researchers to assert the uniqueness of ICT domains, we propose that policy and intervention development would be facilitated by an emphasis on research into the consequences of ICT-related behaviors as well as understanding their causal antecedents.
Ultimately, what are the antecedents of these behaviors?
Assuming that we understand the ICT behavior itself and have discovered that it does indeed matter in a workplace, investigating its antecedents, both direct and indirect, is merited. This is the step that will drive effective policy and intervention formation because antecedents dictate which personal and situational variables influence behaviors and experiences. They guide organizational investment (e.g., training interventions, work redesign). Beyond yielding actionable strategies for organizations, investigations into both proximal and distal antecedents will also, in turn, inform new outcome investigations. Consider Andel et al.’s (Reference Andel, Kessler, Pindek, Kleinman and Spector2019) recent findings on cyberloafing as a coping response to workplace aggression. Informed by this work, subsequent investigations have explored cyberloafing as a potential resource for employees managing stress. Understanding why a behavior manifests led to novel insights and follow-up studies into the effects of those behaviors.
As a literature pursues antecedents, it will inevitably give way to myriad studies each investigating a small section of the nomological network. For some ICT research (e.g., cyberbullying, cyberloafing), this proliferation has already begun; for others, it is coming. If we organize the constellation of antecedents into a structural framework, such as Campbell’s model, we can avoid fragmentation and catalyze, rather than impede, scientific development.
Conclusions
The call of Hu et al. (Reference Hu, Barber, Park and Day2021) to “defrag” and “reboot” ICT research is one of prime importance. Although ICT research has flourished in recent years, a solid understanding of a behavioral domain is necessary before deeming it unique or redundant. Therefore, rather than simply categorizing the limited research available, we recommend taking a cue from Campbell’s model (McCloy et al., Reference McCloy, Campbell and Cudeck1994) of performance as an organizing and prescriptive framework to facilitate developing these early-stage literatures. Specifically, we must first rigorously conceptualize ICT behaviors and develop valid measures for their assessment. Then, we can answer the question of their importance by studying their relationships with vital workplace outcomes. Ultimately, investigations into their indirect and direct antecedents will yield actionable implications to address organizational concerns. By ensuring that we understand what we are investigating, we, as I-O psychologists, can draw on our strengths and most richly contribute to this interdisciplinary realm.
You have
Access
Information communications technology (ICT) has revolutionized workplaces, bringing novel challenges and opportunities to employees across occupational contexts. Perhaps that transformational effect can account for scholarly interest from a wide range of fields, including communications, information systems, psychology, sociology, management, and medicine, among others. Given the breadth of interest and volume of research, it is not hard to imagine disciplines being edged out of the conversation. Thus, Hu et al. (Reference Hu, Barber, Park and Day2021) raise an important point: For industrial-organizational (I-O) psychologists to contribute maximally to this field and create effective applied solutions, they must proceed with an interdisciplinary and critical approach to ICT research. We firmly agree that, for I-O psychology to maintain its voice in the ICT conversation, we need to “defrag” and “reboot” the domain. However, we propose a different approach for creating the coherence that Hu et al. hope to achieve in this young, multifaceted literature. In this comment, we first discuss limitations of Hu et al.’s approach and then propose an alternative approach to guiding future research and practice in ICT.
Evaluating the uniqueness of “ICT research”
One of Hu et al.’s (Reference Hu, Barber, Park and Day2021) intentions was to “determine whether a compelling case can be made for the uniqueness of ICT-related concepts in studying employee[s] and performance in I-O psychology” (p. 371). However, an overemphasis on evaluating the uniqueness of ICT research is too limiting for the current state of the literature. This approach fails to account for two of the most salient characteristics of ICT research: its multifaceted nature and its youth. First, “ICT research” is unique in the breadth of its scope, so it cannot be treated as a coherent behavioral domain, as is job performance. Consider several ICT research areas that are relevant to I-O psychologists: computer security negligence, cyberloafing, hacking, cyberbullying, social media engagement, remote onboarding, and virtual interviewing, among many others. A quick glance at these disparate content areas should dispel any notion that they can be evaluated as one cogent domain.
The second reason that it is impossible to determine whether ICT-related concepts are unique yet because of the nascent state of research into many of these constructs. At best, we may have gathered sufficient evidence to suggest novelty of only the most studied ICT behaviors. That limited research suggests that some are distinct in nature, whereas others may be novel manifestations of established behavioral domains. For example, online social media use, despite its relevance to in-person social behavior, demonstrates unique relationships with mental health symptoms (e.g., Hardy & Castonguay, Reference Hardy and Castonguay2018); similarly, cyberloafing, which is akin to production deviance, exhibits a pattern of relationships with common correlates that are distinct from traditional forms of deviance (Mercado et al., Reference Mercado, Giordano and Dilchert2017). Even more challenging, some ICT behaviors do not have clear analogs (e.g., information-security deviant behavior; Chu & Chau, Reference Chu and Chau2014).
A third and related reason for the difficulty in assessing the uniqueness of ICT is the open-ended nature of what it means for something to be “unique.” I-O psychology has a long and fruitful past of studying highly correlated phenomena as unique. Consider any hierarchically structured domain, such as cognitive ability. The correlations between specific cognitive abilities can be larger than the reliability indices of some psychological constructs. Nevertheless, an entire industry exists to create assessments for measuring various forms of cognitive abilities. We should not require new investigations to first argue novelty but rather to first investigate and subsequently demonstrate either novelty or redundance. Importantly, investigating the similarities and differences between ICT and non-ICT analogous concepts (if they exist) is a crucial step to advance science, but it should not be the first step. If we attempt to determine whether these constructs are indeed unique prior to their rigorous investigation, we may hinder progress unnecessarily. Look no further than what happened in the personality domain in the 1960s (Guion & Gottier, Reference Guion and Gottier1965) when research was halted for decades before consensus was reached on a coherent framework for defining and measuring personality (e.g., Barrick & Mount, Reference Barrick and Mount1991).
A behaviors–experiences framework is not prescriptive
Although the technology behaviors and experiences framework applied by Hu et al. (Reference Hu, Barber, Park and Day2021) may be useful for a backward glance at what has been published, it does not provide a prescriptive look to the future to fruitfully guide how I-O psychologists can optimally contribute to these research domains. It categorizes, rather than drives, research questions. More importantly, a great strength of I-O psychology is its recognition that behaviors and experiences are typically best understood as interdependent, reciprocal elements. These elements are connected through often complex, structural relations [nomological network], which a prescriptive framework would highlight rather than ignore. Separating behaviors and experiences not only suggests an artificial independence but also reinforces the same disciplinary divides against which Hu et al. have cautioned us. Instead, we advocate for applying a prescriptive framework to serve as a baseline structure to facilitate the integration of interdisciplinary investigations into the broader ICT picture. Organizing a domain into its proximal and distal determinants and outcomes has proven instrumental in understanding other research areas (e.g., personality neuroscience, DeYoung, Reference DeYoung2010; genetics and counterproductivity, Krueger et al., Reference Krueger, Hicks, Patrick, Carlson, Iacono and McGue2002; Stanek et al., Reference Stanek, Ones and McGue2017), and we expect a similar utility in the ICT domain as well.
An alternative approach to ICT research
Instead of the approach proposed by Hu et al. (Reference Hu, Barber, Park and Day2021), we propose the following framework inspired by Campbell’s causal model of performance (e.g., McCloy et al., Reference McCloy, Campbell and Cudeck1994). Decades of research investigating employee performance has demonstrated the utility of understanding indirect/distal antecedents, direct/proximal antecedents, the behavior itself, and its outcomes. Applying this simple approach to the study of behavioral domains that are facilitated by ICTs drives meaningful research questions and helps overcome many of the literature’s limitations that were highlighted by Hu et al.
First, what is the behavior?
The foundation of Campbell’s model of performance (McCloy et al., Reference McCloy, Campbell and Cudeck1994) is understanding the behavior (or behavioral domain) itself. Importantly, the ICT domain is not itself a single behavioral domain but instead comprises varied technologically mediated behaviors (e.g., cyberloafing, virtual communications). Developing and validating quality measures of these behavioral domains requires precise conceptualizations and definitions that are informed by multi- and interdisciplinary reviews and inquiries. Such conceptualizations will also highlight distinctions between the wide range of constructs subsumed by ICT research and demonstrate why they cannot be evaluated as one domain.
Understanding and measuring behaviors of interest will require a classic, multifaceted approach to validation. The field is quick to demand evidence of discriminant validity for ICT constructs, claiming—perhaps hoping—that these behaviors are merely novel manifestations of what is known. However, convergence between ICT constructs and more traditional analogs can also catalyze research. Consider, for example, the strong correlation between overall counterproductive work behavior (CWB) and cyberloafing (ρ = .38; Mercado et al., Reference Mercado, Giordano and Dilchert2017). Although they are distinct, cyberloafing is much more accessible to measure, demonstrating both higher base rates in self-report data and the potential for objective data collection, thereby yielding new avenues to investigate more established constructs. Moreover, given their similarities, the well-established nomological network of CWB (Berry et al., Reference Berry, Ones and Sackett2007) provides a useful starting place for investigations into the nomological network of cyberloafing. However, especially as we seek to provide actionable results for practitioners, we must consider not only whether construct scales reflect a latent general factor but also the extent to which they are deficient or contaminated assessments themselves. For example, consider email performance, a component of job performance that is almost never mentioned or studied. Yet, for most office workers, email comprises a substantial proportion of one’s job; according to McKinsey Global Institute (2012), workers who collaborate and coordinate with others spend an average of 28% of their work time managing email. Appraising performance without considering email (i.e., using a deficient measure) would preclude helpful feedback to an otherwise excellent employee who demonstrates severe email delinquency. To provide useful and actionable interventions, it is imperative to understand the entirety of behavioral domains.
As we begin to deeply understand these behaviors, we will be able to integrate appropriate bodies of literature in a way that is infeasible without separating ICT research into coherent behavioral domains. For example, email incivility—a form of interpersonal deviance—has been notably informed by politeness, group dynamics, and CWB (e.g., Lim & Teo, Reference Lim and Teo2009). In contrast, hacking investigations require scholarly foundations in computer skills and technology acceptance and would not be best served by a communications-based theory.
Next, what are the outcomes?
With solid conceptualizations and measures of ICT behaviors, we can proceed to examine potential outcomes. Often, critics ask about the novelty of ICT behaviors when they really want to understand their importance. Because it is one of the most studied ICT behaviors, we can yet again consider cyberloafing as an example. For years, scholars cynically focused on whether cyberloafing was unique, with statements such as “this is surely a difference of degree, not kind [of problem]” (Block, Reference Block2001, p. 226). Unfortunately, this focus on novelty ignored the more essential questions of whether and how cyberloafing influenced productivity, profits, and employee well-being, which are only beginning to be investigated nearly two decades later. Hu et al.’s (Reference Hu, Barber, Park and Day2021) concern that ICT research has not yet aided policy and intervention is well placed, but this is not merely a symptom of the scientist–practitioner gap. Instead, scholars have lagged practice; we cannot advise policy and intervention development until we have evidence on which to base those practices. Rather than calling researchers to assert the uniqueness of ICT domains, we propose that policy and intervention development would be facilitated by an emphasis on research into the consequences of ICT-related behaviors as well as understanding their causal antecedents.
Ultimately, what are the antecedents of these behaviors?
Assuming that we understand the ICT behavior itself and have discovered that it does indeed matter in a workplace, investigating its antecedents, both direct and indirect, is merited. This is the step that will drive effective policy and intervention formation because antecedents dictate which personal and situational variables influence behaviors and experiences. They guide organizational investment (e.g., training interventions, work redesign). Beyond yielding actionable strategies for organizations, investigations into both proximal and distal antecedents will also, in turn, inform new outcome investigations. Consider Andel et al.’s (Reference Andel, Kessler, Pindek, Kleinman and Spector2019) recent findings on cyberloafing as a coping response to workplace aggression. Informed by this work, subsequent investigations have explored cyberloafing as a potential resource for employees managing stress. Understanding why a behavior manifests led to novel insights and follow-up studies into the effects of those behaviors.
As a literature pursues antecedents, it will inevitably give way to myriad studies each investigating a small section of the nomological network. For some ICT research (e.g., cyberbullying, cyberloafing), this proliferation has already begun; for others, it is coming. If we organize the constellation of antecedents into a structural framework, such as Campbell’s model, we can avoid fragmentation and catalyze, rather than impede, scientific development.
Conclusions
The call of Hu et al. (Reference Hu, Barber, Park and Day2021) to “defrag” and “reboot” ICT research is one of prime importance. Although ICT research has flourished in recent years, a solid understanding of a behavioral domain is necessary before deeming it unique or redundant. Therefore, rather than simply categorizing the limited research available, we recommend taking a cue from Campbell’s model (McCloy et al., Reference McCloy, Campbell and Cudeck1994) of performance as an organizing and prescriptive framework to facilitate developing these early-stage literatures. Specifically, we must first rigorously conceptualize ICT behaviors and develop valid measures for their assessment. Then, we can answer the question of their importance by studying their relationships with vital workplace outcomes. Ultimately, investigations into their indirect and direct antecedents will yield actionable implications to address organizational concerns. By ensuring that we understand what we are investigating, we, as I-O psychologists, can draw on our strengths and most richly contribute to this interdisciplinary realm.