In the past few years, the term “21st century skills” has gained increasing popularity in educational research and business practice. Neubert, Mainert, Kretzschmar, and Greiff (Reference Neubert, Mainert, Kretzschmar and Greiff2015) advocated for the utility of assessing 21st century skills in industrial–organizational psychology (I-O psychology) and its superiority over assessing basic psychological constructs, using application-oriented constructs, or conducting job analysis to determine relevant skills for individual work settings. We argue, however, that the issues identified and discussed in the focal article are rather threefold and that, to integrate 21st century skills into I-O psychology and to bridge organizational science and practice, we need to have (a) a standard framework of clearly defined constructs, (b) innovative assessments, and (c) evidence for validity generalization. We elaborate below.

First, we note that 21st century skills are a particularly ill-defined group of constructs that are used broadly to refer to important predictors of readiness and success in the 21st century workplace across multiple domains of individual differences and beyond the knowledge acquired in formal education. This set of constructs needs to be clarified, and a standard framework needs to be developed to organize these constructs. We see this as the most urgent task in the application of 21st century skills. A number of attempts have been made so far to identify and organize the core dimensions of 21st century workforce competencies, either by reviewing the literature (e.g., National Research Council [NRC], 2012) or by analyzing existing data on job requirements and worker characteristics for a wide range of occupations (e.g., Burrus, Jackson, Xi, & Steinberg, Reference Burrus, Jackson, Xi and Steinberg2013; Golubovich, Su, & Robbins, in press). Nonetheless, even the primary resources cited in the focal article to support the use of 21st century skills diverged in how they defined the constructs. For example, Griffin, McGaw, and Care (Reference Griffin, McGaw and Care2012) emphasized the skills that students need for “employment in a digital age,” including complex problem solving (CPS), collaborative problem solving (ColPS), and learning through digital networks. Neubert and colleagues adopted this perspective and further implied that CPS and ColPS build on and yet transcend traditional domains of individual differences. In contrast, the National Research Council (NRC; 2012) defined 21st century skills much more broadly and more closely aligned with traditional constructs commonly assessed in I-O psychology, categorizing 21st century skills into three clusters of competencies in the cognitive (e.g., cognitive processes and strategies), intrapersonal (e.g., intellectual openness, conscientiousness, and core self-evaluation), and interpersonal (e.g., teamwork, leadership) domains. Indeed, the NRC (2012) report took the opposite position from the focal article by stating that “the committee views the various sets of terms associated with the ‘21st century skills’ label as reflecting important dimensions of human competence that have been valuable for many centuries, rather than skills that are suddenly new, unique and valuable today [emphasis added]” (p. Sum-2).

The key issue in question here, therefore, is whether 21st century skills are formative constructs composed of fundamental individual difference variables, or are they something entirely different from these components and—as the focal article argued—difficult to be described by job analysis (see Edwards & Bagozzi, Reference Edwards and Bagozzi2000, for a discussion on formative measurement models). Our perspective is more in line with NRC (2012). We believe that 21st century skills can be captured by important knowledge, skills, abilities, and other characteristics (KSAOs). For example, ColPS is not a new skill that replaces traditional skills; instead, it reflects the behavioral competencies that manifest as a collection of reasoning ability, critical thinking and communication skills, and personality traits linked to effective teamwork and interaction with people. For psychologists who study the psychological mechanisms of organizational behaviors, it is essential to not only identify the proximal behavioral competencies that predict success in the workplace by observing “larger trends” in the shift of work tasks but also understand the ingredients and psychological basis underlying these relationships. The first and foremost challenge for I-O psychologists who are trying to address this gap between science and practice, therefore, is to clearly define and organize the behavioral competencies required by nonroutine, interactive work tasks, and to link them with foundational dimensions of psychological variables.

Meanwhile, we acknowledge the utility of the 21st century skills label in that it calls for integration of constructs and innovative assessments, which brings up our second point: We argue that the novelty in assessments rather than the novelty in constructs would be the major contribution that the conversation about 21st century skills brings to I-O psychology. In the past decade, psychological testing in personnel selection has indeed been shifting from the assessment of separate, distal individual traits to the use of multiconstruct assessment methods in high-fidelity activities or situations, such as assessment centers and situational judgment tests (Ryan & Ployhart, Reference Ryan and Ployhart2014). The innovation in assessment methods, including the use of computerized tests and simulations, has made it possible to assess behavioral competencies required in work tasks more effectively and efficiently than could be done using traditional tests. These computerized multiconstruct assessment methods are suitable for assessing constructs falling under the label of 21st century skills and can be further advanced as a result of the demands for assessing these constructs. On the other hand, the innovations in test format are not exclusive to the assessment of 21st century skills and have equally benefited the assessment of fundamental psychological constructs such as cognitive ability and personality traits. Further, many fundamental psychological constructs that are crucial for the understanding of person–environment fit and that have demonstrated predictive validity for job performance over and above cognitive ability and personality traits, such as interests (Nye, Su, Rounds, & Drasgow, Reference Nye, Su, Rounds and Drasgow2012; Van Iddekinge, Putka, & Campbell, Reference Van Iddekinge, Putka and Campbell2011), have not been integrated into the existing frameworks of 21st century skills, routine procedures of personnel selection, training and development, or organizational leadership. Therefore, although we acknowledge the complementary value of directly assessing nonroutine, interactive behaviors in simulations of work situations that require problem solving, we argue that the field of I-O psychology will need to continue relying on increasingly innovative assessments of fundamental psychological constructs, particularly for the purpose of enhancing person–environment fit.

Finally, Neubert and colleagues suggested that 21st century skills such as CPS and ColPS are transversal and are capable of predicting individual success in the increasingly mobile (vertically) and diverse (laterally) modern careers. This contention implies that the predictive validity of such constructs generalizes across jobs with varying levels of job complexity, in different job families or occupational fields, and—in the context of a globalized economy—across cultures, negating the necessity to perform job analyses within specific jobs. The focal article also implied that these transversal skills are likely predictive of a broad range of workplace outcomes, including job performance, career advancement, and leadership effectiveness. We argue that this proposition of validity generalization is subject to empirical tests (cf. Murphy, Reference Murphy2003). Even in the era of boundaryless careers, it is key to define the boundary for the use of an assessment or selection method in terms of when and in what situations does which construct of 21st century skills work. This process of validity generalization, we argue, would not be any different for introducing 21st century skill assessments into I-O psychology than for evaluating the relationship between fundamental psychological constructs, such personality traits, and job performance (e.g., Barrick & Mount, Reference Barrick and Mount1991). Observing trends in today's world of work provides a starting point to propose the hypothesis that skills such as CPS and ColPS may be predictive of a broad range of workplace outcomes across many jobs. Testing this hypothesis will largely depend on the availability of validity studies and meta-analytic syntheses of empirical data. In the case of many constructs and related assessments falling under the 21st century skills label, the research on their importance for various work outcomes is still limited, and a meta-analysis on their predictive validity is lacking (NRC, 2012). Therefore, it may be premature to draw inferences about the transversal predictive power of 21st century skills at this time.

Before validity generalization analysis for 21st century skills becomes available, job analysis will continue to serve valuable functions in human resource management, including determining minimal job qualifications, informing relevant applicant assessments, and designing training on critical work activities. The limitations of job analysis highlighted in the focal article can start to be addressed by designing job analytic instruments to specifically elicit information about the relevance of those complex and interactive skills of interest and to ask about anticipated changes in job requirements (future-oriented job analysis; Landis, Fogli, & Goldberg, Reference Landis, Fogli and Goldberg1998). Over time, further improvements in job profiling surveys can be driven by recent innovations in assessment validation. Specifically, synthetic validity approaches (see Scherbaum, Reference Scherbaum2005, for an overview) take detailed job analytic data and validity data for various types of assessments and try to anticipate a particular assessment's validity based on certain job components. Assuming that enough data are gathered on the validities of complex and interactive skill assessments for various types of jobs (that vary in their job components), the job components that require these skills can be elucidated. This information, in turn, can be used to further refine job profiling surveys.

Given the increasing attention to 21st century skills in education and business and the aforementioned challenges with regard to the construct, the assessment, and the validation of 21st century skills, what roles can I-O psychologists take in future research on this topic? We propose the following:

1. 1. The field of I-O psychology can play a major role in defining the constructs falling under the label of 21st century skills, establishing the boundaries and a standard framework of such constructs, and unifying the dialogue among researchers and practitioners. It is important that the standard framework should include a taxonomy of core dimensions of competencies that are hypothesized to be important for workplace outcomes, including job performance, career advancement, and leadership effectiveness. This framework should also outline the empirical relationships between these core dimensions of competencies and fundamental psychological constructs, such as cognitive ability, personality traits, and interests.

2. 2. The field of I-O psychology can help shape future assessments of complex and interactive skills required in modern careers and ensure that the assessments fully reflect the above taxonomy of competencies.

3. 3. The field of I-O psychology will be instrumental in collecting evidence on the predictive validity of 21st century skills for various work outcomes and synthesizing these findings using validity generalization methods.

4. 4. One of the most critical arguments for the need for 21st century skills is the gap between the skills that students acquire through formal education and those that employers in an ever-changing world of work expect the workforce to possess. With a standard, evidence-based framework of 21st century skills, the field of I-O psychology can apply it to educational and psychological testing, selection and staffing, and training and development in organizations to mitigate the skill gap and create a smooth talent supply chain.