Melson-Silimon, Harris, Shoenfelt, Miller, and Carter (Reference Melson-Silimon, Harris, Shoenfelt, Miller and Carter2019) raise an important issue that industrial and organizational (I-O) psychologists should take seriously: As we integrate normal and abnormal personality models, we may find that personality assessments adversely impact the mentally disabled. Though not mentioned by Melson-Silimon and colleagues, this is a criticism of the personality testing enterprise that has been ongoing since its inception in the early 20th century. We would like to use this commentary as an opportunity to focus on the future of personality assessment in employment selection. We call attention to the relevant history of the personality testing enterprise, discuss whether personality testing adversely affects the mentally disabled, and discuss the strategic role I-O psychologists play in this enterprise.
Consider the context surrounding the passage of the National Labor Relations Act (NLRA) of 1935 (for details, see Zickar & Kostek, Reference Zickar, Kostek, Christiansen and Tett2013), which outlawed the practice of asking applicants if they would unionize. During this time, organizational sociologist Elton Mayo claimed that irrational thinking and emotional issues explained poor employee performance and union membership (Zickar & Kostek, Reference Zickar, Kostek, Christiansen and Tett2013). Managers, who were eager to prevent unionization, thus saw personality testing as a means of indirectly flouting the law. Catering to these desires, Doncaster Humm and Guy Wadsworth marketed their Humm–Wadsworth Temperament Test for these purposes (see Zickar & Kostek, Reference Zickar, Kostek, Christiansen and Tett2013). Although this test was designed to identify and help employees or job applicants suffering from a mental disorder, it was used by employers to weed out individuals they believed were union sympathizers (antisocial types) or who were communist ideologues (manic-depressive types; see Emre, Reference Emre2018). The need for personality testing was exacerbated after 1978 when the Equal Employment Opportunity Commission published the Uniform Guidelines on Employee Selection. The Uniform Guidelines sparked a search for selection procedures with comparable utility but less adverse impact compared to cognitive ability tests, leading I-O psychologists to personality tests (see Hogan, Reference Hogan2007).
We raise this historical perspective to encourage the I-O psychology community to maintain awareness of the historical context within which we operate. Personality testing has long been criticized by academics, politicians, and also in the popular press (e.g., Emre, Reference Emre2018) for denying mentally disordered individuals a voice in our society (see Zickar & Kostek, Reference Zickar, Kostek, Christiansen and Tett2013). We must recognize that our actions could be interpreted as yet another attempt to help organizations flout the law, which is clearly not our intention. Rather, we suspect that many within our field will ascribe to an ideal in organizational life that most if not all individuals—regardless of class status (e.g., disability)—can compete for and find a valuable place in our society (e.g., Cascio & Aguinis, Reference Cascio and Aguinis2011). As a profession, our research can inform the sorting process by which individuals find ways to contribute meaningfully to organizations (see Oh, Kim, & Van Iddekinge, Reference Oh, Kim and Van Iddekinge2015) and, therefore, society at large (see Lee & Steel, Reference Lee and Steeln.d.).
We submit that framing personality testing at work as a means of identifying ideal workers for the work might advance the conversation in ways that show a sensitivity to the issue at hand. It is important to note that anyone can engage in the behaviors that might be considered ideal for a given work setting. However, such behaviors come more easily to certain individuals: those with the right or “ideal” kind of personality (Aguinis, Mazurkiewicz, & Heggestad, Reference Aguinis, Mazurkiewicz and Heggestad2009).Footnote
1
To help frame this discussion, we think it is important for the field to address two interrelated questions: (a) What is the evidence suggesting that certain trait standings are ideal for a given work setting? and (b) Does screening and/or selecting for ideal personality for the work cause adverse impact? We provide our own take at these questions below and close with a call for more research.
What is the evidence that ideal trait standings for a specific work setting exist?
Only a few studies have examined whether specific trait standings are ideal for a given work setting.Footnote
2
For instance, in a study of U.S. employees working in a call center, Grant (Reference Grant2013) found a nonlinear effect linking extraversion to sales, suggesting that balancing talking with listening was helpful for selling products. Examining a broader sample of job settings, Le et al. (Reference Le, Oh, Robbins, Ilies, Holland and Westrick2011) put forward mixed evidence of nonlinear relationships linking conscientiousness and emotional stability to performance. Their findings suggest that moderate levels of conscientiousness and emotional stability might result in higher performance for low-complexity jobs (e.g., waitstaff). Conversely, for high-complexity jobs (e.g., psychologists), high levels of these traits (e.g., > 2 SDs) might be associated with higher levels of performance. Building on Le et al., Carter et al. (Reference Carter, Dalal, Boyce, O’Connell, Kung and Delgado2014), using ideal point modeling, consistently linked conscientiousness to performance nonlinearly across two different samples. Ideal point modeling strategies are important for our discussion because they more faithfully represent the cognitive processes underlying personality assessment (Chernyshenko, Stark, Drasgow, & Roberts, Reference Chernyshenko, Stark, Drasgow and Roberts2007). For instance, consider the statement “I occasionally seek out better ways of doing work.” Someone might disagree to this item because they either rarely or always seek out better ways of doing work, and such a signature reflects lower and higher levels, respectively, of the personality trait than that tapped into by the item. Compared to conventional factor analytic techniques, which assume that higher levels of agreement always correspond to higher trait standings, ideal point models reliably identify nonlinear effects when they exist (Carter, Dalal, Guan, Lopilato, & Withrow, Reference Carter, Dalal, Guan, Lopilato and Withrow2017).
Unfortunately, the evidence regarding ideal trait standings for a given work setting is far from conclusive. Recently, many of the above findings were called into question by Nickel, Roberts, and Chernyshenko (Reference Nickel, Roberts and Chernyshenko2019). Using ideal point modeling strategies, they failed to link conscientiousness nonlinearly to work outcomes such as job satisfaction and organizational citizenship. Nickel et al. posited that nonlinear effects may be emerging due to sampling error or a failure to account for other explanations for “too much of a good thing” effects (e.g., neuroticism; see the misattribution hypothesis; Nickel et al.). Nickel et al. raise important concerns for I-O psychologists to consider as they conduct research into ideal personality trait standings at work. The concerns they raise are related to several problems that have long plagued personality assessment at work (e.g., item multidimensionality, sampling error, outliers). The extent to which such concerns are bugs rather than features deserves further scholarly attention from I-O psychologists. For instance, the item “I become annoyed when things around me are disorganized,” which has been designed to assess high levels of orderliness (an aspect of conscientiousness; see Chernyshenko et al., Reference Chernyshenko, Stark, Drasgow and Roberts2007), also seems to tap into high levels of volatility (an aspect of neuroticism). This item most accurately describes someone with an orderliness standing ~2.5 SDs above the mean, which is a relatively narrow subset of the population at large. Reliably assessing this extreme trait standing will prove difficult given so few individuals in the population. Although technical solutions (e.g., multidimensional ideal point IRT) may address these issues, the upside for practice may be that ideal point personality assessments assess extreme or even maladaptive trait standings relatively less reliably, rendering detection of personality disorders dubious.
Does screening or selecting for ideal workers cause adverse impact against the mentally disabled?
We submit that just because normal and abnormal models of personality functioning can be integrated (see Suzuki, Samuel, Pahlen, & Krueger, Reference Suzuki, Samuel, Pahlen and Krueger2015), it does not necessarily follow that measurement models designed for use at work will necessarily identify disordered individuals. As we are more likely to assess individuals belonging to a subclinical rather than a clinical population (Wu & LeBreton, Reference Wu and Lebreton2011), it remains to be demonstrated empirically whether ideal point personality assessments actually cause adverse impact in practice. Indeed, this will prove difficult because it will require individuals to disclose an invisible disability in the hiring process. Though research is sorely needed in this area, it has nevertheless been suggested that using personality tests like the ideal point variety is unlikely to violate federal law for at least two reasons (see Dilchert, Ones, & Krueger, Reference Dilchert, Ones and Krueger2014; Wu & LeBreton, Reference Wu and Lebreton2011). First, we are not trying to identify individuals who are unable to work but those who are best able to perform the job needed by organizations. Second, the assessments we are discussing are designed for use with the general population (rather than in a clinical population). In other words, at worst, we may be assessing impairments but not necessarily disabilities (see Wu & LeBreton, Reference Wu and Lebreton2011); therefore, using ideal point personality assessments in pre-employment contexts before conditional job offers is unlikely to violate the Americans with Disabilities Act (Dilchert et al., Reference Dilchert, Ones and Krueger2014; Wu & LeBreton, Reference Wu and Lebreton2011). Nevertheless, we agree with Melson-Silimon et al. (Reference Melson-Silimon, Harris, Shoenfelt, Miller and Carter2019): We should be vigilant and proactive in receiving guidance from the EEOC. History need not repeat itself.
Identify ideal trait standings for a given work setting: A call for more research
Imagine that personality assessments in selection contexts (e.g., structured assessments, Big Five tests, etc.) were tightly regulated such that practical use was broadly considered questionable at best. Indeed, this is similar to what occurred with cognitive ability testing following the establishment of the Uniform Guidelines. Would personality assessment truly no longer occur? We contend that personality assessment would occur in some fashion in the employment process, but the effectiveness would be highly questionable. For instance, Huffcutt et al. (Reference Huffcutt, Conway, Roth and Stone2001), who analyzed 47 interview studies, found that basic personality and applied social skills were the most frequently assessed constructs during interviews. Although this point may seem obvious, it is noteworthy because our research can shape how effectively personality assessments are carried out. For instance, if for all practical purposes conventional personality testing were to be replaced by less structured means of assessing personality (e.g., unstructured or semistructured interviews), then individuals defined by dark personality tendencies (e.g., narcissists) might outperform more deserving applicants (Paulhus, Westlake, Calvez, & Harms, Reference Paulhus, Westlake, Calvez and Harms2013; Peck & Levashina, Reference Peck and Levashina2017), making both employees and organizations worse off. Contamination of this kind is indeed a key driver of research into more useful and faking-resistant methods of personality assessment in the first place. Conversely, if mentally disordered individuals were screened out by interviews (intentionally or unintentionally), such an action would still be considered questionable in light of the ADA.
Furthermore, the Uniform Guidelines impels us to use additional approaches for identifying ideal trait standings (e.g., personality-oriented work analysis, or POWA; see O’Neill, Goffin, & Rothstein, Reference O’Neill, Goffin, Rothstein, Christiansen and Tett2013). POWA has been used to describe the relative importance of personality traits for a given work setting (e.g., the Personality-Related Position Requirements Form, or PPRF; see Raymark, Schmit, & Guion, Reference Raymark, Schmit and Guion1997). Unfortunately, conventional approaches like the PPRF may miss trait standings that are ideal for a work setting. Consider the following item from the PPRF: “work with dissatisfied customers or clients to achieve a mutually agreeable solution.” This item has been classified as assessing work-related requirements for extraversion. In the context of evaluating a sales role, job analysts would likely agree that this behavior is essential for performance in the job. Unfortunately, it might also be interpreted by job analysts as suggesting that extraversion is highly important. However, if working with dissatisfied customers or clients requires striking the right balance between talking and listening, then ambiverts will have an advantage (Grant, Reference Grant2013). We view this shortcoming of the PPRF as an opportunity to refine it. Hence, we echo Melson-Silimon et al. (Reference Melson-Silimon, Harris, Shoenfelt, Miller and Carter2019) in their call for more research into POWA. Importantly, we believe that future research should examine the promise of adapting POWA to identify ideal trait standings and that more research is needed to demonstrate that ideal trait standings for a given work setting exist.
You have
Access
Melson-Silimon, Harris, Shoenfelt, Miller, and Carter (Reference Melson-Silimon, Harris, Shoenfelt, Miller and Carter2019) raise an important issue that industrial and organizational (I-O) psychologists should take seriously: As we integrate normal and abnormal personality models, we may find that personality assessments adversely impact the mentally disabled. Though not mentioned by Melson-Silimon and colleagues, this is a criticism of the personality testing enterprise that has been ongoing since its inception in the early 20th century. We would like to use this commentary as an opportunity to focus on the future of personality assessment in employment selection. We call attention to the relevant history of the personality testing enterprise, discuss whether personality testing adversely affects the mentally disabled, and discuss the strategic role I-O psychologists play in this enterprise.
Consider the context surrounding the passage of the National Labor Relations Act (NLRA) of 1935 (for details, see Zickar & Kostek, Reference Zickar, Kostek, Christiansen and Tett2013), which outlawed the practice of asking applicants if they would unionize. During this time, organizational sociologist Elton Mayo claimed that irrational thinking and emotional issues explained poor employee performance and union membership (Zickar & Kostek, Reference Zickar, Kostek, Christiansen and Tett2013). Managers, who were eager to prevent unionization, thus saw personality testing as a means of indirectly flouting the law. Catering to these desires, Doncaster Humm and Guy Wadsworth marketed their Humm–Wadsworth Temperament Test for these purposes (see Zickar & Kostek, Reference Zickar, Kostek, Christiansen and Tett2013). Although this test was designed to identify and help employees or job applicants suffering from a mental disorder, it was used by employers to weed out individuals they believed were union sympathizers (antisocial types) or who were communist ideologues (manic-depressive types; see Emre, Reference Emre2018). The need for personality testing was exacerbated after 1978 when the Equal Employment Opportunity Commission published the Uniform Guidelines on Employee Selection. The Uniform Guidelines sparked a search for selection procedures with comparable utility but less adverse impact compared to cognitive ability tests, leading I-O psychologists to personality tests (see Hogan, Reference Hogan2007).
We raise this historical perspective to encourage the I-O psychology community to maintain awareness of the historical context within which we operate. Personality testing has long been criticized by academics, politicians, and also in the popular press (e.g., Emre, Reference Emre2018) for denying mentally disordered individuals a voice in our society (see Zickar & Kostek, Reference Zickar, Kostek, Christiansen and Tett2013). We must recognize that our actions could be interpreted as yet another attempt to help organizations flout the law, which is clearly not our intention. Rather, we suspect that many within our field will ascribe to an ideal in organizational life that most if not all individuals—regardless of class status (e.g., disability)—can compete for and find a valuable place in our society (e.g., Cascio & Aguinis, Reference Cascio and Aguinis2011). As a profession, our research can inform the sorting process by which individuals find ways to contribute meaningfully to organizations (see Oh, Kim, & Van Iddekinge, Reference Oh, Kim and Van Iddekinge2015) and, therefore, society at large (see Lee & Steel, Reference Lee and Steeln.d.).
We submit that framing personality testing at work as a means of identifying ideal workers for the work might advance the conversation in ways that show a sensitivity to the issue at hand. It is important to note that anyone can engage in the behaviors that might be considered ideal for a given work setting. However, such behaviors come more easily to certain individuals: those with the right or “ideal” kind of personality (Aguinis, Mazurkiewicz, & Heggestad, Reference Aguinis, Mazurkiewicz and Heggestad2009).Footnote 1 To help frame this discussion, we think it is important for the field to address two interrelated questions: (a) What is the evidence suggesting that certain trait standings are ideal for a given work setting? and (b) Does screening and/or selecting for ideal personality for the work cause adverse impact? We provide our own take at these questions below and close with a call for more research.
What is the evidence that ideal trait standings for a specific work setting exist?
Only a few studies have examined whether specific trait standings are ideal for a given work setting.Footnote 2 For instance, in a study of U.S. employees working in a call center, Grant (Reference Grant2013) found a nonlinear effect linking extraversion to sales, suggesting that balancing talking with listening was helpful for selling products. Examining a broader sample of job settings, Le et al. (Reference Le, Oh, Robbins, Ilies, Holland and Westrick2011) put forward mixed evidence of nonlinear relationships linking conscientiousness and emotional stability to performance. Their findings suggest that moderate levels of conscientiousness and emotional stability might result in higher performance for low-complexity jobs (e.g., waitstaff). Conversely, for high-complexity jobs (e.g., psychologists), high levels of these traits (e.g., > 2 SDs) might be associated with higher levels of performance. Building on Le et al., Carter et al. (Reference Carter, Dalal, Boyce, O’Connell, Kung and Delgado2014), using ideal point modeling, consistently linked conscientiousness to performance nonlinearly across two different samples. Ideal point modeling strategies are important for our discussion because they more faithfully represent the cognitive processes underlying personality assessment (Chernyshenko, Stark, Drasgow, & Roberts, Reference Chernyshenko, Stark, Drasgow and Roberts2007). For instance, consider the statement “I occasionally seek out better ways of doing work.” Someone might disagree to this item because they either rarely or always seek out better ways of doing work, and such a signature reflects lower and higher levels, respectively, of the personality trait than that tapped into by the item. Compared to conventional factor analytic techniques, which assume that higher levels of agreement always correspond to higher trait standings, ideal point models reliably identify nonlinear effects when they exist (Carter, Dalal, Guan, Lopilato, & Withrow, Reference Carter, Dalal, Guan, Lopilato and Withrow2017).
Unfortunately, the evidence regarding ideal trait standings for a given work setting is far from conclusive. Recently, many of the above findings were called into question by Nickel, Roberts, and Chernyshenko (Reference Nickel, Roberts and Chernyshenko2019). Using ideal point modeling strategies, they failed to link conscientiousness nonlinearly to work outcomes such as job satisfaction and organizational citizenship. Nickel et al. posited that nonlinear effects may be emerging due to sampling error or a failure to account for other explanations for “too much of a good thing” effects (e.g., neuroticism; see the misattribution hypothesis; Nickel et al.). Nickel et al. raise important concerns for I-O psychologists to consider as they conduct research into ideal personality trait standings at work. The concerns they raise are related to several problems that have long plagued personality assessment at work (e.g., item multidimensionality, sampling error, outliers). The extent to which such concerns are bugs rather than features deserves further scholarly attention from I-O psychologists. For instance, the item “I become annoyed when things around me are disorganized,” which has been designed to assess high levels of orderliness (an aspect of conscientiousness; see Chernyshenko et al., Reference Chernyshenko, Stark, Drasgow and Roberts2007), also seems to tap into high levels of volatility (an aspect of neuroticism). This item most accurately describes someone with an orderliness standing ~2.5 SDs above the mean, which is a relatively narrow subset of the population at large. Reliably assessing this extreme trait standing will prove difficult given so few individuals in the population. Although technical solutions (e.g., multidimensional ideal point IRT) may address these issues, the upside for practice may be that ideal point personality assessments assess extreme or even maladaptive trait standings relatively less reliably, rendering detection of personality disorders dubious.
Does screening or selecting for ideal workers cause adverse impact against the mentally disabled?
We submit that just because normal and abnormal models of personality functioning can be integrated (see Suzuki, Samuel, Pahlen, & Krueger, Reference Suzuki, Samuel, Pahlen and Krueger2015), it does not necessarily follow that measurement models designed for use at work will necessarily identify disordered individuals. As we are more likely to assess individuals belonging to a subclinical rather than a clinical population (Wu & LeBreton, Reference Wu and Lebreton2011), it remains to be demonstrated empirically whether ideal point personality assessments actually cause adverse impact in practice. Indeed, this will prove difficult because it will require individuals to disclose an invisible disability in the hiring process. Though research is sorely needed in this area, it has nevertheless been suggested that using personality tests like the ideal point variety is unlikely to violate federal law for at least two reasons (see Dilchert, Ones, & Krueger, Reference Dilchert, Ones and Krueger2014; Wu & LeBreton, Reference Wu and Lebreton2011). First, we are not trying to identify individuals who are unable to work but those who are best able to perform the job needed by organizations. Second, the assessments we are discussing are designed for use with the general population (rather than in a clinical population). In other words, at worst, we may be assessing impairments but not necessarily disabilities (see Wu & LeBreton, Reference Wu and Lebreton2011); therefore, using ideal point personality assessments in pre-employment contexts before conditional job offers is unlikely to violate the Americans with Disabilities Act (Dilchert et al., Reference Dilchert, Ones and Krueger2014; Wu & LeBreton, Reference Wu and Lebreton2011). Nevertheless, we agree with Melson-Silimon et al. (Reference Melson-Silimon, Harris, Shoenfelt, Miller and Carter2019): We should be vigilant and proactive in receiving guidance from the EEOC. History need not repeat itself.
Identify ideal trait standings for a given work setting: A call for more research
Imagine that personality assessments in selection contexts (e.g., structured assessments, Big Five tests, etc.) were tightly regulated such that practical use was broadly considered questionable at best. Indeed, this is similar to what occurred with cognitive ability testing following the establishment of the Uniform Guidelines. Would personality assessment truly no longer occur? We contend that personality assessment would occur in some fashion in the employment process, but the effectiveness would be highly questionable. For instance, Huffcutt et al. (Reference Huffcutt, Conway, Roth and Stone2001), who analyzed 47 interview studies, found that basic personality and applied social skills were the most frequently assessed constructs during interviews. Although this point may seem obvious, it is noteworthy because our research can shape how effectively personality assessments are carried out. For instance, if for all practical purposes conventional personality testing were to be replaced by less structured means of assessing personality (e.g., unstructured or semistructured interviews), then individuals defined by dark personality tendencies (e.g., narcissists) might outperform more deserving applicants (Paulhus, Westlake, Calvez, & Harms, Reference Paulhus, Westlake, Calvez and Harms2013; Peck & Levashina, Reference Peck and Levashina2017), making both employees and organizations worse off. Contamination of this kind is indeed a key driver of research into more useful and faking-resistant methods of personality assessment in the first place. Conversely, if mentally disordered individuals were screened out by interviews (intentionally or unintentionally), such an action would still be considered questionable in light of the ADA.
Furthermore, the Uniform Guidelines impels us to use additional approaches for identifying ideal trait standings (e.g., personality-oriented work analysis, or POWA; see O’Neill, Goffin, & Rothstein, Reference O’Neill, Goffin, Rothstein, Christiansen and Tett2013). POWA has been used to describe the relative importance of personality traits for a given work setting (e.g., the Personality-Related Position Requirements Form, or PPRF; see Raymark, Schmit, & Guion, Reference Raymark, Schmit and Guion1997). Unfortunately, conventional approaches like the PPRF may miss trait standings that are ideal for a work setting. Consider the following item from the PPRF: “work with dissatisfied customers or clients to achieve a mutually agreeable solution.” This item has been classified as assessing work-related requirements for extraversion. In the context of evaluating a sales role, job analysts would likely agree that this behavior is essential for performance in the job. Unfortunately, it might also be interpreted by job analysts as suggesting that extraversion is highly important. However, if working with dissatisfied customers or clients requires striking the right balance between talking and listening, then ambiverts will have an advantage (Grant, Reference Grant2013). We view this shortcoming of the PPRF as an opportunity to refine it. Hence, we echo Melson-Silimon et al. (Reference Melson-Silimon, Harris, Shoenfelt, Miller and Carter2019) in their call for more research into POWA. Importantly, we believe that future research should examine the promise of adapting POWA to identify ideal trait standings and that more research is needed to demonstrate that ideal trait standings for a given work setting exist.
Author ORCIDs
Christopher M. Castille, 0000-0002-2198-6417
Rachel Williamson Smith, 0000-0001-7283-8572