The role of selection

As noted in the focal article, much of the scholarly turnover literature focuses on retaining employees after they have been hired. In fact, during a recent panel discussion on the future of turnover research, leading turnover scholars acknowledged that research on predicting turnover pre-hire had not been one of their top priorities thus far (e.g., Porter et al., Reference Porter, Holtom, Maertz, Allen, Hausknecht and Woo2019). The current state of the literature supports this assessment. To date, the bulk of research on retention focuses on three post-hire, context-dependent predictors of turnover—job satisfaction, organizational commitment, and turnover intentions (e.g., Holtom, Mitchell, Lee, & Eberly, Reference Holtom, Mitchell, Lee and Eberly2008). However, these theories cannot be leveraged to inform the development of pre-hire predictors because their applications are largely job or organization specific. Furthermore, the applicant–organization relationship is still in its early stages during selection. As such, individual-level job attitudes such as job satisfaction and commitment are not relevant to applicant decision-making processes. The limited amount of studies devoted to identifying stable individual-level predictors of turnover (e.g., conscientiousness and emotional stability; Barrick & Mount, Reference Barrick and Mount1996) tend to rely on small samples nested within specific industries, which can limit the generalizability of reported findings. Indeed, analyses within our own selection systems (which comprise a deeper, more diverse sampling of various types of jobs, applicants, and organizations) suggest that these predictors will not work in all circumstances.

To their credit, in the focal article, the authors briefly discuss the role of individual worker knowledge, skills, abilities, and other characteristics (KSAOs) in understanding and predicting turnover. However, they question the utility of these predictors out of concern that it “may be rare to have data on all employees in a company, with this being more likely to occur when the scope of analysis is only on a small population.” (Speer et al., Reference Speer, Dutta, Chen and Trussell2019, p. 272). On this point, we disagree. In our experience, when organizations use assessments, they typically cover a substantial percentage of their employees. Even in cases where the assessments are limited to a single job or job family, they are often utilized for high-volume, entry-level positions—roles that typically have the highest rates of turnover. As such, predicting turnover using pre-hire assessments remains a practically relevant pursuit for many organizations. In some industries and job types, such as retail, fast-food, call centers, and sales, year-on-year rates of turnover that exceed 100% are not uncommon.

Nevertheless, we agree that turnover research should be designed in such a way that it can generalize to broad audiences of both scholars and practitioners. Although the authors make a compelling case that attrition modeling can achieve strong prediction of turnover, we are concerned such efforts would be premature in most organizations. Pre-hire is perhaps the best time in an employee’s trajectory at a company to collect strong, psychometrically sound data on psychological constructs of interest. Our own data, validated in a predictive context against hundreds of thousands of hires, has shown retention rate differences of as much as 30 percentage points between high and low bands on predictors of turnover risk at 6 months post-hire. Stable and measurable individual differences underlie this prediction.

Defining turnover

A second goal the authors advocate for is making an immediate impact with available data. However, we believe that it may be more prudent to focus first on building a sufficient infrastructure around turnover as a criterion to allow for the collection of data that can more explicitly target key phenomena of interest. On this point, it is our experience that many organizations lack coherent, descriptive coding schemes for termination reasons. As such, it can be difficult at times to understand exactly what we are trying to predict, either pre-hire or post-hire. We have come to believe that one of the most pressing issues facing those seeking to predict turnover is recognizing that turnover as a criterion is not a monolith. An employee resigning to pursue education opportunities or move across the country because of a spouse’s new job is substantively different than an employee who is being fired for misconduct, absenteeism, or theft. Although many organizations often attempt to disentangle these differences in reasons for leaving, there is no consistently used taxonomy by organizations.

In our role as practitioners, we perform regular turnover analyses for many organizations. During these analyses, the organizations often provide their internal coding schemes that denote why individuals separated from the organization. Yet we see remarkably little overlap in how termination reasons are coded, with some using as few as two classifications and others as many as 40. Even the simple classifications vary significantly: involuntary versus voluntary, regrettable versus nonregrettable, and controllable versus uncontrollable, among dichotomized others. We believe the main issue here is a lack of established guidelines or frameworks to follow. Indeed, many of the organizations we interact with are Fortune 50 companies that would gladly follow best practices for coding terminations if such information were available.

As part of a broader recent internal effort, we sought to compile data from a variety of organizations. Across the 11 organizations in our sample, we identified over 250 unique termination reasons, which, guided by empirical research on established constructs, we subsequently collapsed into approximately 30 general categories. It is true that several taxonomies of turnover exist. However, these categorization schemes (e.g., involuntary vs. voluntary) appear too broad to meaningfully identify specific antecedents. How can we seek to predict turnover when we cannot consistently define it? If we are unable to consistently define different dimensions of a given criterion, what hope do we have of truly understanding or predicting it either pre- or post-hire (Austin & Villanova, Reference Austin and Villanova1992)?

Understanding first, predicting later

Our last point relates to the core of what makes I-O psychologists valuable to organizations. As we continue to move into more advanced methods of both predicting and understanding turnover, an area that requires more attention is model interpretation. Speer et al. (Reference Speer, Dutta, Chen and Trussell2019) should be commended for their discussion and explanation of model types, including logistic regression and ensembled tree methods, as well as validation metrics like area under the curve (AUC). Beyond model-type considerations, we assert that one of the most important requirements today for applied attrition modeling is the ability to interpret these more sophisticated models. In such cases, it is not always enough to know what the prediction was; one must also explain how the conclusion was reached (Doshi-Velez & Kim, Reference Doshi-Velez and Kim2017). This is especially salient when the outcomes may have consequential implications for peoples’ lives, like decisions about hiring, promotion, and career development investment. Given these stakes, we believe the focus on model interpretation is an extremely important part of the process that has been left out of the focal article.

The nonlinearity of both these complex algorithms and test content scoring (i.e., biodata and personality) often makes it difficult to interpret the results of many of the newer algorithms described in the focal article. Traditional model interpretation methods for these techniques focus on global variable importance and leverage methods like the Gini coefficient or information gain/entropy. Global variable importance fails to explain which response patterns within the variable are more or less important. However, there are methods—local interpretation methods—that provide the level of detail that should be understood by practitioners taking advice from the focal article. For instance, the Local Interpretable Model-Agnostic Explanation (LIME; Ribeiro, Singh, & Guestrin, Reference Ribeiro, Singh and Guestrin2016) is designed to provide local interpretation (i.e., why the model made the prediction it did at an individual level), but this method can also be aggregated via variable and response weights to provide a proxy for a more in-depth global interpretation. In doing so, you not only understand which variables were most important to the model but also have insights into the variable response patterns the model used to predict whether the individual would be likely to stay or leave the organization.

Ultimately, we believe that being too quick to embrace new analytic innovations such as the attrition model risks putting the proverbial cart before the horse in terms of focus. The basis of any intervention should always start with understanding and interpretation. Therefore, before we can advocate for new innovations in the prediction of employee turnover, we must first (a) define precisely what is driving turnover in the organization and (b) align one’s selection system with the goal of increasing retention.

Although the presented guide to attrition modeling may, no doubt, help many practitioners, this method should be viewed as a later-stage intervention that builds upon previous bedrock work. We hope scholars will recognize the impact big data and new analytic techniques can bring to our field without losing sight of the value I-O psychologists bring through refined theoretical and conceptual understanding of core constructs.