Considering the surge of popular interest in HR analytics, the focal article by Speer, Dutta, Chen, and Trussell (Reference Speer, Dutta, Chen and Trussell2019) provides a timely perspective on disconnects between science and practice with respect to modeling turnover. Like Speer and colleagues, we have found the academic turnover literature light on details when it comes to addressing the complexities of modeling turnover in practice. Unfortunately, Speer et al. do not mention there is a large body of published literature on attrition modeling that falls outside of academic journals, yet it offers detailed technical treatments and examples of dealing with the nuances of applied attrition modeling. Specifically, over the past several decades, the U.S. military has conducted extensive studies of attrition among enlisted personnel that speak to several issues raised by Speer et al. (e.g., Marshall-Mies et al., Reference Marshall-Mies, Jupton, Hirose, White, Mottern and Eshwar2007; McCloy & DiFazio, Reference McCloy, DiFazio, Campbell and Zook1996; Putka & Strickland, Reference Putka and Strickland2005; Strickland, Reference Strickland2005; White, Harris, Mottern, & Eshwar, Reference White, Harris, Mottern and Eshwar2008). This work has been extensively documented in publicly available technical reports, many of which can be downloaded from the Defense Technical Information Center (DTIC) at https://discover.dtic.mil/. We suggest search terms such as “attrition analysis,” “enlisted attrition,” “modeling attrition,” “Project First Term,” and “Project First Watch.”
Speer et al. (Reference Speer, Dutta, Chen and Trussell2019) are not alone in their omission of this body of published work, as we have rarely seen it cited in the academic literature on turnover. This is unfortunate because this work bridges the science and practice of attrition modeling by offering more fine-grained treatments of several of the issues Speer et al. raised than academic journal articles typically have space to provide. Unlike publishing in academic journals, researchers producing these technical reports are often unburdened by limits imposed on the length of academic manuscripts.
The purpose of our response is twofold. First, we aim to raise awareness that there exists an “other published literature” that provides concrete, research-based examples of addressing several of the issues Speer et al. (Reference Speer, Dutta, Chen and Trussell2019) raised. Second, we aim to dispel the notion that this military research lacks relevance for modeling in civilian organizations, given that many of the methodological issues one must confront when modeling turnover are present regardless of the type of organization.
Two recent, large-scale examples of attrition modeling in the U.S. Military are the U.S. Army’s Project First Term (Strickland, Reference Strickland2005) and U.S. Navy’s Project First Watch (Marshall-Mies et al., Reference Marshall-Mies, Jupton, Hirose, White, Mottern and Eshwar2007). Both projects tracked entire cohorts of enlisted personnel for multiple years (from organizational entry to exit) to model if and when they left their respective organization, and what predicted their departure. Project First Term tracked and modeled attrition among all 63,938 recruits who entered the Army in FY 1999; Project First Watch tracked and modeled attrition among 46,000 recruits who entered Naval training between April and August 2003. Both projects modeled attrition using a rich set of potential predictor variables that ranged from archival administrative and demographic data, typical of what one may find in a human resources information system (HRIS), to surveys administered at key points during one’s tenure (e.g., before and after job-specific training) and annual surveys thereafter in the case of Project First Term. Moreover, as a follow-up to Project First Term, the U.S. Army subsequently conducted a partial replication of the work to evaluate whether attrition models developed on the FY 1999 cohort, cross-validated among the population of 67,940 recruits who entered the Army in FY 2003 (Putka & Strickland, Reference Putka and Strickland2005).
Here, we focus on Project First Term, as its technical reports more clearly detail several of the issues described in Speer et al. (Reference Speer, Dutta, Chen and Trussell2019), as well as our first-hand knowledge of that work, having served as its primary analysts. Speer et al. raise several themes representing disconnects between the turnover science–practice that we confronted head-on and elaborate on thoroughly in the Project First Term technical reports: (a) the importance of differentiating types of turnover, (b) formulation of predictor variables in a data-rich setting, (c) definition of timeframe and the nuances of handling time, and (d) turnover modeling strategy.
Importance of differentiating types of turnover
Speer et al. (Reference Speer, Dutta, Chen and Trussell2019) noted the importance of considering the type of attrition being modeled. In Project First Term, we differentiated between various types of attrition that reflected reasons people left the Army (e.g., poor performance, poor conduct, medical reasons). These distinctions proved critical in two ways. First, we discovered that reasons why individuals tended to leave depended on what point in their career they left. For example, early leavers tended to depart for primarily physical or medical reasons, whereas later leavers tended to depart for performance- or conduct-related reasons. Not surprisingly, this pattern also had implications for what types of variables were most predictive of turnover at different points in one’s tenure. Had we focused on turnover in general, or even the common “voluntary” versus “involuntary” distinction, these insights would have been overlooked and key predictor–criterion relations would have been masked. This illustrates Speer et al.’s point about giving careful attention to type of turnover when designing and conducting modeling attrition in practice, particularly those types of attrition that have meaning within the organization of interest.
Formulation of predictor variables in a data-rich setting
Another challenge Speer et al. (Reference Speer, Dutta, Chen and Trussell2019) highlighted involves formulating a meaningful set of predictor variables—a task that can be particularly challenging when one has many potential inputs. Some variables might tie to existing theories of turnover, whereas other variables may be more germane to the organization in which the modeling is being conducted. Project First Term was no exception. We were faced with formulating a meaningful set of predictors based on a rich set of administrative data and seven surveys administered to employees over the course of their first 48 months of tenure. The surveys comprised hundreds of items, some pertinent to prevailing turnover theory and others more exploratory or experimental in nature. We relied heavily on common data reduction and psychometric methods to identify predictor composites that were of sufficient psychometric quality (e.g., reliable and consistent with the factor structure underlying the data) and could be tied to constructs that had analogues in the research literature or had clear potential value given the Army setting. An entire chapter in Strickland (Reference Strickland2005) is devoted to this data reduction and predictor composite formation work and can help serve as an example for practitioners faced with similar data-rich situations.
Definition of timeframe and the nuances of handling time
Speer et al. (Reference Speer, Dutta, Chen and Trussell2019) raised the critical issue of defining the timeframe of interest for modeling attrition. One of the key differentiating features of Project First Term is the example it provides of handling temporal issues for both sample definition and modeling strategy. We adopted a cohort-based approach in which we tracked all 63,938 individuals who entered in FY 1999 and modeled if and when they left over the next 4 years (using event history analyses on a person-period data set with more than 2 million records). We also modeled turnover that occurred within specific segments of one’s career, such as during training versus after training but before the end of the 4-year study time frame. Using a cohort-based approach enabled us to more clearly see that turnover that happened at different points in time in individuals’ tenures not only tended to be different in terms of frequency (e.g., turnover happened with greater frequency early on and tapered off over time) but also in terms of what variables best predicted leaving (e.g., see Chapter 3 in Strickland, Reference Strickland2005).
Contrast the cohort-approach described above with the typical approach described by Speer et al. (Reference Speer, Dutta, Chen and Trussell2019), where a practitioner may take a snapshot of an organization’s workforce at a given point in time and then model whether employees leave at some point thereafter (e.g., 12 months later). This “snapshot” approach can make it difficult to identify the types of trends we discovered in Project First Term because employees in such samples often are at different points in their jobs and/or organizational tenures (from newer employees to those nearing retirement). For instance, what 12-month turnover looks like and is predicted by may be very different for newer employees than for more seasoned employees, yet it will largely be masked without a cohort-based approach.
Let us take this example a step further and imagine that, as a field, we focused on cumulating knowledge regarding predictors of turnover (e.g., via meta-analyses) based on such snapshot-based samples. Not only would the nuance and variation observed through cohort-based design be invisible, but predictors that may be of value for predicting early turnover but not for later turnover (or vice versa) may wash out and look unpredictive when averaged across studies. Beyond the work referenced above, we recommend reviewing Singer and Willett (Reference Singer and Willett2003) for in-depth discussion of the nuances of modeling binary events such as turnover that unfold over time. We have found this reference very useful in both our private and public sector attrition modeling work when thinking through the inferences that can and cannot be drawn given how “time” is handled in turnover studies.
Turnover modeling strategy
One last issue we want to touch on is modeling strategy. Project First Term provides a detailed example of (a) blending theory-driven and more exploratory models of attrition and (b) using different types of models and samples to address different research questions and practical needs. As noted above, we developed and evaluated event history analysis (EHA) models to examine how attrition unfolded and was differentially predicted across individuals’ first 48 months of tenure. However, we also evaluated both exploratory and structural models of attrition to more specifically isolate drivers of attrition in key time periods of interest to the organization (i.e., attrition during entry training, attrition once on the job; see Chapters 3–7 in Strickland, Reference Strickland2005). Examining both exploratory and theory-driven structural models allowed us to balance inductive and deductive approaches to predicting and explaining attrition more than a focus on any one type of model alone would permit.
One key takeaway from Project First Term is that adopting a “one size fits all” approach to modeling is limiting and can mask key nuances. This is noteworthy, because turnover studies published in journals often focus on testing a specific theory or a set of relations espoused by that theory. Research such as Project First Term helps illustrate the value of modeling attrition and identifying the best predictors of attrition through multiple lenses without being constrained by a more concise “storyline” that academic publishing often entails. Importantly, the exploratory elements of this research enabled us to uncover interesting patterns of relationships among variables using a very large sample of data and cross-validate them in a subsequent, independent, large sample. Such discovery would have been missed had we ascribed to a strict theory-driven modeling approach often reinforced in major academic journals and served to offer new “facts” upon which to build future theory (Campbell & Wilmot, Reference Campbell, Wilmot, Anderson, Ones, Sinangil and Viswesvaran2018; Hambrick, Reference Hambrick2007).
Summary
Although the Project First Term research is now more than 15 years old, it is noteworthy that the issues faced in these studies and the methods adopted (e.g., survival analysis and structural models of attrition) are in line with those discussed by Speer et al. (Reference Speer, Dutta, Chen and Trussell2019) and can prove informative to current practitioners. Indeed, across its nearly 400 pages, the Strickland (Reference Strickland2005) report arguably provides academicians and practitioners a detailed picture of what in-depth attrition modeling in practice looks like when aimed at fulfilling different research and practical objectives (not simply testing a theory or testing a limited set of hypotheses for publication). In closing, we hope this brief response serves to raise awareness of this extensive literature—a literature we believe provides valuable examples for those facing the complexities of modeling attrition in science and practice.
You have
Access
Considering the surge of popular interest in HR analytics, the focal article by Speer, Dutta, Chen, and Trussell (Reference Speer, Dutta, Chen and Trussell2019) provides a timely perspective on disconnects between science and practice with respect to modeling turnover. Like Speer and colleagues, we have found the academic turnover literature light on details when it comes to addressing the complexities of modeling turnover in practice. Unfortunately, Speer et al. do not mention there is a large body of published literature on attrition modeling that falls outside of academic journals, yet it offers detailed technical treatments and examples of dealing with the nuances of applied attrition modeling. Specifically, over the past several decades, the U.S. military has conducted extensive studies of attrition among enlisted personnel that speak to several issues raised by Speer et al. (e.g., Marshall-Mies et al., Reference Marshall-Mies, Jupton, Hirose, White, Mottern and Eshwar2007; McCloy & DiFazio, Reference McCloy, DiFazio, Campbell and Zook1996; Putka & Strickland, Reference Putka and Strickland2005; Strickland, Reference Strickland2005; White, Harris, Mottern, & Eshwar, Reference White, Harris, Mottern and Eshwar2008). This work has been extensively documented in publicly available technical reports, many of which can be downloaded from the Defense Technical Information Center (DTIC) at https://discover.dtic.mil/. We suggest search terms such as “attrition analysis,” “enlisted attrition,” “modeling attrition,” “Project First Term,” and “Project First Watch.”
Speer et al. (Reference Speer, Dutta, Chen and Trussell2019) are not alone in their omission of this body of published work, as we have rarely seen it cited in the academic literature on turnover. This is unfortunate because this work bridges the science and practice of attrition modeling by offering more fine-grained treatments of several of the issues Speer et al. raised than academic journal articles typically have space to provide. Unlike publishing in academic journals, researchers producing these technical reports are often unburdened by limits imposed on the length of academic manuscripts.
The purpose of our response is twofold. First, we aim to raise awareness that there exists an “other published literature” that provides concrete, research-based examples of addressing several of the issues Speer et al. (Reference Speer, Dutta, Chen and Trussell2019) raised. Second, we aim to dispel the notion that this military research lacks relevance for modeling in civilian organizations, given that many of the methodological issues one must confront when modeling turnover are present regardless of the type of organization.
Two recent, large-scale examples of attrition modeling in the U.S. Military are the U.S. Army’s Project First Term (Strickland, Reference Strickland2005) and U.S. Navy’s Project First Watch (Marshall-Mies et al., Reference Marshall-Mies, Jupton, Hirose, White, Mottern and Eshwar2007). Both projects tracked entire cohorts of enlisted personnel for multiple years (from organizational entry to exit) to model if and when they left their respective organization, and what predicted their departure. Project First Term tracked and modeled attrition among all 63,938 recruits who entered the Army in FY 1999; Project First Watch tracked and modeled attrition among 46,000 recruits who entered Naval training between April and August 2003. Both projects modeled attrition using a rich set of potential predictor variables that ranged from archival administrative and demographic data, typical of what one may find in a human resources information system (HRIS), to surveys administered at key points during one’s tenure (e.g., before and after job-specific training) and annual surveys thereafter in the case of Project First Term. Moreover, as a follow-up to Project First Term, the U.S. Army subsequently conducted a partial replication of the work to evaluate whether attrition models developed on the FY 1999 cohort, cross-validated among the population of 67,940 recruits who entered the Army in FY 2003 (Putka & Strickland, Reference Putka and Strickland2005).
Here, we focus on Project First Term, as its technical reports more clearly detail several of the issues described in Speer et al. (Reference Speer, Dutta, Chen and Trussell2019), as well as our first-hand knowledge of that work, having served as its primary analysts. Speer et al. raise several themes representing disconnects between the turnover science–practice that we confronted head-on and elaborate on thoroughly in the Project First Term technical reports: (a) the importance of differentiating types of turnover, (b) formulation of predictor variables in a data-rich setting, (c) definition of timeframe and the nuances of handling time, and (d) turnover modeling strategy.
Importance of differentiating types of turnover
Speer et al. (Reference Speer, Dutta, Chen and Trussell2019) noted the importance of considering the type of attrition being modeled. In Project First Term, we differentiated between various types of attrition that reflected reasons people left the Army (e.g., poor performance, poor conduct, medical reasons). These distinctions proved critical in two ways. First, we discovered that reasons why individuals tended to leave depended on what point in their career they left. For example, early leavers tended to depart for primarily physical or medical reasons, whereas later leavers tended to depart for performance- or conduct-related reasons. Not surprisingly, this pattern also had implications for what types of variables were most predictive of turnover at different points in one’s tenure. Had we focused on turnover in general, or even the common “voluntary” versus “involuntary” distinction, these insights would have been overlooked and key predictor–criterion relations would have been masked. This illustrates Speer et al.’s point about giving careful attention to type of turnover when designing and conducting modeling attrition in practice, particularly those types of attrition that have meaning within the organization of interest.
Formulation of predictor variables in a data-rich setting
Another challenge Speer et al. (Reference Speer, Dutta, Chen and Trussell2019) highlighted involves formulating a meaningful set of predictor variables—a task that can be particularly challenging when one has many potential inputs. Some variables might tie to existing theories of turnover, whereas other variables may be more germane to the organization in which the modeling is being conducted. Project First Term was no exception. We were faced with formulating a meaningful set of predictors based on a rich set of administrative data and seven surveys administered to employees over the course of their first 48 months of tenure. The surveys comprised hundreds of items, some pertinent to prevailing turnover theory and others more exploratory or experimental in nature. We relied heavily on common data reduction and psychometric methods to identify predictor composites that were of sufficient psychometric quality (e.g., reliable and consistent with the factor structure underlying the data) and could be tied to constructs that had analogues in the research literature or had clear potential value given the Army setting. An entire chapter in Strickland (Reference Strickland2005) is devoted to this data reduction and predictor composite formation work and can help serve as an example for practitioners faced with similar data-rich situations.
Definition of timeframe and the nuances of handling time
Speer et al. (Reference Speer, Dutta, Chen and Trussell2019) raised the critical issue of defining the timeframe of interest for modeling attrition. One of the key differentiating features of Project First Term is the example it provides of handling temporal issues for both sample definition and modeling strategy. We adopted a cohort-based approach in which we tracked all 63,938 individuals who entered in FY 1999 and modeled if and when they left over the next 4 years (using event history analyses on a person-period data set with more than 2 million records). We also modeled turnover that occurred within specific segments of one’s career, such as during training versus after training but before the end of the 4-year study time frame. Using a cohort-based approach enabled us to more clearly see that turnover that happened at different points in time in individuals’ tenures not only tended to be different in terms of frequency (e.g., turnover happened with greater frequency early on and tapered off over time) but also in terms of what variables best predicted leaving (e.g., see Chapter 3 in Strickland, Reference Strickland2005).
Contrast the cohort-approach described above with the typical approach described by Speer et al. (Reference Speer, Dutta, Chen and Trussell2019), where a practitioner may take a snapshot of an organization’s workforce at a given point in time and then model whether employees leave at some point thereafter (e.g., 12 months later). This “snapshot” approach can make it difficult to identify the types of trends we discovered in Project First Term because employees in such samples often are at different points in their jobs and/or organizational tenures (from newer employees to those nearing retirement). For instance, what 12-month turnover looks like and is predicted by may be very different for newer employees than for more seasoned employees, yet it will largely be masked without a cohort-based approach.
Let us take this example a step further and imagine that, as a field, we focused on cumulating knowledge regarding predictors of turnover (e.g., via meta-analyses) based on such snapshot-based samples. Not only would the nuance and variation observed through cohort-based design be invisible, but predictors that may be of value for predicting early turnover but not for later turnover (or vice versa) may wash out and look unpredictive when averaged across studies. Beyond the work referenced above, we recommend reviewing Singer and Willett (Reference Singer and Willett2003) for in-depth discussion of the nuances of modeling binary events such as turnover that unfold over time. We have found this reference very useful in both our private and public sector attrition modeling work when thinking through the inferences that can and cannot be drawn given how “time” is handled in turnover studies.
Turnover modeling strategy
One last issue we want to touch on is modeling strategy. Project First Term provides a detailed example of (a) blending theory-driven and more exploratory models of attrition and (b) using different types of models and samples to address different research questions and practical needs. As noted above, we developed and evaluated event history analysis (EHA) models to examine how attrition unfolded and was differentially predicted across individuals’ first 48 months of tenure. However, we also evaluated both exploratory and structural models of attrition to more specifically isolate drivers of attrition in key time periods of interest to the organization (i.e., attrition during entry training, attrition once on the job; see Chapters 3–7 in Strickland, Reference Strickland2005). Examining both exploratory and theory-driven structural models allowed us to balance inductive and deductive approaches to predicting and explaining attrition more than a focus on any one type of model alone would permit.
One key takeaway from Project First Term is that adopting a “one size fits all” approach to modeling is limiting and can mask key nuances. This is noteworthy, because turnover studies published in journals often focus on testing a specific theory or a set of relations espoused by that theory. Research such as Project First Term helps illustrate the value of modeling attrition and identifying the best predictors of attrition through multiple lenses without being constrained by a more concise “storyline” that academic publishing often entails. Importantly, the exploratory elements of this research enabled us to uncover interesting patterns of relationships among variables using a very large sample of data and cross-validate them in a subsequent, independent, large sample. Such discovery would have been missed had we ascribed to a strict theory-driven modeling approach often reinforced in major academic journals and served to offer new “facts” upon which to build future theory (Campbell & Wilmot, Reference Campbell, Wilmot, Anderson, Ones, Sinangil and Viswesvaran2018; Hambrick, Reference Hambrick2007).
Summary
Although the Project First Term research is now more than 15 years old, it is noteworthy that the issues faced in these studies and the methods adopted (e.g., survival analysis and structural models of attrition) are in line with those discussed by Speer et al. (Reference Speer, Dutta, Chen and Trussell2019) and can prove informative to current practitioners. Indeed, across its nearly 400 pages, the Strickland (Reference Strickland2005) report arguably provides academicians and practitioners a detailed picture of what in-depth attrition modeling in practice looks like when aimed at fulfilling different research and practical objectives (not simply testing a theory or testing a limited set of hypotheses for publication). In closing, we hope this brief response serves to raise awareness of this extensive literature—a literature we believe provides valuable examples for those facing the complexities of modeling attrition in science and practice.