Sterba and Bauer's Keynote Article (Reference Sterba and Bauer2010 [this issue]) likely represents the clearest exposition of its kind on the assumptions, strengths, and limitations of model-based person-oriented methods and thus represents a significant contribution to the literature. Clearly, they should be a must read for developmental psychopathologists about to embark on person-oriented longitudinal analyses. At the same time, Sterba and Bauer's arguments may be so convincing that the reader may be steered away from person-oriented analyses in favor of conventional growth modeling with interactions included in the model. The purpose of this commentary is to highlight the work of Muthén and colleagues, who provide an equally compelling logic in laying out the steps that substantive researchers can take in building a scientifically strong case for either assuming continuously varied growth “… or that [trajectory groups] actually exist” (e.g., Kreuter & Muthén, Reference Kreuter and Muthén2008; Muthén, Reference Muthén2003; Reference Muthén and Kaplan2004).
Consistent with Sterba and Bauer (Reference Sterba and Bauer2010), Muthén and colleagues (Muthén, Reference Muthén2003, Reference Muthén and Kaplan2004; Kreuter & Muthén, Reference Kreuter and Muthén2008) argue that substantive theory is the first step in building the case that trajectory groups actually exist. For example, in Schaeffer, Petras, Ialongo, Poduska, and Kellam (Reference Schaeffer, Petras, Ialongo, Poduska and Kellam2003), which was cited by Sterba and Bauer (Reference Sterba and Bauer2010), the authors posited four growth trajectory classes of aggressive–disruptive behavior in keeping with Paterson, Reid, and Dishon's (Reference Patterson, Reid and Dishion1992) model of the development of antisocial behavior from childhood to adulthood: (a) an early starter class characterized by a consistently high level of aggressive–disruptive behavior across the elementary and middle school years, (b) a late starter class that first manifests a low level of aggressive–disruptive behavior in the early to middle elementary school years and then increases dramatically in the late elementary and middle school years, (c) a desister class in which children who manifest high levels of aggressive–disruptive behavior over the early to middle elementary school years subsequently exhibit a dramatic decrease in such behavior in the late elementary and middle school years, and (d) a normative class characterized by low levels of aggressive disruptive behavior across the elementary and middle school years. Along these lines, Hunter, Muthén, Cook, and Leuchter (Reference Hunter, Muthén, Cook and Leuchter2010) used growth mixture modeling to analyze data from a trial of antidepressant medication. The authors hypothesized that they would find at least two classes for the placebo group: a class that showed no improvement in depression over time and a second class that consistent with theory and substantive reasoning on placebo response demonstrated improvement in depressive symptoms similar to that seen in treatment responders. Thus, in both these examples, theory rather than the data drove the researchers to hypothesize that trajectory groups existed and that they were not “… better off assuming continuously varied growth a priori …” (Raudenbush, Reference Raudenbush2005, p. 136).
Muthén and colleagues (Kreuter & Muthén, Reference Kreuter and Muthén2008; Muthén, Reference Muthén2003, Reference Muthén and Kaplan2004) propose a second step for researchers to take in building a case for the existence of trajectory classes. This second step involves testing whether theoretically relevantly covariates predict class membership probabilities. Schaeffer et al. (Reference Schaeffer, Petras, Ialongo, Poduska and Kellam2003) described a set of covariates that was consistent with Patterson et al.'s (Reference Patterson, Reid and Dishion1992) theory of potential antecedents and moderators of growth trajectory membership and found the expected relationship between the covariates and class membership, particularly for the contrast between the normative and early starter classes. This served to further ensure the researchers that multiple classes of growth existed. A further example of the use of covariates in predicting class membership comes from a personal communication from Petras, Maysyn, Kellam, and Ialongo. In this example, the authors examined whether random assignment to one of two universal, school-based preventive interventions or a control condition in the fall of first grade was associated with class membership in terms of the growth of aggressive–disruptive behavior from the spring of first grade through the spring of third grade. In contrast to the control condition, both interventions were associated with a greater likelihood of being in a normative class characterized by low levels of aggressive disruptive behavior, although this association was only significant at the p < .05 level for one of the interventions.
A third step proposed by Muthén and colleagues is establishing the predictive validity of the classes. Consistent with them, Schaeffer et al. (Reference Schaeffer, Petras, Ialongo, Poduska and Kellam2003) tested whether class membership predicted later incarceration in the adult criminal system as well as involvement in the juvenile justice system. They found that membership in the early and late starter classes was associated with a significantly and dramatically higher probability of these two distal outcomes relative to the normative class. Again, further evidence in support of their hypothesis that multiple classes existed. It is important that Schaeffer et al. (Reference Schaeffer, Petras, Ialongo, Poduska and Kellam2003) could have used propensity score analysis within the context of their growth mixture analysis, which would have allowed them to draw stronger inferences with respect to the causal relationship between class membership and the distal outcomes in their study. As defined by Rosenbaum and Rubin (Reference Rosenbaum and Rubin1983), the propensity score represents the conditional probability of being in the “treatment” group relative to the control group given the observed covariates. Propensity score analysis is typically used in quasiexperimental or passive longitudinal designs, where study participants cannot be randomly assigned to the “treatment” condition. The propensity score serves the purpose of balancing the observed covariates between the treatment and “control” groups to rule out alternative explanations for the observed relationship between the treatment and the outcome. Parental divorce or age of pubertal onset are examples of treatment variables a developmental psychopathologist might study in terms of their causal impact on later outcomes in children. Clearly, one cannot randomly assign children to having parents who divorce or not or to a specific age of pubertal onset. Haviland and colleagues (Haviland & Nagin, Reference Haviland and Nagin2005, Reference Haviland and Nagin2007; Haviland, Nagin, & Rosenbaum, Reference Haviland, Nagin and Rosenbaum2007) provide examples of the integration of growth mixture modeling and propensity score analysis, which researchers could use to achieve the aim of drawing stronger causal inferences with respect to the relationship between trajectory class membership and distal outcomes. This would then allow for stronger inferences with respect to the existence of multiple growth trajectory classes.
Muthén and colleagues' fourth step consists of comparing the results of alternative modeling approaches and model fit. To that end, Kreuter and Muthén (Reference Kreuter and Muthén2008) provide a tutorial on how to carry out their approach to model comparison, using two data sets well known in the criminologic literature: the Cambridge (Farrington & West, Reference Farrington, West, Kerner and Kaiser1990) and the Philadelphia cohort study (see Nagin & Land, Reference Nagin and Land1993). Kreuter and Muthén (Reference Kreuter and Muthén2008) contend that standard growth curve analysis as well as the semiparametric, group-based approach (Nagin, Reference Nagin1999) can be treated as specific cases of general growth mixture modeling. This in turn provides the researcher with the ability to compare a conventional, or single-class, growth model with a growth mixture model, using model-fit indicators, such as the log-likelihood test, Bayesian information criteria (Schwartz, Reference Schwartz1978), and the Bootstrap Likelihood Ratio Test (McLachlan & Peel, Reference McLachlan and Peel2000). Kreuter and Muthén (Reference Kreuter and Muthén2008) acknowledge that there is no formal statistical test for comparing the results of alternative modeling approaches, but the Lo–Mendel–Rubin, Bayesian information criteria, and Bootstrap Likelihood Ratio Test represent the most accepted methods at this point in time.
As noted above, in addition to comparing alternative models in terms of the Lo–Mendel–Rubin, Bayesian information criteria, and Bootstrap Likelihood Ratio Test, Muthén and colleagues contend researchers must examine and compare the actual fit to the data in terms of the standardized residuals for the observed response pattern frequency and the effect of single influential cases on model fit. The latter is determined by computing an influence statistic for each observation in accord with Cook (Reference Cook1986) and Liski (Reference Liski1991). Within the context of their tutorial, Kreuter and Muthén (Reference Kreuter and Muthén2008) give examples of the use of standardized residual analysis for response pattern frequency in determining model fit, which are particularly informative in that they involve cases in which the growth mixture model does a better job in predicting in response pattern frequency than a conventional growth model. One of the examples involves the Philadelphia cohort data, wherein one of the response variables consists of police contacts over the ages of 4–26. In this example, Kreuter and Muthén compare the standardized residuals for a standard growth model and three growth mixture models. One response pattern they give as an example is characterized by one police contact between the ages of 18 and 19 and none before or after. The conventional growth model as well as all three of the growth mixture models yielded an overestimate of the observed frequency of this pattern. However, whereas the overestimate for the conventional growth model was about 20%, the overestimates for the three growth mixture models ranged from less than 1% to just over 9%. Kreuter and Muthén (Reference Kreuter and Muthén2008) point out that the nature and scaling of the data must be taken into account in comparing alternative models. One cannot assume that the model for comparing binary or count outcomes is acceptable for comparing data consisting of ordered categorical or continuous outcomes. Model comparisons strategies for the latter two can be found in three of Muthén's works (Reference Muthén, Marcoulides and Schumacker2001a, Reference Muthén, Collins and Sayer2001b, Reference Muthén and Kaplan2004).
A fifth step offered by Muthén and colleagues centers on considering whether the size and nature of the sample will allow the researcher to find multiple classes when they actually exist. Kreuter and Muthén (Reference Kreuter and Muthén2008) provide an example wherein they only used a random subset of 500 of the more than 13,000 cases available for analysis from the Philadelphia cohort study. Compared to the growth mixture analyses involving the entire sample, where a four-class model appeared to provide the best fit to the data, a conventional, or single class, growth model with random effects provided the best fit for the random subset of 500. Thus, the researcher must consider whether the size and nature of the sample will allow for sufficient power to detect multiple classes. This may be especially true in the case of relatively low base rate outcomes such as criminal arrests in community samples.
In summary, the steps elaborated above are clearly not exhaustive. Moreover, they are only of value when taken as whole. Indeed, success at any one step, whether it be that class membership predicted the distal outcomes of interest or that theoretically relevant covariates were found to be associated with class membership, is not sufficient to assume that multiple classes do exist. This amount of model testing may prove daunting to substantive researchers. However, there is no alternative if a researcher wants to make a convincing case that trajectory groups actually exist and that s/he is not “… better off assuming continuously varied growth a priori and therefore never tempting our audience to believe [… ] that groups actually exist” (Raudenbush, Reference Raudenbush2005, p. 136).
You have
Access
Sterba and Bauer's Keynote Article (Reference Sterba and Bauer2010 [this issue]) likely represents the clearest exposition of its kind on the assumptions, strengths, and limitations of model-based person-oriented methods and thus represents a significant contribution to the literature. Clearly, they should be a must read for developmental psychopathologists about to embark on person-oriented longitudinal analyses. At the same time, Sterba and Bauer's arguments may be so convincing that the reader may be steered away from person-oriented analyses in favor of conventional growth modeling with interactions included in the model. The purpose of this commentary is to highlight the work of Muthén and colleagues, who provide an equally compelling logic in laying out the steps that substantive researchers can take in building a scientifically strong case for either assuming continuously varied growth “… or that [trajectory groups] actually exist” (e.g., Kreuter & Muthén, Reference Kreuter and Muthén2008; Muthén, Reference Muthén2003; Reference Muthén and Kaplan2004).
Consistent with Sterba and Bauer (Reference Sterba and Bauer2010), Muthén and colleagues (Muthén, Reference Muthén2003, Reference Muthén and Kaplan2004; Kreuter & Muthén, Reference Kreuter and Muthén2008) argue that substantive theory is the first step in building the case that trajectory groups actually exist. For example, in Schaeffer, Petras, Ialongo, Poduska, and Kellam (Reference Schaeffer, Petras, Ialongo, Poduska and Kellam2003), which was cited by Sterba and Bauer (Reference Sterba and Bauer2010), the authors posited four growth trajectory classes of aggressive–disruptive behavior in keeping with Paterson, Reid, and Dishon's (Reference Patterson, Reid and Dishion1992) model of the development of antisocial behavior from childhood to adulthood: (a) an early starter class characterized by a consistently high level of aggressive–disruptive behavior across the elementary and middle school years, (b) a late starter class that first manifests a low level of aggressive–disruptive behavior in the early to middle elementary school years and then increases dramatically in the late elementary and middle school years, (c) a desister class in which children who manifest high levels of aggressive–disruptive behavior over the early to middle elementary school years subsequently exhibit a dramatic decrease in such behavior in the late elementary and middle school years, and (d) a normative class characterized by low levels of aggressive disruptive behavior across the elementary and middle school years. Along these lines, Hunter, Muthén, Cook, and Leuchter (Reference Hunter, Muthén, Cook and Leuchter2010) used growth mixture modeling to analyze data from a trial of antidepressant medication. The authors hypothesized that they would find at least two classes for the placebo group: a class that showed no improvement in depression over time and a second class that consistent with theory and substantive reasoning on placebo response demonstrated improvement in depressive symptoms similar to that seen in treatment responders. Thus, in both these examples, theory rather than the data drove the researchers to hypothesize that trajectory groups existed and that they were not “… better off assuming continuously varied growth a priori …” (Raudenbush, Reference Raudenbush2005, p. 136).
Muthén and colleagues (Kreuter & Muthén, Reference Kreuter and Muthén2008; Muthén, Reference Muthén2003, Reference Muthén and Kaplan2004) propose a second step for researchers to take in building a case for the existence of trajectory classes. This second step involves testing whether theoretically relevantly covariates predict class membership probabilities. Schaeffer et al. (Reference Schaeffer, Petras, Ialongo, Poduska and Kellam2003) described a set of covariates that was consistent with Patterson et al.'s (Reference Patterson, Reid and Dishion1992) theory of potential antecedents and moderators of growth trajectory membership and found the expected relationship between the covariates and class membership, particularly for the contrast between the normative and early starter classes. This served to further ensure the researchers that multiple classes of growth existed. A further example of the use of covariates in predicting class membership comes from a personal communication from Petras, Maysyn, Kellam, and Ialongo. In this example, the authors examined whether random assignment to one of two universal, school-based preventive interventions or a control condition in the fall of first grade was associated with class membership in terms of the growth of aggressive–disruptive behavior from the spring of first grade through the spring of third grade. In contrast to the control condition, both interventions were associated with a greater likelihood of being in a normative class characterized by low levels of aggressive disruptive behavior, although this association was only significant at the p < .05 level for one of the interventions.
A third step proposed by Muthén and colleagues is establishing the predictive validity of the classes. Consistent with them, Schaeffer et al. (Reference Schaeffer, Petras, Ialongo, Poduska and Kellam2003) tested whether class membership predicted later incarceration in the adult criminal system as well as involvement in the juvenile justice system. They found that membership in the early and late starter classes was associated with a significantly and dramatically higher probability of these two distal outcomes relative to the normative class. Again, further evidence in support of their hypothesis that multiple classes existed. It is important that Schaeffer et al. (Reference Schaeffer, Petras, Ialongo, Poduska and Kellam2003) could have used propensity score analysis within the context of their growth mixture analysis, which would have allowed them to draw stronger inferences with respect to the causal relationship between class membership and the distal outcomes in their study. As defined by Rosenbaum and Rubin (Reference Rosenbaum and Rubin1983), the propensity score represents the conditional probability of being in the “treatment” group relative to the control group given the observed covariates. Propensity score analysis is typically used in quasiexperimental or passive longitudinal designs, where study participants cannot be randomly assigned to the “treatment” condition. The propensity score serves the purpose of balancing the observed covariates between the treatment and “control” groups to rule out alternative explanations for the observed relationship between the treatment and the outcome. Parental divorce or age of pubertal onset are examples of treatment variables a developmental psychopathologist might study in terms of their causal impact on later outcomes in children. Clearly, one cannot randomly assign children to having parents who divorce or not or to a specific age of pubertal onset. Haviland and colleagues (Haviland & Nagin, Reference Haviland and Nagin2005, Reference Haviland and Nagin2007; Haviland, Nagin, & Rosenbaum, Reference Haviland, Nagin and Rosenbaum2007) provide examples of the integration of growth mixture modeling and propensity score analysis, which researchers could use to achieve the aim of drawing stronger causal inferences with respect to the relationship between trajectory class membership and distal outcomes. This would then allow for stronger inferences with respect to the existence of multiple growth trajectory classes.
Muthén and colleagues' fourth step consists of comparing the results of alternative modeling approaches and model fit. To that end, Kreuter and Muthén (Reference Kreuter and Muthén2008) provide a tutorial on how to carry out their approach to model comparison, using two data sets well known in the criminologic literature: the Cambridge (Farrington & West, Reference Farrington, West, Kerner and Kaiser1990) and the Philadelphia cohort study (see Nagin & Land, Reference Nagin and Land1993). Kreuter and Muthén (Reference Kreuter and Muthén2008) contend that standard growth curve analysis as well as the semiparametric, group-based approach (Nagin, Reference Nagin1999) can be treated as specific cases of general growth mixture modeling. This in turn provides the researcher with the ability to compare a conventional, or single-class, growth model with a growth mixture model, using model-fit indicators, such as the log-likelihood test, Bayesian information criteria (Schwartz, Reference Schwartz1978), and the Bootstrap Likelihood Ratio Test (McLachlan & Peel, Reference McLachlan and Peel2000). Kreuter and Muthén (Reference Kreuter and Muthén2008) acknowledge that there is no formal statistical test for comparing the results of alternative modeling approaches, but the Lo–Mendel–Rubin, Bayesian information criteria, and Bootstrap Likelihood Ratio Test represent the most accepted methods at this point in time.
As noted above, in addition to comparing alternative models in terms of the Lo–Mendel–Rubin, Bayesian information criteria, and Bootstrap Likelihood Ratio Test, Muthén and colleagues contend researchers must examine and compare the actual fit to the data in terms of the standardized residuals for the observed response pattern frequency and the effect of single influential cases on model fit. The latter is determined by computing an influence statistic for each observation in accord with Cook (Reference Cook1986) and Liski (Reference Liski1991). Within the context of their tutorial, Kreuter and Muthén (Reference Kreuter and Muthén2008) give examples of the use of standardized residual analysis for response pattern frequency in determining model fit, which are particularly informative in that they involve cases in which the growth mixture model does a better job in predicting in response pattern frequency than a conventional growth model. One of the examples involves the Philadelphia cohort data, wherein one of the response variables consists of police contacts over the ages of 4–26. In this example, Kreuter and Muthén compare the standardized residuals for a standard growth model and three growth mixture models. One response pattern they give as an example is characterized by one police contact between the ages of 18 and 19 and none before or after. The conventional growth model as well as all three of the growth mixture models yielded an overestimate of the observed frequency of this pattern. However, whereas the overestimate for the conventional growth model was about 20%, the overestimates for the three growth mixture models ranged from less than 1% to just over 9%. Kreuter and Muthén (Reference Kreuter and Muthén2008) point out that the nature and scaling of the data must be taken into account in comparing alternative models. One cannot assume that the model for comparing binary or count outcomes is acceptable for comparing data consisting of ordered categorical or continuous outcomes. Model comparisons strategies for the latter two can be found in three of Muthén's works (Reference Muthén, Marcoulides and Schumacker2001a, Reference Muthén, Collins and Sayer2001b, Reference Muthén and Kaplan2004).
A fifth step offered by Muthén and colleagues centers on considering whether the size and nature of the sample will allow the researcher to find multiple classes when they actually exist. Kreuter and Muthén (Reference Kreuter and Muthén2008) provide an example wherein they only used a random subset of 500 of the more than 13,000 cases available for analysis from the Philadelphia cohort study. Compared to the growth mixture analyses involving the entire sample, where a four-class model appeared to provide the best fit to the data, a conventional, or single class, growth model with random effects provided the best fit for the random subset of 500. Thus, the researcher must consider whether the size and nature of the sample will allow for sufficient power to detect multiple classes. This may be especially true in the case of relatively low base rate outcomes such as criminal arrests in community samples.
In summary, the steps elaborated above are clearly not exhaustive. Moreover, they are only of value when taken as whole. Indeed, success at any one step, whether it be that class membership predicted the distal outcomes of interest or that theoretically relevant covariates were found to be associated with class membership, is not sufficient to assume that multiple classes do exist. This amount of model testing may prove daunting to substantive researchers. However, there is no alternative if a researcher wants to make a convincing case that trajectory groups actually exist and that s/he is not “… better off assuming continuously varied growth a priori and therefore never tempting our audience to believe [… ] that groups actually exist” (Raudenbush, Reference Raudenbush2005, p. 136).