Comparative construct validity of quantitative models of psychopathology

A fundamental limitation of the work presented by Cramer et al. pertains to their selective approach to model fitting. They present the results of fitting network models, but not the results of fitting other psychometric models. Their strategy is instead to argue on an a priori basis that other models (e.g., latent variable models) should not be considered as potential models.

This strategy is obviously problematic. The selection of a quantitative model is an empirical matter, not a matter that can be decided a priori. As Cramer et al. point out (e.g., in their use of the Akaike Information Criterion [AIC] as an index of fit; sect. 2, para. 8), a desirable model can be thought of as balancing at least two properties: (1) the ability to reproduce the observed data and (2) the efficiency or parsimony with which the model can achieve the first property, more efficient models having greater scientific utility if they are equivalent in their ability to reproduce the observed data (Markon & Krueger Reference Markon and Krueger2004).

With regard to model fit, the network models proposed by Cramer et al. are unlikely to emerge as optimal models when compared with latent variable models. This is because the network models fit by Cramer et al. contain a multitude of parameters; they are lacking in parsimony when compared with latent variable models. Models lacking in parsimony often provide a poor relative fit to data and are thereby lower in scientific utility because they amount to little more than re-expressions of observed data (Barron & Cover Reference Barron and Cover1991). Because they capitalize on chance, they also tend to generalize poorly to new samples of measures or persons. Heavily parameterized models lack the ability to articulate organizing scientific constructs that have proven indispensable in building theories and evaluating the correspondence between theories and data.

Consider, for example, the model shown in Cramer et al.'s Figure 4, portraying network-model derived connections among the symptoms of major depression and generalized anxiety disorder. Cramer et al. attempt to argue that the multiplicity of parameters portrayed in Figure 4 (the size of the nodes, the darkness of the circumferences, the thickness of the edges, and the darkness of the edges) provides valuable information in focusing scientific inquiry aimed at understanding and ameliorating depression and anxiety. However, the fit of this highly complex model is never compared with other models that articulate organizing scientific constructs (i.e., latent variable models). By virtue of their greater parsimony and articulation of organizing constructs (e.g., the latent construct of neuroticism as the nexus of anxiety and depression; Griffith et al., in press), latent variable models point us toward key targets for scientific inquiry.

Such guidance is obviously critical in pursuing research aimed at reducing the public health burden of mental disorder (Lahey Reference Lahey2009). Put simply, how would one use the information in Figure 4 to explain to a policy maker how we might go about spending public funds wisely in the service of working to ameliorate the burden of depression and anxiety? By funding hundreds of separate projects focused on understanding each line in the figure? We doubt such a conversation would prove generative, or that scientific inquiry framed by Figure 4 would prove enlightening. We would also encourage the reader to contemplate how complicated Figure 4 would look if a more comprehensive set of psychopathological symptoms (e.g., symptoms of other mood and anxiety disorders) were modeled along with the specific symptoms that were the focus of Cramer et al.'s efforts.

Although comparative models of internal structure are an important step in model development, a next step is to evaluate the ability of a model to explain external constructs. Space considerations prevent us from describing the extensive literature on the construct validity of latent variable models of psychopathology at length. We can provide only a few examples but encourage the reader to consult the citations we give for more details. As examples, latent variable models of psychopathology can account for phenomena such as gender differences in the prevalence of specific syndromes (Kramer et al. Reference Kramer, Krueger and Hicks2008), the genetic and environmental effects that both connect and distinguish specific syndromes (Kendler et al. Reference Kendler, Prescott, Myers and Neale2003), and the generality and specificity of biobehavioral correlates of psychopathology (Patrick et al. Reference Patrick, Bernat, Malone, Iacono, Krueger and McGue2006).

Ultimately, some comparisons between network models and latent variable models may be moot, because the two frameworks are more similar than Cramer et al. suggest and may be identical in many cases. Importantly, network models of the sort espoused by Cramer et al. would be latent variable models if they included multiple measures of each symptom, rendering each of their nodes a latent variable. Thus, latent variable models are not only more parsimonious, but also more comprehensive than the models discussed by Cramer et al. Moreover, some sophisticated network models have proposed constructs very similar to latent variables (Kemp & Tenenbaum Reference Kemp and Tenenbaum2008). Theoretical discussions about the relative merits of latent variable versus network models are fundamentally misleading. The important questions are what models to compare empirically, how to make those comparisons, and what those empirical comparisons reveal.