I wish to congratulate Cramer et al. for writing an audacious article that offers much food for thought. Among other things, the target article provides a welcome expansion on ideas presented by Borsboom (Reference Borsboom2008). I focus on areas that could use further fleshing out and on psychometric and methodological issues, leaving clinical issues for others with expertise in that area.
The causal interpretation of the network models puzzles me most. Cramer et al. state a tentative causal interpretation as: “the stronger the association between symptoms, the more likely that one symptom will lead to another” (sect. 6, para. 11); but the article does not develop the causal interpretation and instead focuses on modeling symmetric associations. If the authors intend a symmetric notion of causation, this seems like a very strong restriction on the models that rules out cases such as Borsboom's (Reference Borsboom2008) chain model in which panic attacks cause concern, but not vice versa. Asymmetric causation brings us back to something akin to a path diagram, although the positing of nonlinear causal connections by using a threshold parameter would add an interesting additional element to the causal model. However, even asymmetric bidirectional causal models face conceptual hurdles (Rozeboom Reference Rozeboom2009). Certainly one needs to conceptually distinguish causal parameters from associations (McDonald Reference McDonald2002), even if one then intends to argue in favor of equating them. A good deal of work remains to clarify various possible causal interpretations and evaluate their relative merits.
A second aspect of the article that I found difficult involved the critique of latent variable models. Cramer et al. make a reasonable case against a single common factor for all symptoms listed for a disorder in the Diagnostic and Statistical Manual of Mental Disorders (DSM). However, a critical latent variable modeler would inspect the symptom correlation matrix showing the same patterns of association described in the network, warning against a single common factor, or catch the problems when assessing model fit. As a result, much of the critique of latent variable models seems tilted in favor of networks by making a mismatched comparison of a proper network model against an improper latent variable model. Perhaps improper use of latent variable models occurs in the diagnostic literature, but then the criticism applies to the misuse of such models, not the models themselves.
The advocacy against latent variable models risks becoming an advocacy in favor of analyzing causal relationships between observed variables rather than latent variables, and this holds several potential pitfalls. One motivation for using latent variable models involves the ability to remove the attenuating effects of measurement error from the estimates of causal relationships. Replacing latent variable models with causal networks of observed variables negates this advantage. As a thought experiment, imagine modeling a network of variables measured without any error. Now imagine a knob that allows you to gradually turn up the amount of random measurement error mixed into the observed variables. As you turn the knob, you can expect the estimated causal connections between the nodes to decrease, eventually to the point that alternative networks become increasingly indistinguishable based on the observed data.
An alternative would involve developing multiple measures of each symptom. One could then treat each symptom as a latent variable with its own common factor model, and model the causal relationships between symptoms as relationships between these latent variables (Bollen Reference Bollen1989). This would allow the research to both control for measurement error and model direct causal relationships between the symptoms. As such, it remains unclear that shifting the focus to causal relationships between symptoms requires giving up measurement models and their advantages, but these two issues seem conflated in Cramer et al.'s article. I see no motivation for encouraging a return to observed regularities as the primary object of scientific explanation.
Cramer et al.'s comments regarding different causal networks for different individuals seem to cut against the presented network analyses and in favor of a return to some kind of latent variable model. Latent class analysis, or latent mixture modeling, provides a means of identifying stable subclasses of individuals sharing the same causal structure. Cramer et al. sensibly suggest modeling individual differences on the network parameters, allowing each individual his or her own causal structure. However, if clusters of individuals share the same causal structure, the latent mixture would provide a more parsimonious model than one that states that everybody differs from everybody else.
Borsboom (Reference Borsboom2008) suggested understanding symptoms as parts of disorders but took this as an alternative measurement model. Cramer et al. definitively reject the idea that symptoms measure disorders. I did not find the motivation for this shift clear from Cramer et al.'s presentation and would like to see further work fleshing out Borsboom's original idea that symptoms can both constitute and measure a disorder, or at least that symptom severity can measure disorder severity despite this constitutive part-whole relationship.
A final minor point involves the use of random split-sample cross-validation to support conclusions regarding the stability of the results of the network analyses. It seems unsurprising that a random split of such a large sample would produce the same results in each half, but it remains unclear why this provides support for the stability of the results. Resolving the issue requires greater clarity regarding the intended notion of stability. However, a more informative approach might instead make use of the methods outlined by Shadish et al. (Reference Shadish, Cook and Campbell2002) for systematically examining the consistency of the results across various possible moderating variables available in the data set. Such non-random stratification of the sample might provide a much more informative and stringent assessment of the stability of the results.
Again, I wish to congratulate Cramer et al. for having written such a far-reaching article. I hope that my comments can play a constructive role in moving the research program forward and look forward to further developments.
You have
Access
I wish to congratulate Cramer et al. for writing an audacious article that offers much food for thought. Among other things, the target article provides a welcome expansion on ideas presented by Borsboom (Reference Borsboom2008). I focus on areas that could use further fleshing out and on psychometric and methodological issues, leaving clinical issues for others with expertise in that area.
The causal interpretation of the network models puzzles me most. Cramer et al. state a tentative causal interpretation as: “the stronger the association between symptoms, the more likely that one symptom will lead to another” (sect. 6, para. 11); but the article does not develop the causal interpretation and instead focuses on modeling symmetric associations. If the authors intend a symmetric notion of causation, this seems like a very strong restriction on the models that rules out cases such as Borsboom's (Reference Borsboom2008) chain model in which panic attacks cause concern, but not vice versa. Asymmetric causation brings us back to something akin to a path diagram, although the positing of nonlinear causal connections by using a threshold parameter would add an interesting additional element to the causal model. However, even asymmetric bidirectional causal models face conceptual hurdles (Rozeboom Reference Rozeboom2009). Certainly one needs to conceptually distinguish causal parameters from associations (McDonald Reference McDonald2002), even if one then intends to argue in favor of equating them. A good deal of work remains to clarify various possible causal interpretations and evaluate their relative merits.
A second aspect of the article that I found difficult involved the critique of latent variable models. Cramer et al. make a reasonable case against a single common factor for all symptoms listed for a disorder in the Diagnostic and Statistical Manual of Mental Disorders (DSM). However, a critical latent variable modeler would inspect the symptom correlation matrix showing the same patterns of association described in the network, warning against a single common factor, or catch the problems when assessing model fit. As a result, much of the critique of latent variable models seems tilted in favor of networks by making a mismatched comparison of a proper network model against an improper latent variable model. Perhaps improper use of latent variable models occurs in the diagnostic literature, but then the criticism applies to the misuse of such models, not the models themselves.
The advocacy against latent variable models risks becoming an advocacy in favor of analyzing causal relationships between observed variables rather than latent variables, and this holds several potential pitfalls. One motivation for using latent variable models involves the ability to remove the attenuating effects of measurement error from the estimates of causal relationships. Replacing latent variable models with causal networks of observed variables negates this advantage. As a thought experiment, imagine modeling a network of variables measured without any error. Now imagine a knob that allows you to gradually turn up the amount of random measurement error mixed into the observed variables. As you turn the knob, you can expect the estimated causal connections between the nodes to decrease, eventually to the point that alternative networks become increasingly indistinguishable based on the observed data.
An alternative would involve developing multiple measures of each symptom. One could then treat each symptom as a latent variable with its own common factor model, and model the causal relationships between symptoms as relationships between these latent variables (Bollen Reference Bollen1989). This would allow the research to both control for measurement error and model direct causal relationships between the symptoms. As such, it remains unclear that shifting the focus to causal relationships between symptoms requires giving up measurement models and their advantages, but these two issues seem conflated in Cramer et al.'s article. I see no motivation for encouraging a return to observed regularities as the primary object of scientific explanation.
Cramer et al.'s comments regarding different causal networks for different individuals seem to cut against the presented network analyses and in favor of a return to some kind of latent variable model. Latent class analysis, or latent mixture modeling, provides a means of identifying stable subclasses of individuals sharing the same causal structure. Cramer et al. sensibly suggest modeling individual differences on the network parameters, allowing each individual his or her own causal structure. However, if clusters of individuals share the same causal structure, the latent mixture would provide a more parsimonious model than one that states that everybody differs from everybody else.
Borsboom (Reference Borsboom2008) suggested understanding symptoms as parts of disorders but took this as an alternative measurement model. Cramer et al. definitively reject the idea that symptoms measure disorders. I did not find the motivation for this shift clear from Cramer et al.'s presentation and would like to see further work fleshing out Borsboom's original idea that symptoms can both constitute and measure a disorder, or at least that symptom severity can measure disorder severity despite this constitutive part-whole relationship.
A final minor point involves the use of random split-sample cross-validation to support conclusions regarding the stability of the results of the network analyses. It seems unsurprising that a random split of such a large sample would produce the same results in each half, but it remains unclear why this provides support for the stability of the results. Resolving the issue requires greater clarity regarding the intended notion of stability. However, a more informative approach might instead make use of the methods outlined by Shadish et al. (Reference Shadish, Cook and Campbell2002) for systematically examining the consistency of the results across various possible moderating variables available in the data set. Such non-random stratification of the sample might provide a much more informative and stringent assessment of the stability of the results.
Again, I wish to congratulate Cramer et al. for having written such a far-reaching article. I hope that my comments can play a constructive role in moving the research program forward and look forward to further developments.