Cramer et al. argue that we have been looking at mental disorders through the wrong psychometric lenses, and that we need a general psychometric theory of disorders as networks. Their article is valuable in as much as it raises the possibility of connecting psychometric theory to network models. However, we focus on two points of contention. Before doing so, we stress agreement on an overarching point: There is insufficient evidence that any mental disorder is a single continuous latent variable. As stated by Borsboom et al. (Reference Borsboom, Mellenbergh and Van Heerden2004, p. 1063), the “ontological position that the attribute being measured exists and affects the outcome of the measurement procedure” is a strong scientific claim that is difficult, but necessary, to establish (see Michell Reference Michell1999).
Having said that, our first point of contention is this: We do not think the authors convincingly argue that the network perspective “nullifies the need to invoke latent variables as an explanation of the covariance between symptoms” (sect. 1, para. 5). The authors recognize that “non-symptom causal processes” like homeostasis may mediate and “partly explain” relations between symptoms (sect. 2, para. 9). However, a network model needs to explicitly incorporate such causal mechanisms to form the basis of a scientific theory, and this would make causal mechanisms fundamental, not incidental. Thus, Holland (Reference Holland1998, p. 132) refers to mechanisms and their defining transition functions as the primitives of a network. In Kauffman's (Reference Kauffman1993) reaction network model of autocatalytic sets, for example, nodes may represent molecules, and edges may represent reactions: chemical reactions are the causal relations in the network. Without causal mechanisms, it is not clear how the network presented in the target article is anything more than a “method to visualize” sets of symptoms defined by the statistical associations in the empirical data (sect. 3). It would seem that, as soon as one invokes causal mechanisms, one must invoke latent variables like psychological and physiological states. Thus, we question whether the network perspective nullifies the need to invoke latent variables, in some form, to explain covariation among symptoms.
Our second point of contention is that Cramer et al. have not formally demonstrated the technical premise for claiming their network perspective is a radically different conceptualization to the latent variable perspective. The premise is that “the presence of direct causal relations between symptoms contradicts the essential assumptions that underlie psychology's main class of measurement models” (sect. 1, para. 5). This premise is asserted in the target article and references, but not formally demonstrated. Our concern is not whether the conceptualization is radically different. We are concerned that, without proper demonstration, the premise is potentially misleading.
Briefly, on a more basic issue, we think it worth making more explicit that latent variable models in Item Response Theory (IRT) require that inter-individual covariation between levels of a trait and item scores is “entirely attributable to the [latent variable]” (sect. 2, para. 3). We assume, then, that the authors' premise pertains specifically to intra-individual processes, given that in a source cited in the article, Borsboom (Reference Borsboom2008, p. 1101) questions whether symptoms are effects of a common cause “at the level of the individual person.” In asserting their premise, the authors focus specifically on the assumption of local independence. Borsboom (Reference Borsboom2008) uses the analogy of temperature to explain local independence, likening the reading of thermometers to indicators of latent variables. For a fixed temperature, there is no covariation among the readings. Borsboom (Reference Borsboom2008, p. 1099) says “the same implication exists in latent variable models, where this property is called local independence (‘local’ in the sense that one position on the attribute is considered at a time, and ‘independence’ because the indicators are statistically independent in the subpopulation of people who occupy this position).” As Borsboom explicitly states, this is an implication of local independence: See Lord and Novick (Reference Lord and Novick1968, p. 361) for a formal definition.
Expressed in these terms, the implication is that for a subpopulation at a single position there should be no inter-individual covariation among item responses. However, to argue that this lack of covariation precludes causality would be to risk arguing by selective observation. The reason is that IRT models require that an individual's responses to items are correlated with the locations of the items on the latent variable. One of the pioneers of IRT, Rasch, explained this implication of his dichotomous IRT model by showing there were positive correlations between item estimates and the log-odds of correct responses to the items, for groups of respondents with similar raw scores (see Rasch Reference Rasch1960/1980, Figure 7, p. 89). He treated the raw score groups as relatively homogeneous with respect to the latent variable. Thus, even at the level of the individual, local independence does not preclude covariation, attributable to a latent variable, among responses. One cannot, therefore, argue that local independence precludes causation at the level of the individual on the basis that no such covariation may exist. Whether there is a plausible scientific basis for proposing that one or more latent variables cause the relations among indicators is a separate matter, which is justifiably raised in the target article and references. However, it would be unfortunate if readers are led to believe that local independence necessarily means one cannot apply existing psychometric theory if it is posited that intra-individual relations among symptoms are due to a latent variable.
Moreover, a body of work in IRT focuses on polytomous models for items with explicitly dependent categories, such as categories that form a rating scale. These models accommodate direct dependence between the response categories that would otherwise violate the assumption of local independence (Andrich Reference Andrich and Embretson1985; Reference Andrich, Alagumalai, Durtis and Hungi2005; Verhelst & Verstralen Reference Verhelst and Verstralen2008). It may be possible to develop methods to apply these models where there are direct causal relations among symptoms.
In summary, first, we contest that the network perspective proposed by Cramer et al. nullifies the need to invoke latent variables. Second, we argue for the need to thoroughly examine whether existing models can be applied before calling for a new “general psychometric theory” (sect. 8, para. 9).
You have
Access
Cramer et al. argue that we have been looking at mental disorders through the wrong psychometric lenses, and that we need a general psychometric theory of disorders as networks. Their article is valuable in as much as it raises the possibility of connecting psychometric theory to network models. However, we focus on two points of contention. Before doing so, we stress agreement on an overarching point: There is insufficient evidence that any mental disorder is a single continuous latent variable. As stated by Borsboom et al. (Reference Borsboom, Mellenbergh and Van Heerden2004, p. 1063), the “ontological position that the attribute being measured exists and affects the outcome of the measurement procedure” is a strong scientific claim that is difficult, but necessary, to establish (see Michell Reference Michell1999).
Having said that, our first point of contention is this: We do not think the authors convincingly argue that the network perspective “nullifies the need to invoke latent variables as an explanation of the covariance between symptoms” (sect. 1, para. 5). The authors recognize that “non-symptom causal processes” like homeostasis may mediate and “partly explain” relations between symptoms (sect. 2, para. 9). However, a network model needs to explicitly incorporate such causal mechanisms to form the basis of a scientific theory, and this would make causal mechanisms fundamental, not incidental. Thus, Holland (Reference Holland1998, p. 132) refers to mechanisms and their defining transition functions as the primitives of a network. In Kauffman's (Reference Kauffman1993) reaction network model of autocatalytic sets, for example, nodes may represent molecules, and edges may represent reactions: chemical reactions are the causal relations in the network. Without causal mechanisms, it is not clear how the network presented in the target article is anything more than a “method to visualize” sets of symptoms defined by the statistical associations in the empirical data (sect. 3). It would seem that, as soon as one invokes causal mechanisms, one must invoke latent variables like psychological and physiological states. Thus, we question whether the network perspective nullifies the need to invoke latent variables, in some form, to explain covariation among symptoms.
Our second point of contention is that Cramer et al. have not formally demonstrated the technical premise for claiming their network perspective is a radically different conceptualization to the latent variable perspective. The premise is that “the presence of direct causal relations between symptoms contradicts the essential assumptions that underlie psychology's main class of measurement models” (sect. 1, para. 5). This premise is asserted in the target article and references, but not formally demonstrated. Our concern is not whether the conceptualization is radically different. We are concerned that, without proper demonstration, the premise is potentially misleading.
Briefly, on a more basic issue, we think it worth making more explicit that latent variable models in Item Response Theory (IRT) require that inter-individual covariation between levels of a trait and item scores is “entirely attributable to the [latent variable]” (sect. 2, para. 3). We assume, then, that the authors' premise pertains specifically to intra-individual processes, given that in a source cited in the article, Borsboom (Reference Borsboom2008, p. 1101) questions whether symptoms are effects of a common cause “at the level of the individual person.” In asserting their premise, the authors focus specifically on the assumption of local independence. Borsboom (Reference Borsboom2008) uses the analogy of temperature to explain local independence, likening the reading of thermometers to indicators of latent variables. For a fixed temperature, there is no covariation among the readings. Borsboom (Reference Borsboom2008, p. 1099) says “the same implication exists in latent variable models, where this property is called local independence (‘local’ in the sense that one position on the attribute is considered at a time, and ‘independence’ because the indicators are statistically independent in the subpopulation of people who occupy this position).” As Borsboom explicitly states, this is an implication of local independence: See Lord and Novick (Reference Lord and Novick1968, p. 361) for a formal definition.
Expressed in these terms, the implication is that for a subpopulation at a single position there should be no inter-individual covariation among item responses. However, to argue that this lack of covariation precludes causality would be to risk arguing by selective observation. The reason is that IRT models require that an individual's responses to items are correlated with the locations of the items on the latent variable. One of the pioneers of IRT, Rasch, explained this implication of his dichotomous IRT model by showing there were positive correlations between item estimates and the log-odds of correct responses to the items, for groups of respondents with similar raw scores (see Rasch Reference Rasch1960/1980, Figure 7, p. 89). He treated the raw score groups as relatively homogeneous with respect to the latent variable. Thus, even at the level of the individual, local independence does not preclude covariation, attributable to a latent variable, among responses. One cannot, therefore, argue that local independence precludes causation at the level of the individual on the basis that no such covariation may exist. Whether there is a plausible scientific basis for proposing that one or more latent variables cause the relations among indicators is a separate matter, which is justifiably raised in the target article and references. However, it would be unfortunate if readers are led to believe that local independence necessarily means one cannot apply existing psychometric theory if it is posited that intra-individual relations among symptoms are due to a latent variable.
Moreover, a body of work in IRT focuses on polytomous models for items with explicitly dependent categories, such as categories that form a rating scale. These models accommodate direct dependence between the response categories that would otherwise violate the assumption of local independence (Andrich Reference Andrich and Embretson1985; Reference Andrich, Alagumalai, Durtis and Hungi2005; Verhelst & Verstralen Reference Verhelst and Verstralen2008). It may be possible to develop methods to apply these models where there are direct causal relations among symptoms.
In summary, first, we contest that the network perspective proposed by Cramer et al. nullifies the need to invoke latent variables. Second, we argue for the need to thoroughly examine whether existing models can be applied before calling for a new “general psychometric theory” (sect. 8, para. 9).