We agree with the view that Cramer et al. develop in the target article: that naïve latent variable models often fall woefully short of ideal. Unfortunately, their proposed solution and accompanying test case suffer from a number of flaws.

Cramer et al. begin with a straw man: They assume that, in a latent variable model, symptoms cannot also influence one another. Unless we define “latent variable model” to exclude such effects, there is no reason to impose such a constraint on our models. Mathematically, it is straightforward for latent variable models to have both latent common causes of measured variables and direct influences of measured variables on other measured variables. This is often the case for actual causal structures; for example, when there is confounding in observational or quasi-experimental studies.

Cramer et al. further claim that a “latent variable model renders all symptoms equally central and thus exchangeable” (sect. 5, para. 3). This claim is difficult to understand. “Central” is neither a causal nor a statistical notion; “exchangeable” is a statistical notion that, if meant, would be quite inappropriate in this usage. Cramer et al. might mean that in latent variable models all symptoms have the same variance, or the same dependence on any latent variables, or in their probability distributions conditional on values of latent variables, or in their probabilities conditional on one another. Each of these claims is violated in many latent variable models in the social sciences and elsewhere, and all of these claims are false unless “latent variable model” is arbitrarily defined so as to satisfy them. But that would be to focus on a model class that no one ought to accept a priori in the first place.

Cramer et al. focus on a stiff but appropriate standard for (their version of) latent variable models: The models should get the causal relations right. Unfortunately, they do not apply that same standard to their favored alternative. Instead, they resort to representing simple associations, with occasional suggestions that the relationships they so specify could be causal (e.g., sleep deprivation causes tiredness). Simple associations cannot, in general, be used reliably to estimate causal relations: they ignore possible screening off (conditional independence) relations, measurement errors, and latent confounding, and they give no direction to causal relations when they exist. Cramer et al. focus on two extreme model classes and ignore the enormous space of (learnable and estimable) models that lie between these poles; those models can include all of latent variables, direct causal connections, and feedback cycles (of varying speeds).

Consider instead searching for graphical causal models from their data. Absent latent variables, graphical causal models – both cyclic and acyclic – specify conditional independence relations. These relations can be used to search for cyclic and acyclic causal models. The theory of cyclic graphical models is difficult and underdeveloped, and for binary variables no adequate search procedure is available, but the target article does not engage what is known (e.g., Lacerda et al. Reference Lacerda, Spirtes, Ramsey, Hoyer, McAllester and Nicholson2008; Pearl & Dechter Reference Pearl, Dechter, Horvitz and Jensen1996; Richardson Reference Richardson, Horvitz and Jensen1996). For acyclic graphs, there are many correct search algorithms (e.g., PC, Spirtes & Glymour Reference Spirtes and Glymour1991; FCI, Spirtes et al. Reference Spirtes, Glymour and Scheines1993; Conservative PC or CPC, Ramsey et al. Reference Ramsey, Zhang, Spirtes, Dechter and Richardson2006; Greedy Equivalence Search [GES], Meek Reference Meek1997). The PC algorithm, for example, is an asymptotically pointwise consistent search procedure under independent and identically distributed (i.i.d.) sampling when there are no latent variables and the structure is acyclic. The PC algorithm will sometimes return double-headed arrows; asymptotically, the appearance of such structures in the PC output indicates latent common causes of the connected variables. We can at least begin to explore the possibilities with PC.

The data that Cramer et al. use to illustrate their approach are missing more than 70% of the possible values, and there is no explanation in their article of how those missing values are treated. There are several possibilities: A search can be conducted using only cases with no missing values, but that would include only about 10% of the cases; missing values can be replaced at random according to some prior distribution; missing values for a variable can be replaced using a probability distribution equal to the frequency distribution of that variable in the available data; or, as in the PC algorithm, relevant statistics can be computed using available data and ignoring missing values. For the data Cramer et al. provide, and a .05 alpha value for conditional independence decisions, the PC algorithm yields the graph in Figure 1:

Figure 1.

PC output for Cramer et al.'s data

The variables form three distinct clusters: The larger two correspond to the two focal diagnostic categories, connected only by the two measures of sleep. The MDD (major depressive disorders network) measures form two disconnected components. Four variables (gSleep, gConc, gFatig, gIrri) form a chain of double-headed arrows, suggesting that there may be an unobserved common cause. The BPC (Build Pure Clusters) search algorithm (Silva et al. Reference Silva, Scheines, Glymour and Spirtes2006) estimates whether a set of variables shares a latent common cause; BPC is asymptotically correct for binary variables whose values are two-valued projections of a Gaussian distribution. BPC finds that gSleep, gConc, gFatig, and gIrri do have a latent common cause, as do a separate cluster of MDD measures (mRep, mRest, mSuic). Similar results are found if missing values are replaced using the base frequency of each variable value.

This analysis is scarcely complete. For example, were the data good enough to warrant it, one could apply the FCI (Fast Causal Inference) algorithm (Spirtes et al. Reference Spirtes, Glymour and Scheines1993), which is correct when there are both latent common causes and direct influences of measured variables on one another.

In our view, Cramer et al. unnecessarily restrict the modeling options, do not offer a plausible, reliable method for causal inference, and fail to explore what the data from their own example might reveal.