1. Introduction
The architects of the third edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM) are often held to have sacrificed validity for reliability in constructing the manual’s categories (Kendell and Jablensky Reference Kendell and Jablensky2003; Andreasen Reference Andreasen2007). According to this view, the DSM went wrong around 1980 when it adopted an operationalist stance focusing on atheoretical observational criteria, an ecumenical approach that made it easier to apply diagnoses consistently across practitioners and contexts. With etiological explanations being so discouraged, the argument continues, the real contours of psychopathology have not been demarcated, and psychiatry has not been able to identify disease entities akin to those in the rest of medicine. The solution to psychiatry’s validity crisis, it has been suggested, is to refocus psychiatric research on causal mechanisms (Murphy Reference Murphy2006; Kendler, Zachar, and Craver Reference Kendler, Zachar and Craver2009).
In what follows I argue that while the DSM’s operationalism hinders progress toward explanations of psychopathology, it does so primarily in an indirect manner. More than its historical rejection of etiology, it is the manner in which the manual is employed in the research setting that makes the discovery of underlying mechanisms difficult. Owing to its widespread use in the framing of scientific hypotheses about mental disorder, the DSM has largely determined the objects of psychiatric inquiry. In particular, its diagnostic criteria are widely used to gather test populations for psychiatric studies. When the DSM is employed in this way, the implicit assumption is that the operationalized criteria for diagnosing clinical types will also successfully pick out populations about which relevant biomedical facts can be discovered. I will refer to this as the assumption of diagnostic discrimination.
The first aim of this article (section 2) is to make explicit the role of the assumption of diagnostic discrimination in psychiatric research. My second aim is to argue for pessimism about the likelihood of diagnostic discrimination in biomedical psychiatry, on historical (section 3) and methodological grounds (section 4). I show how the assumption is successfully rejected in the Research Domain Criteria (RDoC) project, a new classification tool for psychiatric researchers introduced by the National Institute of Mental Health (NIMH). Finally, in section 5 I consider three possible rebuttals.
2. What Is the Assumption of Diagnostic Discrimination?
While the absence of valid categories in psychiatry is often commented on, there is little consensus about what the term “validity” means in the psychiatric context (Olbert Reference Olbert2014). The term originates in psychometrics, where it is used to calibrate how well a test measures what it is intended to measure. Since diagnostic categories can be said to be measurement instruments only in a loose sense (Blashfield and Livesley Reference Blashfield and Livesley1991), psychiatrists tend to speak of validity instead as an attribute of inferences made about purported disease entities. This move is eased by the fact that for most psychiatric diagnoses, the DSM category serves as both measure and definition (Van Loo and Romeijn Reference Van Loo and Romeijn2015). Historically, the term “valid” has referred to psychiatric constructs that are “a reflection of the true state of nature” and has been evaluated through the presence of validators such as characteristic course, family aggregation, genetic abnormalities, or neural mechanisms (APA 2000, 10). Under the dominant biomedical paradigm in psychiatry, a valid diagnostic construct is one that categorizes patients who all share the same underlying physiological dysfunction.
Critics of the DSM like to point out that so far, no complete mechanistic account of a mental disorder has been satisfactorily demonstrated (Kapur, Phillips, and Insel Reference Kapur, Phillips and Insel2012). My question here does not concern whether diagnoses have been validated in this sense, but rather whether the categories of the DSM, when used as instruments to collect test populations for research purposes, successfully amass patients about whom relevant facts can be gathered. Rather than a metaphysical question about psychiatric kinds, mine is an epistemological question about whether the DSM’s constructs are appropriate targets for validation.
Optimists on this front are often committed to what I term the assumption of diagnostic discrimination, that is, the assumption that our diagnostic tests group patients together in ways that allow for relevant facts about mental disorder to be discovered. By “tests” I refer to the DSM itself or diagnostic screens based on its criteria. Diagnostic discrimination could be proposed about other diagnostic methods (e.g., the Psychodynamic Diagnostic Manual), but my focus here is on the DSM. For the purposes of this discussion, relevant facts are those about the underlying mechanisms causing the signs and symptoms with which patients present that count as a significant discovery within the experimental context. They are the sorts of validators that biomedical scientists hope to find: genetic signatures, neurological or cognitive dysfunctions, focal brain lesions, and so forth. Diagnostic discrimination may be a more or less justified assumption for those interested in other sorts of inferences, as I will consider briefly in section 5.1.
I borrow the term “diagnostic discrimination” from psychometrics, where it is defined as the statistical assessment of how a diagnostic test compares with a gold standard, measured by the test’s specificity, sensitivity, predictive value, and likelihood ratios (Knottnerus and Buntinx Reference Knottnerus and Buntinx2009, 4). Discrimination in this sense is currently inapplicable in psychiatry, which lacks tests by which to calibrate its diagnostic instruments (Van Loo and Romeijn Reference Van Loo and Romeijn2015). In my argument it is invoked as an aspirational term, signifying an ideal rather than a measure. I am interested in the particular epistemic stance that evinces optimism about whether our diagnostic categories effectively group together patients homogeneous with regard to the objects of interest for biomedical psychiatry. The extent to which the DSM’s criteria are discriminative for the purposes of biomedical research is, of course, an empirical question. My aim is not to offer any specific assessments, but rather to raise some concerns about the warrant for prima facie optimism about discrimination.
Unfortunately, critical attention to the problem of the DSM’s validity is entirely compatible with a naive commitment to diagnostic discrimination. Study designs that aim at revising the DSM’s categories may still use DSM criteria to select research samples, assuming that those samples will be homogeneous for the targeted pathogenic mechanisms. Even those profoundly dissatisfied with the DSM’s categories may nonetheless employ its criteria in order to locate latent constructs that they hope to use to revise and perfect the manual. The DSM’s central role in laboratory research is not only entrenched by historical precedent but also held firm by the hand of the biomedical marketplace; funding bodies have traditionally preferred research that is directly pertinent to perceived clinical needs. This has led to a focus on the iterative validation of diagnostic constructs in the pursuit of causal mechanisms that can undergird new therapies.
3. The Case for Pessimism: A Historical Argument
With somatic medicine as the benchmark, discriminative diagnoses have long been considered the ideal targets for validation in psychiatry (Klerman Reference Klerman and Shershow1978); the architects of the DSM-III prioritized the construction of diagnostic categories based on “distilled clinical research experience” as the “first and crucial taxonomic step” toward identifying valid constructs (Feighner et al. Reference Feighner, Robins, Guze, Woodruff, Winokur and Muñoz1972, 57). Influential theorists of psychiatric validity have imagined a bootstrapping model, in which the first phase of achieving validity involves settling on a clinical description of diagnostic kinds (Kendell and Jablensky Reference Kendell and Jablensky2003). Andreasen, for example, writes that only “once a reliable method is applied to define symptoms or delineate a potential diagnostic category or dimension of psychopathology” can “these variables then be validated by examining their relationship to external measures” (Reference Andreasen1995, 162).
Thus, the DSM has, historically, provided the independent variables for studies attempting to validate psychiatric constructs. However, the manual’s origins do not offer confidence that they will be discriminative in the relevant way for biomedical research. The aim of the first edition of the DSM, published in 1952, was to collect statistical, rather than biomedical, information. Despite the ideal of a scientifically objective system, diagnoses are historically embedded concepts, traceable to different strata of the discipline’s past—some of which long precede genetics and neuroscience.
Throughout the history of the manual, ambitious task forces have attempted to revise the DSM’s categories on the basis of contemporary methods and knowledge, rather than in the terms of decades-old census projects and nineteenth-century theory. While Feighner et al. sought to reground psychiatric nosology on empirical foundations, their criteria (which formed the template for the DSM-III) were in fact an amalgam of data and received clinical intuition, with many old taxonomic divisions being inherited unchallenged (Kendler and Muñoz Reference Kendler and Muñoz2010). Similarly, the main architects for the most recent revision, the DSM-5, announced the need to “transcend the limitations of the current DSM paradigm” so that the new DSM could provide research criteria “not constrained by the requirements of the neo-Kraepelinian categorical approach currently adopted” (Kupfer, First, and Regier Reference Kupfer, First and Regier2008, xxii). In the end, however, with some exceptions, the nosological structure remained stable.
The history of the DSM cannot support a general stance of optimism about diagnostic discrimination for biomedical research purposes. In the next section I argue that the structure of the DSM also gives reasons for pessimism, drawing on criticisms made by a growing number of psychiatric researchers that their disappointing failure to validate the DSM’s categories is due to the fact that there is nothing for them to validate. Or, to put these judgments about the ontology of psychiatric kinds in my own epistemological terms, the diagnostic tests for psychiatric constructs are not discriminative in the relevant sense, insofar as little of interest from the perspective of biomedicine can be discovered about patients sharing a diagnosis.
4. The Case for Pessimism: A Conceptual Argument
The first thing to be noted about the DSM’s structure is that if etiopathogenic facts about mental disorders are forthcoming, they will probably not stand in simple causal relationships to the DSM’s constructs. As of its third edition, the DSM’s categories have been polythetic, requiring patients to present with only n symptoms out of a longer list in order to meet the threshold for a given disorder. This has allowed diagnostic criteria to cast wider nets and for reliability to be improved. But as a result, the DSM’s criteria allow for incredible diversity. For example, the DSM-5 permits patients to be diagnosed with posttraumatic stress disorder if they present with any one of 636,120 possible profiles (Olbert, Gala, and Tupler Reference Olbert, Gala and Tupler2014).
Relevant facts will explain this diversity either by revealing homogeneity beneath promiscuous clinical descriptions or by offering disjunctive accounts of the mechanisms that undergird them. While it is not the norm in somatic medicine, symptomatic variation can be found among patients sharing a diagnosis—lupus is one example—so heterogeneity on its own does not prove that the DSM’s diagnoses are not discriminative (Olbert Reference Olbert2014). But the lack of compelling confirmation of psychiatry’s taxonomic boundaries by genetics, epidemiology, neurophysiology, and other allied sciences is worrying, raising the question of whether the manual is useful for anything more than identifying phenotypic clusters (Meehl Reference Meehl, Millon and Klerman1986). Taxometric and epidemiological studies reveal that the enormous heterogeneity in symptoms and course actually contains recognizable subtypes that appear more frequently than others; however, underlying differences in causal pathways or mechanisms that could explain these trends have not been found (Nandi, Beard, and Galea Reference Nandi, Beard and Galea2009).
Recently, a new round of critics has suggested that the heterogeneity of test populations collected on the basis of DSM diagnostic criteria is working against the discovery of laboratory markers for psychopathology (Hyman and Fenton Reference Hyman and Fenton2003). The view is that explanations that facilitate intervention and recovery are better found at other levels—for example, the level of the symptom, the gene, or the neural mechanism. Sanislow et al. have written that “dependence on conventional nosologies leaves the enterprise of understanding mechanisms of psychopathology in the awkward position of assuming the validity of single disorders and organizing research accordingly” (Reference Sanislow, Pine, Quinn, Kozak, Garvey, Heinssen, Wang and Cuthbert2010, 2). But this is not quite right. The validity of the diagnostic construct is not taken for granted in such cases, but rather is the object of investigation. The point is better put in terms of diagnostic discrimination: the assumption that populations delineated by DSM categories are ripe for validation according to current biomedical standards.
In response to these criticisms, the NIMH has constructed a new classification system that allows researchers to apply for funding without structuring their studies around DSM categories. Under the RDoC rubric, psychiatric investigators present their experiments as targeting fundamental components of mental functioning (or “research domains”) that are drawn from allied sciences, instead of using DSM constructs. Research domains contribute one axis to the proposed matrix, which is subdivided into more specific “constructs”—for example, “reward valuation,” “performance monitoring,” or “attachment formation and maintenance.” The other axis is “units of analysis,” ranging from “genes” to “behavior.”
By encouraging the funding of research that investigates certain research domains at certain units of analysis, RDoC changes the targets of validation from “clinical endpoints that have remained unchanged for decades” to any sort of phenomenon relevant to psychopathology that may be viewed either as an extreme on a spectrum of human variation or as a dysfunctional structure or process (Hyman and Fenton Reference Hyman and Fenton2003, 351). Rather than immediately impacting diagnostics, RDoC aims to encourage a profound shift in the way research samples are conceived of and assembled. In some cases, the translational approaches encouraged by the NIMH require the study of mechanisms that cut across traditional diagnostic categories, and RDoC researchers can gather whatever populations are pertinent to their domain of interest. So, for example, a group researching fear circuitry (construct of interest: fear/acute threat; domain: negative valence systems; unit: circuits) might use as their test population patients seeking medical help for anxiety, regardless of whether they meet any specific diagnostic criteria.
Of interest here is that in order to liberate psychiatric research from the constraints of an unhelpful taxonomy, the NIMH has placed its bets for discrimination of research targets beyond the pages of the DSM. Debates over which sorts of objects are most worthy of study may continue to be played out under RDoC through the distribution of funding dollars, but these judgments will be constrained by current epistemological and methodological commitments, rather than assumptions about the merit of inherited classifications. In contrast, when the DSM is used to design experimental protocols and present them to funding bodies, it can act as an epistemic bottleneck (Hyman and Fenton Reference Hyman and Fenton2003), restricting research that cross-cuts or challenges existing diagnostic boundaries and excluding innovative explanatory approaches. If the DSM’s categories are discriminative in the relevant sense, such a narrowing of focus is a boon to research. If not, the DSM is analogous to the lamppost in the tale of the man who makes the mistake of looking for his keys where the light is, instead of where he lost them.
5. Possible Defenses of Optimism
I have argued that the DSM may retard progress in psychiatry not merely by pursuing operationalized definitions over etiological ones but also by limiting the ability of researchers to make original valid inferences about the causes underlying psychiatric disorder.Footnote 1 This effect is due to the widely held but, I have argued, unjustified assumption in psychiatry that the manual’s categories are the appropriate grounds on which to draw test populations for research purposes. In this section I consider three possible objections to my argument. The first is that if warrant for diagnostic discrimination can be found in neither DSM’s history and structure nor biomedical psychiatry’s track record, it can be found in clinical practice. The second is that some assumptions about discrimination must be made, and that the bottlenecking effects that these assumptions have on progress are a necessary cost of doing science. The third is that by giving up on validating the DSM’s categories, psychiatry would lose track of its true targets, making the assumption of diagnostic discrimination an imperative for properly conducted psychiatric research.
5.1. The Clinical Case for Diagnostic Discrimination
It has been assumed that if clinicians are able to separate patients into discrete kinds based on their symptomology, there is good reason to anticipate that scientific validators will ultimately reinforce these divisions (Robins and Guze Reference Robins and Guze1970). However, it seems that many clinicians themselves do not believe that the DSM meaningfully sorts their patients. Studies of the actual usage of the manual suggest that clinicians find it primarily helpful for securing treatment options and mostly ignore its complex polythetic structure (First and Westen Reference First and Westen2007). Practitioners engage in diagnostic “bracket creep” to tweak coverage benefits and duck the restrictions that insurance companies put on their ability to utilize their expert judgment (Bowker and Star Reference Bowker and Star1999). Ethnographic research reveals that diagnoses often follow after treatment decisions, rather than guiding them (Whooley Reference Whooley2010, 461). If the manual’s ubiquity in clinical practice is due to its integral role in the larger machinery of industrial and corporate health care, rather than its accurate representation of clinical types, any argument for diagnostic discrimination on these grounds is unsound.
Further evidence that the manual’s diagnostic constructs do not accurately represent clinical concepts of disorder comes from the widespread alarm over the deprecation of phenomenology due to the DSM’s reductive approach (Andreasen Reference Andreasen2007). The DSM’s operationalized descriptions neglect the fact that “in addition to manifesting the relatively direct consequences of neurobiological abnormalities,” patients “react to their abnormalities in all kinds of ways that may sometimes require the categories of meaning and experience in order to be understood or explained” (Sass, Parnas, and Zahavi Reference Sass, Parnas and Zahavi2011, 16). Some phenomenologically oriented clinicians and philosophers of psychiatry have suggested that these experiential aspects of mental illness should themselves be targets for validation (Mishara and Schwartz Reference Mishara and Schwartz2010). Taken together, these criticisms suggest that the DSM categories do not reflect the clinical picture sufficiently to justify optimism about their utility in the research setting.
5.2. The Inevitability of Diagnostic Discrimination
Another possible objection is that the assumption of discrimination is inevitable in psychiatric investigation, and that the DSM is not (uniquely) culpable. Studies dividing subjects into groups must always depend on measures assumed to be discriminative for the construct in question. Strategies like RDoC, it could be argued, simply replace the diagnostic constructs of the DSM with other sorts of constructs, in this case the subcategories of its proposed domains.
My aim is not to dismiss the importance of discrimination in research, nor to suggest that psychiatry can or should do without constructs altogether, but rather to challenge the assumption that the DSM’s criteria are discriminative for biomedical research purposes. While RDoC also relies on constructs, its architects have emphasized that these constructs are, first, completely open to revision and, second, explicitly designed to be broad enough to include the major paradigms currently at play within psychiatric research. If the NIMH does not fulfill its promise to amend and expand the matrix’s research domain criteria in accordance with shifts in the field, it could well end up with calcified categories that restrict research in the way that the DSM’s have.
Notably, RDoC does not limit the conceivable objects of psychiatric research, which are not the same as the loci on its matrix. Rather than taxonomizing objects for psychiatric investigation, RDoC arranges domains of functioning in which such objects are located, providing for each a consensus definition and orienting researchers toward the available measures or elements across the units of analysis that could be used as variables for gathering populations for studies.Footnote 2 Accordingly, researchers have a significant amount of autonomy in the design of their research. As in all scientific research, their choice of construct and the tests they use to measure for it should be scrutinized closely through peer review.
5.3. The Value of Diagnostic Kinds for Psychiatric Research
A final objection worth considering is whether giving up on diagnostic kinds is worth it—whether the gains to research productivity that would come from having discriminative targets have too high an epistemological or ethical cost. It can be argued that keeping psychiatry focused on diagnoses is the best way to avoid the reduction of the mentally ill to their component parts, which neglects the phenomenological core of psychopathology (McLaren Reference McLaren2011). The NIMH has made little secret of its preference for analysis at the level of brain circuits, based on the reasoning that it is at this level that science is most rapidly gaining insight into the underlying correlates of behavior (Insel Reference Insel2013). However, this approach has garnered accusations that RDoC is “mindless” (Frances Reference Frances2013), that is, symptomatic of “the profession’s intent to complete its abandonment of the mind as the localization and source of our suffering” (Greenberg Reference Greenberg2013, 342).
In response, Bolton (Reference Bolton2013) has argued that the NIMH’s claim that “all mental diseases are brain diseases” need not be reductionistic, insofar as the brain can be seen as integrated into a complex network of causal relations that extend beyond the individual. Advocates of the RDoC framework suggest that it might give empirical grounding to psychotherapeutic as well as pharmotherapeutic approaches (Morris and Cuthbert Reference Morris and Cuthbert2012, 31). However, especially in light of the NIMH’s increasingly enthusiastic pursuit of basic science even as “fundamental and important questions regarding health services, psychosocial treatments, conceptual issues, public health, and patient initiatives remain marginally funded,” it remains to be seen whether the NIMH will be truly ecumenical in the distribution of research dollars across the columns of their matrix (Sadler Reference Sadler, Paris and Phillips2013, 29).
The RDoC project’s purported reductionism differs in an important way from the epistemic bottleneck of the DSM, however, insofar as it increases the conceptual and methodological distance between the laboratory and the clinic, rather than collapsing it. Solomon (Reference Solomon, Kendler and Parnas2014) has argued that while expert disagreement can be generative in science, stable consensus is to be valued in medicine, where the loss of epistemological authority can be dangerous. Her claims are vindicated by the widely expressed view that the minor modifications in each new edition of the DSM, resulting from rapidly shifting professional compromises, can be harmful to patients (Frances Reference Frances2009).Footnote 3 If the pretense is abandoned that psychiatry’s scientific and practical objects are one and the same, the fits and starts of biomedical research need not immediately impact clinical nosology. As Kenneth Schaffner has suggested, clinical research might continue to make progress on refining our understanding of psychopathology at “higher levels of aggregation,” while projects facilitated by RDoC work to reveal the “many-many relations” that make validity such a challenge (Reference Schaffner, Kendler and Parnas2012, 184). However, if the DSM stops playing its role as an epistemic hub (Kutschenko Reference Kutschenko2011), the integration of psychiatric knowledge into therapeutics will need to be reimagined—a project well beyond the scope of this article.
6. Conclusion: Implications for Philosophy of Psychiatry
There has recently been much effort to resolve the metaphysical nature of mental disorders, taken as a class, with diverse viewpoints declaring them to be constructed, human, practical, natural, or homeostatic property cluster kinds (Kendler et al. Reference Kendler, Zachar and Craver2009; Kincaid and Sullivan Reference Kincaid and Sullivan2014). Insofar as the objects of diagnostic tests can be seen as either theoretical constructs or real entities, both realists and instrumentalists can beg the question of diagnostic discrimination, and a focus on kindhood has drawn philosophical attention away from the heterogeneity of psychiatric objects. Since psychiatrists are increasingly pursuing piecemeal causal explanations about constructs beyond the level of the diagnosis, philosophers should follow Kincaid (Reference Kincaid2008) in leaving the question of diagnostic kindhood behind. Instead, I endorse a modest conventionalism about psychiatric classification (Van Loo and Romeijn Reference Van Loo and Romeijn2015) and a turn of attention to the ways in which psychiatry stabilizes its diverse objects across disciplinary boundaries in the absence of the DSM’s authoritative voice (Sullivan Reference Sullivan, Kincaid and Sullivan2014).
