Introduction
Preparations for the publication of the Diagnostic and Statistical Manual of Mental Disorders (DSM), 5th edition (DSM-V) are underway. Many suggestions to change criteria for existing diagnoses are likely to be considered. While the principles guiding criteria revision have not been clearly explicated, we believe that existing diagnostic criteria should be revised when a conceptual problem has been identified, or a more valid or simpler method of defining the disorder has been developed. When a more valid or simpler definition has been proposed, we also believe that no changes to the criteria should be made in the absence of replication of initial findings.
The symptom inclusion criteria for the diagnosis of major depressive disorder (MDD) have remained essentially unchanged during the past 35 years. The nine symptom criteria for primary affective disorder enumerated by the Washington University group (Feighner et al. Reference Feighner, Robins, Guze, Woodruff, Winokur and Munoz1972) were retained, albeit with slight modification, in the Research Diagnostic Criteria (Spitzer et al. Reference Spitzer, Endicott and Robins1978), DSM-III (APA, 1980) and subsequent editions of the DSM (APA, 1987, 1994). The field trial for DSM-IV did not study possible changes to the MDD criteria (Keller et al. Reference Keller, Klein, Hirschfeld, Kocsis, McCullough, Miller, First, Holzer, Keitner, Marin and Shea1995; Rush, Reference Rush, Widiger, Frances, Pincus, Ross, First, Davis and and Kline1998). In fact, there are only a few psychometric studies of the MDD criteria (Breslau & Davis, Reference Breslau and Davis1985; Buchwald & Rudick-Davis, Reference Buchwald and Rudick-Davis1993; Zimmerman et al. Reference Zimmerman, McGlinchey, Young and Chelminski2006b).
The reason for even considering a change in the DSM-IV symptom criteria for MDD after all these years is that there are two practical problems with these criteria – they are somewhat lengthy and there are difficulties in applying some of the criteria in patients with co-morbid medical illnesses because of symptom non-specificity. Studies have identified significant gaps in the knowledge or application of the MDD criteria. Bowers et al. (Reference Bowers, Jorm, Henderson and Harris1992) interviewed experienced general practitioners in Australia regarding the signs and symptoms of depression looked for when a patient presents for depression. None of the physicians listed more than six of the nine MDD symptom criteria and only one-third reported more than three symptoms. In a large survey of 2500 Australian general practitioners who were asked to list which symptoms they used to diagnose depression, only one-quarter listed at least five MDD criterion symptoms (Krupinski & Tiller, Reference Krupinski and Tiller2001). Even after an educational program, only two-thirds of residents in obstetrics and gynecology indicated that they used the formal diagnostic criteria (Learman et al. Reference Learman, Gerrity, Field, van Blaricom, Romm and Choe2003) though this was significantly higher than the 38% rate prior to the educational program. In a study of third-year internal medicine residents' knowledge of the MDD diagnostic criteria, only five of the nine criteria were reported by more than 50% of the physicians in response to the open-ended question, ‘What are the symptoms of a major depressive episode enumerated in DSM-IV?’ and only-third of the residents listed five or more of the nine MDD symptom criteria (Medow et al. Reference Medow, Borowsky, Dysken, Hillson, Woods and Wilt1999). In another study of first-, second- and third-year medical, psychiatry and clinical psychology residents, Rapp & Davis (Reference Rapp and Davis1989) found that only two of the nine criteria were listed by more than 50% of the medical residents. The residents in psychiatry and clinical psychology were better able to recall the MDD symptom criteria, though only five of the nine criteria were listed by at least 50% of the psychiatry and psychology residents. Gerrity et al. (Reference Gerrity, Cole, Dietrich and Barrett1999) examined the impact of a depression education program on primary care physicians' knowledge about depression and their behavior towards depressed patients. Two actors presented unannounced in the physicians' practices as standardized patients with MDD. In the control group, representing usual clinical practice, at least five criteria for MDD were assessed in only one-third of the patient encounters. In the intervention group, at least five criteria were assessed in 70% of the encounters. Thus, the education program significantly increased the likelihood that primary care physicians determined whether patients met the DSM-IV MDD symptom criteria, though a significant minority of physicians still did not do so after the educational program. In a survey of psychiatrists' reported use of the DSM-IV criteria, Zimmerman & Galione (in press) found that even experienced psychiatrists reported that they often do not determine if the MDD criteria are met when diagnosing depression.
A second problem with the MDD symptom criteria occurs with their use in patients with medical co-morbidities (Kathol et al. Reference Kathol, Noyes, Williams, Mutgi, Carroll and Perry1990b; Chochinov et al. Reference Chochinov, Wilson, Enns and Lander1994; Cavanaugh, Reference Cavanaugh1995; Akechi et al. Reference Akechi, Nakano, Akizuki, Okamura, Sakuma, Nakanishi, Yoshikawa and Uchitomi2003). Somatic criteria such as fatigue, appetite disturbance and sleep disturbance may be sequelae of medical illnesses rather than depression. Because of this possible symptom contamination, alternative criteria sets have been proposed that substitute affective and cognitive symptoms for vegetative ones (Kathol et al. Reference Kathol, Mutgi, Williams, Clamon and Noyes1990a; Chochinov et al. Reference Chochinov, Wilson, Enns and Lander1994; Koenig et al. Reference Koenig, George, Peterson and Pieper1997).
In a previous report from the Rhode Island Methods to Improve Diagnostic Assessment and Services (MIDAS) project we developed a briefer list of the symptom criteria of MDD that was composed entirely of the DSM-IV mood and cognitive symptoms: low mood, loss of interest or pleasure, guilt/worthlessness, impaired concentration/indecision and suicidal thoughts (Zimmerman et al. Reference Zimmerman, Chelminski, McGlinchey and Young2006a). The simplified definition did not include the DSM-IV somatic/vegetative symptoms which are more difficult to evaluate in physically ill patients. After determining the cut-off score that maximized concordance with the original DSM-IV definition, high levels of agreement between the simplified and original definitions of MDD were found in the initial derivation sample and a cross-validation sample. The goal of the present study from the MIDAS project was to replicate these findings in another large sample of psychiatric out-patients, and to extend the findings to a sample of patients presenting for the treatment of pathological gambling and a sample of candidates for bariatric surgery.
Method
The Rhode Island MIDAS project represents an integration of research methodology into a community-based out-patient practice affiliated with an academic medical center (Posternak et al. Reference Posternak, Zimmerman and Solomon2002; Zimmerman et al. Reference Zimmerman, Mattia and Posternak2002; Zimmerman, Reference Zimmerman and First2003). To date, 2900 psychiatric out-patients have been evaluated with a semi-structured diagnostic interview in the Rhode Island Hospital Department of Psychiatry out-patient practice. The first 1800 patients were included in our initial study of a simpler MDD definition (Zimmerman et al. Reference Zimmerman, Chelminski, McGlinchey and Young2006a) and are not included in the present report. Thus, the present sample consisted of 1100 psychiatric out-patients. The data in Table 1 show that the majority of the patients were white (87.9%), female (60.2%), married (42.4%) or single (32.0%) and graduated high school (61.5%). The most frequent current DSM-IV diagnoses were MDD (40.8%), social phobia (24.7%), generalized anxiety disorder (17.7%) and panic disorder (17.0%).
Demographic characteristics of the three samples
GED, General Equivalency Diploma; s.d., standard deviation.
The second sample consisted of 210 patients presenting to the Rhode Island Gambling Treatment program. The majority of these patients were white (91.0%), male (55.7%), married (46.2%) or divorced (19.0%) and graduated high school (63.4%) (Table 1).
The third sample consisted of 1200 candidates for bariatric surgery who were evaluated with the same interview schedule as the other subjects. The majority of these patients were white (80.3%), female (84.2%), married (52.4%) or single (22.3%) and graduated high school (73.2%) (Table 1).
All patients were interviewed by a diagnostic rater who administered the Structured Clinical Interview for DSM-IV (SCID; First et al. Reference First, Spitzer, Gibbon and Williams1995). Because we were interested in the psychometric performance of the DSM-IV symptom criteria for MDD we modified the SCID and eliminated the skip-out that curtails the depression module for patients who did not report either depressed mood or loss of interest or pleasure. Thus, we inquired about all of the symptoms of depression for all patients. For compound criteria that encompass more than one symptom (e.g. indecisiveness or impaired concentration; increased sleep or insomnia) we made separate ratings of each component of the diagnostic criterion. Thus, the nine DSM-IV symptom criteria were broken down into 17 separate items. The Rhode Island Hospital institutional review committee approved the research protocol and all patients provided informed, written consent.
As an ongoing part of the MIDAS project, joint-interview diagnostic reliability information was collected on 48 participants. The reliability coefficients of the symptoms of depression ranged from 0.54 to 0.94 (mean κ=0.80).
Data analysis
Previously we developed a simpler definition of MDD exclusive of somatic symptoms with the goal of maximizing concordance with the current DSM-IV definition (Zimmerman et al. Reference Zimmerman, Chelminski, McGlinchey and Young2006a). (Technically, our previous and the current research have focused on the symptom criteria of a major depressive episode. For stylistic simplicity we refer to this as a simpler definition of MDD.) We approached the development of a new definition in six ways, each of which yielded comparable results (agreement rates with the original DSM-IV definition ranged from 92.6% to 95.4%). In the present report we examined the simplest of these definitions: at least three of the following five symptoms are present (low mood, loss of interest, guilt or worthlessness, impaired concentration or indecisiveness, and death wishes or suicidal thoughts), one of which is low mood or loss of interest. In the original report, the level of agreement between this definition and the DSM-IV definition of MDD was 93.7% in the derivation sample and 94.0% in the cross-validation sample. For each of the three samples studied in the present report we computed sensitivity, specificity, overall level of agreement and the chance-corrected level of agreement.
In our original report we excluded the patients who had MDD that was in partial remission because we were unsure if these patients should be placed in the depressed or not depressed group, and we were also unsure if a discrepancy between alternative approaches to classifying these patients should be considered diagnostic error. To illustrate how a patient might be considered in partial remission according to one definition but not another, consider a case in which a patient developed a full depressive episode 6 months prior to the evaluation but for the past 2 months has only experienced the following four symptoms of depression: depressed mood, guilt, anhedonia and death wishes. Such a patient would be considered as having MDD in partial remission according to the DSM-IV criteria, but would fully meet the symptom criteria of our simpler definition of MDD. After further reflection, we decided that all patients should be included in the analysis. Thus, a patient diagnosed with MDD in partial remission according to the DSM-IV criteria who met the case-defining threshold for the simpler definition, would be counted as a disagreement between the two diagnostic approaches. In the initial report we also excluded from the analysis the patients who met the symptom inclusion criteria for a major depressive episode but not the exclusion criteria because application of the exclusion criteria would result in agreement between alternative definitions even if the symptom algorithms disagreed as to whether a patient was a case. In the present report we focused on agreement between the symptom algorithms of a major depressive episode regardless of whether the exclusion criteria would exclude a diagnosis of MDD. Thus, a discrepancy between the symptom algorithms would be counted as a disagreement even if the exclusion criteria were met.
Results
In the psychiatric out-patient sample the overall level of agreement between the simplified and DSM-IV definition of MDD was 91.8% (Table 2). Comparably high levels of concordance were found in gamblers (94.3%) and candidates for bariatric surgery (99.1%). The chance-corrected level of agreement was above 0.80 in each sample, and both the sensitivity and specificity of the new, simpler definition was above 80% in each sample. Across all 2510 patients the level of agreement between the simplified and DSM-IV definition was 95.5% and the κ coefficient was 0.87.
Concordance between a simpler definition of major depressive disorder and the DSM-IV symptom criteria for major depression in three samples
DSM-IV, Diagnostic and Statistical Manual of Mental Disorders, 4th edn.
Discussion
After eliminating the four somatic criteria from the DSM-IV definition of MDD, leaving the five mood and cognitive features, a high level of concordance was found between this simpler definition of MDD with the original DSM-IV classification. This new definition offers two advantages over the DSM-IV definition – it is briefer and therefore more likely to be recalled and applied in clinical practice, and it is free of somatic symptoms, thereby making it easier to apply with medically ill patients.
The high level of concordance between the simpler and DSM-IV definitions of MDD has now been replicated five times. In our initial derivation study, based on a sample of 805 psychiatric out-patients, the cut-off of three criteria was established for the five-item definition, and the concordance rate with the DSM-IV definition was 93.7% (κ=0.87) (Zimmerman et al. Reference Zimmerman, Chelminski, McGlinchey and Young2006a). The initial results were replicated in a sample of 789 psychiatric out-patients, and similarly high levels of agreement were found between the simpler and DSM-IV definition of MDD (overall agreement=94.0%, κ=0.88) (Zimmerman et al. Reference Zimmerman, Chelminski, McGlinchey and Young2006a). Andrews et al. (Reference Andrews, Slade, Sunderland and Anderson2007) examined agreement between the simpler definition and the original DSM-IV criteria in the Australian National Survey of Mental Health and Well-Being. In the first analysis of all 10 641 respondents they found an overall agreement rate of 99.6% with a sensitivity of 92.9%, specificity of 99.8% and κ of 0.93. They conducted a second analysis after limiting the sample to the 1013 subjects who met criteria for a current mood, anxiety or substance-use disorder in order to increase the sample's correspondence to a clinical sample and found nearly identical results (sensitivity=92.9%, specificity=98.8%, overall agreement=96.8%, κ=0.93). In the present study, we found high levels of concordance between the simpler definition and the DSM-IV criteria in another large psychiatric out-patient sample, a sample of patients presenting for the treatment of pathological gambling and morbidly obese patients evaluated as candidates for bariatric surgery. Thus, high concordance between the new definition of MDD and the DSM-IV definition has been found in three psychiatric out-patient samples, a general population community sample, a sample of gamblers who often were depressed but did not present with depression as their primary complaint and a sample of obese subjects who often had medical co-morbidity.
These studies of concordance have not addressed the issue of validity. We did not attempt to derive a more valid symptom-based definition of MDD because we are pessimistic that current validation strategies will yield a clearly more valid definition. Any new definition of MDD is likely to overlap in large-part with an existing definition, and the high level of overlap will make it difficult to demonstrate superior validity. Illustrative of the difficulty in demonstrating improved validity, Kendler & Gardner (Reference Kendler and Gardner1998) examined the validity of different thresholds for defining MDD, and found little difference in the validity of cut-offs of three, four, five and six criteria.
It could be argued that the current MDD criteria should not be changed in the absence of improved validity. This argument might have particular merit for criteria that have remained essentially the same for 35 years. This alludes to the issue raised in the Introduction of the paper regarding the principles guiding the modification of diagnostic criteria. While a simpler definition of MDD will not enhance validity in research studies using careful assessment procedures based on semi-structured diagnostic interviews, we believe that in clinical practice a simpler set of criteria for diagnosing MDD might improve validity because we suspect that MDD is sometimes underdiagnosed in medically ill patients because of the uncertainty as to whether or not to count the somatic criteria. On the other hand, we suspect that MDD is sometimes overdiagnosed when clinicians, particularly non-psychiatrist physicians, do not fully evaluate the diagnostic criteria and diagnose MDD when fewer than the minimum number of features are present. If the distinction between MDD and ‘subthreshold’ variants of depression such as depressive disorder not otherwise specified and adjustment disorder with depressed mood is valid and has treatment implications, then overdiagnosis has clinical implications. Thus, the increased clinical utility of a simpler definition of MDD could potentially enhance diagnostic accuracy and validity in clinical practice.
Of course, it is possible that even with an abbreviated set of diagnostic criteria that clinicians still will not formally apply them but instead will continue to make non-criteria-based judgments regarding the presence or absence of depression. There are likely multiple reasons why clinicians, particularly primary-care clinicians, do not use the DSM-IV criteria for MDD when diagnosing depression, with the length and complexity of the criteria being only one reason. Additional research clarifying the reasons for not using the criteria could help guide the advisory groups revising current diagnostic systems.
We examined the concordance between a simpler and the DSM-IV definition of MDD by applying different algorithms to a set of symptom data. As such, method variance due to raters' elicitation and interpretation of symptom data was eliminated. We believe that this represents the most appropriate approach towards examining the agreement between alternative symptom algorithms because it eliminates all other potential sources of variance. An alternative approach, in which patients would be evaluated twice, first with the DSM-IV criteria and second with the simpler MDD criteria, would result in lower levels of concordance because error variance attributable to the imperfect reliability of assessments would be introduced.
What might be the implications of modifying the diagnostic criteria for major depression? Because the new definition is so highly concordant with the current DSM-IV definition, a change would not have a meaningful impact on epidemiological prevalence rates. By reducing the number of criteria by nearly one-half, less time would be needed to fully assess the criteria. Diagnostic interviews in epidemiological studies, and screening self-report scales, could be shorter. Measures of treatment outcome, however, would not necessarily be briefer. Scales such as the Hamilton Rating Scale for Depression (Hamilton, Reference Hamilton1960) already include items that are not part of the diagnostic criteria. Thus, it would not be necessary to modify outcome measures. Moreover, monitoring the course of depression in clinical practice should not change. Just as it is important for a clinician to assess symptoms such as anxiety, irritability and hopelessness, which are not currently components of the DSM-IV diagnostic criteria, in their depressed patients, clinicians should continue to monitor appetite and sleep disturbance and energy levels, even if they are not official diagnostic criteria. However, it must be acknowledged that it is possible that if these features of depression are deleted from the diagnostic criteria they might be less closely monitored by treating clinicians. While cautionary messages against reifying the DSM diagnostic criteria are promulgated, it would be naive to believe that symptoms that are eliminated from the diagnostic criteria will be monitored with the same degree of vigilance as they had been previously. Another possible unintended consequence of eliminating the somatic criteria from the MDD criteria is the reduced appreciation of the somatic expression of psychiatric illness. To the degree that the DSM is used to teach trainees about psychiatric diagnosis, the elimination of somatic symptoms from the MDD criteria might interfere with clinicians recognizing depression in their patients who present with somatic complaints.
The symptoms that we are recommending for elimination from the MDD diagnostic criteria have been considered core features of the disorder for as long as the disorder has been described. Moreover, the centrality of some of these features such as sleep disturbance (Kupfer, Reference Kupfer1995) and psychomotor disturbance (Parker & Brotchie, Reference Parker and Brotchie1992) as markers of underlying pathophysiology has been suggested. In deciding how to proceed in the next version of the DSM, the conceptual and practical advantages of a briefer set of criteria that is easier to apply to all patients, particularly medically ill patients, needs to weighted against the disadvantages of deviating from tradition and the risk of overlooking symptoms that are important to assess in depressed patients even though they are no longer diagnostic criteria. Is a potential gain in clinical utility, in the absence of data demonstrating superior validity, sufficient to warrant criteria modification?
A limitation of the present study is that it was conducted in a single out-patient practice in which the majority of the patients were white, female and had health insurance. Replication of the results in samples with different demographic characteristics is warranted. Also, replication in a sample of medically ill patients is important because the elimination of the somatic criteria might have its greatest influence in these patients. While many of the candidates for bariatric surgery had co-morbid medical illnesses, we did not systematically characterize the severity of medical illness and thus were unable to examine its influence on the agreement between the simpler and DSM-IV definitions of MDD.
Declaration of Interest
None.
