Examination of the Chicago Multiscale Depression Inventory and Initial Validation of a Positive Scale
The Chicago Multiscale Depression Inventory (CMDI) is a self-report measure that was designed to enhance measurement of depression in medical populations with primary diagnosis of non-psychiatric illness (Nyenhuis et al., Reference Nyenhuis, Luchetta, Yamamoto, Terrien, Bernardin, Rao and Garron1998, Reference Nyenhuis, Rao, Zajecka, Luchetta, Bernardin and Garron1995). The core 42 items of the measure consist of negatively valenced Mood (i.e., sad, cheerless), Evaluative (i.e., useless, resented), and Vegetative (i.e., sluggish, unable to concentrate) subscales of 14 items each that may be interpreted separately or in combination to examine unique features of depression, including dysphoric mood, negatively biased thought processes, and somatic dysfunctions. The measure also includes eight positively valenced items (i.e., joyful, energetic) that were initially included to break up the negative items and prevent reporting bias across the three negative subscales.
Although depression frequently occurs in medical conditions such as multiple sclerosis (MS) and Parkinson’s disease, accurate assessment is complicated by the presence of neurovegetative symptoms that overlap in diagnostic criteria for depression. For example, symptoms such as fatigue, difficulty concentrating, and appetite changes are often central to MS but they are also considered symptoms of depression. Thus, when diagnosing co-morbid depression in non-psychiatric medical patients, clinicians often focus on mood symptoms and may only consider neurovegetative/somatic symptoms that appear beyond expected levels for the population. Early development of the CMDI included evaluation of the scale structure through factor analysis and standardization of the scale with MS patients, adult outpatients with major depression, and normal controls matched to the MS group. Further validation and standardization was conducted with healthy college students, and a follow-up study (Chang et al., Reference Chang, Nyenhuis, Cella, Luchetta, Dineen and Reder2003) demonstrated validity of the CMDI subscales in patients with MS. Although the initial CMDI development focused on examination of the negative subscales, in this study we also explore psychometric properties of the Positive item scale that is embedded within the CMDI in healthy adult and MS patient samples. Inclusion of such a scale as an outcome measure could enhance validity of the CMDI by incorporating assessment of symptoms related to experience of pleasure that are central to diagnosis of depression (American Psychiatric Association, 2013).
Completion of the CMDI involves rating the extent to which each item describes the individual during the past week, including today, on a scale of 1 to 5 where 1 is “Not at All” and 5 is “Extremely.” The CMDI total score is typically calculated through a sum of the 42 negative items that form three subscales—Mood, Evaluative, and Vegetative. These subscales have been the primary measures of interest from the CMDI, with higher scores indicating higher levels of depression. Internal consistency reliability of the CMDI reported in initial validation studies (Nyenhuis et al., Reference Nyenhuis, Luchetta, Yamamoto, Terrien, Bernardin, Rao and Garron1998, Reference Nyenhuis, Rao, Zajecka, Luchetta, Bernardin and Garron1995) is high, with Cronbach’s alpha of .89 for the total scale and .91, .77, and .77 for Mood, Evaluative, and Vegetative subscales, respectively. In medical populations with non-psychiatric illness, the Mood and Evaluative scales are often used in combination without the Vegetative scale to avoid over-diagnosis of depression in the presence of significant somatic symptoms that are also prominent in non-psychiatric illness. Previous research has also demonstrated reliability and validity of the CMDI in specific populations, such as MS, with reported Cronbach’s alpha of .82 and .95 for the Mood and Evaluative scales, respectively (Chang et al., Reference Chang, Nyenhuis, Cella, Luchetta, Dineen and Reder2003).
In addition to negative items, the CMDI also contains positive items that could potentially comprise a scale. The items are rated on the same one to five increments as the negative items, and thus a higher score would indicate greater endorsement of positive affect and experience. These items have not typically been included in calculation of the CMDI total score. To calculate a total score across the whole CMDI, the Positive scale items should be reverse scored before adding to the other items. The Positive scale has received little attention in previous research, including in original works by Nyenhuis et al. (Reference Nyenhuis, Luchetta, Yamamoto, Terrien, Bernardin, Rao and Garron1998, Reference Nyenhuis, Rao, Zajecka, Luchetta, Bernardin and Garron1995), which did not provide reliability and validity data for the scale. Instead, these items were included to prevent uniform responses to items across the negative subscales (Chang et al., Reference Chang, Nyenhuis, Cella, Luchetta, Dineen and Reder2003). However, depression has long been characterized as a disorder of both positive and negative affect, which appear to be distinct dimensions, rather than opposite ends of the same dimension (Olino, Klein, Lewinsohn, Rohde, & Seeley, Reference Olino, Klein, Lewinsohn, Rohde and Seeley2008; Watson, Clark, & Carey, Reference Watson, Clark and Carey1988). Distinct positive and negative emotion dimensions have also been found in cross-cultural studies (Iwata et al., Reference Iwata, Mishima, Shimizu, Mizoue, Fukuhara, Hidano and Spielberger1998). Cognitive neuroscience studies also indicate that these affective processes may involve overlapping but not identical neural structures (Phan, Wager, Taylor, & Liberzon, Reference Phan, Wager, Taylor and Liberzon2002; Wager, Phan, Liberzon, & Taylor, Reference Wager, Phan, Liberzon and Taylor2003).
Positive affect is defined by a combination of emotional, behavioral, motivational, and physiological characteristics related to pursuit and enjoyment of rewards (Forbes & Dahl, Reference Forbes and Dahl2005). Low positive affect could contribute to anhedonia, loss of capacity to experience pleasure or engage in pleasurable activities. Anhedonia is a key criterion in the diagnosis of a depressive disorder (American Psychiatric Association, 2013), distinct from symptoms like sad mood and fatigue, and is also an element in proposed criteria for apathy (Starkstein et al., Reference Starkstein, Merello, Jorge, Brockman, Bruce and Power2009). Anhedonia may be considered a motivational state leading to the behavioral expression of low positive affect (Forbes & Dahl, Reference Forbes and Dahl2005). Thus, validation of the CMDI Positive scale and inclusion of these items in the total score could provide a more effective clinical assessment of an individual’s symptom presentation than assessment of negative symptoms alone.
The combination of increased negative affect and decreased positive affect in depression may be evident in sad mood and self-effacing cognitions, along with reduced motivation and enjoyment in pleasurable activities. Individuals may develop depression with more or less negative affect relative to low positive affect. For example, an individual could display sad mood at moderate levels while showing an extreme lack of motivation to engage in typically pleasurable activities. Another individual may not meet criteria for depression, lacking in any significant negative affectivity, while showing diminished positive affect and lack of drive. Distinguishing decreased positive affect from increased negative affect can be difficult through behavioral observation, but the dimensions may be more clearly evaluated through validated quantitative measures.
While other scales like the Positive and Negative Affect Scale (PANAS) (Watson, Clark, & Tellegen, Reference Watson, Clark and Tellegen1988) were also developed to assess these two dimensions of affect, the CMDI offers another validated measure that it is amenable to use in medical populations by allowing for separate analysis of the vegetative symptom scale. The objective of the present set of studies is to conduct the first validation of the CMDI Positive scale, a set of items already embedded within the CMDI. The full CMDI could fulfill a need for a measure that is theoretically and empirically valid for assessing the symptoms of positive and negative affect and experience that are considered important and relatively independent components of depression. In Study 1, we measured the factor structure and internal consistency reliability of the CMDI Mood, Evaluative, Vegetative, and Positive scales in separate healthy control and MS patient groups. Mean scores on each scale were then compared between groups. Finally, scales were correlated with other established self-report measures of mood symptoms. In Study 2, we further established the criterion validity of the CMDI Positive scale through correlation with the PANAS Positive Affect scale in healthy and MS patient groups.
Method
Study 1: Participants and Procedures
Study 1 consisted of a sample of healthy adults who were demographically matched to a sample of individuals with MS. The 72 healthy adults were 83.3% female, 100% Caucasian, with mean age 45.4 years (SD 11.24) and education 15.5 years (SD 2.3). Participants were recruited through flyers posted in the community, an advertisement in a university-wide email list-serve, and referrals from enrolled participants. Interested participants completed a telephone screen to determine that they were eligible for the study. Exclusionary criteria were history of nervous system disorder, medical condition that could substantially affect cognition or motor function, severe physical or sensory impairments that might significantly interfere with cognitive testing, alcohol/drug abuse, or developmental history of a learning disability or attention-deficit/hyperactivity disorder. Individuals who were not excluded were then scheduled for the study.
Ninety-seven community-based adults with MS were also included in Study 1. The sample was 82.5% female, 100% Caucasian, with mean age 47.3 years (SD 9.0), education 14.3 years (SD 2.0), Expanded Disability Status Scale (EDSS) 4.6 (SD 1.6), and 76.3% relapsing-remitting MS. Diagnoses were clinically confirmed using Polman et al. (Reference Polman, Reingold, Banwell, Clanet, Cohen, Filippi and Wolinsky2010) criteria. Participants were recruited through flyers posted in the community and an advertisement placed in the Central Pennsylvania Chapter of the National Multiple Sclerosis Society MS Connection newsletter. Interested participants completed a telephone interview to determine eligibility. Exclusionary criteria were the same as for the healthy controls except for inclusion of MS diagnosis. MS patients were also excluded for experience of disease relapse or corticosteroid use within 4 weeks before the assessment. Measures for this study were administered within a larger battery of neuropsychological tests and self-report measures of psychosocial functioning for a study of depression in MS. Participants received $75 compensation and MS patients also received a clinical neuropsychological evaluation report.
Forty-two items from three subscales of the CMDI relating to negative affect and experience (Mood, Evaluative, and Vegetative) were obtained following methods outlined by Nyenhuis et al. (Reference Nyenhuis, Rao, Zajecka, Luchetta, Bernardin and Garron1995) in which subscales were factorially derived in a sample of healthy, community-based adults. The remaining eight items relating to positive affect and experience formed a new Positive subscale. CMDI responses from each sample were subject to Principal Components Analysis (PCA) and internal consistency reliability analysis of the subscales. Intercorrelations of CMDI subscales were calculated for each sample. Mean scores on the CMDI subscales were compared between the MS and matched healthy participant samples. Correlations between CMDI subscales and additional validated self-report measures of depression and anxiety, the Beck Depression Inventory - Fast Screen (BDI-FS) and the State-Trait Anxiety Inventory (STAI) Form Y, were also calculated for each sample.
The BDI-FS is a seven-item questionnaire that was designed to evaluate recent depression symptoms in medical patients (Beck, Steer, & Brown, Reference Beck, Steer and Brown2000). The measure focuses on dysphoria, anhedonia, suicidal ideation, and cognition-related symptoms and eliminates neurovegetative items that are found within the Beck Depression Inventory-II. The BDI-FS has been validated for use in multiple clinical populations, including MS (Benedict, Fishman, McClellan, Bakshi, & Weinstock-Guttman, Reference Benedict, Fishman, McClellan, Bakshi and Weinstock-Guttman2003). Participants rate on a scale from 0 to 3 the extent that they identify with a statement that best describes how they have been feeling the past two weeks (i.e., Pessimism: 0= I am not discouraged about my future, 3= I feel my future is hopeless and will only get worse).
The STAI is a 40-item questionnaire that instructs participants to rate the extent that each item describes them on a 1 to 4 scale (1= almost never, 4= almost always) (Spielberger, Reference Spielberger1983). The questionnaire is divided into 2 subscales. For the first 20 items, the “State Anxiety” scale, participants rate items according to how they currently feel, “at this moment”. The next 20 items form the “Trait Anxiety” scale, in which participants rate how they “generally feel”. Higher scores on the scales indicate greater anxiety symptoms. Internal consistency reliability coefficients for the STAI scales range between 0.86 and 0.95 and it has been validated in multiple clinical and community samples.
Study 2: Participants and Procedures
Study 2 examined criterion validity of the CMDI Positive scale through comparison with the PANAS. This study included a sample of 68 healthy undergraduate students who received psychology course credit for participation. A sample of 34 individuals with MS was formed by combining a subset of 22 patients from Study 1 who received both the CMDI and the PANAS during their clinic visit and 12 additional patients who were recruited to augment the sample size of individuals who completed the PANAS. The objective of this study was to contribute to initial validation of the CMDI Positive scale in separate healthy and patient samples, rather than specifically assessing demographically matched samples. The healthy sample was 60.3% female, 77.9% Caucasian, 8.8% Asian-American, 4.4% African-American, 2.9% Multiracial, and 5.9% “Other Ethnicity,” with mean age 19.3 years (SD 2.4). The MS sample was 76.5% female, 100% Caucasian, with mean age 53.2 years (SD 11.2), education 14.7 years (SD 2.1), EDSS 4.5 (SD 1.5), and 70.6% relapsing-remitting MS.
The PANAS consists of a 10-item Positive Affect scale (i.e., excited) and a 10-item Negative Affect scale (i.e., distressed). Participants rate the extent that they have experienced the feeling in each item over the past week using a scale from 1 to 5 where 1 is “Very slightly or not at all” and 5 is “Extremely.” Validation of the PANAS reported high internal consistency reliability of the Positive Affect Scale (alpha =.88). Only one specific positive item, “active,” overlaps between the CMDI and the PANAS positive scales. Correlation coefficients were calculated for the two scales in each sample. These studies were approved by our university’s Institutional Review Board.
Results
Study 1: Internal Consistency, Construct Validity and Clinical Utility
Descriptive and reliability analyses
Means and standard deviations are reported in Table 1 for the CMDI subscales, BDI-FS, and STAI subscales within the Study 1 healthy and MS samples. Cronbach’s alpha coefficients were calculated to measure internal consistency reliability of the CMDI 50-item total scale and subscales. In the healthy sample, alpha coefficients for the four subscales were all high: Positive, .78 (8 items); Mood, .95 (14 items); Evaluative, .91 (14 items); and Vegetative .85 (14 items). In the MS sample, alpha coefficients were also high: Positive, .83; Mood, .96; Evaluative, .92; and Vegetative, .83. Alpha coefficients for the full 50-item CMDI total scale were high in the healthy (.88) and MS (.90) samples. Patients with MS scored lower on the Positive scale than the healthy participants (t(167)=−4.99; p<.01; Cohen’s d =.78).
Group comparison of mean total scores for the CMDI scales, BDI-FS, and STAI scales between Study 1 healthy control and MS participants
Principal components analysis
Each subscale was subject to PCA with varimax rotation. Items were considered to significantly contribute to the subscale factor if they loaded onto the factor at greater than 0.4. All 8 items had significant loadings on the Positive factor in the healthy and MS samples. Thirteen of 14 items had significant loadings on the Mood factor in the healthy sample, with the one exception of “miserable”; all 14 items had significant loadings in the MS sample. All 14 items had significant loadings on the Evaluative factor in the healthy and MS samples. For the Vegetative factor, 12 of 14 items had significant loadings in the healthy sample and 11 of 14 items had significant loadings in the MS sample. “Poor appetite” and “uninterested in sex” had loadings less than 0.4 in the healthy sample and in the MS sample, and the item “easily awakened” also did not have a significant loading in the MS sample. Table 2 displays results of the PCA for the Positive scale.
CMDI Positive scale PCA results for Study 1 healthy control and MS participants
Convergent and discriminant validity
Correlations across CMDI subscales, the BDI-FS, and the STAI are reported in Table 3 for the healthy sample and in Table 4 for the MS sample. In the healthy sample, all CMDI subscales had intercorrelations that were statistically significant at p<.01. The specific correlations with the Positive scale were: Mood, r=−.40; Evaluative, r =−.34; and Vegetative, r =−.40. The Positive scale also had statistically significant correlations with the BDI-FS, r =−.56, p<.01; and the STAI State scale, r =.26, p<.05; but not with the STAI Trait scale.
Correlation matrix for CMDI scales, BDI-Fast Screen, STAI State, and STAI Trait in Study 1 healthy control participants
**Pearson correlation coefficient significant, p<.01.
*Correlation significant, p<.05.
Correlation matrix for CMDI scales, BDI-Fast Screen, STAI State, and STAI Trait in Study 1 MS participants
**Pearson correlation coefficient significant, p<.01.
*Correlation significant, p<.05.
Intercorrelations between CMDI subscales were also statistically significant in the MS sample at p<.01. Specific correlations with the Positive scale were: Mood r =−.55, Evaluative r =−.55, and Vegetative r =−.23. The Positive scale also had a statistically significant correlation with the BDI-FS (r=−.52; p<.01), but it was not significantly correlated with the STAI State or Trait scales.
Measurement of depression
Participants were divided into two groups of depressed and non-depressed based on a criterion cutoff score greater than or equal to 4 on the BDI-FS indicating significant depression. This is the cutoff recommended in the BDI-FS manual, as well two different validation studies of the BDI-FS in MS (Benedict et al., Reference Benedict, Fishman, McClellan, Bakshi and Weinstock-Guttman2003; Strober & Arnett, Reference Strober and Arnett2015). Mean BDI-FS score of the depressed, neurologically healthy group was high (M=6.83; SD=2.56) in comparison to the non-depressed group (M=0.85; SD=1.14; t(5)=−5.67, p<.01; Cohen’s d=3.02). Depressed MS patients reported a mean score of 5.89 (SD=2.12) compared to 1.25 (SD=1.02) in the non-depressed group (t(46)=−12.47; p<.01; Cohen’s d=2.79). The CMDI Positive scale differentiated depressed and non-depressed groups in the healthy sample (t(70)=3.38; p<.01; Cohen’s d =1.28) and in the MS sample (t(95)=5.63; p<.01; Cohen’s d =1.19). Specifically, positivity was significantly lower in the depressed groups than in the non-depressed groups. Furthermore, in a linear regression analysis, the CMDI Positive scale predicted statistically significant variance (8% of 49% R 2 total variance) in BDI-FS total depression score beyond the variance predicted by the total score of the CMDI negative scales (∆F(1,69) =11.90; p<.01) in the healthy sample. In the MS sample, the Positive scale contributed some additional variance (1% of 61% R 2 total variance) in predicting BDI-FS depression but the effect did not reach the threshold for statistical significance when preceded by entry of the negative total scale score (∆F(1,94)=3.37; p=.07).
Study 2: Positive Scale Criterion Validity
Study 2 additionally analyzed criterion validity of the CMDI Positive scale through comparison with the PANAS Positive Affect scale in healthy college student and MS patient samples. Mean scale scores and correlations are reported for each sample in Table 5. There was a strong, statistically significant correlation between the scales in the healthy college student sample (r=.84; p<.01) and in the MS patient sample (r=.89; p<.01). The CMDI Positive and PANAS Positive Affect scales have one overlapping item (active). When the total score for each scale was calculated without this item, the correlation between scales remained statistically significant (Healthy: r=.81; p<.01; MS: r=.86; p<.01).
Mean item scores and correlations for the CMDI Positive scale and PANAS Positive Affect scale in healthy and MS participants
Note. All correlations (r) significant, p<.01.
Discussion
The CMDI was initially developed to improve accuracy in measurement of depression, especially in populations with non-psychiatric medical illness. Each CMDI subscale captures a unique feature of depression, and the scales can be used individually or in combination according to the goals of the clinician or researcher. This study advanced the use of the CMDI by providing reliability and validity data for a Positive item scale that is already embedded within the measure, along with the Mood, Evaluative, and Vegetative scales that assess negative symptoms and experience associated with depression. Since assessment of positive affect and ability to engage in pleasurable activities are important elements in diagnosis of major depression, interpretation of scores on the positive items could add value to the CMDI and its quantitative evaluation of depression symptoms.
The psychometric properties and clinical value of the CMDI subscales were evaluated in healthy and MS samples. The internal consistency reliability of the total 50-item measure was high, and each of the subscales also showed high reliability. We also used PCA to assess whether specific items within each subscale loaded onto the subscale factor. All Positive scale items met the criterion threshold for loading onto the scale factor in both the healthy and MS samples, and results for the Evaluative scale were similar. However, the analysis also indicated that certain items in the Mood and Vegetative scales may not significantly augment the factors; the Vegetative scale in particular had two to three items that did not have significant factor loadings in either the healthy or MS samples. The Vegetative scale items were also found to be less factorially valid in previous research by Nyenhuis et al. (Reference Nyenhuis, Luchetta, Yamamoto, Terrien, Bernardin, Rao and Garron1998) and may be excluded from calculation of the subscale score, although they are generally still included in the total score. The items in this subscale may assess symptoms that are less homogenous than the items in the other subscales.
Consistent with predictions, the Positive scale had a significant inverse correlation with each of the other CMDI scales, but it was not as strongly correlated with them as they were with each other. The finding is consistent with previous research indicating that positive and negative affect have distinct dimensions (Olino et al., Reference Olino, Klein, Lewinsohn, Rohde and Seeley2008; Watson et al., Reference Watson, Clark and Carey1988); while there appears to be some overlap in the general construct of affect, positive and negative are not simply opposites. An important element in validating this Positive scale was to show its clinical value. We achieved this by showing that scores on the CMDI Positive scale differentiated individuals with MS from matched healthy controls, with MS patients reporting less positivity. The Positive scale was also useful for specifically evaluating depression. Within each of these populations, individuals who reported less positivity on the scale were more likely to report depression on the BDI-FS. Furthermore, regression analysis showed that the Positive scale seems to contribute to measurement of depression beyond information gained from the negative scales by capturing unique variance in predicting depression that was not accounted for by the negative scales. While this analysis showed statistical significance in the healthy individuals, the effect did not reach the statistical threshold in MS patients. This was most likely due to the fact that the negative scales accounted for more variance in depression in the MS compared with the healthy control group (61% compared to 49%) and, as such, there was less variance left to explain when the Positive scale was entered into the analysis. Thus, our results indicate that symptoms of negative emotion and experience appear to be more strongly correlated with overall depression in MS than in healthy controls.
In this study, the Positive scale had a significant positive correlation with a measure of state anxiety, and the Vegetative scale had a significant negative correlation with state anxiety; these CMDI scales did not show a significant correlation with a measure of trait anxiety. In comparison, the Evaluative scale had a significant negative correlation with trait anxiety. Our findings provide evidence that state and trait anxiety may have different relationships to dimensions of positive and negative affect, which may be explored in future studies. There is some work that indicates that anxiety typically has no relationship (positive or negative) to positive affect (Olino et al., Reference Olino, Klein, Lewinsohn, Rohde and Seeley2008; Watson et al., Reference Watson, Clark and Carey1988). However, the research literature on the relationship of positive and negative affect to anxiety does not clearly distinguish between state and trait anxiety. Thus, we recommend further research that evaluates these dimensions. It seems plausible that increased positive affect could contribute to the time-limited increases in arousal associated with state anxiety. In contrast, elevated negative mood could lead to more persistent trait anxiety.
A goal of this study was to include a sample that was representative of the populations in which the CMDI is likely to be administered. We achieved this goal in Study 1 with a sample of healthy adults who were demographically matched in education, age, and gender with a sample of MS patients. However, there were limitations to the sample in Study 2, which was completed with healthy college students who on average are younger and may have better cognitive functioning than a broader medical population. Still, results provided additional criterion validity data for the Positive scale, and findings in the MS patients were very similar to the student sample. It may be useful to further evaluate the CMDI in a general healthy adult population and in other conditions where mood disorders are of concern.
Taken together, this study indicates that the CMDI scales, including the Positive scale, have very good reliability and validity across healthy adults and MS patients. Overall, people with MS reported greater negative symptoms and fewer positive symptoms than healthy people. Also, negative and positive affective dimensions showed some independence, which is consistent with previous factor analytic research and with clinical criteria for diagnosis of major depression. Existing scales lack assessment of positive affect dimensions or may not be amenable for use in patients with significant neurovegetative symptoms that overlap with depression criteria; each of the CMDI scales may be used separately or in combination for evaluation of individual dimensions of symptoms. Thus, including the eight-item CMDI Positive scale with the negative scales would provide comprehensive and, potentially, more valid information about an individual’s depression symptoms. These scales may be advantageous for further research on individual differences in emotional functioning and contributions of positive and negative dimensions to clinical presentation.
Acknowledgments
Special thanks to Gray Vargas, Margaret Cadden, Amanda Rabinowitz, Frank Hillary, Koraly Perez-Edgar, and Jennifer Graham for their help with various aspects of the project. This work was supported by the National MS Society (award number PP1829). The authors do not have conflicts of interest to report.
