Significant outcomes
∙ Ratings on the Psychotic Depression Assessment Scale (PDAS) obtained via the dedicated semi-structured PDAS interview appear to be clinically valid, scalable and responsive measures of the severity of psychotic depression in older adults.
∙ Resolution of psychotic features [Brief Psychiatric Rating Scale (BPRS5) total score≤1) is indicative of remission of psychotic depression.
Limitations
∙ Limited sample size (62 older adults with psychotic depression).
∙ Test–retest stability and inter-rater reliability was not tested.
∙ Only patients from Belgium were included in the study. Thus, the validity of the PDAS should be tested in other populations as well.
Introduction
Major depression with psychotic features (psychotic depression) is a severe and debilitating mental disorder that differs substantially from major depression without psychotic features (non-psychotic depression) (Reference Gournellis, Lykouras, Fortos, Oulis, Roumbos and Christodoulou1–Reference Rothschild7). Yet, in clinical studies of psychotic depression, the severity of the illness has most often been assessed by means of rating scales developed for non-psychotic depression, for example different versions of the Hamilton Depression Rating Scale (HAM-D) (Reference Hamilton8) and the Montgomery-Asberg Depression Rating Scale (MADRS) (Reference Montgomery and Asberg9). However, these scales do not cover the psychotic dimension of psychotic depression sufficiently (Reference Ostergaard, Rothschild, Flint, Mulsant, Whyte, Leadholm, Bech and Meyers10).
Recently, a rating scale dedicated to the assessment of the severity of psychotic depression, the PDAS, was developed (Reference Østergaard, Meyers, Flint, Mulsant, Whyte, Ulbricht, Bech and Rotschild11). The 11 PDAS items were extracted empirically from the HAM-D17 and the BPRS (Reference Overall and Gorham12). The initial studies of the PDAS showed that the scale had good clinical validity (the PDAS total score correlated strongly with that of global severity measurements), scalability (each of the 11 items added unique information regarding the severity of psychotic depression) and responsiveness (the PDAS total score dropped significantly during treatment) in the measurement of psychotic depression (Reference Østergaard, Meyers, Flint, Mulsant, Whyte, Ulbricht, Bech and Rotschild11,Reference Ostergaard, Meyers, Flint, Mulsant, Whyte, Ulbricht, Bech and Rotschild13,Reference Ostergaard, Pedersen, Uggerby, Munk-Jorgensen, Rothschild, Larsen, Gotzsche, Sondergaard, Bille, Bolwig, Larsen and Bech14).
The PDAS contains two subscales, a depression subscale (HAM-D6) consisting of the six items from the melancholia subscale (Reference Bech, Gram, Dein, Jacobsen, Vitger and Bolwig15,Reference Timmerby, Andersen, Sondergaard, Ostergaard and Bech16) of the HAM-D17 (depressed mood, feelings of guilt, work and activities, psychomotor retardation, psychic anxiety and general somatic symptoms) and a psychosis subscale (BPRS5) consisting of five items from the BPRS (emotional withdrawal, suspiciousness, hallucinatory behaviour, unusual thought content and blunted affect) (Reference Overall and Gorham12). Item scores range from 0 to 4, yielding a total score range from 0 to 44. Severity cut-offs based on the PDAS total score have been established in a prior study. The cut-off for remission is <8, scores from 8 to 15 correspond to mild psychotic depression, scores from 16 to 23 correspond to moderate psychotic depression and scores >23 correspond to severe psychotic depression (Reference Ostergaard, Rothschild, Flint, Mulsant, Whyte, Vermeulen, Bech and Meyers17).
In the majority of the studies of the PDAS, ratings on the 11 items were extracted from ratings on the rather lengthy HAM-D17 and BPRS (35 items in total) (Reference Ostergaard, Meyers, Flint, Mulsant, Whyte, Ulbricht, Bech and Rotschild13,Reference Ostergaard, Pedersen, Uggerby, Munk-Jorgensen, Rothschild, Larsen, Gotzsche, Sondergaard, Bille, Bolwig, Larsen and Bech14,Reference Ostergaard, Rothschild, Flint, Mulsant, Whyte, Vermeulen, Bech and Meyers17). Therefore, to facilitate rating on the PDAS and to increase reliability of PDAS ratings, a dedicated semi-structured interview (Reference Köse Çinar and Dinesen Østergaard18) was developed. The results of the first study in which PDAS ratings were obtained by means of the dedicated semi-structured interview were promising (Reference Köse Çinar and Dinesen Østergaard18). However, an important limitation of this study performed in a younger adult population was that the analysis of clinical validity was based on comparison of ratings on the PDAS and the Clinical Global Impressions – Severity of illness (CGI-S) scale performed by the same rater. This may have resulted in an overestimation of the clinical validity of PDAS ratings (Reference Köse Çinar and Dinesen Østergaard18). Therefore, the aim of the present study was to investigate whether the clinical validity of PDAS ratings obtained via the dedicated semi-structured interview could be confirmed in a study where PDAS and CGI-S ratings were performed independently in a sample of older adults. Specifically, the following research questions were addressed: (I) Are PDAS ratings obtained via the semi-structured PDAS interview clinically valid? (II) Is the scalability of the PDAS maintained when using the semi-structured PDAS interview? (III) Are PDAS ratings obtained via the semi-structured interview sensitive to change in the severity of illness (responsiveness)? (IV) Is resolution of psychotic features according to a BPRS5 score≤1 indicative of remission of psychotic depression according to the PDAS?
Material and methods
Participants
Between 1 January 2016 and 31 December 2016, patients with psychotic depression aged 60 years or above who were admitted to the university psychiatric hospital Duffel, Belgium (Flanders) or the psychiatric department of the general hospital Sint-Jan Brugge-Oostende AV, Belgium (Flanders) were invited to participate in the study. The diagnostic inclusion criterion was a current episode of unipolar psychotic depression according to DSM-IV criteria [patients were considered as suffering from psychotic depression, when the clinical diagnosis made by an experienced psychiatrist and the diagnosis obtained by a trained psychiatric resident using version 5 of the Mini International Neuropsychiatric Interview (MINI) (Reference Sheehan, Lecrubier, Sheehan, Amorim, Janavs, Weiller, Hergueta, Baker and Dunbar19) were both psychotic depression]. The exclusion criteria were (I) current substance abuse (including alcohol) or (II) a diagnosis of a neurodegenerative disease including dementia. The presence of substance abuse was confirmed by the MINI and neurodegenerative disease was assessed using heteroanamnestic information, medical imaging and cognitive screening as part of routine clinical practice. All participants gave oral and written informed consent. The Medical Ethical Committee of the University of Antwerp approved the study (case number EC1536).
Instruments
The assessments of psychopathology were performed within 1 week after admission and at discharge (maximum 1 week before discharge). The following ratings were performed. PDAS: trained nurses rated severity of psychotic depression using the semi-structured PDAS interview (https://psychoticdepressionassessmentscale.com). Before the initiation of the study, the PDAS raters participated in a training rating session based on a video recorded PDAS interview. MADRS: psychologists or experienced practice nurses rated the severity of depression using the MADRS (Reference Montgomery and Asberg9) independently of the PDAS ratings. We chose the MADRS instead of the HAM-D17 as a depression reference measure for the PDAS, since the PDAS contains six items from the HAM-D17. Also, the MADRS is commonly used in clinical practice and research in older adults with depression as has demonstrated sufficient validity in this population (Reference Borza, Engedal, Bergh, Benth and Selbæk20,Reference Engedal, Kvaal, Korsnes, Barca, Borza, Selbaek and Aakhus21). CGI-S: senior psychiatrists scored clinical global severity on the CGI-S, a seven-point anchored scale frequently used in studies on the severity of depression and in rating scale validation (Reference Kadouri, Corruble and Falissard22). All raters were part of the staff at the hospitals where the participants were admitted.
Statistical analysis
(I) Clinical validity of PDAS ratings obtained via the semi-structured PDAS interview
Clinical validity was assessed by means of Spearman’s correlation analysis of the total scores of the PDAS versus the CGI-S ratings and the total scores on the MADRS, respectively. For these analyses, pooled data of baseline and endpoint ratings were used in order to cover the entire severity spectrum (see Fig. 1).
Scatterplots of baseline (blue) and endpoint (green) scores on severity rating scales. BPRS, Brief Psychiatric Rating Scale; CGI-S, Clinical Global Impression Scale – Severity; HAM-D, Hamilton Depression Rating Scale; Montgomery-Asberg Depression Rating Scale; PDAS, Psychotic Depression Assessment Scale.
(II) Scalability of PDAS ratings obtained via the semi-structured PDAS interview
Scalability is present when the symptoms represented by the items of a rating scale are ordered according to their relationship to the severity of the latent syndrome that is measured (Reference Bech23). When an instrument is scalable, each symptom item adds unique information regarding the severity of the underlying syndrome. Scalability is therefore a prerequisite for adding individual item scores to a total score. In this study, scalability was assessed using Mokken analysis, generating a Loevinger coefficient of homogeneity for the PDAS total score. In the context of the Mokken analysis, a Loevinger coefficient of homogeneity ≥0.40 is indicative of scalability (Reference Bech23).
(III) Responsiveness of the PDAS obtained via the semi-structured PDAS interview
To determine the responsiveness of the PDAS, we calculated the proportion of participants who met the PDAS remission criterion (total score <8) (Reference Ostergaard, Rothschild, Flint, Mulsant, Whyte, Vermeulen, Bech and Meyers17) at baseline and endpoint, respectively. Also, we investigated whether the MADRS and CGI-S scores of participants meeting the remission criterion on the PDAS were compatible with remission of illness. Statistical differences between baseline and endpoint were assessed using the Wilcoxon’s signed-rank test.
(IV) Resolution of psychotic features and remission of psychotic depression
In the analysis of whether resolution of psychotic features at endpoint was indicative of remission of psychotic depression, we defined resolution of psychotic features as a BPRS5 total score≤1 based on the results from Park et al. (Reference Park, Ostergaard, Choi, Kim, Jun, Lee, Kim, Yim and Park24), who demonstrated that the BPRS5 score accurately separated psychotic depression (BPRS5 total score >1) from non-psychotic depression (BPRS5 total score≤1). Based on a 2×2 confusion matrix, we calculated Cohen’s κ, sensitivity and specificity of the BPRS5 ≤1 cut-off in classifying remission using PDAS total scores <8 as reference (Reference Ostergaard, Rothschild, Flint, Mulsant, Whyte, Vermeulen, Bech and Meyers17).
All analyses were conducted with IBM Statistics version 23, except for the Mokken analysis, which was performed using the dedicated Mokken Scale Analyses for Polytomous Items Program (Reference Molenaar, Debels and Sijtsna25).
Results
Table 1 shows the characteristics of 62 participants with unipolar psychotic depression (66% had recurrent psychotic depression). The mean age was 74.3 (SD=7.1; range 60–90) years and most participants (63%) were female. Mean length of stay was 51.8 (SD=35.1) days. We obtained endpoint data from 56 of the 62 participants (three had an unplanned discharge, two were transferred to a general hospital due to worsening of somatic conditions and one participant died). Missing data at baseline were CGI-S (n=2) and PDAS (n=1), and at endpoint CGI-S (n=1), MADRS (n=2) and PDAS (n=1).
Characteristics of the study participants (n=62)
BPRS5, total scores on the psychotic subscale Brief Psychiatric Rating Scale 5 (BPRS5); CGI-S, total scores on the Clinical Global Impression Scale – Severity; ECT, electroconvulsive therapy; HAM-D6, total scores on the depressive subscale Hamilton Depression Rating Scale (HAM-D6); MADRS, total scores on the Montgomery-Asberg Depression Rating Scale; PD, psychotic depression; PDAS, total scores on the Psychotic Depression Assessment Scale.
Data are shown as mean (±SD) or number (%) as appropriate.
Table 1 also contains data regarding treatment during the admission. All participants were treated with an antidepressant and 80% with a combination of an antidepressant and an antipsychotic. The majority of the participants (64%) received electroconvulsive therapy and 85% of those receiving electroconvulsive therapy were also treated with a combination of an antidepressant and an antipsychotic.
(I) Clinical validity of PDAS ratings obtained via the semi-structured PDAS interview
Figure 1 shows the scatter plots of the following pair-wise correlations (Spearman’s correlation coefficients in parentheses): PDAS versus CGI-S (0.76), PDAS versus MADRS (0.79), PDAS versus HAM-D6 (0.96), PDAS versus BPRS5 (0.85), HAM-D6 versus CGI-S (0.74) and BPRS5 versus CGI-S (0.63). All correlations were highly statistically significant (p<0.001).
(II) Scalability of PDAS ratings obtained via the semi-structured PDAS interview
The Mokken analysis yielded a Loevinger coefficient of homogeneity of 0.46, which should be interpreted against the predefined threshold for scalability (≥0.40).
(III) Responsiveness of the PDAS obtained via the semi-structured PDAS interview
On admission, none of the participants met the remission criterion on the PDAS (total score <8), whereas at endpoint 54% of the sample had reached remission (Table 2). The participants reaching remission according to the PDAS had a mean total score of 6.8 on the MADRS (remission<10) and 1.8 on the CGI-S (remission: CGI-S<3), respectively. For the entire population, the mean PDAS scores dropped from 19.7 (SD=5.7) on admission to 7.6 (SD=5.2) at discharge, corresponding to a mean reduction of 12.1 (SD=6.3) points. Similarly, the mean reductions on the MADRS and CGI-S were 18.8 (SD=9.4) points and 4.0 (SD=1.0) points, respectively. The Wilcoxon signed-rank test showed that there was a statistically significant difference between admission and endpoint scores on PDAS (Z=−6.27, p<0.001), HAM-D6 (Z=−6.26, p<0.001), BPRS5 (Z=−5.87, p<0.001), MADRS (Z=−6.26, p<0.001) and CGI-S (Z=−6.36, p<0.001).
Endpoint severity ratings of participants in remission (n=30) according to the Psychotic Depression Assessment Scale (PDAS) (score<8)
BPRS5, total scores on the psychotic subscale Brief Psychiatric Rating Scale (BPRS5); CGI-S, total scores on the Clinical Global Impression Scale – Severity; HAM-D6, total scores on the depressive subscale Hamilton Depression Rating Scale (HAM-D6); MADRS, total scores on the Montgomery-Asberg Depression Rating Scale; PDAS, total scores on the PDAS.
(IV) Resolution of psychotic features and remission of psychotic depression
Resolution of psychotic features (BPRS5 total score≤1) classified remission as defined by the PDAS with the following level of agreement: Cohen’s κ=0.71 (SE=0.095, p<0.001), sensitivity=0.83, specificity=0.88. The 2×2 confusion matrix is shown in Table 3.
2×2 Confusion matrix on remission (BPRS5 definition vs. PDAS definition)
BPRS5, total scores on the psychotic subscale Brief Psychiatric Rating Scale (BPRS5); PDAS, total scores on the Psychotic Depression Assessment Scale.
Discussion
This study in 62 older adults with psychotic depression showed (I) that PDAS ratings obtained via the semi-structured PDAS interview are clinically valid, (II) that the scalability of the PDAS is maintained when using the semi-structured interview for rating, (III) that PDAS ratings obtained via the semi-structured interview are responsive to change in the severity of illness and (IV) that resolution of psychotic features (defined as a BPRS5 total score ≤1) is indicative of remission of psychotic depression.
The correlation coefficient of 0.76 between the PDAS total score and the independently rated CGI-S score found in this study is in line with results from prior studies and indicates good clinical validity (Reference Østergaard, Meyers, Flint, Mulsant, Whyte, Ulbricht, Bech and Rotschild11,Reference Ostergaard, Pedersen, Uggerby, Munk-Jorgensen, Rothschild, Larsen, Gotzsche, Sondergaard, Bille, Bolwig, Larsen and Bech14,Reference Köse Çinar and Dinesen Østergaard18). The scatterplot of the PDAS versus the CGI-S appears to show a ceiling effect, with some relatively low CGI-S scores correlating to quite high PDAS scores. This ceiling effect seems to be driven predominantly by the psychosis subscale (BPRS5) rather than by the depression subscale (HAM-D6), as evident in the scatterplots of the total scores on these two subscales versus the CGI-S score (Fig. 1). This indicates that when making a global (severity) assessment of PD (here represented by the CGI-S), as is often done in clinical practice, mainly the depressive symptoms are weighed. This tendency to underestimate psychotic symptoms is consistent with evidence showing that PD is often misdiagnosed as non-psychotic depression (Reference Rothschild, Winer, Flint, Mulsant, Whyte, Heo, Fratoni, Gabriele, Kasapinovic and Meyers26), which makes targeted care difficult. A similar ceiling effect was evident for the correlation between PDAS and MADRS. This highlights the importance of using the PDAS as a severity measure for psychotic depression, also in older adults, as it represents the only empirically derived rating scale covering both the depressive and psychotic symptoms (Reference Ostergaard, Rothschild, Flint, Mulsant, Whyte, Leadholm, Bech and Meyers10).
The Mokken analysis of the PDAS data resulted in a Loevinger coefficient of homogeneity of 0.46, which is indicative of scalability and consistent with the findings of prior studies of the PDAS (Reference Østergaard, Meyers, Flint, Mulsant, Whyte, Ulbricht, Bech and Rotschild11,Reference Ostergaard, Pedersen, Uggerby, Munk-Jorgensen, Rothschild, Larsen, Gotzsche, Sondergaard, Bille, Bolwig, Larsen and Bech14,Reference Köse Çinar and Dinesen Østergaard18), including the only study in which PDAS ratings were obtained via the semi-structured PDAS interview (Reference Köse Çinar and Dinesen Østergaard18). Thus, adding the individual symptom item scores on the PDAS to a total score is psychometrically valid (Reference Bech23).
According to the PDAS, 54% of the participants were in remission at the end of the study and the participants’ mean PDAS total score dropped from 19.5 at baseline to 7.5 at endpoint. The changes from baseline to endpoint on the other severity measures were of a similar magnitude. In a recent study of the PDAS in psychotic depression (Reference Köse Çinar and Dinesen Østergaard18), the proportion of participants obtaining remission at endpoint was 74%. The explanation for the lower remission rate found in our study is probably that our sample consisted of older adults only. It has been reported that in older adults, especially those above 70 years, it is difficult to obtain sufficient treatment response and full recovery from psychotic depression (Reference Victoria, Whyte, Butters, Meyers, Alexopoulos, Mulsant and Flint27,Reference Bingham, Whyte, Meyers, Mulsant, Rothschild, Banerjee and Flint28).
A recent study by Park et al. (Reference Park, Ostergaard, Choi, Kim, Jun, Lee, Kim, Yim and Park24) showed that the total score on the BPRS5 psychosis subscale of the PDAS accurately separated psychotic depression (BPRS5 total score >1) from non-psychotic depression (BPRS5 total score ≤1) in a sample of younger depressed adults. In this study, we assessed whether BPRS5 total scores ≤1 could also accurately classify older adults with psychotic depression as being in remission (defined as a PDAS total scores <8). Indeed, we found that this cut-off classified remission with a sensitivity of 0.83 and a specificity of 0.88. Thus, resolution of psychotic symptoms as operationalised by having a BPRS5 total score ≤1 is an accurate marker of remission of psychotic depression and may be used as such in clinical practice.
A limitation of this study is that it only included participants from Belgium. However, based on studies of the performance of PDAS in psychotic depression in North America (Reference Østergaard, Meyers, Flint, Mulsant, Whyte, Ulbricht, Bech and Rotschild11), Denmark (Reference Ostergaard, Pedersen, Uggerby, Munk-Jorgensen, Rothschild, Larsen, Gotzsche, Sondergaard, Bille, Bolwig, Larsen and Bech14) and Turkey (Reference Köse Çinar and Dinesen Østergaard18), it seems that the scale has cross-national validity. Another limitation of our study is that test–retest stability and inter-rater reliability was not tested. In a prior study using the PDAS, the inter-rater reliability was very high (intra-class correlation coefficient of 0.98) (Reference Ostergaard, Pedersen, Uggerby, Munk-Jorgensen, Rothschild, Larsen, Gotzsche, Sondergaard, Bille, Bolwig, Larsen and Bech14). Also, since the raters were part of the staff at the hospitals where the participants were admitted, their knowledge of the participants’ psychopathology will inherently have increased from admission to discharge, potentially leading to more accurate ratings at endpoint. It would have been ideal to have had systematic information from informants (relatives/staff members) to supplement the interviews when rating the psychopathology both at baseline and endpoint. This was unfortunately not practically feasible.
The most important strength of this study is that it is the first to establish the validity of PDAS ratings obtained via the semi-structured PDAS interview using independent global severity ratings (CGI-S) as reference. Also, it is the first study to establish the validity of PDAS in an older adults with psychotic depression, a subgroup characterised by more severe and complex psychopathology (Reference Gournellis, Oulis and Howard2).
In conclusion, this study demonstrates that the PDAS appears to be a clinically valid, scalable and responsive measure of psychotic depression among older adults. As the PDAS is the only empirically derived rating scale for the measurement of severity in psychotic depression, we believe that it currently represents the most ideal rating scale for this severe subtype of depression, although further psychometric validation – including in other languages – would be beneficial.
Acknowledgements
We greatly appreciate the support and efforts of the management and staff of the collaborating institutions.
Authors’s Contribution
The study was designed in collaboration between all authors. T.V., L.L. and Y.M. were responsible for the collection of data. The analyses were planned/carried out and manuscript was drafted by T.V., R.C.v.d.M. and S.D.Ø. Manuscript was revised critically for important intellectual content by the remaining authors. All authors approved the final version of the manuscript before submission.
Ethical Standards
The authors confirm that the study complies with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975 (revised in 2008).
