Significant outcomes
∙ This study of 50 patients with psychotic depression (34 unipolar and 16 bipolar) is the first to validate the semi-structured Psychotic Depression Assessment Scale (PDAS) interview.
∙ The semi-structured PDAS interview provides clinically valid, psychometrically valid, and responsive ratings of the severity of psychotic depression.
∙ At baseline, none of the 50 participants met the PDAS remission criterion (total score <8). At endpoint (mean of 6.6 weeks (SD=1.6) after baseline) 74% (95% CI: 60–85) of the participants met this criterion.
Limitations
∙ Both the ratings on the symptom rating scales (PDAS, HAM-D17, and BPRS) and the global scales (CGI-S and CGI-I) were performed by the same rater (R.K.C.).
∙ The study was conducted at one site only – and used the Turkish version of the semi-structured PDAS interview. This limits the generalisability of the findings.
∙ The number of participants with bipolar psychotic depression was limited (n=16). Larger studies of the validity of the semi-structured PDAS interview for this population are warranted.
Introduction
Both unipolar depression and bipolar depression can present with psychotic features (psychotic depression). In both unipolar psychotic depression and bipolar psychotic depression, the clinical picture is characterised by severe depression with psychomotor disturbance (retardation in particular), delusions (most often mood-congruent) and hallucinations (Reference Rothschild1,Reference Ostergaard, Leadholm and Rothschild2). Patients with psychotic depression are severely ill and at high risk of suicide (Reference Gesi, Carmassi, Miniati, Benvenuti, Massimetti and Dell’Osso3–Reference Leadholm, Rothschild, Nielsen, Bech and Ostergaard5), requiring acute and intense treatment and monitoring (Reference Rothschild1).
A valid rating scale is essential when monitoring the severity of mental disorders such as psychotic depression. However, until recently, there was no specific rating scale avaliable to measure the severity of psychotic depression (Reference Ostergaard, Rothschild and Flint6). Based on this observation, we developed the Psychotic Depression Assessment Scale (PDAS) (Reference Ostergaard, Meyers and Flint7) via a reanalysis of the data from the Study of Pharmacotherapy of Psychotic Depression (STOP-PD) (Reference Meyers, Flint and Rothschild8). The 11-item PDAS covers both depressive and psychotic symptoms among patients with psychotic depression (Reference Ostergaard, Meyers and Flint9). The scale consists of the six-item melancholia subscale (HAM-D6) (Reference Bech, Gram, Dein, Jacobsen, Vitger and Bolwig10) of the 17-item Hamilton Depression Rating Scale (HAM-D17) (Reference Hamilton11), plus five psychosis-related items from the Brief Psychiatric Rating Scale (BPRS) (Reference Overall and Gorham12). The 11 items are: depressed mood, guilt feelings, work and activities, psychomotor retardation, psychic anxiety and somatic symptoms (general) from the HAM-D17, and hallucinatory behaviour, unusual thought content, suspiciousness, emotional withdrawal and blunted affect from the BPRS (Reference Ostergaard, Meyers and Flint7). Following the analyses based on the STOP-PD sample, we confirmed that the PDAS is a clinically valid, responsive and scalable measure of psychotic depression in a Danish sample of patients (Reference Ostergaard, Pedersen and Uggerby13).
In most prior studies of the PDAS, the scores on the 11 items have been extracted from separate ratings on the HAM-D17 and BPRS. However, ideally, the PDAS could be rated without having to conduct very time consuming interviews needed to rate HAM-D17 and the BPRS. To this end, a semi-structured interview focussing specifically on the 11 PDAS items, has been developed and is currently available in English, Danish, Dutch, Turkish, Korean, Japanese, and Brazilian-Portuguese at https://psychoticdepressionassessmentscale.com/. However, this semi-structured interview has not yet been subjected to formal clinical and psychometric validation. Therefore, the aim of this study was to perform such validation. Specifically, the following research questions were addressed:
(I) Are ratings obtained via the semi-structured PDAS interview clinically valid?
(II) Is the scalability (psychometric validity) of the PDAS maintained when using the semi-structured interview for rating?
(III) Are PDAS ratings obtained via the semi-structured interview sensitive to change in the severity of illness (responsiveness)?
Methods
Study sample
Based on the sample size (n=50) of a recent validation study of the PDAS (Reference Ostergaard, Pedersen and Uggerby13), we aimed at recruiting a total of 50 patients with psychotic depression (unipolar or bipolar) from the inpatient clinic of the Department of Psychiatry, Trakya University Hospital, Edirne, Turkey.
We have previously found that the PDAS appears to be valid for both unipolar and bipolar psychotic depression (Reference Ostergaard, Pedersen and Uggerby13). Therefore, we did not distinguish between the two phenotypes in our psychometric analyses (research questions I–III).
Potential participants were initially examined independently by two psychiatrists. Subsequently, the diagnosis of psychotic depression (unipolar or bipolar) was confirmed through use of the Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders, fourth edition (SCID-I) (Reference First, Spitzer, Gibbon and Williams14) by an experienced psychiatrist (R.K.C.) in accordance with DSM-5 criteria. Other inclusion criteria were: being 18 years of age or older, understanding written and spoken Turkish. The exclusion criteria were: meeting criteria for mixed affective episode, hypomania, mania, schizophrenia, or schizoaffective disorder, any of the substance (except nicotine) use disorders, organic mental disorders, or mental retardation. The study was approved by the local ethical committee (TÜTF-BAEK 2015/117). The study conformed to the provisions of the Declaration of Helsinki, and only subjects who gave informed consent (and/or consent was provided by their legal guardians) participated in the study.
Measures
Turkish versions of the PDAS, the HAM-D17 (Reference Hamilton11), the BPRS (Reference Overall and Gorham12), and the Clinical Global Impressions scale – severity of illness (CGI-S) (Reference Guy15) were used in data collection (and rated in this order). The 50 participants enrolled in the study were interviewed and rated twice: as soon after admission as possible and just before discharge from the hospital using the same scales as mentioned above. Thus, a total of 100 interview/rating sessions were completed. All the interviews/ratings were performed by an experienced psychiatrist (R.K.C.). Data on sociodemographics and psychiatric comorbidity was extracted from the SCID-I records.
Statistical approach for each of the three research questions
(I) Are ratings obtained via the semi-structured PDAS interview clinically valid?
This was tested by means of Spearman’s correlation analysis of total scores on the PDAS and CGI-S scores at study baseline. The correlation between total scores on the HAM-D17 and CGI-S, and total scores on the BPRS and CGI-S were tested for comparison.
(II) Is the scalability (psychometric validity) of the PDAS maintained when using the semi-structured interview for rating?
Mokken scale analysis (Reference Mokken16) was used to evaluate the scalability of the PDAS and the HAM-D17 and BPRS for comparison. This non-parametric item response theory model is based on Loevinger’s coefficient of homogeneity, calculated as the weighted average of the individual item coefficients (Reference Mokken16). The Loevinger coefficient of homogeneity expresses whether each item adds unique information regarding the latent dimension being measured (here the severity of psychotic depression). A coefficient of homogeneity of 0.40 or higher is considered as a demonstration of scalability – that is, the items are additive and the total score of the rating scale is a meaningful measure for the severity of the syndrome of interest (Reference Loevinger17). Separate Mokken analyses were performed using data from baseline and endpoint, respectively.
(III) Are PDAS ratings obtained via the semi-structured interview sensitive to change in the severity of illness (responsiveness)?
This was tested by assessing the baseline to endpoint change in total scores on the PDAS, by means of the Wilcoxon test and comparing the obtained result with CGI-I ratings at endpoint. Furthermore, the proportion of participants having reached remission on the PDAS (total score <8 (Reference Ostergaard, Rothschild and Flint18)) at baseline and endpoint was compared.
Statistical programmes
The scalability of the PDAS was tested by means of Mokken analysis using the dedicated Mokken Scale Analyses for Polytomous Items Programme (Reference Molenaar, Debels and Sijtsma19). Wilcoxon’s test of changes of scale scores over time was performed using the SAS statistical package (version 9.00, 2002). Correlation between scale scores were tested by means of Spearman’s correlation analysis, also performed using SAS. All other statistical analyses were performed using the SPSS version 22.0 (see Table 1).
Comparison of rating scale scores, demographic and clinical characteristics in patients with unipolar psychotic depression (UPD) and bipolar psychotic depression (BPD) at baseline
PDAS, Psychotic Depression Assessment Scale; HAM-D17, 17-item Hamilton Depression Rating Scale; BPRS, Brief Psychiatric Rating Scale; CGI-S, Clinical Global Impressions scale – severity.
Statistics: The Kolmogorov–Smirnov one sample test was used to test for normality. The Independent Samples t-test and Mann–Whitney U test were used to compare the groups. Continuity Correction χ2 test was used for comparison of categorical variables. Two-sided p-values of <0.05 were considered statistically significant throughout. There were complete psychometric data available for all 50 participants with the following exceptions: for one participant, there was only PDAS data available. Furthermore, for one participant there was one item score missing on HAM-D17 at baseline, and for another participant, there was one item score missing on the BPRS at endpoint.
*p<0.05 (Mann–Whitney U test).
Results
Sociodemographics and clinical characteristics of the sample at baseline
A total of 50 patients with psychotic depression (34 unipolar + 16 bipolar) were recruited for the study. The baseline sociodemographic and clinical characteristics of the participants stratified on subtype (unipolar/bipolar) are shown in Table 1.
Results of the three research questions
(I) The clinical validity of the PDAS
The Spearman coefficients for the correlation between total scores on the PDAS and CGI-S at study baseline was 0.66 (p<0.001). The Spearman coefficients for the correlation between total scores on the HAM-D17 and CGI-S was 0.49 (p<0.001) and the Spearman coefficient for the correlation between total scores on the BPRS and CGI-S was 0.60 (p<0.001).
(II) The scalability of the PDAS
The Loevinger coefficients of homogeneity derived from the Mokken analyses are shown in Table 2.
Results of the Mokken analysis of the PDAS, HAM-D17, and BPRS
While the HAM-D17 and the BPRS met the criterion for scalability (Loevinger’s coefficient of homogeneity ≥0.40) at neither baseline nor endpoint, the PDAS was scalable at endpoint (Loevinger’s coefficient of homogeneity=0.45).
(III) The responsiveness of the PDAS
The mean time between baseline and endpoint ratings in the study was 6.6 weeks (SD=1.6). Over this time, the total scores on the three symptom rating scales all decreased significantly (p<0.001). Specifically, the mean PDAS score dropped from 22.8 (SD=5.7) at baseline to 4.5 (SD=4.2) at endpoint. Similarly, the mean HAM-D17 score dropped from 26.2 (SD=7.0) to 5.3 (SD=4.4), and the mean BPRS score dropped from 37.8 (SD=12.6) to 6.5 (SD=6.0). The mean CGI-I score at endpoint was 2.1 (SD=0.9), corresponding to ‘much improved’ (Reference Guy15). Furthermore, at baseline, none of the 50 participants met the PDAS remission criterion (total score <8). At endpoint, 74% (95% CI: 60–85) of the participants met this criteron (76% of the patients with unipolar psychotic depression and 69% of the patients with bipolar psychotic depression).
Discussion
The aim of this study was to test the validity of PDAS ratings obtained using the semi-structured PDAS interview in a sample of patients with psychotic depression. Notably, the age (Reference Ostergaard, Bertelsen, Nielsen, Mors and Petrides20,Reference Ostergaard, Petrides and Dinesen21) and sex distribution (Reference Ostergaard, Pedersen and Uggerby13) of the sample is in line with prior studies on psychotic depression. It entails that is seems resonable to compare the findings of the present study with those from other studies on psychotic depression.
Discussion of the three research questions
(I) The clinical validity of the PDAS
The clinical validity of the PDAS was tested by assessing the correlation between PDAS scores and CGI-S scores at baseline. The correlation was strong and within range of those reported in other studies of the PDAS, where less specific interviews were used to collect the ratings (Reference Ostergaard, Pedersen and Uggerby13).
(II) The scalability of the PDAS
As opposed to the HAM-D17 and the BPRS, the PDAS was scalable at study endpoint. The lack of scalability at baseline may be due to low variability in PDAS scores. Actually, for this very reason, ratings from week 4 are most often used when assessing scalability based on data from clinical trials (Reference Ostergaard, Meyers and Flint7). This is roughly equivalent to the endpoint in the present study [6.6 weeks (SD=1.6)]. Furthermore, as a sensitivity analysis, we pooled PDAS ratings from baseline and endpoint (to increase variability) and reran the Mokken analysis. The resulting Loevinger coefficient of homogeneity was 0.61 (compared with 0.17 at baseline and 0.45 at endpoint), which further supports the scalability of PDAS ratings obtained via the semi-structured interview.
(III) The responsiveness of the PDAS
The mean PDAS total scores decreased from 22.8 (SD=5.7) at baseline to to 4.5 (SD=4.2) at endpoint, which reflected the CGI-I ratings at endpoint (mean=2.1 corresponding to ‘much improved’). Furthermore, at baseline, none of the 50 participants met the PDAS remission criterion (total score <8). At endpoint, 74% (95% CI: 60–85) of the total sample met this criteron. This demonstration of sensitivity to change in the severity of psychotic depression is consistent with findings from prior studies of the PDAS in which less specific interviews were used to collect the ratings (Reference Ostergaard, Pedersen and Uggerby13).
It is notable that 74% of the participants with psychotic depression met PDAS remission criteria at endpoint, which was 6.6 weeks after baseline on average (SD=1.6). This is an impressive treatment response for a a group of severely ill patients. Since response to specific treatments was not the focus of this study, we do not have systematic data on the treatment of the patients. However, patients are commonly referred to the Department of Psychiatry at Trakya University Hospital in Turkey for electroconvulsive therapy (ECT), so the vast majority of the patients with psychotic depression in the sample studied here received ECT. Thus, the treatment response observed here is in accordance with prior studies showing that ECT is an extremely effective treatment for patients with psychotic depression (Reference Loo, Mahon, Katalinic, Lyndon and Hadzi-Pavlodic22,Reference Petrides, Fink and Husain23) as also reflected in most treatment guidelines on psychotic depression (Reference Leadholm, Rothschild, Nolen, Bech, Munk-Jorgensen and Ostergaard24).
There are some limitations to this study. First and foremost, both the ratings on the symptom rating scales (PDAS, HAM-D17, and BPRS) and the global scales (CGI-S and CGI-I) were performed by the same rater (R.K.C.). Ideally, these two sets of ratings had been performed by independent raters (to avoid that the ratings on PDAS, HAM-D17, and BPRS drive the CGI-S and CGI-I ratings), but this was not practically feasible. This obviously increases the risk of circular reasoning (regarding clinical validity). However, the degree of correlation between the PDAS and CGI-S reported in this study (clinical validity) is within range of that found in other studies correlating PDAS scores with global severity mesaures (Reference Ostergaard, Meyers and Flint7,Reference Ostergaard, Pedersen and Uggerby13), so we have no reason to believe that our demonstration of clinical validity is due to circular reasoning.
Another important limitation of this study is that it was conducted at one site only – and used the Turkish version of the semi-structured PDAS interview (as well as the Turkish version of the PDAS rating criteria). This of course limits the generalisability of our findings and validation of the semi-structured PDAS interview in other languages should be prioritised.
Despite the limitations outlined above, the results of this study indicate that PDAS ratings obtained via the semi-structured PDAS interview are clinically valid, scalable, and responsive to change in the severity of psychotic depression over time.
Acknowledgements
None.
Authors’ Contributions
Both authors contributed to the design of this study. All psychometric ratings were performed by R.K.C. Both authors were involved in the writing of the manuscript and approved the final version before submission.
Financial Support
S.D.Ø. is supported by a grant from the Lundbeck Foundation (R165-2013-15320).
Conflicts of Interest
The authors declare no conflicts of interest.
