Introduction
It is estimated that between 5% and 17% of the older adults suffers from a mental disorder (Ritchie et al., Reference Ritchie2004; Trollor et al., Reference Trollor, Anderson, Sachdev, Brodaty and Andrews2007), of which anxiety disorders, depression and somatoform disorders are the most common (Beekman et al., Reference Beekman1998; Reference Beekman, Copeland and Prince1999; Hilderink et al., Reference Hilderink, Collard, Rosmalen and Oude Voshaar2012). Meta-analyses of randomized controlled trials have established the efficacy of both psychological and pharmacological interventions for depressive disorders and anxiety disorders in later life (Pinquart et al., Reference Pinquart, Duberstein and Lyness2006; Pinquart and Duberstein, Reference Pinquart and Duberstein2007). Whether these results can be extrapolated to routine clinical care remains unknown.
Most effort in clinical trials is put on the internal validity by using strict in- and exclusion criteria, specialized treatment settings and well-controlled circumstances to increase client and therapist adherence to treatment protocols (De Beurs et al., Reference de Beurs2011; Van der Lem et al., Reference van der Lem, van der Wee, van Veen and Zitman2012). Emphasis on high internal validity may result in a low generalizability. A large routine outcome monitor study in the Netherlands showed that only 25% of depressed clients aged 18–65 years treated in secondary mental health care, met criteria to be included in a randomized controlled trial (Van der Lem et al., Reference van der Lem, van der Wee, van Veen and Zitman2012). Such naturalistic studies in secondary health care and pragmatic clinical trials show less favorable results compared to RCTs (Sinyor et al., Reference Sinyor, Schaffer and Levitt2010; Van der Lem et al., Reference van der Lem, van der Wee, van Veen and Zitman2012). In order to improve the effect of mental health care, knowledge of predictors of treatment success and failure are important. Several predictors have been identified in previous research, including sociodemographic characteristics (age (Pinquart and Duberstein, Reference Pinquart and Duberstein2007), marital status (Hamilton and Dobson, Reference Hamilton and Dobson2002), educational level (Haringsma et al., Reference Haringsma, Engels, van der Leeden and Spinhoven2006)), severity of symptoms at pre-treatment (Hamilton and Dobson, Reference Hamilton and Dobson2002; Haringsma et al., Reference Haringsma, Engels, van der Leeden and Spinhoven2006), comorbid psychiatric or physical conditions (Oslin et al., Reference Oslin2004; Schuurmans et al., Reference Schuurmans, Comijs, Emmelkamp, Weijnen, van den Hout and van Dyck2009). Interpretation, however, is limited as different trials have identified not only a different set of predictors, but even opposite effects have been reported. These inconsistencies might be partly explained by the highly selective patient samples in the individual studies.
The present study aims to investigate the effectiveness of treatment in older adults with mental disorders in a naturalistic setting within one year of treatment, using data collected in routine psychiatric outpatient practice. Also factors that predict the course of treatment will be studied. The findings will be examined for the overall sample and within the largest subgroup of clients – those diagnosed with a primary diagnosis of depression.
Method
Design
For the present study, data were used from the MEntal health care Monitor Older adults (acronym MEMO). MEMO is a large-scale national study aimed at assessing the quality of mental health care provided to older adults in day-to-day practice in the Netherlands. In the Netherlands, old age psychiatry for clients suffering from neurodegenerative diseases and associated behavioral disturbances (also called psychogeriatric disorders) is organized separately from other psychiatric disorders such as substance use disorder, primary psychotic disorders, mood disorders, anxiety disorders, personality disorders, et cetera (also called gerontopsychiatric disorders). MEMO is restricted to clients with gerontopsychiatric disorders as clients with psychogeriatric disorders are not treated on divisions for gerontopsychiatric disorders and both groups require different instruments to investigate quality of care. If a client first presents to a gerontopsychiatric division with a ‘functional’ disorder, which later proves to be an early stage of dementia, this client will be transferred to the psychogeriatric division. In the Netherlands, mental health care organizations deliver both inpatient and outpatient care to older adults with psychiatric disorders. In this study multidisciplinary teams providing outpatient care were included for monitoring their clients, because most older adults receive this type of mental health care. Each multidisciplinary team consists of a psychiatrist, psychologist and specialized nurses. Other disciplines, such as a social worker, family therapist or art therapist, participate in varying degrees. Outpatient mental health care is delivered conform the available multidisciplinary guidelines in The Netherlands and includes both psychopharmacological and psychological treatment.
A total of 14 mental health care organizations nationwide participated in MEMO and included a total of 1413 clients in the period June 2008 – March 2010. The minimum age to be referred to outpatient old age psychiatry varies between organizations from 60 to 65 years. All newly referred clients had to be assessed by their clinician at baseline and routinely monitored every four months up to end of treatment or till 12 months in case of continued treatment. If clients did not agree to the use of their anonymous data for scientific research, their data were not included in MEMO. In order to improve the representativeness of the sample, we a priori decided only to include data from organizations that were able to monitor at least 60% of their included clients. For this reason seven organizations were excluded from analysis, resulting in a total client number of 941 across seven organizations.
As clients were neither subjected to interventions nor compliance with behavioral rules, the METiGG medical ethics committee decided that the “Medical Research Involving Human Subjects Act” was not applicable to MEMO and the study did not require ethical approval. More details on the design of MEMO can be found elsewhere (Veerbeek et al., Reference Veerbeek, Oude Voshaar, Depla and Pot2013a).
Primary outcome measures
HoNOS 65+ - The Health of the Nation Outcome Scales for older adults (HoNOS 65+) has been developed in England and successfully translated to Dutch, to evaluate the outcome of treatment for older adults with respect to both social and mental functioning, irrespective of the type of psychiatric diagnosis (Burns et al., Reference Burns1999; Staring et al., Reference Staring, Stobbe and Mulder2003). HoNOS 65+ is a twelve-item observation scale that can be filled out by a clinician in less than ten minutes. The twelve scales include behavioral disturbance, non-accidental self-injury, problem drinking or drug use, cognitive problems, problems related to physical illness or disability, problems associated with hallucinations and/or delusions, problems associated with depressive symptoms, other mental and behavioral problems, problems with social or supportive relationships, problems with activities of daily living, overall problems with living conditions, problems with work and leisure activities – quality of daytime environment. The total score ranges from zero to forty-eight and represents overall functioning, with higher scores indicating a greater number of problems. Research has shown acceptable psychometric properties in older populations (Pirkis et al., Reference Pirkis, Burgess, Kirk, Dodson, Coombs and Williamson2005). Before data collection, all clinicians were trained in how to administer HoNOS 65+ reliably. During MEMO, annual booster sessions were organized for all clinicians.
GDS-15 - Clients with a primary diagnosis of depressive disorder were also monitored with the Dutch version of the shortened Geriatric Depression Scale (GDS-15) (Sheikh and Yesavage, Reference Sheikh and Yesavage1986; Kok et al., Reference Kok, Heeren and van Hemert1993). The original 30-item version of the Geriatric Depression Scale (GDS) (Yesavage et al., 1983) has been developed as a screening tool, but has also been found to be a valid and reliable instrument to monitor treatment effects (Smalbrugge et al., Reference Smalbrugge, Jongenelis, Pot, Beekman and Eefsting2008). The GDS does not have somatic items interfering with (highly prevalent) somatic illnesses at older age and is easy to administer (all items are rated as ‘yes’ or ‘no’). The GDS-15, has similar psychometric properties compared to the original 30-item version (Sheikh and Yesavage, Reference Sheikh and Yesavage1986). The Dutch version of the GDS-15 has been validated in both nursing home residents and among older persons in the general population. Both samples showed sufficient sensitivity, internal consistency and test-retest reliability (Jongenelis et al., Reference Jongenelis2005; van de Rest et al., Reference van de Rest, van der Zwaluw, Beekman, de Groot and Geleijnse2010). The total score of GDS-15 ranges from zero to fifteen; a score of 6 or more may indicate a depressive disorder.
Determinants of treatment outcome
Socio-demographic characteristics, comorbidity and life-events were examined as determinants of treatment outcome. Socio-demographic characteristics include gender, age, nationality (Dutch, Western immigrant, non-Western immigrant), living arrangement (alone, with partner, with others), and highest education attained (primary school or less, secondary education, higher education).
Psychiatric and somatic comorbidity were derived from the DSM-IV-TR classification assigned by the treating physician after the diagnostic procedure had been concluded. For the present paper, we coded the presence of a psychiatric disorder as axis I comorbidity (yes/no) and any personality disorder and/or personality traits as axis II comorbidity (yes/no). Somatic comorbidity was categorized on axis III as ‘none’, ‘one’ and ‘two or more’.
Life events during the period of treatment (for example loss of spouse, financial problems, having moved homes, chronic illness, et cetera) were measured at post-treatment by a slightly adapted version of the life events list used in the Longitudinal Aging Study Amsterdam (De Beurs et al., Reference de Beurs, Beekman, Geerlings, Deeg, van Dyck and van Tilburg2010). The presence of life-events was categorized as ‘none’, ‘one’ and ‘two or more’.
Statistical analysis
As the HoNOS 65+ sum score were positively skewed and normalized after log-transformation, all analyses were based on the log-transformed values. In absence of a final measure, the last available follow-up measurement was extrapolated as post treatment.
Paired t-tests were used to calculate differences between pre- and post-treatment total scores on HoNOS 65+ and GDS-15. HoNOS 65+ subscales were not used to calculate on which domains the main improvements were, since a recently published study found no support for the division of HoNOS 65+ items into subscales (Veerbeek et al., Reference Veerbeek, Oude Voshaar, Depla and Pot2013b). To measure the magnitude of the observed effect, effect sizes on HoNOS 65+ and GDS-15 were calculated as Mpre-Mpost/sdpre (Becker, Reference Becker1988). Effect sizes ≥.56 can assumed to be large, while effect sizes between 0.33 and 0.56 are moderate and effect sizes < 0.33 are small (Lipsey, Reference Lipsey1990).
Prognostic variables for effect size were calculated in two ways. First univariate, using linear regression with effect size as dependent variable and gender (1 = male), age, nationality (1 = immigrant), living arrangement (1 = with partner/others), highest education attained (1 = high), psychiatric comorbidity on axis I (1 = yes), comorbid personality disorder on axis II (1 = yes), somatic comorbidity on axis III, total score at intake and life events as independent variable in separate analyses. Dummy variables representing the seven organizations were included in each analysis. Second, backwards multivariate regression analysis was used with effect size as dependent variable and all the variables mentioned before as independent variables to attain the best prognostic model for effect size.
All analyses were based on complete case series as well as on a worst-case scenario. In the worst case scenario the effect size of clients with any missing data is set at zero (no effect).
All analyses described above were performed with SPSS Version 19.
Results
Participants
Figure 1 shows participant flow of 941 clients through the study. At intake there were no differences in demographic variables (age, gender, living arrangement, nationality, level of education) between the 237 clients who were excluded and the 704 clients who were included in the study. Total score at intake on HoNOS 65+ was higher for the clients who were excluded (t = −2.34; df = 890; p = 0.02).
Participant flow and follow-up.
Mean age of the 704 participants included in the sample was 75 years (range 53–106), two thirds were female, and depressive disorders (40%), adjustment disorders (24%) and anxiety disorders (11%) were most prevalent as primary diagnosis (see Table 1 for further description of the sample). Compared to the total group of included clients, psychiatric and somatic comorbidity appeared to be lower in clients with a primary diagnosis of depression.
Description of the total group and clients with a primary diagnosis of depression: demographics, diagnoses and comorbidity
Means and effect sizes
Table 2 shows the mean total score on HoNOS 65+ of the total group and on HoNOS 65+ and GDS-15 of clients with a primary diagnosis of depression. According to HoNOS 65+ the functioning of the total group of clients and clients with a primary diagnosis of depression improved significantly after receiving outpatient mental health care (t = 18.34; df = 703; p <.001 and t = 12.44; df = 283; p < 0.001 respectively). Clients with depression also showed a significant reduction in depressive symptoms (t = 12.16; df = 235; p < 0.001).
Mean total score at pre- and post-treatment and effect size of the total group and clients with depression on HoNOS 65+ and GDS-15
Mean effect size of the total group was d = 1.08. Mean effect sizes of clients with depression were d = 1.23 for HoNOS 65+ and d = 0.92 for GDS-15, respectively. The majority of clients showed a large effect size, while approximately one third showed a small effect size (Table 2). Clients who got worse were also included in the latter category.
Determinants of outcome
For the total group the variables total score on HoNOS 65+ at intake, comorbid personality disorder, comorbid somatic disorder and life events had a significant impact on the change in HoNOS 65+ scores from pre- to post-treatment (Table 3). Clients with a higher total score at intake, clients with no (traits of a) personality disorder, clients with less severe somatic comorbidity and clients who experienced a lower number of critical life events during treatment showed a larger improvement in functioning than their counterparts. These variables account for 14% of the variance in the outcome on HoNOS 65+.
Prognostic variables in a univariate and multivariate regression model for the primary outcome ‘effect size’, divided to the total group (HoNOS 65+) and clients with depression (HoNOS 65+ and GDS-15)
*Adjusted for organization.
For clients with depression the same variables were found as significant predictors on HoNOS 65+ and explained 20% of the variance. On GDS-15 the variables total score at intake, somatic comorbidity and life events had a significant impact on change in scores and explained 13% of the variance in outcome. Clients who had more depressive symptoms at intake, less severe somatic comorbidity and clients who experienced a lower amount of critical life event during treatment showed a greater reduction in depressive symptoms than their counterparts.
Worst-case scenario
As described before data from 237 clients were excluded from the previous results, because treatment was ended prematurely or pre- or post-test measurements were missing. When these clients were taken into account in analyses mean effect size for the total group was d = 0.82. There was no change in variables that predicted outcome on HoNOS 65+ in the total group (Table 4).
Prognostic variables in a multivariate regression model for the primary outcome ‘effect size’ in a worst case scenario, divided to the total group (HoNOS 65+) and clients with depression (HoNOS 65+ and GDS-15)
*Adjusted for organization.
Mean effect sizes of clients with depression were d = 0.98 for HoNOS 65+ and d = 0.63 for GDS-15, respectively. As in the total group, there was no change in variables that predicted outcome on both HoNOS 65+ and GDS-15.
Discussion
The current study shows that functioning of older persons referred for psychiatric problems to an outpatient clinic for old age psychiatry substantially improves: large effect sizes were found in 60% of the clients. A better outcome was determined by a lower level of functioning at pre-treatment, the absence of personality problems less severe somatic comorbidity and a lower amount of life-events experienced during treatment.
Treatment effect in old age psychiatry
In the current study large effect sizes were found in the total group and in the largest subgroup of clients, namely those suffering from depression. Two previous studies also reported effect sizes on HoNOS 65+. A study in The Netherlands carried out in one mental health care organization reported a large effect size (d = 0.80) for outpatient clients (Broersma and Sytema, Reference Broersma and Sytema2010). In this study, patients with psychogeriatric disorders were included as well.
In the other study, no effect size was reported for the total score on HoNOS 65+, only for six of the twelve individual items for older adults who received another type of care, i.e. community mental health care (Gee et al., Reference Gee, Croucher and Beveridge2010). The effect sizes of these individual items ranged between −.24 and −.59.
In a previous meta-analysis large within group effect sizes of pharmacological interventions (d = 0.59) and behavioral interventions (d = 0.78) in older adults with depression were also found, although lower then the ones found in our study (Pinquart and Duberstein, Reference Pinquart and Duberstein2007). Comparison with effect sizes reported in other meta-analyses is hard, due to differences between the experimental and control group, instead of pre-test-post-test differences within the experimental group (Lipsey and Wilson, Reference Lipsey and Wilson1993; Pinquart et al., Reference Pinquart, Duberstein and Lyness2006). When taken together the participants who dropped out of treatment, or where pre- or post-test were missing into account, effect sizes in all groups decreased but remained large.
In the subgroup of clients with depression, the effect size of self-rated depressive symptom severity was high, although a little bit lower compared to the observer-rated effect of overall functioning. This is in line with a previous meta-analysis of pharmacological treatment in older adults with depression, which also showed a larger effect size for clinician-rated depression (d = 1.35) compared to client-rated depression (d = 0.99), although opposite results were found for psychological treatment (d = 0.54 for clinician-rated depression and d = 0.81 for client-rated depression) (Pinquart and Duberstein, Reference Pinquart and Duberstein2007).
Determinants of outcome
Results of the present study revealed that in the total group of clients the variables level of functioning at baseline, comorbid somatic disorder, comorbid personality disorder and the experience of life events during treatment predict the effect size of change in level of functioning.
One explanation for the finding that clients with worse functioning at pre-treatment show a larger improvement may be that their scores allow for larger improvement, but results could also be caused by regression to the mean. This means that someone with an extreme score on baseline tends to have a score closer to the mean at follow-up measure. Therefore, firm conclusions cannot be drawn. Results from randomized controlled trials on this topic are inconclusive, with more severely ill older adults responding less (Ackerman et al., Reference Ackerman, Greenland, Bystritsky and Small1997; Hamilton and Dobson, Reference Hamilton and Dobson2002; Joel et al., Reference Joel and Begley2014), equally (Stein et al., Reference Stein, Kasper, Andersen, Nil and Lader2004; Korte et al., Reference Korte, Bohlmeijer, Cappeliez, Smit and Westerhof2012) or better (Haringsma et al., Reference Haringsma, Engels, van der Leeden and Spinhoven2006) to treatment compared to older adults with less severe symptoms at baseline. A recent meta-analysis showed that in older adults with chronic depression (>10 years) more severe depression at baseline contributed to a better response to antidepressant treatment (Nelson et al., Reference Nelson, Delucchi and Schneider2013). Nonetheless, even when corrected for pre-treatment functioning, several determinants of outcome were identified.
In the current study the majority of the clients present with somatic comorbidity. Research showed that physical illness, such as diabetes mellitus or coronary heart disease, are associated with increased prevalence rates of depression and clients with depression are at a higher risk of developing these medical illnesses (Penninx and Van Dyck, Reference Penninx and van Dyck2010; Katon, Reference Katon2011). In line with our results, the presence of chronic medical illnesses is previously reported to have a negative effect on the treatment of depression (Katon, Reference Katon2011).
In this study one in four clients had comorbid (traits of a) personality disorder and this was related to a smaller effect size in the total group and subgroup of clients with depression. Previous research in younger adults with a depressive disorder showed that clients with a comorbid personality disorder were more likely to have a poor outcome than clients without a comorbid personality disorder (Rosenbluth et al., Reference Rosenbluth, Macqueen, McIntyre, Beaulieu and Schaffer2012). Research in older adults also pointed out the negative influence of personality traits to the treatment and chronicity of depression (Hayward et al., Reference Hayward, Taylor, Smoski, Steffens and Payne2013) and anxiety (De Beurs et al., Reference de Beurs, Beekman, Deeg, van Dyck and van Tilburg2000). This indicates that in the treatment of late life psychiatric disorders underlying personality traits should be taken into account.
The occurrence of stressful life events during treatment had a negative impact on outcome in both the total group and clients with a depressive disorder specifically. This is in line with previous research that showed the negative role of stressful life events in the development of mood and anxiety disorders in older adults, although results are inconsistent on the influence of life events on symptom severity after the development of a disorder (De Beurs et al., Reference de Beurs, Beekman, Deeg, van Dyck and van Tilburg2000; Rosenbluth et al., Reference Rosenbluth, Macqueen, McIntyre, Beaulieu and Schaffer2012).
Interestingly, determinants of a better treatment response found in randomized controlled trials among older adults, like a higher age (Pinquart and Duberstein, Reference Pinquart and Duberstein2007), being married (Hamilton and Dobson, Reference Hamilton and Dobson2002) and a higher level of education (Haringsma et al., Reference Haringsma, Engels, van der Leeden and Spinhoven2006) were not confirmed by the present study. Also nationality did not predict outcome, probably due to the fact that almost all clients were indigenous.
Strength and limitations
Strength of the current study is that we used naturalistic data, gathered in routine clinical practice in mental health care organizations across the country. The Dutch healthcare system provides unrestricted access to outpatient mental health care and there were hardly any exclusion criteria (Van der Lem et al., Reference van der Lem, van der Wee, van Veen and Zitman2011). This enhances representativeness and generalizability of the results to outpatient practice for old age psychiatry. In this light, we also included clients under the age of 60 in our sample, as their mental health problems were regarded as age-specific, e.g. a depressive disorder in the context of a severe stroke, and therefore treated on the division of old age psychiatry instead of the division for younger adults.
Nonetheless, the main limitation of this design is the lack of a comparison group, which hinders adequate control for the naturalistic course of mental health problems. Some effects may thus be explained by regression to the mean. However, it is unlikely that the large effect sizes we found can be fully explained by these mechanisms. Moreover, taking the long-term follow-up we have, it is not possible to include a sound comparison group as a one-year waiting list control group of referred patients would be unethical. A population based sample of older persons with psychiatric problems cannot be matched sufficiently on severity and care-needs. Another limitation is the loss to follow-up, although this is a problem in all studies with a naturalistic design (Van der Lem et al., Reference van der Lem, van der Wee, van Veen and Zitman2011; Van der Lem et al., Reference van der Lem, van der Wee, van Veen and Zitman2012). Also, we cannot ascertain that the participating organizations included all clients that met inclusion criteria, or that they left ‘difficult’ clients out. However, since it was expected that this would bias results in a positive way, analyses were also performed in a ‘worst-case scenario’. This dropped the mean effect sizes found, but they could still be labeled as large. The variance explained by our prediction models for overall functioning was not very high, 14% in the total group and 20% in the subgroup of clients with depression, but given the fact that reality is many times more complex and many factors come into play, the variances can be considered acceptable. Furthermore, the current study reflects whether clients profit from care, but it is unknown what treatment clients received specifically. Although there are guidelines according to which clients with specific disorders should be treated in mental health care, it is unknown to which extent these were followed by the participating organizations.
Conclusion
Taking into account the representativeness of our sample (by hardly having any exclusion criteria), the present study shows that within one year of treatment the majority of older adults profit from the care they receive for their mental health problems in secondary outpatient mental health care. Therefore, older adults should be stimulated to seek help for their mental health problems. The negative effect of comorbid physical diseases as well as personality problems on treatment outcome, argues for the development of specific treatment strategies for older persons focused on comorbid somatic diseases and older persons with personality problems.
Conflict of interest
None.
Description of authors’ roles
M. Veerbeek assisted in designing the study, carried the study out, analyzed the data and wrote the paper. R. Oude Voshaar supervised the study and statistical analyses and wrote this paper. A.M. Pot designed the study and supervised the carrying out of the study and the writing of the paper.
Acknowledgments
This study was financed by the Dutch Ministry of Health, Welfare and Sport. We are grateful for this financial assistance in enabling MEMO.
