Introduction
Depression is common in older people and is associated with significant disability and social isolation (McDougall et al., Reference McDougall2007). However, it continues to be under-recognized both in the primary care setting (Chew-Graham et al., Reference Chew-Graham, Baldwin and Burns2004) and in those receiving community health and social services (Bruce et al., Reference Bruce2002) and so may remain untreated, or the appropriate responses to it, in terms of the planning of personal or home care, may not be given. One response to under-recognition has been to use screening scales to aid detection (Lynch et al., Reference Lynch, Clarkson, Blenkiron and Fraser2002). Such tools are often used in the context of research studies where their properties, for example their validity compared with established scales, are measured. Several studies, for example in the US home health care setting, have evaluated the effects of using depression screening scales allied to training in the greater recognition of depression, leading to more appropriate referrals to specialist services and consequent improvements in patient outcomes such as emotional and physical functioning (Bruce et al., Reference Bruce2007; Ell et al., Reference Ell2007; Brown et al., Reference Brown, Raue, Roos, Sheeran and Bruce2010). Another way to address the problems of under-recognition is for governments to instigate national policies aimed at improving, overall, the assessment process. This was the case, for example, in Australia with the adoption of multidisciplinary Aged Care Assessment Teams (ACATs) in the 1980s. These teams have responsibility for assessing the appropriateness of admission to nursing homes or hostels (residential homes) through a coordinated, comprehensive assessment (Cape and Gibson, Reference Cape and Gibson1994). Such polices are aimed at providing greater structure to the often multiple assessments of older people in order to improve recognition of important needs, such as difficulties with activities of daily living (ADLs), incontinence, social and environmental difficulties, cognitive impairment and mood.
There are difficulties in evaluating such approaches to improving recognition in that national policies cannot be randomized as they tend to apply to whole populations. Researchers therefore have little say over the implementation of the intervention and its timing and those subject to it cannot be allocated to groups in ways that minimize the influence of confounding factors such as differences in patient or user circumstances. Instead, community-based observational studies are vital to informing policy-makers and practitioners about the effects of such policies under routine conditions. However, as opposed to the wealth of evidence from specially conducted research studies on improving recognition of depression, there is little evidence of the effects of national policies on changes to overall assessments made in the community and few observational studies are done in this area. In particular, there is little evidence concerning whether polices to change community-based assessments improve recognition of needs such as depression in older people.
In England, a national policy introducing shared tools, procedures and a greater standardization of the assessment process was instigated in April 2004 – the Single Assessment Process (SAP) – aimed at bringing together and coordinating the disparate assessments of multiple professionals, greater uniformity and more effective information sharing to improve the assessment process for older people (Department of Health, 2002a). The policy represented an attempt to build on the potential benefits of multidimensional geriatric assessment programs (Stuck et al., Reference Stuck, Siu, Wieland, Adams and Rubenstein1993). In this study we undertook an observational study within one English social services area to test the impact of this policy in terms of: (i) whether depression was initially identified in statutory community care assessments by care managers, and (ii) whether it was correctly recognized, based on congruence of this identification with a gold standard research measure.
Methods
Intervention
The SAP was a national policy in England designed to integrate the assessment processes of different professionals with the aim of providing greater structure and standardization to the assessment process. The policy involved the use of shared tools and assessment protocols. Before the SAP (in this study, referred to as Time 1, where data were collected between July 1998 and November 2000), assessments were largely conducted by social services’ care managers in professional isolation and were relatively unstructured, although in the assessments locally there were required headings under which care managers had to detail their concerns regarding the older people referred to them and to list services felt necessary to support them at home. After the SAP (referred to as Time 2, where data were collected between February 2006 and July 2006, between 21 and 27 months after the policy's initial introduction, reflecting that the contract for the evaluation was instigated once the policy was underway and to allow for it to become embedded in local practices) the locality approach included assessment training, a greater focus on sharing information with health colleagues (including clinicians) and the use of a locally developed structured assessment tool. In the latter respect, this local approach was similar to that of the majority (66%) of local authorities which implemented the policy (Department of Health, 2008). The study therefore examined the impact of the policy (between Time 1 and Time 2) on the detection of depression and associated difficulties; the policy's impact at the local level was conceived of as particularly involving the effects of enhanced information sharing and the use of a more structured assessment tool.
Setting
The study was conducted within Manchester City Council Adult Social Care, a large social services local authority serving a relatively deprived city population in north-west England. The resident population of Manchester is over 400,000 with around 25,000 people aged 75 years and over. Like all local authorities, Manchester was expected to have processes in place to support the SAP policy from 1 April 2004. These processes included: expectations of closer professional working between clinicians, care managers, nurses and others; shared recording procedures; new assessment tools; and a shared structure to the assessment process for health and social care agencies around four types of assessment – “contact”, “overview”, “specialist” and “comprehensive”.
In this study, professional community care assessments were evaluated and conducted by care managers, who are qualified social workers responsible for coordinating the work of multiple professionals and care providers. These assessments are statutorily required and are the lead responsibility of the local social services authority. Twenty-three care managers across six teams served by the authority were recruited to refer cases to the research team. Those eligible were older people over 60 years old and resident within the City of Manchester who had recently had an assessment under legislation (Department of Health, 1990). At two time periods, one before and one after SAP implementation, care managers identified older people where an assessment had been completed for substantial levels of care or who were judged to have complex needs, with active consideration being given to care home admission (the most vulnerable older people). These criteria included, for example, older people for whom intensive packages of home care, involving more than 10 hours a week, were planned and those experiencing complex needs, indicators of which were a recent unexplained history of falling, not eating, immobility, incontinence, social withdrawal, confusion or wandering. Thus, all older people referred were included on the basis of similar judgments regarding complexity or case-mix. These criteria were devised to encourage suitable referrals in terms of recent health or care needs that would comprise a more “comprehensive” assessment under the policy (Department of Health, 2002a). A full description of the criteria has been published (Challis et al., Reference Challis2004). The data on the existence and identification of a range of needs prior to the policy were available from a randomized trial of assessment practice for older people, the findings of which have already been reported (Challis et al., Reference Challis2004).
Data collection
The study objectives were to determine the effects of the policy on: (i) initial identification of possible depression in community care assessments, and (ii) correct recognition (congruence with the gold standard assessment). The primary outcome measure was the accuracy of recognition, measured by comparing depression identified in community care assessments with ratings by independent standardized interview at each of the two time periods.
During interviews with each referred older person (and any carer if requested), three researchers (PC, MA, CS) used a range of instruments for examining the probable presence of difficulties. These instruments covered domains likely to be important in an overall assessment of older people's needs such as those covered by the statutory community care assessment. The instruments were:
• The 15-item Geriatric Depression Scale (GDS; Sheikh and Yesavage, Reference Sheikh, Yesavage and Brink1986) at a cut-off point of >5, reflecting a case definition of probable depression (Yesavage, Reference Yesavage1988). This is a concise screening tool for depression in older people requiring “yes” or “no” answers to questions on the existence of various symptoms of depression.
• The standardized Mini-Mental State Examination (MMSE; Molloy and Standish, Reference Molloy and Standish1997) at a cut-off point of ≤17 denoting a probable case of moderate-severe dementia (Ely et al., Reference Ely, Brayne, Huppert, O'Conner and Pollitt1997). This is a brief (30-item) test of cognition, covering verbal abilities; orientation in time, memory and attention, and performance abilities; naming objects; execution of written or spoken instructions; writing and copying a complex polygon.
• The Barthel Index for physical function, a simple (10-item) index of independence from help (physical or verbal) in ADLs (Mahoney and Barthel, Reference Mahoney and Barthel1965); and
• The Clifton Assessment Procedures for the Elderly Behavior Rating Scale (CAPE BRS) for communication difficulties and social disturbance (Pattie and Gilleard, Reference Pattie and Gilleard1979), both scored dichotomously as presence/absence of difficulties. This schedule (18 items) examines current functioning and behavior, covering self care, mobility and mental state. The scale has been shown to discriminate between older people with different levels of dependency and different care outcomes.
Ratings of the existence of needs from these measures acted as independent gold standard judgments of the true presence of needs at the two time periods.
To obtain information on needs identified in community care assessments, the same researchers screened the assessment documents relating to each referred case within the agency. Data were extracted from the assessment document and relevant supporting information (such as correspondence) using a pro-forma listing each relevant need, and whether the difficulty had been identified at any level of severity in the assessment record (including when this had been stated after consultation with health colleagues). The outcomes examined were: the proportion of cases assessed as having, or suspected of having, depression in community care assessments (“identified” depression); the proportion of correctly identified cases after confirmation by the standardized interview (“recognized” depression or true positives), and the overall accuracy of recognition (kappa statistic) at each time period.
Sample size calculation
We used sample size calculations for comparing two coefficients of agreement in identifying depression between the two periods (Donner, Reference Donner1998). Given previous reports of low accuracy in identifying depression in this setting (Stewart et al., Reference Stewart, Challis, Carpenter and Dickinson1999), a relatively conservative estimate was used. In order to detect a significant difference, at the 5% level, from a coefficient of 0.20 (“slight” agreement) to a coefficient of 0.60 (“moderate” agreement) (Landis and Koch, Reference Landis and Koch1977), assuming the prevalence of need to be 0.50, a sample size of 83 in each group at a power of 80% was required.
Data analysis
Descriptive statistics (means and proportions) were used to summarize data at both time periods, in terms of both the characteristics of the samples and the existence of depression, cognitive impairment and associated difficulties (within-sample prevalence from the standardized measures).
To test the impact of the SAP policy we analyzed each of the proportion of cases identified as having depression in community care assessments and the proportion of correctly recognized depression cases, confirmed by standardized interview, at each time period. The accuracy, or precision, of the community care assessment was then evaluated by means of the positive predictive values, i.e. the proportion of correctly identified cases among those assessed as having depression, at each time period.
Overall accuracy in identifying depression was analyzed by inter-rater reliabilities between community care assessment and standardized measures at each time period using the kappa statistic (κ). Unadjusted kappa values may not take account of prevalence and the propensity of different observers to identify depression. Thus adjusted kappa values were calculated (Dunn, Reference Dunn1989). Kappa coefficients at Time 1 and Time 2 were compared using a test statistic that follows a χ2 distribution, with appropriate probability values therefore reportable (Donner et al., Reference Donner, Eliasziw and Klar1996). This analysis tested the hypothesis that the accuracy of recognition was significantly higher post SAP implementation.
We also carried out logistic regression analyses to model the effects of “exposure” to the policy (Time 2 versus Time 1) on both the probability of identifying depression and also correctly recognizing it (confirmed from standardized interview), controlling for its prevalence and several associated difficulties and circumstances as potential confounding effects. Two separate models were examined in each case: the first controlled for the prevalence of depression and difficulties associated with it – cognitive impairment and ADLs such as mobility, toileting and incontinence (Iliffe et al., Reference Iliffe, Tai, Haines, Booroff, Goldenberg, Morgan and Gallivan1993), communication problems and social disturbance; the second model controlled also for gender and whether the older person lived alone. These analyses were designed to determine the effects of the changed assessment arrangements allowing for the possible differing prevalence of conditions at the two time periods. In effect they controlled for the potential to identify and then accurately recognize depression more easily if these difficulties are present. All analyses were done with SPSS version 14.
Results
Characteristics and prevalence of difficulties
Over the pre-SAP period, after exclusions, 127 older people were referred for whom a full community care assessment was available. Of these, 106 (83%) agreed to take part and were interviewed. Corresponding figures for the post-SAP period were 134 referrals with 110 (82%) being interviewed. Reasons for non-interview were primarily that the older person, or by proxy through their informal carer, refused to take part in an interview although two older people in the pre-SAP period had moved from the area, thus no longer being under the jurisdiction of the local authority. The demographic characteristics of those interviewed did not differ significantly from those who refused or were unavailable for interview at either time period (p > 0.05 for all variables).
Table 1 reports the characteristics of those interviewed before and after the introduction of the policy, showing equivalence between the samples. Older people in both periods were relatively old (a mean age of 82 years), predominantly female, white, widowed or single, living alone, and living in their own homes (either owned or rented). These characteristics mirror those found in surveys of older people entering care homes, adding validity to the entry criteria for the study (Netten et al., Reference Netten, Darton, Bebbington and Brown2001). However, there were differences in the prevalence of some needs, with more older people at Time 2 having problems with cognitive impairment, toileting and incontinence, communication difficulties, and social disturbance. Mobility problems were fewer at Time 2. Probable depression was equally prevalent at the two time periods.
Characteristics of participants; values are numbers (percentages) unless otherwise stated
* Defined as a problem “sometimes” or “frequently” from subscales of the CAPE BRS: communication (two statements on difficulties with understanding and communicating by speaking, writing, or gestures); social disturbance (five statements on difficulties with objectionable behavior day/night, accusations, hoarding, and disturbed sleep).
SAP = Single Assessment Process; MMSE = Mini-Mental State Examination; GDS = Geriatric Depression Scale; CAPE BRS = Clifton Assessment Procedures for the Elderly Behavior Rating Scale.
Impact of the policy on the number of identified and recognized cases
In terms of monitoring the impact of the policy, of the 100 older people at Time 1 who completed the GDS, 12 (12%) were identified as possibly having depression in community care assessments. This figure for Time 2 (with slightly more people completing the GDS, giving a total of 103) was 28 (27%). Of these, however, the number of correctly recognized cases, confirmed by the GDS, was 8 at Time 1 and 15 at Time 2 (Table 2). Thus, the positive predictive value (PPV) for depression in community care assessments was 8/12 = 67% at Time 1 and 15/28 = 54% at Time 2. This finding suggests no improvement in accuracy of assessments post SAP (pre-SAP κ = 0.28, 95% CI: 0.06–0.38; post-SAP κ = 0.26, 95% CI: 0.04–0.42, p = 0.93). Of interest was the relatively large proportion of “false negatives” (cases where community care assessments reported no depression but the standardized interview confirmed that depression was present) at each time period (25% at Time 1 and 20% at Time 2). Adjusted kappa values also suggested only “slight” agreement between community care assessments and the standardized measures, this remaining appreciably the same before and after SAP implementation.
Identification of depression in community care assessments and/or by standardized interview
Positive predictive values: pre-SAP = 67%, post-SAP = 54%.
Adjusted kappa: pre-SAP = 0.28, post-SAP = 0.26, p = 0.93.
SAP = Single Assessment Process.
The results of logistic regression analyses (Tables 3 and 4) showed that the probability of identifying depression was significantly increased after the policy (OR = 2.94, 95% CI: 1.31–6.75). However, the correct recognition of depression did not improve after the policy (OR = 1.82, 95% CI: 0.59–6.61). The co-existence of depression and cognitive impairment made identification, but not correct recognition, more likely (depression: OR = 3.38, 95% CI 1.53–7.38; cognitive impairment: OR = 2.44, 95% CI 0.10–5.88). Other factors characterizing personal circumstances (gender and whether living alone), communication and social disturbance, and problems with ADLs did not impact on the likelihood of identifying or recognizing depression.
Associations between the policy intervention and identification of possible depression in community care assessments
Summary: Model 1 χ2 = 26.18 (p = 0.001), Nagelkerke R2 = 0.19; Model 2 χ2 = 28.29 (p = <0.01), Nagelkerke R2 = 0.21.
SAP = Single Assessment Process.
Associations between the policy intervention and accurate recognition of depression in community care assessments
Summary: Model 1 χ2 = 63.69 (p = <0.001), Nagelkerke R2 = 0.53; Model 2 χ2 = 64.31 (p = <0.001), Nagelkerke R2 = 0.54.
SAP = Single Assessment Process.
Discussion
Despite being a relatively common condition, identifying and correctly recognizing depression amongst older people has been signaled as a problem (Chew-Graham et al., Reference Chew-Graham, Baldwin and Burns2004). In primary care, only about a third to a half of cases of depression are recognized as such by general practitioners (Volkers et al., Reference Volkers, Nuyen, Verhaak and Schellevis2004). The situation is similarly challenging among older people assessed for, or receiving community support. In England in the late 1990s, assessment documents of local authorities responsible for community care assessments, which are intended to include information on health status, such as mood, as well as social difficulties, only included such information in detail in 6% of cases (Stewart et al., Reference Stewart, Challis, Carpenter and Dickinson1999). For older people receiving home support after assessment, one English study found that 44 (26%) of 169 older people receiving home care had case-level depression. However, only seven (16%) of these received antidepressants and only four (9%) were in contact with psychiatric services (Banerjee and Macdonald, Reference Banerjee and Macdonald1996). In the US home health care setting, home care nurses in one study accurately documented the presence of depression in only 13 out of 35 cases, indicating that they missed almost two-thirds of cases (Brown et al., Reference Brown, Bruce, McAvay, Raue, Lachs and Nassisi2004). These figures for home health care are similar to those in the present study, where three-quarters of cases were missed before the policy and over half after the policy. This indicates that, despite efforts towards improving the situation, recognition of depression in community settings as well as in primary care remains low.
Interventions directed at improving this situation in primary care have included feedback to general practitioners of the results of screening tests but with equivocal results (Peveler and Kendrick, Reference Peveler and Kendrick2001). One reason for contradictory evidence concerning the outcomes for patients of screening for depression in primary care is that these outcomes are dependent on what the general practitioner does with the information from a screening questionnaire (Lynch et al., Reference Lynch, Clarkson, Blenkiron and Fraser2002). Correct recognition of the disorder, through screening tools for example, is thus only useful if it becomes a practical aid to treatment and management. That is, the fact of recognition needs to be supplemented with decision-making tools, such as protocols, that signal the appropriate referral routes for particular patients based on their scores on a screening instrument. Screening, in this sense, was enshrined in policy in England with the instigation of the over-75s health check in general practice from 1990. Efforts at recognizing depression in particular were linked to this policy (Arthur et al., Reference Arthur, Jagger, Lindesay, Graham and Clarke1999). The evidence, however, indicated variable implementation and unpopularity with GPs who felt the additional workload outweighed the benefits (Clinical Standards Advisory Group, 1998) and the policy was subsequently abandoned. In community settings, research evidence suggests benefits of integrating social services and health assessments, including better recognition of mental disorder (Challis et al., Reference Challis2004). In this regard, the study reported here investigated whether a national policy relating to the broader assessment of older people for community support had an appreciable effect on identification and correct recognition of depression.
We found, in common with these other studies, that the correct recognition of depression was not improved after introduction of the policy. The policy was expected to involve closer working relationships and enhanced information-sharing between health and social care professionals responsible for assessing older people. A rationale for the policy was that shared assessment tools would assist accurate detection of conditions like depression where its identification is included as a subset to a broader assessment (Philp, Reference Philp1997). However, the data presented here show that this promise seems not to have been realized. This may have consequences for future management as, if not properly recognized, depressive symptoms may persist and lead to increased isolation and breakdown in family relationships. Under-recognition may also lead to a lack of appropriate management and necessitate more complex care packages and subsequent unplanned referral to other professionals such as specialist clinicians (Banerjee and Macdonald, Reference Banerjee and Macdonald1996). This would have resource consequences for agencies and incur personal costs to the user.
The analyses showed, however, that the policy did increase the probability that care managers responsible for community care assessments initially identified (rather than correctly recognized) depression. One reason for this may have been a general “Hawthorne effect” (Mayo, Reference Mayo1949) around the policy's introduction; i.e. that these professionals may have been more sensitized to the idea of depression being present merely by virtue of the attention given by managers in implementing the policy. The care managers may thus have reacted by modifying their behavior and considering the possibility of depression in their assessments simply in response to the fact that this had been signaled as important to look for (such as part of changed assessment documentation), rather than reflecting a true change in their abilities to recognize cases correctly. Another potential explanation for the increased likelihood of identifying cases may have been that the depression (when it was identified, particularly at Time 1 when relatively unstructured assessments were used) could have been secondary to other physical health conditions in this population. This may, in part, offer an explanation for the increased probability of care managers identifying depression, although the regression analyses for identification did control for the existence of physical and cognitive impairments that may characterize these health conditions. However, it remained the case that the policy's introduction did not increase the likelihood of these staff correctly recognizing depression.
Limitations
As the study design was an observational one, it was difficult to isolate the effect of the policy change from other influences ongoing at the time of the study, such as changes in other local procedures, referral systems (especially between health and social care), professional workload and commitment. Although the groups were similar, some needs were more prevalent after the introduction of the policy. This fact may have reflected the influence of other policies, such as the introduction of national guidance on eligibility (Department of Health, 2002b), which may have led to the application of more restrictive criteria by the social services authority by which older people are deemed to be eligible for service. The threshold required to receive assistance may have been raised and therefore older people entering the system post SAP may have been frailer than previously, perhaps making initial identification of needs more likely. The regression analysis of identification of possible depression showed that prevalence had an effect, although there was a negligible effect of prevalence on the correct recognition of depression. The adjusted kappa ratings did take account of this possible confounding effect in the study but the design makes it difficult to separate out unequivocally the effects of the policy from those of other influences such as those cited here.
The study was also conducted in only one agency where data on assessments were available before the introduction of the policy. Caution is thus required in generalizing to agencies in other localities who may have implemented different policies and procedures for assessment within the overall framework of the policy (Department of Health, 2008). In this respect, one component of the intervention in this agency, the use of a locally developed rather than standardized assessment tool, may also have been a potential limitation, although the use of a local tool was in common with 66% of local authorities implementing the policy. There was a lack of central prescription in the policy, however, with certain standardized tools being accredited for use but with a degree of discretion being granted to local agencies in how they interpreted the guidance. Thus, the nature of the intervention itself was a complex one with associated difficulties in its replication to other areas.
Conclusion
This study evaluated the impact of a national policy in England on the identification and recognition of depression in one locality. Although the policy appeared to make the identification of depression more likely among staff responsible for coordinating community-based assessments, it did not impact on their ability to recognize depression correctly. Recognizing and responding to depression in a coordinated way in the community thus remains poorly addressed. Specific strategies, including training, would seem to be an important step towards improving detection and thus towards correct management rather than, as here, general guidance and procedures concerning overall assessments. For example, the use of tools to routinely elicit the specific symptoms and pointers to depression in older people in this setting may be a fruitful development. Such a strategy, allied to training in the use of such tools, guidance in the management of cases that are confirmed by using the tool, and specific instructions as to the follow-up questions that may be asked of the older person to confirm the existence of depression, has been employed in US home health care agencies with encouraging results (Bruce et al., Reference Bruce2007). Training in the context of the present setting in England could usefully build on such initiatives and would require a clear emphasis on those difficulties, commonly seen by care managers, which may complicate assessment of depression such as medical conditions, pain and other physical disabilities. Training techniques that emphasized instruction, role play and real-world assessment scenarios, perhaps using video, would also offer useful strategies to these professionals in improving recognition of depression as part of their statutory duty to provide assessments of older people.
Conflict of interest
None.
Description of authors’ roles
P. Clarkson helped to design the study, supervised data collection and interviewed participants, wrote the paper and analyzed the results. M. Abendstern collected data, interviewed participants and assisted with writing. C. Sutcliffe collected data, interviewed participants and assisted with writing. J. Hughes helped to design the study, supervised data collection and assisted with writing. D. Challis helped to design the study, oversaw the statistical analysis and assisted with writing.
Acknowledgments
We thank all care managers in the social services teams for referring participants to the study; Caroline Marsh, Alan Calvert and Roger Lightup for sanctioning the research and data collection through Manchester City Council Adult Social Care; the older people and their carers for agreeing to participate; and Professor Graham Dunn for statistical advice. The study was funded by the Department of Health in England. The views expressed here are those of the authors and are not necessarily shared by the Department of Health.
