Introduction
A wide and ever-increasing number and variety of screening measures are used in residential aged care facilities (RACFs). For the purposes of this paper, RACFs include long-term care facilities and are taken to include any congregate living residence, created for older adults and others with chronic illnesses, disabilities and/or deficits in activities of daily living (ADL) or instrumental activities of daily living (IADL) that necessitate 24-hour nursing or social care. We refer to the people who live in long-term care facilities as “residents”, and to the care provided in these facilities as “residential care” when speaking generically (i.e. unless referring specifically to nursing care, medical care, recreational therapy, etc.).
Appropriate and sensitive use of the instruments within the RACF environment is critical to quality care delivery. Screening measures include instruments routinely used for clinical purposes in these facilities, as well as tools associated with organizational systems (for example, outcome monitoring), which may involve the management of funding schemes that are locally mandated. The purpose of this paper is to identify those issues involving screening assessment that are of broad relevance to health care professionals working in the residential aged care sector. This paper will not attempt to suggest instruments appropriate for use in RACF contexts; such a review is far beyond the scope of the present paper, and probably beyond the reasonable bounds of a single journal article. The reader will be directed to guidelines and review articles where appropriate. The main focus of the present paper, however, will be a delineation of the key issues to consider concerning the use of screening instruments in the RACF environment, as well as how decisions about the most clinically meaningful and empirically supported decisions concerning their use might be arrived at, with some latitude for tailoring use to meet the needs of a particular client mix and a particular RACF context.
There are important contextual issues that are relevant to the effective use of screening instruments in RACFs. First, the multiple uses and types of instruments make increasing demands on a range of professionals working in RACFs. These environments are often resource poor and staff training in the appropriate use of screening measures may be quite limited and not viewed as a high priority. Practice issues such as standardized administration, the use of normative data and cut scores, and the distinction between screening and diagnostic assessment may be unfamiliar to many staff members in RACFs. Second, staff knowledge of mental health problems and provision of appropriate interventions are often chronically and severely limited in such settings. In particular, knowledge of severe mental disorders (e.g. bipolar disorder, schizophrenia, schizoaffective disorder) is often inadequate, as is their knowledge of evidence-based interventions for these disorders and behavior problems associated with these and other disorders (e.g. dementia). This is true even though such conditions are over-represented in the population of such facilities. Thus it is critical for the whole range of professionals working in such facilities to have a sound knowledge of the key issues affecting mental health in older adults, particularly dementia, psychosis, and mood disorders. Screening for these and other mental health issues can serve the useful purpose of identifying the need for further investigation by relevant staff and, ideally, triggering appropriate treatment responses. Third, the high level of physical frailty among RACF residents poses significant challenges for assessment in general, including the use of screening instruments. Although these instruments are typically brief and fairly straightforward, residents with sensory, psychomotor, and communicative impairments may have difficulty, for example, on cognitive screens that involve reading, writing, or following commands.
Furthermore, the majority of RACF residents display varying degrees of cognitive impairment. Some screening instruments have good psychometric properties among residents with moderate cognitive impairment. However, for many screening instruments, it is unclear at what point the cognitive impairment compromises the psychometric integrity of the instrument. Harper (Reference Harper, Kane and Kane2000) provides an excellent overview of strategies for assessing older adults who cannot communicate, including the strengths and weaknesses of various approaches. Similarly, psychiatric comorbidities, such as the high comorbidity of depression and anxiety in older adult populations (e.g. Lenze et al., Reference Lenze2000), are also common in RACF residents. It is important to be cognizant of comorbidities when interpreting results of screening. The prevalence of highly comorbid, but reversible, conditions such as anxiety and depression in RACF residents underscores the importance of screening for such conditions in this population.
Screening instruments have a long history in psychology and much is known that gives guidance on desirable psychometric properties and other characteristics. Recurring themes in psychological assessment are the importance of the biopsychosocial context of the assessment and of the goals of the assessment (Messick, Reference Messick, Wainer and Braun1988; Reference Messick and Atkin1991; Reference Messick1998), particularly in terms of the decisions that are to be made on the basis of the assessment (for an overview of recent findings in decision-making see Garb, Reference Garb2005). Screening tools should not be used simply to collect information on people. How the scores arising from the screening instruments are to be used and what the consequences of the decisions made on the basis of such screens are the important questions that need to be answered prior to the implementation of any screening program, whether for the purpose of, for example, the widespread assessment of all residents of a facility or the selective screening of suspected serious conditions in one or a few people.
It is also critical to determine whether screening is appropriate in a particular context, or whether the actual domain or function of interest needs to be comprehensively assessed. In many cases screening is viewed as a preliminary assessment that can lead to more comprehensive assessment based upon the findings of the screen(s). If a diagnosis is desired, a screening inventory might be a good first start. There are many elaborate tests of disorders and cognitive skills that are represented by single items on screening instruments, particularly cognitive screening instruments. The determination of the purpose for screening, identifying the correct assessment tool and the most appropriate person to administer and interpret it, and having systems in place to act of the findings of the screening are prime considerations.
Purpose, or the “why” of clinical screening
Screening is different from comprehensive assessment. A useful concept here is that of the bandwidth-fidelity compromise (Cronbach, Reference Cronbach1960), or, in other words, the trade-off between depth versus breadth. Such trade-offs can depend on a variety of factors including setting, patient characteristics and time constraints. One can screen for a wide range of psychopathological entities or conditions in no particular depth, or focus more tightly on a circumscribed symptom cluster or disorder. One cannot do both in a single, brief assessment, and so compromises must be made in using screening instruments. Sensitivity and specificity are also relevant here. We want high sensitivity for a screening instrument, particularly when screening something that we dare not miss (e.g. suicidal behaviors or ideation).
One general purpose of screening is to determine whether a person might have a particular condition. Screens should neither be conceived of nor used for the purpose of obtaining diagnoses of conditions, as this requires more complex instruments administered by more highly trained specialists. However, the gauging of symptom presence or severity is a highly useful function that can be achieved using the appropriate screening tool in order to identify those residents in whom assessment by more highly trained specialists is needed. Another specific circumstance in which more comprehensive assessment is required is when the suspicion has been raised that the accuracy of the information provided by the person being assessed is questionable.
As mentioned above, many psychiatric conditions and some instances of cognitive decline are at least amenable to treatment, if not reversible. Treatment for psychiatric conditions is often lacking in RACFs (e.g. Canadian Coalition for Seniors’ Mental Health, 2006); however, when treatments are utilized, screening instruments can play an important role in evaluating the effectiveness of interventions. Routine use of such tools may improve staff attitudes about the value of treatment for residents, especially if treatment gains can be objectively demonstrated.
Screening instruments can be applied both to people residing in RACFs and to community-dwelling individuals in transition to these facilities. Application of the same or similar screening tools before and after the transition to long-term care can point out dramatic changes in functioning (e.g. development of depression), which can then be the focus of timely interventions. Screens can be directed at the individual in question, or to their family and close associates, either if proxy responses are necessary due to circumstances, or if the viewpoint of knowledgeable informants is directly sought. There is general recognition that proxy reports are influenced by many factors, including the informant used, the degree of contact between the individual to be assessed and his/her proxy, and the social context such as the caregiving relationship (Harper, Reference Harper, Kane and Kane2000). Typically, there is better concordance for more observable behaviors and traits as opposed to internal states. In addition, informants are often better at providing a reliable rating of the frequency of an observation rather than its severity. Much of the research has focused on detecting depression in the presence of cognitive impairment, and the majority of studies have found that collateral sources report more symptoms than do patients. A recent study compared the self-report of persons with mild cognitive impairment (MCI) or Alzheimer's disease (AD) – Mini-mental State Examination (MMSE) mean score of 24 – with the reports of family members or close associates (Chopra et al., Reference Chopra, Sullivan, Feldman, Landes and Beck2008). Results indicated significant disagreement on all items of the GDS-15, with the exception of the item pertaining to memory problems. Thus, proxy reports should be used cautiously, particularly when screening for mood symptoms.
Appropriate screening tools can quickly and efficiently provide reliable and objective information on specific clinical or functional parameters. Such instruments allow for the collection of objective data that may clarify or modify unstructured clinical impressions or ambiguous information from family members. Such screens can provide answers to basic questions about mental, psychiatric and behavioral functioning as well as changes in these states over time. Brief screens also may be used for administrative purposes, for example, in the collection of minimum data sets that are used for administrative or funding purposes. The latter use is generally less reactive for the persons being screened because they may only require information from care staff with no direct interaction with the resident and may be more universally and reflexively applied.
As with all measures, there is no justification for using a screen if that instrument has not been designed and normed on the particular population of interest. This is particularly important for a clinical instrument that is translated into a different language (due to attendant linguistic and cultural issues among other factors) as well as when a measure is used on a different clinical population than its original development specified. Since the RACF is a relatively specific setting, the importation of instruments from other settings must be done with care. Evidence must be available that justifies and supports the use of any screening instrument that was developed for use in other settings and subsequently imported into the long-term care environment
Who administers, who receives, and who interprets screens?
As opposed to comprehensive assessment batteries, screening tools may be designed to be administered by a range of health care professionals rather than a particular professional (e.g. complex neuropsychological batteries administered by neuropsychologists or trained psychometricians). While such instruments may appropriately be used by a variety of disciplines, familiarity with ideas about standardized administration and particular administration, scoring and interpretation guidelines are always necessary. Administration of brief screening instruments is usually relatively straightforward with a minimum of complex test materials or the need for special equipment. In contrast, only appropriately trained and qualified professionals should administer more comprehensive instruments, particularly neuropsychological measures. For example, the need for precise timing of an individual's response to a question in a screening tool is usually minimal. Such response times may be an important measurement feature as well as an important diagnostic indicator in more complex tests. Many screens are also specifically designed to be given to individuals at the bedside or in less than ideal conditions, such as home environments. Recently, instruments have been designed specifically to take account of cultural contexts so as to increase accuracy and clinical utility. Some have even been designed to be administered over the telephone or via an interpreter.
Administration and interpretation of any measure should be included either in the test manual (if it is commercially available) or the published literature (if it is freely available in a journal publication). Broad procedures for an acceptable approach to the administration and interpretation of all types of instruments have been published in various test administration guidelines (for example, the American Psychological Association's Standards for Educational and Psychological Testing, currently under revision). Information on a broad range of measures is also available in compendia such as the Mental Measurements Yearbook (MMY). Ideally a screening instrument will have had its scoring criteria and normative data, and the resulting interpretation of a given score, carefully and clearly delineated. For example, if cut points are offered, they should be derived empirically through the application of a procedure such as the analysis of receiver operating characteristic curves (Silverman et al., Reference Silverman, Musa, Martin, Lave, Adams and Ricci1995; Reckase, Reference Reckase1996). The implementation and evaluation of these algorithms may require specialized knowledge and resources. Also, using a screen on a different clinical population or in a different context from that in which it has been validated requires further supportive evidence. Older adults are a heterogeneous population, and in clinical settings the types of patients encountered may vary widely; the characteristics of the person being assessed should be considered; if they are highly divergent from the population the test was designed for, caution should be exercised. Thus, for example, the demonstrated utility of a screening measure in an outpatient geriatric assessment population does not mean that the use of the same measure to screen for the same disorder in persons with dementia in a residential care setting can be defended.
In addition, the use of screening tools both clinically and ethically requires attention not only to how measures are given and interpreted but also to what is done with the information obtained. Specifically, there should be adequate systems and infrastructure to develop and act on the relevant knowledge gained. For example, screening for depression by a trained nurse working in a geriatric assessment clinic with experienced geriatric psychiatrists available for follow-up treatment necessarily involves different organizational systems and may result in varying outcomes than when a similar instrument is used by less psychometrically skilled ward-level nursing staff in a care facility without access to potential specialized treatment providers. The widespread use of screening instruments by RACF staff does require specific attention to ethical issues, in particular, a resident's right to privacy, confidentiality of assessment results, and informed consent as reviewed in ethical codes of conduct, including the Standards for Psychological Services in Long-Term Care Facilities (Lichtenberg et al., Reference Lichtenberg1998). In residents who have been declared incapacitated with regard to consenting to assessment, informed consent will usually need to be obtained from the resident's authorized legal representative. In residents who can give consent, clear explanations of the purpose and nature of the questions to be asked are necessary.
“Universal screening” is screening that is applied to all individuals as a matter of policy or in reaction to a certain set of circumstances or at a particular point in time. The ultimate costs and utility of such screening depends on the availability of structures in the system to react to the findings. The economics of implementation of any screening program are dependent upon local resources and infrastructure. There is another assumption that care plans may follow from such universal prompts, but whether resources are in place to allow this to occur is an important consideration. Such issues for the implementation of screening programs go beyond the limited purposes of this document.
Where and when to screen?
Circumstances of when and where a screening instrument is administered are governed by the purpose of the assessment, as well as by who is completing the instrument and who is administering the measure. The choice of a particular screening measure may, in part, be dictated by local circumstances, including the availability of more highly trained specialists and access to particular instruments. Screening may be part of some protocols, such as an admissions suite of tests, which govern timing of administration; it is important for staff to realize that screens for various disorders (e.g. mood disorders) may have to be re-administered if circumstances change for a resident.
As per guidelines for the administration of any instrument, the conditions of the administration should be such that the performance obtained is as free as possible from extraneous bias (for example, measures should be administered in as quiet an environment as is feasible). Such issues as time of day effects (particularly for repeated assessments), and potential medication effects on performance should be monitored. The context of the administration is important in terms of the skills of the person administering the instrument and attendant support systems and interpretive guidelines.
An important question involves the trigger for a mental health screening. For example, there may be one or several triggers to invoke screening of a person whose ability to continue to reside in community is in doubt. Some assessments may require formal requests sent in to consultants; here, issues such as waiting lists might impact on care (although such issues are more likely when lengthy assessment batteries are requested). Triggers for capacity evaluations may be complex as often legal or social agencies are involved. Issues surrounding assessment of capacity are beyond the scope of the present paper; the reader is referred to ABA and APA (2008) and Qualls and Smyer (Reference Qualls and Smyer2007) for further information. However, in competency as well as most forms of assessment, including screening, the issue of the accuracy of self-report information is crucial.
How are screens interpreted?
As mentioned, screening instruments are usually designed for relatively straightforward interpretation. Some are designed with simple, single cut-off scores or broad categorical interpretations (e.g. mild, moderate, severe). Comprehensive assessment approaches are more likely to require the calculation of individually indexed scores (i.e. using age- and education-based normative data) and so require more extensive training for appropriate use. Behaviors observed during assessment or underlying circumstances influencing performance are not routinely examined or considered in interpretation. This illustrates one of the frequently overlooked dangers of screening assessment, namely that they inherently carry an increased risk over comprehensive assessments of incorrect conclusions, particularly when important supplementary information is ignored or misinterpreted. Often, the particular context or individual characteristics of the person are not used to modify the interpretation of the obtained score. This is not to say that normative data do not exist for most screening tools, nor that more fine-grained consideration of the impact of individual differences on interpretation should not be considered. For example, while MMSE scores may be interpreted using simple cut-off scores, such scores obtained without reference to age and education specific normative data (Crum et al., Reference Crum, Anthony, Bassett and Folstein1993) are invalid, as are interpretations stemming from them.
What happens when a person is incorrectly categorized is often not considered in clinical practice. These errors are more likely with screening measures because their brevity limits their ability to detect some cases that can be detected with more extensive assessment. Screening may therefore be part of a two-stage process in which the second stage involves more extensive assessment (Streiner, Reference Streiner2003). If the person passes the screening measure but is still considered to have a trigger for second-stage assessment or detailed clinical assessment, further investigation is clearly warranted. Sensitivity and specificity (which should be known in established screening programs) would inform this, and can be determined for both stages of two-stage, sequential assessment. On a cautionary note, the lack of mental health resources within RACFs is a significant barrier to practice and there is the potential for inappropriately using screening instruments as diagnostic indicators, rather than implementing the two-stage process.
Evaluation of the clinical utility of a particular measure
For any assessment tool, including screening measures, the careful evaluation of its psychometric properties (e.g. reliability and validity) and clinical utility is required whenever the instrument is used in a new context. Sensitivity, specificity, and positive and negative predictive powers are basic and important psychometric considerations for a measure and provide much useful information on the clinical utility of a screening instrument. The sensitivity of a measure reflects the proportion of actual cases which are correctly identified as such (e.g. the percentage of sick people who are identified as having the condition); the specificity of a measure refers to the proportion of non-cases which are correctly identified (e.g. the percentage of well people who are identified as not having the condition). Positive and negative predictive powers refer to, respectively, the likelihood of the presence or absence of a disorder given a particular test result. It is important to note that in low prevalence situations, positive findings are more likely to be false positives; the degree to which this is problematic most likely relates to the seriousness of missing a positive finding.
Each of these psychometric constructs can prove more or less important depending upon the goal of screening. In most cases, sensitivity is going to be the most important factor, as it will provide the desired bandwidth mentioned earlier (in other words, the goal is to not miss the condition in question). However, when it is important not to have a false alarm (e.g. when no further assessment is likely), then specificity becomes relatively more important (but this does not mean that sensitivity becomes unimportant). In the end, it is important for those using screening instruments to understand the relative merits of such issues as sensitivity and specificity and to have information regarding these available. When agencies or governments consider the merits of screening, additional issues such as positive predictive value, or the precision rate (i.e. the proportion of individuals with positive test results who are correctly identified) are important. We need to know what happens to these parameters, for example, with different cut-off scores on various screening tools.
Consideration of the clinical characteristics of the particular person being assessed is important because the utility of a measure is sensitive to how closely the person being evaluated matches with the sample on whom the measure was originally validated. Instruments vary widely with respect to how broadly they may be applied in the clinical setting. Of the many things that might affect utility, the base rate of the condition in question is most important. Prior research on the screening tool may have established particular figures for reliability and validity, but generalizability across settings must be carefully considered. Understanding of the degree to which such factors as the administrator's skills, the environment and time of day in which the instrument is administered, and individual characteristics of the person being assessed influence the psychometric characteristics of an instrument is critical to informed administration. Unfortunately, with the exception of major depressive disorder, subthreshold depression, and the dementias, there is a paucity of good epidemiological data on the base rates of various mental health problems in RACFs (e.g. anxiety disorders). Thus, it is often difficult to determine whether instruments evaluated in other settings with specified base rates can be reliably used within the RACF context.
Methods
The International Psychogeriatric Association Task Force on Mental Health Services in Long-Term Care Facilities aims to enhance care of persons in RACFs. As one component of this work, task force members from diverse parts of the world and a variety of disciplines have come together to develop and evaluate guidelines on aspects of long-term care. Task force members, including the first author, presented a preliminary form of their work at a day-long workshop on nursing home guidelines and best practice at the International Psychogeriatric Association World Congress in Montreal, Canada (1–5 September 2009); at that forum comments on the paper were noted for incorporation into the final manuscript. The present paper reflects our group's work on clinical issues around screening for mental disorders in the residential aged care environment.
With respect to residential aged care, a variety of guidelines on a range of topics have been produced both nationally and internationally. For example, the Canadian Coalition for Seniors’ Mental Health (CCSMH) has produced comprehensive clinical practice guidelines for the assessment of mood and behavior in RACFs (CCSMH, 2006). Recently a group in Australia has produced recommendations across a suite of instruments used to assess dementia in RACFs (Sansoni et al., Reference Sansoni2007). Similarly, numerous review articles and intervention studies focused on psychosocial and pharmacological interventions used in such settings have been published; many of these use screening instruments to demonstrate efficacy or change. Some have focused on assessments of cognition, emotional or behavioral functioning of residents in long term care facilities (e.g. Snowden et al., Reference Snowden, Sato and Roy-Byrne2003). There are numerous articles describing the properties of instruments frequently used in RACFs or detailing comparisons of such instruments and their efficacy (e.g. Cummings et al., Reference Cummings, Mega, Gray, Rosenberg-Thompson, Carusi and Gornbein1994; Koehler et al., Reference Koehler2005). Finally, there are a variety of compendia of psychological test measures, some of which are focused on older adults in residential care settings (e.g. Teresi et al., Reference Teresi, Lawton, Holmes and Ory1997) as well as compendia of tests aimed at older adults more generally (e.g. Burns et al., Reference Burns, Lawlor and Craig2004). However, the present authors have found no published guidelines exclusively relevant to screening instruments used in nursing home environments. Clinical norms for the use of assessment tools including screening instruments in a variety of contexts from providing care in the community to the provision of services in a RACF or prior to admission to a facility are present in the literature (e.g. Teresi et al., Reference Teresi, Lawton, Holmes and Ory1997) as well as provided by the authors and publishers of specific instruments. The authors took note of these sources of information; however, searches using a variety of search terms in several electronic databases (e.g. PubMed) failed to provide any valid citations when the search term “screening” was used in the context of long-term care.
Results
Guidelines for use of screening measures in the RACF environment
Our group could locate no international guidelines specifically relating to mental health screening tools for use in RACFs. Guidelines pointing out best practice usage of screens for affective and behavioral disturbance in residents of RACFs are contained in the CCSMH (2006) guidelines and focus primarily on depression and behavioral screens. These Canadian guidelines include evidence levels for use of screens in particular circumstances and a brief list of some of the most common of these instruments is provided. The focus is on best practice usage in RACF settings. In contrast, most diagnostic guidelines, such as those for Alzheimer's disease established by the NINCDS-ADRDA (McKhann et al., Reference McKhann1984; Dubois et al., Reference Dubois2007), while describing utility in clinical usage, in fact make a point not to recommend specific instruments. There are more general guidelines on screening tools, both in the extant literature as well as in the form of on-line guidelines (e.g. those compiled for depression by the U.S. Agency for Research and Health Care Quality; http://www.ahrq.gov/Clinic/uspstf/uspsdepr.htm).
Administrative screening instruments such as those used in national minimum data sets have their own development processes and supporting documents (Murphy and O'Brien-Pallas, Reference Murphy and O'Brien-Pallas2004). For example, in the U.S.A., the Federal Government has mandated use of the Minimum Data Set (MDS), supplemented or supported with various screening instruments, for clinical assessment of all residents in Medicare or Medicaid certified nursing homes (regardless of source of payment for the individual resident). The aim of this practice is to provide a systematic assessment of each resident's functional capabilities for use by care staff. MDS information is transmitted electronically by RACFs to state-based and ultimately national MDS databases (see Hawes et al., Reference Hawes, Morris, Phillips, Mor, Fries and Nonemaker1995, for discussion of reliability estimates of MDS).
Peak bodies (e.g. national psychological associations) often publish guidelines for the use of measures in general, including screening instruments. Local health care organizations, including third party payers such as insurance companies, may recommend or mandate the use of a particular screening instrument in specific circumstances. Of relevance here is the fact that many RACFs are operated by either government or non-government agencies that have their own internal guidelines for screening assessments.
There are a variety of resources available on screening tools. These include reference texts, compilations of screening tests and individual reviews and analyses in the published literature (e.g. Teresi et al., Reference Teresi, Lawton, Holmes and Ory1997; Burns et al., Reference Burns, Lawlor and Craig2004). There are also systematic attempts at establishing comprehensive screening programs for clinical use. For example, the development and dissemination of the interRAI suite of assessment tools is a noteworthy example and has been instrumental in promoting a common approach to assessment internationally (see www.interrai.org).
A separate issue involves the translation of tests into different languages. Translating measures into different languages is a difficult and complex process that involves establishing both linguistic equivalence and demonstrating adequate psychometric properties in the target group (Stewart, Reference Stewart and McKay2008). Important conceptual and methodological issues should be considered in any test adaptation, particularly with regard to the validity of test scores in cross-cultural contexts (Hambleton et al., Reference Hambleton, Merenda and Spielberger2006). van de Vijver and Tanaka-Matsumi (Reference van de Vijver, Tanaka-Matsumi and McKay2008) provide numerous helpful strategies for addressing construct, method, and item biases in the context of cross-cultural assessment. Moreover, there are guidelines for the appropriate practices to follow when translating a measure (World Health Organization, 2007). One cannot rely upon the properties established in the original language to be appropriate in the new language or context. Metaphorical language use varies considerably between cultures. Establishing the efficacy of the instrument in the new context also is required. Local dialects and potential confounders such as varying educational achievement across regions, even in the face of a common language, must be taken into account. It is also important to note that there are particular interpretation concerns whenever informal translators are used.
Despite increased efforts both nationally and internationally to raise the standards of care in RACFs, and a growing number of meta-analyses and guidelines surrounding such care, the issues surrounding assessment instrument use and interpretation in such facilities have been largely ignored. The increased proliferation of assessment tools available both commercially as well as in the published literature suggests that a critical evaluation of the use and potential misuse of such measures in RACFs and the potential consequences of this is timely.
Discussion
Screening instruments for use in RACFs
As mentioned, there are two main types of screening tools commonly used in RACFs: omnibus tools which screen over a number of domains and more targeted tools. In addition, there are measures with a primary focus on cognition, on psychiatric symptoms, on behaviors and on functional status. Each of these will be discussed in the following sections. Several reference books exist, devoted either exclusively or in part to the description of such measures (e.g. Burns et al., Reference Burns, Lawlor and Craig2004); however, exhaustive listing of screening measures appropriate for use in RACFs is beyond the scope and intent of the present paper.
We believe that recommendations of specific assessment instruments would be inappropriate for a variety of reasons. No screening measure will be suitable for application with all individuals or in all contexts. Instruments need to be selected for a specific purpose, and need to have demonstrated empirical evidence for the use of that measure for the stated purpose in that clinical application (e.g. Meyer et al., Reference Meyer2001). However, clinical utility, for example, while mentioned often in the literature, can be very difficult to demonstrate empirically and probably represents an “aspirational” goal. The measure used should take into consideration characteristics of the individual that are relevant to its administration and interpretation, such as educational level attained. In addition, over time, new instruments emerge or existing instruments are modified or revised. Measures of cognitive impairment and depression, for example, are already available in relatively large numbers, including many designed specifically for patient groups commonly encountered in RACFs, with a few designed specifically for use in the RACF environment. Other domains such as anxiety or psychosis are developing areas for screens for older adults in general, but as yet relatively few instruments have been empirically validated for use in the RACF environment (e.g. Edelstein et al., Reference Edelstein2008). Again, attention to the psychometric properties of a measure as well as being cognizant of factors which might affect appropriate interpretation of scores are critically important for best practice and outcomes.
More popular instruments, or ones that have been in use for longer periods of time, are more likely to have been used in a variety of studies, to have been translated into various languages, and may even be linked to funding or prescribing regimes. For example, in many countries specific instruments must be administered for the start and continuation of cholinesterase inhibiting medications for the treatment of dementia. But such regulations may result in the use of certain measures in contexts for which they were not designed or for which there is little empirical evidence.
For each of the broad categories of instruments described below, there exist both self-rated and informant measures, as well as clinician-rated scales, such as the Cornell Scale for Depression in Dementia (Alexopoulos et al., Reference Alexopoulos, Abrams, Young and Shamoian1988). In addition, some measures ask for a comparison between current and past functioning. Some instruments also exist in several versions or lengths – examples might include the various numbers of items on depression screens or versions of measures for specific settings. In addition, some users might wish to assess repeatedly over time, to gauge the efficacy of an intervention or the progression of symptoms; here, carry-over effects might become an important consideration. In a similar fashion, the recall of material from a related measure may also increase measurement error. Sometimes, changing the order of administration of the measures can reduce this effect. Serial testing is best carried out when multiple equivalent forms of the measure are available, or the measure has otherwise been designed to take account of the effects of multiple administrations. It is particularly important that instruments used for serial testing show excellent test-retest reliability, so that the results can be attributed to change rather than error variance. Users of screening instruments need to consider the exact purpose for which information is needed in order to select the best test for the purpose.
From whom information is gathered is also critical for all assessments, but particularly for screening instruments because they are often given inappropriate weight or credibility within settings. Health care workers who are intimately involved in patient care may have a different perspective to nursing managers or visiting consultants. Information systematically collected from persons who have known an individual over an extended period of time may be most instructive in terms of interpreting current behaviors and cognitive and emotional functioning.
As mentioned, basic tenets of good assessment suggest that any form of screening requires careful consideration of the purpose and timing of the assessment, who will interpret the resulting data and how the data will be used. Screens are usually designed to answer relatively circumscribed clinical questions, in a relatively brief period of time. The following sets out the broad ideal features of such instruments; obviously, actual tests vary in the degree to which they achieve these objectives.
Omnibus instruments
Instruments designed to screen many domains simultaneously (e.g. cognition as well as mood) are commonly referred to as omnibus instruments and can have several advantages over shorter, single-domain instruments. The data collected across multiple domains provide a more holistic picture of functioning and can ensure that key areas of interest are not missed, as could be the case in a more piecemeal approach. In addition, the relationship between various aspects of functioning can be easily examined. Omnibus instruments are usually longer than single-purpose instruments; some instruments with skip rules (single items which, if passed, allow an entire section of the test to be skipped) can afford broad coverage of domains while maximizing efficiency of time use.
The overall complexity of administration of omnibus instruments varies. Such instruments may contain discrete scales with discrete scores; a few such instruments also provide a single global score. Omnibus instruments offer a trade-off in most circumstances between depth or completeness of domains surveyed, balanced against the increased time needed to administer the instrument and the level of training or knowledge required of the person administering the instrument. A particular consideration in the RACF environment is the fact that many residents, particularly frail older adults or those with more advanced cognitive decline, may be unable to complete the measures in one sitting. Breaking up the test administration of such an omnibus instrument over multiple testing sessions introduces potential additional error variance into scores and so impacts the utility of the data obtained. Thus, in most circumstances, use of such measures – for example, intelligence batteries or neuropsychological omnibus instruments – may yield information of less value obtained at greater cost than briefer, more targeted measures.
Screens for cognitive functioning
Screens for cognitive functioning may include instruments that specifically purport to sample performance over several cognitive domains (e.g. memory, executive functioning) or may concentrate on a single domain. Such screening measures are usually only designed to give a rough estimate of functioning, in contrast to traditional cognitive or neuropsychological instruments that are generally used as part of a more comprehensive assessment battery. We repeat that such screening instruments should not be used for major decisions such as arriving at a diagnosis or determining the potential of a person to benefit from interventions due to the increased margins for error that are inherent in such screening tests. In some screening instruments other psychometric considerations, such as the restricted number of items and thus the decreased potential to demonstrate change reliably, should inform appropriate clinical use of the measure. So, for example, the reliability of each of the domains (e.g. memory) assessed with instruments like the MMSE is questionable because of the restricted number of items; this is (in part) why it should not be used to diagnose dementia. Similarly, extremely brief inventories would be poor at capturing meaningful cognitive or psychiatric change due to an intervention.
Some instruments may simply provide a single global score reflecting a broader construct of “cognitive functioning.” Such instruments are often used clinically to ascertain the appropriateness of referral for more thorough testing or for provision of a service or placement or examination of the screening test protocol by a more knowledgeable clinician. These types of instruments can also be used in settings where more complex data gathering is impractical, or for shorthand communication of complex phenomena between professionals. An advantage of routine use of a small number of appropriate clinical screening scales in an RACF setting is that staff may begin to be more attuned to fluctuations in the domains of interest, and may come to a more shared vocabulary and construct of what various levels of cognitive, emotional and behavioral functioning look like in their patients. This can assist in such important tasks as handovers of patients at shift changes and introduction of new staff into the health care team.
Screens for psychiatric symptoms
A distinction here must be made between instruments that measure a discrete psychiatric symptom versus a wider range of domains of psychopathology. For example, there are instruments that screen for depressive symptoms alone, and instruments that sample across a range of psychiatric symptoms. Similar to cognitive screens, some psychiatric screens are labeled “global mental health” screens.
The use of specific tests depends upon the setting and characteristics of the individuals tested, as well as the purpose of testing. For older adults, there are some data to suggest that the way in which they might describe psychiatric symptoms is different to younger cohorts (Tweed et al., Reference Tweed, Blazer, Ciarlo, Wallace and Woolson1992; Shapiro et al., Reference Shapiro, Roberts and Beck1999). For example, the term “anxiety” might not be used explicitly, and physical symptoms might instead be reported to communicate psychological distress. Although beyond the scope of the current paper, this points once again to the need for careful instrument development with the specific populations of interest. For example, there is some evidence that older adults experience depression and anxiety differently to younger adults (Christensen et al., Reference Christensen1999). There is also evidence that older adults’ physiological responses to anxiety arousing stimuli differ from those of younger adults (Lau et al., Reference Lau, Edelstein and Larkin2001). In light of the foregoing, the content validity of instruments created with younger adults is questionable when used with older adults.
Some conditions (such as delirium) are signaled by a range of cognitive, psychiatric and behavioral symptoms. Such conditions are important for all levels of staff to be aware of, particularly if (as with delirium) they are relatively common in the RACF setting as well as being potentially life-threatening. Delirium is an example of a condition requiring not so much a screening approach but rather an integrated set of behavioral observations and informant ratings, particularly for residents with advanced dementia. Adequate management of conditions such as delirium also necessitates good communication between health care professionals within the RACF across time.
Screens for behaviors
A large array of observable behaviors may be of interest in a clinical setting such as the RACF or in older adults in the community who may be eligible for placement in a RACF. Often, certain behaviors are suggestive of specific psychiatric or neurological disorders – for example, delirium – or may trigger decisions by family caregivers regarding placement in an RACF. Thus while some behavioral screens are more general, many target specific behaviors or behavior clusters. It is important to note that it is often a combination of a particular behavior (e.g. aggression, wandering) displayed by the older adult, in combination with behaviors, coping skills and attitudes of family caregivers, that often leads to an admission to a long term care facility. Similarly, staff attitudes, knowledge and coping strategies, coupled with environmental factors and management policies, often combine with specific resident behaviors to lead to negative outcomes such as aggressive outbursts (either by patients or staff). Increasing recognition of a more systemic view of behaviors of concern in RACFs has in turn led not only to a number of measures of patient behaviors, but also to instruments measuring attitudes and coping in RACF staff. Although beyond the scope of this paper, awareness of such tools may be of great utility if gathering a broader picture of the complex dynamics within a RACF is desired.
One important issue with respect for behavior of residents within an RACF is that the stimulus or cue that initiates the behavior, particularly in persons with dementia, may be difficult to ascertain. Some instruments do a more satisfactory job than others of allowing for the objective reporting of behaviors free of subjective qualifiers such as “intentional” or “excessive”.
The period of observation of behaviors is very important. Uncommon but highly significant behaviors (e.g. inappropriate sexual behaviors) may be relatively rare events but may cause high levels of distress among carers both in the community and the RACF. Thus these behaviors may be missed if the time period of observation is inadequate. Some behaviors may be tied to time of day, medication dosing schedules, or self-care regimes, or to the behavior of specific carers (their behavior, their appearance, their tone of voice, facial expression, or touch), such that multiple ratings across shifts and times are necessary. Likewise, as mentioned previously, determining who might serve as the best informant about behaviors of concern is critical, as some persons may be better placed to report on specific aspects of behaviors in either time or place (e.g. past toileting regimes versus current responses to care).
Frequency or counts of behaviors are generally more reliable than the impact or level of distress of behaviors (de Vaus, Reference de Vaus2002). However, both the clinical and organizational consequences of impact as well as their frequency cannot be ignored. There are also data to suggest that the impact of behaviors differs across settings, For example, apathy is often cited by family as one of the most distressing behaviors associated with dementia, whereas in RACFs it is not of as much concern as, for example, aggression (Mega et al., Reference Mega, Cummings, Fiorello and Gornbein1996). This raises the interesting question of how problem behaviors are defined and by whom. Apathy is much easier to manage in an institutional environment than aggression, and may, in fact, be reinforced (Baltes, Reference Baltes, Ables, Gift and Ory1994). Anhedonia and other symptoms of depression may be viewed as normative in these environments, due to the high prevalence of mood disorders, and may not be readily identified by staff as problem behaviors.
Again, the purpose of assessment should be considered (for example, whether common or rare behaviors are of interest). In addition, the institution needs to have protocols in place to address the consequences of behaviors when they occur, such as sexual intimacy or behaviors that result in injuries to staff. In addition, the results of assessments ideally should be shared not only among staff but also with family members, who may be quite bewildered and concerned about specific behaviors displayed by persons with dementia.
Screens for functional status
This set of screening tools encompasses those measuring activities of daily living (both IADLs and ADLs); here there is a continuum between true brief screens and more comprehensive instruments. In this domain, all of these instruments might be considered as proxies for the actual ability to carry out a particular task. However, many instruments actually require performance of the tasks, and these may offer additional insight into the true capabilities of the person. For example, a measure to determine capacity to manage finances might include asking a person to interpret a telephone bill. Health care capacity measures often include a set of potential scenarios to help ground the person's ability to describe their wishes regarding treatment. Many ADL instruments include an observational component.
If the provision of rehabilitation services is available, screening of functional status may operate as a trigger for a referral for such services. Screening of functional capacities may also be useful in the community to flag signs of decline which might be clinically addressed, thus allowing a person to stay in their own home for as long as possible.
Conclusions
Recommendations for implementation of better practice of screening within the RACF environment
In light of the preceding discussion of important points for consideration in the ethical, clinically effective and best practice components of using screening instruments in RACFs, several recommendations are now offered. They are offered with the knowledge that the structure and functioning of facilities can vary widely, with respect to management philosophy, national or cultural considerations, and the unique synthesis of staff, residents and their families. Moreover, both the extant research literature as well as funding regimes and legislative and accreditation mandates evolve and shift over time; thus, any discussion of clinical practice and recommendations to improve such practice must also be updated and revisited over time.
1. Screening should be undertaken in the context of comprehensive assessment and humane care and the particular circumstances of the RACF. Screening for the sake of screening not only serves no purpose, but any assessment is a strain on the individual, uses resources, and requires ethical use of assessment instruments. Screening programs should be adequately resourced so that the clinical utility can be properly evaluated and demonstrated. A perennial complaint in RACFs is the limited amount of time available for tasks, and screening is no exception and must be properly supported. Part of this support is in the training and support of staff in the use of screening instruments, which may have indirect gains of increased sense of efficacy on the part of the staff member and increased meaningful communication between staff, residents and their family members.
2. Increased research should be undertaken on the clinical efficacy of screening tools. The determination of use of screening tools should be based on empirical evidence, but for many instruments the evidence is spotty and head-to-head comparisons of instruments are often lacking. In developed countries the sheer number of screens available may be daunting, and in developing countries access to useful tools may be limited. More work on determining the clinical efficacy of screening instruments is required, particularly with respect to clinical outcomes, for example in terms of symptom reduction or delay in disease progression. Furthermore, we do not fully understand the many potential “side effects” of screening (e.g. false positive diagnoses, erroneous treatment of mentally well people, creation of stigma). There is a need to encourage more research reports on the use of screening instruments within the RACF environment internationally.
There may be utility in adapting existing instruments to the RACF environment, particularly if the instruments were designed for older populations originally. However, as with all instruments (and as alluded to in the text above), such adaptation requires rigorous assessment of the performance characteristics of such adapted scales in the RACF context before relying upon them to assist clinical decision-making.
3. The development of a resource bank to support screening tool selection and implementation is important and requires support by all levels of administration. A public domain resource bank would help developing countries to provide a higher standard of care. Screening measures used should be periodically reviewed to take advantage of new knowledge and developments in the field. Information on cultural considerations with respect to the use of screening measures in various countries would be of great utility. Information and limitations of translations of various instruments would also be useful. Such a resource bank again might contribute to wider research efforts in the field internationally.
4. Policies should be developed concerning the training of staff in the appropriate use, administration, and interpretation of screening tests. Each facility should have policies on the standards for training and implementation or use of screening instruments. These policies should be developed with the input of staff and should be reviewed periodically.
Directions for future research and policy considerations
For some domains many screening measures exist (e. g. those for dementia and depression) for use in RACF settings. By contrast some areas have relatively few instruments (e.g. anxiety and psychosis). More research in the development and evaluation of instruments in these areas is important.
The development of more internationally applicable instruments may be appropriate. For example, the WHO-QOL (Power et al., Reference Power, Quinn and Schmidt2005; Peel et al., Reference Peel, Bartlett and Marshall2007) illustrates a productive international collaboration to develop an instrument with good psychometrics and the ability to be widely used. An international consensus-based approach is useful as it could allow large-scale trials of efficacy to be conducted which would be generalizable across multiple settings around the world. Broad consensus across political and organizational structures, cultures and nations is difficult but can be achieved with appropriate resources and strong leadership. The Kimberly Indigenous Cognitive Assessment (KICA; LoGiudice et al., Reference LoGiudice2006) is a good model for the developing world in terms of instrument development.
Furthermore, guidelines for trials of the efficacy, potential side-effects and clinical outcomes could be developed to guide future research in this area. This would stimulate new research in the area, and could result in a body of empirical evidence for instrument choice, particularly as many choices of instrument may be made on the basis of perceived extent of usage (often resulting in instruments inappropriately invested with best practice or “gold standard” status in the absence of an empirical evidence base). Again, an international consensus approach would be important and cross-national trials informative.
Education on the appropriate use of screening tools, including how best to select instruments based on their development, psychometric properties and utility, is necessary for appropriate and ethical administration and interpretation of clinical measures in the RACF. Better communication between family and facility care staff can assist with interpreting the results of screening tools. In turn, appropriate interpretation of screening measures can facilitate understanding of patients’ behaviors and functioning by family.
On an individual level, health care providers can ensure that screening instruments are administered in a clinically appropriate manner. Individuals can ensure they possess the competence to administer, score and interpret any screening tests used. There is a need to follow relevant local guidelines and standards for ethical professional practice.
At the level of peak bodies and organizations, the establishment of guidelines is important. For example, the standards for educational and psychological testing developed jointly by the American Psychological Association, the American Educational Research Association and the National Council on Measurement in Education reflect both measurement theory and clinical issues in the current literature as well as legal considerations involved in testing in the U.S.A. (American Psychological Association, 1999).
Finally, at the level of individual organizations there is a need to develop locally relevant policies related to the development, use and evaluation of screening instruments and screening programs. At all times care should be taken to ensure that the purpose and use of clinical assessments are clear.
Conflict of interest declaration
Nancy A. Pachana is a Deputy Editor of International Psychogeriatrics.
Description of authors’ roles
N. A. Pachana, E. Helmes and G. J. A. Byrne formulated the basic premise of the manuscript and wrote the paper. B. Edelstein, C. Konnert and A-M. Pot contributed major sections, revised drafts and assisted with writing the paper.
Acknowledgments
We would like to thank sincerely the following colleagues for their input to this paper: Professor Desley Hegney, National University of Singapore; Ian James, Newcastle Centre for the Health of the Elderly, U.K.; Oscar Ribeiro, Aveiro University, Portugal; Dick Verburg, VU University Medical Center, Amsterdam; and Johannes Wancata, Medical University of Vienna.
