Introduction
Behavioral observation in an inpatient setting has been claimed to contribute to cognitive assessment, supplementing information obtained from medical history and cognitive testing (Spiegel et al., Reference Spiegel1991; Dellasega, Reference Dellasega1998; Langley, Reference Langley, Kane and Kane2000; Inouye et al, Reference Inouye, Foreman, Mion, Katz and Cooney2001; Foreman et al, Reference Foreman, Fletcher, Mion, Trygstad, Mezey, Fulmer, Abraham and Zwicker2003; Bouwens et al., Reference Bouwens2008). The level and nature of everyday cognitive functioning can provide valid information for medical diagnoses, nursing interventions, interpretation of neuropsychological tests, discharge arrangements and placement (Langley, Reference Langley, Kane and Kane2000; Marson and Herbert, Reference Marson, Hebert, Attix and Welsh2006). In nursing, daily practice involves observing admitted patients' behavior around the clock during patient-care activities, such as assisting with personal care, transfers or meal times. It has been claimed that reliable and valid assessment of cognitive functioning through daily observation is problematic, because overt behavioral deficits and disabilities may not be directly linked to a specific underlying neurocognitive impairment (Lezak et al., Reference Lezak, Howiesen, Loring, Hannah and Fischer2004). For instance, observed “forgetting” may be the result of memory impairment, or it may also be due to cognitive deficits in other domains (e.g. impaired attention) or psychiatric symptoms (such as depression). In daily practice, however, monitoring of patients' behavior can provide a basis for initial screening for cognitive decline.
In this study, we focused on the validity of daily observation of memory function by means of a standardized observation scale in relation to neuropsychological memory tests. Although many behavioral scales include items addressing memory function, observation scales including a specific memory subscale are limited in number. Examples of memory (sub)scales are the Cognitive Problem Scale (CPS), part of the Resident Assessment Instrument (RAI; Morris et al., Reference Morris1994) and the Disoriented Behaviour subscale of the Multidimensional Observation Scale for Elderly Subjects (MOSES; Helmes et al., Reference Helmes, Csapo and Short1987). Furthermore, memory (sub)scales are typically validated against nonspecific cognitive measures, such as the Mini-mental State Examination (MMSE). Only the Nurses' Observation Scale for Geriatric Patients (NOSGER) memory subscale has been additionally validated against several specific neuropsychological memory tests (Spiegel et al., Reference Spiegel1991; Wahle et al., Reference Wahle, Häller and Spiegel1996), demonstrating satisfying concurrent validity. As we are interested in the value of observation scales as a way to assess memory function, we also explored the value of other observation scales. In the present study, we examined the construct validity of the memory subscale of the Nurses' Behavioural Rating Scale for Geriatric Inpatients (GIP) (Verstraten, Reference Verstraten1988). In Dutch nursing homes, the GIP is widely used in geriatric clinical practice and its psychometric qualities have been studied extensively (de Jonghe et al., Reference de Jonghe, Kat and de Reus1994; Reference de Jonghe, Kat, Rottier and de Reus1995; Reference de Jonghe, Calis and Boom-Poels1996). The GIP memory subscale focuses on geriatric inpatients as well as the NOSGER, but the subscale items differ (see Table 1).
NOSGER and GIP memory subscales
Methods
Sample and setting
Consecutive patients admitted to the geriatric ward of the Radboud University Nijmegen Medical Centre from July to December 2007 were recruited if they fulfilled the inclusion criteria of understanding simple instructions and being able to complete the 45-minute neuropsychological test battery. Further, exclusion criteria were applied as follows: having delirium (Delirium Observation Scale/DOS < 3; Schuurmans et al., Reference Schuurmans, Shortridge-Baggett and Duursma2003); being bedridden; having severe dementia (Clinical Dementia Rating Scale/CDR = 3; Hughes et al., Reference Hughes, Berg, Danziger, Coben and Martin1982); having severe hallucinations; and not being able to complete the 45-minute neuropsychological test battery. All nurses participating in the study were registered nurses (n = 25), and some were qualified clinical geriatric nurses with a two-year specialist training (n = 15).
GIP
For the memory observation scale, we used the six items from the Memory subscale from the GIP (range 0 to 3; lower scores indicating more memory problems). The GIP contains 82 items in total, subdivided into 14 subscales on a 4-point Likert scale, covering aspects of cognitive and non-cognitive functioning in geriatric patients. The content validity of the total GIP has been assessed; the construct validity showed satisfactory correlation with the Cognitive Screening Test (de Jonghe et al., Reference de Jonghe, Calis and Boom-Poels1996). The internal consistency was good (Cronbach's α = 0.86), while the inter-rater reliability of the memory subscale was fair to moderate (κ = 0.47). Furthermore, the memory subscale ratings were highly related to clinical diagnosis: mean scores in patient groups with dementia, psychosis, depression were 9.54, 3.61 and 1.72, respectively. In our study, the nurse observed the patient in routine clinical care during an eight-hour shift. The memory subscale sheet was filled in during day and evening shifts for three consecutive days, thereby yielding six assessments per patient. The GIP memory score was calculated by computing the mean of the six items per observation period, and then computing the mean of the six observation periods, resulting in a score between 0 and 3, with low scores indicating more memory problems.
Data gathering and analysis
In a 45-minute neuropsychological test battery, four different types of memory tasks were conducted: visual paired-associate learning, word-list learning, and story recall and spatial memory. The following tests, which are valid and reliable measures of different aspects of memory function and are widely used in Dutch clinical geriatric practice, were selected: Visual Association Test (VAT; Lindeboom et al., Reference Lindeboom, Schmand, Tulner, Walstra and Jonker2002), the Eight-Word Verbal Learning Test (8WT, from the Amsterdam Dementia Screening; ADS; van der Pol-Vetten and Liem, Reference van der Pol-Vetten and Liem1992), and the subtests Route Recall and Story Recall from the Rivermead Behavioural Memory Test (RBMT; Wilson et al., Reference Wilson, Cockburn and Baddeley1985), respectively. The VAT is a brief visual associative learning task based on imagery mnemonics (range 0–12). The 8WT measures the quality of learning a list of words in several rounds (range 12–55). The Route Recall (range 0–16) and Story Recall (range 0–42) subtests of the RBMT provide ecologically valid measures of spatial memory and verbal memory respectively (Lezak et al., Reference Lezak, Howiesen, Loring, Hannah and Fischer2004). All tests were administered by a trained junior psychologist who was blind to the memory GIP observation ratings, on one of the three days of the GIP observation period.
Additionally, we determined the global cognitive status using the MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975), and screened for depressive symptoms through the Geriatric Depression Scale (GDS-15; Yesavage et al., Reference Yesavage1982), both administered together with the neuropsychological tests. Severity of dementia was determined by means of the CDR score on the day of discharge. Comorbidity was measured through the Cumulative Illness Rating Scale-Geriatric (CIRS-G; range 0–52; higher scores indicate a higher level of comorbidity; Miller et al., Reference Miller1992).
Pearson's correlation coefficients were computed to determine the correlation between the scores on the memory subscale and the four neuropsychological test performances. Statistically significant coefficients higher than 0.4 were considered as meaningful correlations (Slick, Reference Slick, Strauss, Sherman and Spreen2006). We expected no significant correlation between the memory observation score and the depression scale. Differences in GIP memory scores in groups without dementia (CDR = 0) and with dementia (CDR = 1 or 2), in groups with different level of memory impairment (MMSE > 23, MMSE = 18–23; and MMSE<18), and in groups with and without depressive symptoms (GDS > 5, GDS ≤ 5) were analyzed by means of t-test and ANOVA test.
Sample
During the study period 129 patients were admitted to the geriatric ward, 63 of whom fulfilled the inclusion criteria. For 13 patients, it was not possible to collect data because of early discharge or logistic reasons, resulting in a total of 50 patients who completed the full test battery. Of these, 64% were female; the mean age was 83 years (SD = 6.7, range 72–97). The lowest level of education was primary school (30%), and the highest level was university education (10%). The mean number of different drugs per day was 4.6 (SD = 2.5, range 0–10).
Results
The mean GIP memory score was 2.3; the minimum score was 1.4 and the maximum 2.9, standard deviation 0.42 (range 0–3, lower scores indicating more memory problems) (n = 50). The intraclass correlation between the six measurement moments was 0.59. The mean severity score of comorbidity was 14 (SD = 5.4, range: 3–28) on the CIRS-G scale, reflecting the considerable disease burden. According to the CDR criteria, 36% of the respondents had no dementia (CDR = 0), 30% had questionable or mild dementia (CDR = 0.5 or 1), and 30% moderate dementia (CDR = 2). A quarter of the patients reported significant depressive symptoms on the GDS-15 (GDS > 5). Based on the MMSE, 38% of the respondents had no or mild indications for cognitive impairment (MMSE ≥ 24), 32% had moderate impairment (MMSE 18–23) and 30% had severe cognitive impairment (MMSE ≤ 17).
Analyses revealed that the correlation between the results on the memory observation scale and the performance on four neuropsychological memory tests were statistically significant and higher than 0.4 (between 0.45 and 0.71), as shown in Table 2. The correlations among the four memory tests themselves varied between 0.51 and 0.80 (p < 0.05). Furthermore, the correlation with the MMSE was 0.63, and with the dementia severity scale (CDR) it was −0.46, both statistically significant (p < 0.01). No significant correlation was found between the memory observation scale and depression score GDS-15 (p > 0.05).
Correlations between daily observations, four neuropsychological memory tests, MMSE, CDR and GDS-15
** Correlation is significant at the 0.01 level (2-tailed).
* Correlation is significant at the 0.05 level (2-tailed).
GIP = Nurses' Behavioral Rating Scale for Geriatric Inpatients; VAT = Visual Association Test; 8WT = Eight Word Test; RBMT = Rivermead Behavioural Memory Test; MMSE = Mini-mental State Examination; CDR = Clinical Dementia Rating scale; GDS = Geriatric Depression Scale.
The mean GIP memory scores differed statistically significantly (p < 0.01) between patients with dementia and those without dementia: 2.0 and 2.4, respectively. The mean GIP memory scores also differed statistically significantly (p < 0.5) between different groups of memory impairment: 2.6, 2.4 and 1.9 respectively. The mean GIP memory scores in the group with or without depressive symptoms did not differ, being 2.3 and 2.4, respectively.
Discussion
This study is one of the first studies to explore the value of an observation scale for daily memory function. Correlations between the daily observation scale scores and neuropsychological tests performances were all in the fair to moderate range. The correlations between GIP and the neuropsychological tests were similar in strength to the correlations among the four memory tests, suggesting that the daily observation may be a valid way to assess memory function. This is in line with the NOSGER study in which results of observation items were correlated with results of neuropsycological tests (Spiegel et al., Reference Spiegel1991; Wahle et al., Reference Wahle, Häller and Spiegel1996). However, the correlations between observation scores and measures for global cognitive decline (MMSE and CDR) were moderate as well. This may suggest that the observation scale may be tapping global cognitive status rather than specific memory impairment. To examine this in greater depth, we recommend comparing the results of the observation scale with neuropsychological tests examining other cognitive domains (such as attention or executive function).
A strength of the observation scale is that no correlation was found with the depression scores. As differentiation between memory impairment and depression is rather complicated in clinical practice, this finding is an important added value.
Overall, the results of our study are in line with the results of the NOSGER validation study which found that an observation scale for memory function may have value for providing information on underlying memory impairment. The bedside nurses needed no extra education to fill out the GIP form and they reached fair to moderate levels of correlation with neuropsychological tests. The results of our study can be generalized to geriatric inpatients with considerable variety in the severity of cognitive decline and overall disease burden. We recommend further research for the validity of observation scales in other settings of elderly care (psychiatric hospital, home care, nursing homes, homes for the elderly, and outpatient clinics).
A valid and reliable assessment of cognitive function by nurses is important because nurses tailor their interventions to the patient's cognitive abilities, and are often asked to contribute to the diagnostic process as well. The observational results may be used in triage, thereby contributing to diagnosis by pointing to the need for neuropsychological assessment. Daily observation is an easy, inexpensive and highly feasible assessment procedure for nurses and can be conducted continuously during daily care. It is not burdensome to the patient and can even be performed with very ill patients. Therefore, memory observation scales have significant potential to contribute to the quality of care for the increasing populations of older patients.
Conflict of interest
None.
Description of author's roles
Anke Persoon, Roy Kessels and Liesbeth Joosten-Weyn Banningh designed the study. Jill Verkoelen collected the data and analyzed the data together with Anke Persoon. Anke Persoon and Roy Kessels wrote the paper. Marcel Olde Rikkert and Theo van Achterberg critically reviewed the study design and the paper.
