Introduction
Although complaints about memory decline are quite common among older adults, they do not always signal the onset of neurodegenerative diseases (Minett et al., Reference Minett, Da Silva, Ortiz and Bertolucci2008). Memory complaints may be associated with personality characteristics, mood, anxiety, and metamemory (one's own beliefs, feelings and knowledge about memory) (Vianna Paulo and Yassuda, Reference Vianna Paulo and Yassuda2010). For clinicians and family members, it may be challenging to judge whether memory complaints indicate changes from previous levels of performance. Objective testing may help elucidate whether cognitive performance is within the expected range, considering age and education.
However, objective testing of memory and other cognitive functions has always faced criticisms, among them the fact that traditional tests do not reflect everyday functioning (Makatura et al., Reference Makatura, Lam, Leahy, Castillo and Kalpakjian1999). This is a critical issue for professionals in the field of gerontology, because they are frequently requested to assist families in making decisions about living arrangements, driving cessation, occupational limitations, and other autonomy-related issues.
In Latin American countries psychometric testing is a complex issue. Countries such as Brazil present a vast territory with striking cultural and socioeconomic differences. Adapting, validating and producing performance norms for a psychometric instrument may be a daunting task when one considers a sample that would be representative of such a diverse population. Further difficulties are faced when the aging population is considered. Aging in southern Brazil is completely different from aging in northern areas, since health, education and leisure opportunities may vary substantially (Neri, Reference Neri2007). Solutions to these problems may include validating and normatizing instruments for particular populations, finding instruments that are not affected by education, and developing mathematical ways to correct for educational or cultural biases.
In Brazil, partly due to the challenges described above, few cognitive instruments have been adequately validated for the assessment of cognitive decline in the older population (see Nitrini et al., Reference Nitrini, Caramelli, Bottino, Damasceno, Brucki and Anghinah2005 for a discussion of this issue). Significant progress has been made in the study of screening instruments for the diagnosis of dementia (Brucki et al., Reference Brucki, Nitrini, Caramelli, Bertolucci and Okamoto2003; Flaks et al., Reference Flaks2006; Lourenço et al., Reference Lourenço, Ribeiro-Filho, Moreira, Paradela and Miranda2008). With the sharp rise in the percentage of older adults in the Brazilian population in the next decades, and the expected increase in the prevalence of dementing syndromes, a study of cognitive batteries that may help differentiate healthy aging from early onset dementia is warranted.
The Rivermead Behavioural Memory Test (RBMT) was created to bridge the gap between traditional memory tests and observational measures of everyday performance (Wilson et al., Reference Wilson, Cockburn and Baddeley1985; Reference Wilson, Cockburn, Baddeley and Hiorns1989). The RBMT is a short memory battery composed of 12 tests which require the patient to remember a name and a last name, a short newspaper article, and a route, to recognize objects and faces, to remember to request a hidden belonging, to ask a question when an alarm rings, and to answer time and space orientation questions. Because it simulates everyday tasks, the RBMT is regarded as an ecological memory battery.
Previous studies have shown that the RBMT is a valid and reliable tool to detect memory deficits in patients with different neuropsychological profiles, such as patients with brain injury (Makatura et al., Reference Makatura, Lam, Leahy, Castillo and Kalpakjian1999), epilepsy (Pérez and Godoy, Reference Pérez and Godoy1998), Alzheimer's disease (Kazui et al., Reference Kazui2005), and older adults with age-associated changes (Ostrosky-Solis et al., Reference Ostrosky-Solis, Jaime and Ardila1998). It correlates well with observational and questionnaire-based measures of memory, and it has adequate test-retest stability, and inter-rater agreement (Wilson et al., Reference Wilson, Cockburn, Baddeley and Hiorns1989; Makatura et al., Reference Makatura, Lam, Leahy, Castillo and Kalpakjian1999). Additionally, the battery offers four alternative versions which are useful for the evaluation of interventions and continued follow-up assessment. The RBMT has also been adapted and validated in different cultures, maintaining its psychometric characteristics (Alonso and Prieto, Reference Alonso and Prieto2004; Kazui et al., Reference Kazui2005; Küçükdeveci et al., Reference Küçükdeveci, Kutlay, Elhan and Tennant2008). A recent study has also suggested that it is not significantly biased by education (Yassuda et al., Reference Yassuda2009).
A Brazilian version of the RBMT has been elaborated (Oliveira and Schmidt, Reference Oliveira and Schmidt1999); however, to our knowledge, only preliminary analyses of its psychometric properties have been conducted (Yassuda et al., Reference Yassuda2006). Therefore, the aim of the present study was to assess some of the psychometric characteristics of the RBMT when applied to a sample of Brazilian older adults with different cognitive profiles, and to investigate whether it can contribute to the early detection of cognitive decline. Reliability was assessed by calculating internal consistency among the 12 RBMT subitems, and validity was assessed by investigating whether RBMT scores contributed significantly to clinical diagnosis of patients as normal controls (NC), mild cognitive impairment (MCI), and Alzheimer's disease (AD). Correlations between RBMT and the Cambridge Cognitive Test (CAMCOG) memory subscale scores (Roth et al., Reference Roth1986), also applied in this sample, were calculated to assess convergent validity. Cutoff scores were also proposed for AD and MCI, and the RBMT accuracy to detect such conditions was evaluated.
Methods
Participants
Baseline data for 195 older adults (51 men) who are regularly followed up at the Psychogeriatric Unit of the Institute of Psychiatry, University of São Paulo, Brazil, were included in the present analyses. Participants were originally recruited from community sources (community centres, newspaper and radio advertisements), where a study on memory and aging was advertised. The sample also included participants who spontaneously sought medical attention related to memory complaints, or who were referred from other hospital units for assessment of suspected cognitive decline. Few participants (less than 5%) were volunteers who wanted to donate their time to a research study, and were mainly relatives who came to accompany a family member with memory concerns. Most of this clinical sample was therefore composed of older adults who were concerned about possible cognitive decline.
NC, MCI and AD patients came from different sources. However, because participant origin has not been included as a variable in our database, it was not possible to determine objectively whether specific sources have contributed differently to each diagnostic group, and how many participants came from each source. All subjects signed an informed consent prior to enrolment. This study was approved by the Local Ethical Committee and was performed in accordance to the Helsinki Declaration. This sample has been described in previous articles (Diniz et al., Reference Diniz2008; Yassuda et al., Reference Yassuda2009).
Screening tests and instruments
Patients were examined by clinicians specialized in the evaluation of dementing disorders, including geriatric psychiatrists, neurologists and neuropsychologists. Psychiatrists performed general mental state examination with the Brazilian version of CAMCOG (Roth et al., Reference Roth1986), which is a compilation of screening tests. The CAMCOG includes the Mini-mental State Examination (MMSE) (Folstein et al., Reference Folstein, Folstein and McHugh1975). The Brazilian version of the CAMCOG has shown adequate psychometric and diagnostic properties to detect dementia and mild cognitive impairment (Bottino et al., Reference Bottino, Almeida, Tamai, Forlenza, Scalco and Carvalho1999; Nunes et al., Reference Nunes2008). The 21-item Hamilton Depression Scale (HAM-D) was administered to rule out depressive symptoms (Hamilton, Reference Hamilton1960).
A complete neuropsychological examination was conducted by trained psychologists on a different day and it included the Rivermead Behavioural Memory Test (Wilson et al., Reference Wilson, Cockburn and Baddeley1985; Oliveira and Schmidt, Reference Oliveira and Schmidt1999; Yassuda et al., Reference Yassuda2006; Diniz et al., Reference Diniz2008). A description of the neuropsychological protocol can be found in previous articles (Diniz et al., Reference Diniz2008; Yassuda et al., Reference Yassuda2009).
The Brazilian version of the RBMT uses the visual stimuli of the original test for faces and pictures. Verbal stimuli, such as names and stories, were adapted to Brazilian culture, yet, they have remained highly similar to the original ones, in terms of task difficulty and structure. The Brazilian test manual reports that the authors produced a direct translation from the original test for verbal stimuli and instructions. This initial version of the test was applied to a group of adult volunteers to verify its adequacy for the Brazilian population. After this pilot application, some adjustments were implemented and this adapted version was applied to a new group of adult volunteers. New adjustments were made to generate the final version of the test. The translation and adaptation procedures were carried out by two neuroscientists who completed their graduate education in English-speaking countries.
Evidence of functional decline was based on the scores of the Informant Questionnaire of Cognitive Disorders of the Elderly (IQCODE) (Jorm and Jacomb, Reference Jorm and Jacomb1989; Abreu et al., Reference Abreu, Nunes, Diniz and Forlenza2008) and on the Blessed Dementia Scale (BDS) (Blessed et al., Reference Blessed, Tomlinson and Roth1968). Clinicians also took into account caregivers’ and patients’ reports on limitations to activities of daily living.
For this study, the variables of interest were the RBMT screening and profile scores and the CAMCOG total and memory subscale scores. The RBMT screening score ranges from 0 to 12, and the profile score ranges from 0 to 24. The CAMCOG total score varies from 0 to 107, and the memory subscale score of the CAMCOG, which involves memorization of new information among other memory tasks, varies from 0 to 27. The MMSE was also used to characterize the sample, ranging from 0 to 30 points.
Procedures
Consensus diagnoses were reached by an expert multidisciplinary team, taking into account clinical, neuropsychological, and laboratory and neuroimaging data. All information available was used to characterize cognitive preservation, the presence of MCI or dementia during consensus meetings, and no specific test was used as gold standard. Participants were classified as normal controls (NC), mild cognitive impairment (MCI), and Alzheimer's disease (AD). Dementia was diagnosed according to DSM–IV criteria (American Psychiatric Association, 1994), and AD was diagnosed according to the NINCDS–ADRDA criteria (McKhann et al., 1994). Diagnosis of MCI was made according to Petersen's criteria (Petersen, Reference Petersen2004): (1) subjective cognitive complaint, preferably corroborated by an informant; (2) objective impairment in the performance on the cognitive tests of the assessment battery, but not severe enough to reach dementia diagnosis; (3) preserved global intellectual function; (4) preserved or minimal impairments in activities of daily living. Patients classified as NC did not report any functional impairment and their cognitive test results were within normal ranges in all tests. MCI patients reported overall functional preservation; however, their cognitive performance was outside normal ranges, in at least one test (for a complete description of cutoff scores used for MCI classification, see Yassuda et al., Reference Yassuda2009). AD patients presented cognitive and functional impairment.
Patients diagnosed with other types of dementia were excluded from the sample (n = 12) and referred to other hospital units. Patients with other psychiatric disorders, such as depression (n = 18), and other neurologic disorders (n = 5), were also excluded from the sample, as were patients with missing data for the variables included in the present study.
Laboratory tests were carried out for every patient to rule out potentially reversible causes of cognitive impairment, including: thyroid function, complete blood count, blood chemistry, folic acid and vitamin B12, blood lipid profile, syphilis tests. Neuroimaging studies (CT scans or MRI) were completed according to clinical judgment.
Statistical analyses
Presence of normal distribution was tested with Kolmogorov-Smirnov tests. Age-followed normal distribution and between group differences were tested with parametric ANOVAs with Tukey post-hoc analyses. Education, CAMCOG and MMSE did not follow normal distribution and between-group differences were assessed with Kruskal-Wallis analyses and with Bonferroni post-hoc analyses. As diagnostic groups were different in age and education and the RBMT variables did not display normal distributions, multinomial logistic regression analyses (with age and education included in the model) were used to test whether each cognitive score was important to determine caseness according to diagnostic groups (Neter et al., Reference Neter, Kutner, Li and Nachtsheim2005). Internal consistency analyses were carried out using Cronbach's α. As a measure of convergent validity, the RBMT scores were correlated with the scores for the CAMCOG memory subscale with Spearman's correlations. To assess the accuracy of the RBMT to detect AD and MCI, non-parametric ROC curve analyses were performed with the sample divided into two educational ranges: participants with eight or less years of formal education (n = 88) and more than eight years (n = 116). This classification was chosen because eight years of education in Brazil represents complete elementary school, and cut off scores for specific education strata have important clinical implications.
Results
The sample included in the present analysis was composed of 195 older adults (NC = 78, MCI = 76, AD = 41). The sociodemographic characteristics and cognitive profile of the total sample are described in Table 1. The AD group was significantly older than the other groups, and the NC group had significantly higher levels of education. There were significant differences among the three diagnostic groups for cognitive status, assessed by the CAMCOG and the MMSE. To test for gender differences, men and women were compared for the RBMT screening and profile scores. No significant differences were found and gender was not included in the analyses that followed.
Means and standard deviation in parenthesis for age, median and inter-quartile intervals (IQI) for education and cognitive variables for the total sample (N = 195).
a. ANOVA with Tukey post hoc tests: NC and MCI are statistically similar but different from AD.
b. Kruskal-Wallis with Bonferroni comparisons: MCI and AD are statistically similar but different from NC.
c Kruskal-Wallis with Bonferroni comparisons: All three groups are statistically different.
Bold indicates statistically similar values.
NC = normal controls; MCI = mild cognitive impairment; AD = Alzheimer's disease; CAMCOG = Cambridge Cognitive Test; MMSE = Mini-mental State Examination; RBMT = Rivermead Behavioural Memory Test.
Table 2 presents medians and inter-quartile intervals for RBMT screening and profile scores, and for each RBMT subtest score for the three diagnostic groups. Multinomial logistic regression analyses indicated that total scores and each subtest score contributed significantly to determine whether patients were classified as AD, MCI or NC, even when age and education were included in the model as covariates. Multiple group comparisons, from the multinomial logistic regression analyses, revealed that the subtests Appointment, Pictures, Immediate and Delayed Story, Immediate and Delayed Route, Delayed Message, and Date contributed to differentiate MCI from NC participants. For the other subtests, NC and MCI participants had equivalent performance, but scored significantly higher than the AD group.
Medians and inter-quartile intervals (IQI) in parenthesis for RBMT variables for the total sample (N = 195)
Note. All p values refer to the multinomial logistic regression controlling for age and education; they indicate whether each sub test contributes to determine diagnostic membership.
a Post hoc tests: All three groups are statistically different.
b Post hoc tests: NC and MCI are statistically similar but different from AD. Bold indicates statistically similar values.
Bold indicates statistically similar values.
NC = normal controls; MCI = mild cognitive impairment; AD = Alzheimer's disease; CAMCOG = Cambridge Cognitive Test; MMSE = Mini-mental State Examination; RBMT = Rivermead Behavioural Memory Test.
For the RBMT profile score internal consistency was high for the total sample (Cronbach's α = 0.91). When the sample was split into the three diagnostic groups, Cronbach's α was high for the AD group (0.84) and moderate for the MCI (0.62) and the NC (0.62) groups. For the RBMT screening score, similar results were found. Cronbach's α was 0.87 for the total sample, 0.85 for the AD group, 0.55 for the MCI group and 0.60 for the NC group.
Spearman's correlations indicated a robust and significant association between the RBMT total scores and the CAMCOG memory subscale score (profile, ρ = 0.68, p < 0.001; screening, ρ = 0.65, p < 0.001), the CAMCOG total score (profile, ρ = 0.74, p < 0.001; screening, ρ = 0.71, p < 0.001), and the MMSE (profile, ρ = 0.65, p < 0.001; screening, ρ = 0.65, p < 0.001).
Results for the non-parametric ROC curve analyses are presented in Table 3, for participants with eight or less years of education, and in Table 4 for participants with more than eight years of education. Results suggest that the RBMT has high accuracy in both education strata to differentiate AD from NC. Accuracy is lower when the RBMT is used to distinguish MCI from NC. Negative predictive values are lower when the RBMT is used to differentiate MCI from AD in the group with lower educational experience.
ROC curve analyses results for participants with eight or less years of education (n = 87)
Note. NC n = 19, MCI n = 38, AD n = 30.
NC = normal controls; MCI = mild cognitive impairment; AD = Alzheimer's disease; RBMT = Rivermead Behavioural Memory Test; PPV = positive predictive value; NPV = negative predictive value; AUC = area under the curve; CI = confidence interval.
ROC curve analyses results for participants with more than eight years of education (n = 113)
Note. NC n = 58, MCI n = 39, AD n = 16.
NC = normal controls; MCI = mild cognitive impairment; AD = Alzheimer's disease; RBMT = Rivermead Behavioural Memory Test; PPV = positive predictive value; NPV = negative predictive value; AUC = area.
Discussion
The objective of the present study was to evaluate the contribution of the RBMT to the diagnosis of MCI and AD, and to assess its psychometric characteristics in a clinical sample of Brazilian older adults.
Present results suggested that this instrument in its Brazilian version can differentiate clinical groups. Previous studies have demonstrated the RBMT can separate NC from patients with cognitive decline due to AD (Pérez and Godoy, Reference Pérez and Godoy1998; Matsuda et al., Reference Matsuda, Kazui, Hirono and Mori2002), brain damage due to different causes (Wilson et al., Reference Wilson, Cockburn, Baddeley and Hiorns1989), and temporal lobe epilepsy (Pérez and Godoy, Reference Pérez and Godoy1998). More recently, Kazui and colleagues (Reference Kazui2005) have shown that the RBMT can contribute to the diagnosis of MCI in a sample of Japanese older adults. In agreement with our results, MCI patients in general performed significantly lower than NC and significantly higher than AD patients, as indicated by the RBMT profile and screening scores. Comparing study results, one can see that the Japanese MCI group had lower mean RBMT scores than the present sample, which may be due to sampling differences. In the Japanese study, as the authors acknowledged, the MCI patients were severe cases whose performance was more similar to AD patients than NC patients. In our sample, the performance of MCI participants is closer to that of NC, and conversion to dementia has been below expected rates (Diniz et al., Reference Diniz, Nunes, Yassuda and Forlenza2009).
Kazui and colleagues (Reference Kazui2005) suggested that delayed memory tests in the RBMT may be particularly useful in identifying MCI cases. Our results point out that prospective memory subitems, such as Appointment, and immediate and delayed memory tasks based on free recall, such as Story and Route (not based on recognition, such as Pictures and Faces) may be the most useful. Previous analyses of the RBMT prospective memory items in a sample of older adults with early stage dementia have suggested that prospective memory is particularly vulnerable to neurodegenerative diseases, and may show changes even before episodic (retrospective) memory (Huppert and Beardsall, Reference Huppert and Beardsall1993). Recent studies involving RBMT prospective memory subtests in a sample of Brazilian AD patients and NC have suggested that deficits are of higher magnitude in retrospective memory tests, although AD patients show poorer performance in both when compared to controls (Martins and Damasceno, Reference Martins and Damasceno2008). Present analyses have suggested that Names, Faces and Orientation were among the subtests that failed to discriminate MCI from NC. This may be due to the fact that Faces relies on recognition, and some of the Orientation questions are based on over-learned semantic memory items.
Internal consistency results have replicated previous findings (Alonso and Prieto, Reference Alonso and Prieto2004; Küçükdeveci et al., Reference Küçükdeveci, Kutlay, Elhan and Tennant2008), and it was fairly high when the entire sample was included in the analyses. However, in separate diagnostic group analyses, correlations among RBMT items were moderate for NC and MCI groups, while it was high for the AD group. To our knowledge, previous studies have not evaluated this aspect of the RBMT reliability. In less impaired patients, higher inter-item variability may be due to the fact that the RBMT assesses different memory subsystems, such as episodic and prospective memory, in addition to space and time orientation, and participants may display asymmetrical performance in them. The test also involves visual and auditory stimuli, and different testing paradigms, such as recognition and free recall, which may increase inter-item variability in subtest scores. This variability may be limited in AD patients who may have lower scores in most subtests due to floor effects. The presence of ceiling effects for some NC and MCI participants may have generated a restriction of range in the scores and it is also a possible explanation for lower coefficients. One should also consider that for a test which aims to assess everyday performance by means of diverse memory tasks, internal consistency may be lower as tasks may recruit different memory subsystems which might follow differential patterns of age-related decline.
Previous studies have shown that the RBMT correlates well with other memory measures (Wilson et al., Reference Wilson, Cockburn, Baddeley and Hiorns1989; Pérez and Godoy, Reference Pérez and Godoy1998; Makatura et al., Reference Makatura, Lam, Leahy, Castillo and Kalpakjian1999). In the present study, to evaluate convergent validity, RBMT and CAMCOG memory subscale scores were correlated and coefficients were high and significant for the total sample. It is noteworthy that the RBMT scores were also strongly correlated to the total CAMCOG score and the MMSE. This finding seems to suggest that the RBMT shares variance with general mental status screening tools, such as the CAMCOG and the MMSE. In fact, the RBMT includes space and time orientation questions and prospective memory items (heavily dependent on attentional control), not usually included in most memory tests.
Sensitivity and specificity analyses for proposed cut off scores have suggested that the RBMT screening and profile scores display similar accuracy. In addition, results suggest this tool has high accuracy to identify AD and moderate accuracy to identify MCI in both educational levels. This finding seems to support previous results from our laboratory (Yassuda et al., Reference Yassuda2009) indicating that the RBMT is less influenced by educational experience than other cognitive instruments, such as the CAMCOG, the MMSE, and the Wechsler Adult Intelligence Scale-revised (WAIS-R) Vocabulary and Block Design. As hypothesized previously, memory may be less vulnerable to education effects. Alternatively, the RBMT may access memory using paradigms that parallel everyday challenges, making educational experience less relevant. These findings have important clinical implications, particularly for developing countries, and for developed nations with a significant proportion of older immigrants, because they suggest that the RBMT may be used in older populations with diverse educational backgrounds. In Brazil, present findings need to be interpreted with caution, as one needs to take into account the cultural differences which exist between southern urban areas, where the present data were collected, and the rest of the country.
In general, the results support the clinical use of the Brazilian version of the RBMT with older patients in the diagnosis of dementia and early cognitive decline associated with MCI. It should be pointed out that the RBMT was capable of distinguishing MCI from NC participants, showing moderate sensitivity and high specificity, rendering it a specific rather than a screening test. Other cognitive instruments such as the CAMCOG, the Clock Drawing Test, and verbal fluency have not been able to distinguish these two groups in this sample (Diniz et al., Reference Diniz2008). Nevertheless, it should be noted the RBMT had poorer accuracy in detecting MCI among elderly people with lower levels of education as negative predictive values were lower.
Unfortunately, sample size did allow stratification of the sample into different age groups, and this is a limitation of the present study. Future studies should investigate test-retest stability, inter-rater agreement, and the equivalence of the four available versions of this instrument, aspects which could not be incorporated in this study.
Conflict of interest
None.
Description of authors’ roles
Mônica Sanches Yassuda participated in study design, collected data, was responsible for data analyses and wrote the paper; Mariana Kneese Flaks, Luciane Fátima Viola, Fernanda Speggiorin Pereira, Claudia Maia Memória and Paula Villela Nunes participated in study design, collected data, and reviewed the paper. Orestes Vicente Forlenza participated in study design, coordinated the study and revised the paper.
Acknowledgments
This work was supported by grants from FAPESP process no. 2007/00110-0, and from the Associação Beneficiente Alzira Denise Hertzog (ABADHS).
