Introduction
As new treatments for early Alzheimer’s disease (AD) emerge, it is critical to focus efforts on screening in primary care settings, where the majority of older adults receive medical care. This requires tools to identify early AD and to distinguish it from other kinds of dementias. Latinos are currently the fastest growing segment of older adults in the United States and are projected to have a six-fold increase in AD and related dementias by 2050 (Alzheimer Association, 2004). They are at high risk for AD, VaD, and AD/VaD due to low education and socio-economic status and the aggregation of vascular risk factors including diabetes, hypertension, stroke, and heart disease in their communities (Haan et al., Reference Haan, Mungas, Gonzalez, Ortiz, Acharya and Jagust2003; Luchsinger et al., Reference Luchsinger, Reitz, Honig, Tang, Shea and Mayeux2005; Skoog et al., Reference Skoog, Nilsson, Persson, Lernfelt, Landahl, Palmertz and Svanborg1996). Access to diagnosis and treatment is hampered by language proficiency, personal beliefs, and economic status, delaying diagnosis by as many as 5 years (Ortiz & Fitten, Reference Ortiz and Fitten2000). This is especially worrisome since the onset of AD appears 6 to 8 years earlier in Latino compared with non-Latino patients (Clark et al., Reference Clark, DeCarli, Mungas, Chui, Higdon, Nunez and van Belle2005). Any cost-effective strategy for screening older adults for cognitive impairment must be both sensitive and specific in the educationally and ethnically diverse primary care population (Manly, Reference Manly2006; Mungas, Reference Mungas2006; Teresi, Stewart, Morales, & Stahl, Reference Teresi, Stewart, Morales and Stahl2006).
The picture version of the Free and Cued Selective Reminding Test with Immediate Recall (pFCSRT+IR)(Grober & Buschke, Reference Grober and Buschke1987) has much to recommend it as a candidate measure. Among English speakers, test performance, discriminative and predictive validity for dementia does not differ with race or education (Grober, Lipton, Katz, & Sliwinski, Reference Grober, Lipton, Katz and Sliwinski1998; Grober, Sanders, Hall, & Lipton, 2010; Ivnik, Smith, & Lucas, Reference Ivnik, Smith and Lucas1997) making it a useful clinical tool in ethnically and educationally diverse cohorts. Unlike most episodic memory tests, pFCSRT+IR begins with a study phase in which patients identify and name pictured objects (e.g., grapes, desk) in response to category cues (fruit, furniture) which are used in the test phase to prompt recall of items not free recalled. In African American and Caucasian primary care patients, free recall identified very mild dementia with high sensitivity and specificity that did not differ by race or education (Grober, Sanders, Hall, & Lipton, 2010). The pFCSRT+IR has been widely used to detect amnestic mild cognitive impairment (aMCI) and dementia, (Grober, Buschke, Crystal, Bang, & Dresner, Reference Grober, Buschke, Crystal, Bang and Dresner1988; Katz et al., Reference Katz, Lipton, Hall, Zimmerman, Sanders, Verghese and Derby2012; Petersen, Smith, Ivnik, Kokmen, & Tangalos, Reference Petersen, Smith, Ivnik, Kokmen and Tangalos1994), predict future dementia and Alzheimer’s Disease (AD) (Derby et al., Reference Derby, Burns, Wang, Katz, Zimmerman, L’Italien and Lipton2013; Grober, Lipton, Hall, Crystal, Reference Grober, Lipton, Hall and Crystal2000; Sarazin et al., Reference Sarazin, Berr, De Rotrou, Fabrigoule, Pasquier, Legrain and Dubois2007), and distinguish AD dementias from nonAD dementias (Grober, Hall, Sanders, & Lipton, Reference Grober, Hall, Sanders and Lipton2008; Pillon, Deweer, Agid, & Dubois, Reference Pillon, Deweer, Agid and Dubois1993; Traykov et al., Reference Traykov, Baudic, Raoux, Latour, Rieu, Smagghe and Rigaud2005). Its use in both in clinical practice and in clinical trials has increased recently due, in part, to accumulating data demonstrating its association with CSF biomarkers (Rami et al., Reference Rami, Fortea, Bosch, Sole-Padulles, Llado, Iranzo and Molinuevo2011; Wagner et al., Reference Wagner, Wolf, Reischies, Daerr, Wolfsgruber, Jessen and Wiltfang2012), neuroimaging findings (Koric et al., Reference Koric, Ranjeva, Felician, Guye, de Anna, Soulier and Ceccaldi2013; Petersen et al., Reference Petersen, Jack, Xu, Waring, O’Brien, Smith and Kokmen2000; Sarazin et al., Reference Sarazin, Chauvire, Gerardin, Colliot, Kinkingnehun, De Souza and Dubois2010; Zimmerman et al., Reference Zimmerman, Pan, Hetherington, Katz, Verghese, Buschke and Lipton2008), functional imaging (Diamond et al., Reference Diamond, Miller, Dickerson, Atri, DePeau, Fenstermacher and Sperling2007; Lekeu et al., Reference Lekeu, Van der Linden, Chicherio, Collette, Degueldre, Franck and Salmon2003; McLaren et al., Reference McLaren, Sreenivasan, Diamond, Mitchell, Van Dijk, Deluca and Atri2012), and autopsy-markers of AD (Grober et al., Reference Grober, Dickson, Sliwinski, Buschke, Katz, Crystal and Lipton1999). Performance on the test defines the core clinical phenotype in the revised criteria for prodromal AD (Dubois et al., Reference Dubois, Feldman, Jacova, DeKosky, Barberger-Gateau, Cummings and Scheltens2007, Reference Dubois, Feldman, Jacova, Cummings, Dekosky, Barberger-Gateau and Scheltens2010).
The FCSRT+IR has been administered in several languages including French (Auriacombe et al., Reference Auriacombe, Helmer, Amieva, Berr, Dubois and Dartigues2010; de Souza et al., Reference de Souza, Chupin, Lamari, Jardel, Leclercq, Colliot and Sarazin2012; Mahieux et al., Reference Mahieux, Onen, Berr, Volteau, Habert and Legrain2009; Sarazin et al, Reference Sarazin, Berr, De Rotrou, Fabrigoule, Pasquier, Legrain and Dubois2007; Tounsi et al., Reference Tounsi, Deweer, Ergis, Van der Linden, Pillon, Michon and Dubois1999), German (Lindenberger & Reischies, Reference Lindenberger and Reischies1999; Wagner et al., Reference Wagner, Wolf, Reischies, Daerr, Wolfsgruber, Jessen and Wiltfang2012), Portuguese (Lemos et al., 2014), and Spanish (Pena-Casanova et al., Reference Pena-Casanova, Gramunt-Fombuena, Quinones-Ubeda, Sanchez-Benavides, Aguilar and Badenes2009) as a screening test for very mild dementia and the states that precede it. In this study, we assessed validity of the Spanish language version of the pFCSRT+IR in identifying very mild dementia in Latino patients in a primary care setting. The Mini Mental State Exam (MMSE: Folstein, Folstein, & McHugh, Reference Folstein, Folstein and McHugh1975) was selected as a comparative screening test because of its wide, validated forms in English and Spanish and well known operating characteristics (Teresi et al., Reference Teresi, Golden, Cross, Gurland, Kleinman and Wilder1995). We hypothesized that the pFCSRT+IR would be more sensitive and specific than the MMSE for dementia screening.
Methods
Setting
The Memory Screening Project was carried out in the Adult Primary Care Clinic of the Jacobi Medical Center that serves a diverse patient population in the Bronx, NY. The project screened over 400 Latino and 300 non-Latino patients. Inclusion criteria were age of 65 or older and fluency in English or Spanish. Any patient with a medical diagnosis of dementia at the time of screening was excluded. A 20-min screening battery that included the pFCSRT+IR was coordinated with each patient’s regularly scheduled clinic appointment. An experienced Latina bilingual examiner approached eligible patients at their scheduled appointment, recruited interested patients, and tested them at their convenience, before or after seeing their physician. One year after their screening visit, a subgroup of patients underwent a diagnostic battery that included neuropsychological tests and interviews with a family member or friend. The selection of these patients was based on the availability of the examiner at the time of their appointment with their primary care physician and if they were willing to take the time for the diagnostic assessment. The MMSE was administered at this visit. The study was approved by the Institutional Review Boards at the Albert Einstein College of Medicine and Jacobi Medical Center
Clinical Diagnosis
Dementia diagnosis was based on standardized criteria from the DSM-IV and required impairment in memory plus at least one additional cognitive domain, accompanied by functional decline (American Psychiatric Association, 1994). The cognitive and functional status of each participant was established by consensus of a neuropsychologist (E.G.) and a geriatrician (A.E.), using scores from the diagnostic test battery (Table 1).
Diagnostic battery
Note. CERAD=Consortium to Establish a Registry in Alzheimer’s Disease; CDR=Clinical Dementia Rating; ADCS=Alzheimer’s Disease Cooperative Study.
A report was generated for each patient containing the test scores and percentiles for the neuropsychological tests in the diagnostic battery based on the performance of clinic patients without dementia. Confirmation of memory and cognitive impairment and presence of functional decline were determined by informant responses to a structured clinical interview covering six domains of cognitive and daily functioning (Morris, Reference Morris1993) augmented by their responses to the Alzheimer’s Disease Cooperative Study ADL scale (Galasko et al., Reference Galasko, Bennett, Sano, Marson, Kaye and Edland2006; Sano et al., Reference Sano, Egelko, Jin, Cummings, Clark and Pawluczyk2006). Raters reviewed the report, made an independent determination of the patient’s diagnosis, and then rated the patient’s cognitive performance and activities of daily living using the CDR scale (Morris, Reference Morris1993). Diagnosis (no dementia, dementia) of any patient for whom the raters disagreed on DSM IV criteria or CDR box scores was resolved at consensus conferences. Diagnoses were made without knowledge of FCSRT+IR or MMSE scores.
pFCSRT+IR
The pFCSRT+IR (Grober & Buschke, Reference Grober and Buschke1987) begins with a study phase in which participants search a card containing four pictures (e.g., grapes) for an item that goes with a unique category cue (e.g., fruit). After all four items are identified, the card is removed and immediate cued recall of the four items is tested. The search and identification procedure is continued for the next group of four items until all 16 items have been identified and retrieved in immediate recall. There are three trials of recall, each consisting of free recall followed by cued recall for items not retrieved by free recall for a maximum score of 48. Items not retrieved by cued recall are re-presented as reminders. Each separate trial is followed by 20 s of interference. Total recall is the sum of free and cued recall. A Spanish version for each of the three-pictured lists was constructed from the English lists using standard back-translation methods. Separately, each pictured item was shown to 15 older Spanish speakers whose naming of them confirmed the translations. The dependent measure was the sum of free recall across the three test trials.
MMSE
The MMSE was introduced more than 35 years ago to identify cognitive impairment by assessing mental status and includes questions on memory, orientation, language, and attention, for a total of 30 points (Folstein et al., Reference Folstein, Folstein and McHugh1975). The Spanish and English MMSE version used here is described in detail elsewhere (Gurland, Wilder, Cross, Teresi, & Barrett, Reference Gurland, Wilder, Cross, Teresi and Barrett1992). The score includes world (mundo) backward instead of serial subtractions because of the lower refusal rates. The MMSE was administered at the diagnostic assessment.
Statistical Procedures
Characteristics at the screening visit were reported using descriptive statistics. To compare means between those with and without dementia and determine effect sizes, univariate analysis of variance was used. Chi square test was used for categorical variables. The correlation between education and test scores was assessed with Pearson’s correlation coefficient. Receiver operating characteristic (ROC) curves were generated for MMSE scores and free recall sum to visualize differences in their sensitivity and specificity across the full range of cut scores. The diagnostic accuracy of the tests was compared by examining the area under the ROC curve (AUC). Since this method may be insensitive to differences at high sensitivity and specificity, another approach is to hold specificity (sensitivity) constant at a clinically relevant value and then compare sensitivities (specificities) across tests using the Pearson’s Chi Square test. Youden’s index, the sum of sensitivity and specificity minus one, was used to select the optimal cut score for each test (Youden, Reference Youden1950).
Results
The sample included 112 Latino patients with no prior diagnosis of dementia who were tested in Spanish. Of these, 24 (21%) met DSM IV criteria for dementia as determined by the investigators using the case conference process described above. The Case Conference clinical team had no knowledge of the MMSE or pFCSRT+IR scores. Severity of dementia was very mild (CDR 0.5) for 15 patients and mild (CDR 1.0) for 9 patients. Of the 88 patients determined not to have dementia by diagnostic assessment, 50 had no cognitive impairment (CDR 0) and 38 had very mild impairment (CDR 0.5). Table 2 shows the demographic and performance scores for the 112 patients by dementia status.
Baseline demographic characteristics and cognitive performance by dementia status at the diagnostic evaluation in Latinos not previously diagnosed
aPuerto Rican and Dominican
bShort Acculturation Scale (Marin et al, Reference Marin, Sabogal and Vanoss1987)
cSum of animal, fruit, and vegetable
CERAD: Consortium to Establish a Registry in Alzheimer’s Disease
CDR: Clinical Dementia Rating Scale
pFCSRT+IR: Picture version of the Free and Cued Selective Reminding Test with Immediate Recall
MMSE: Mini Mental Status Exam
GDS: Geriatric Depression Scale
Patients with dementia were older than patients without dementia. Most patients (85%) were born in Puerto Rico or the Dominican Republic. Time to the diagnostic evaluation did not differ between the groups nor did they not differ in years of education or the number of depressive symptoms they endorsed.
Years of education did not influence free recall performance across all patients (r=.04; p=.66) or in patients with and without dementia (r=.22, p=.30, and r=.02; p=.86). MMSE scores were related to years of education in the whole sample (r=.25; p=.01) and in patients without dementia (r=.33; p=.003) but not in demented patients (r=.03, p=.89).
Figure 1 displays the ROC curves for the MMSE and the pFCSRT+IR across the full range of cuts scores. The area under the curve was higher for the pFCSRT+IR than the MMSE (86% vs. 72%; p<.026). Classification accuracy also differed at clinically relevant values. Equating the specificity of both tests to 81%, which required a cut score of ≤27 for free recall and ≤24 for MMSE, sensitivity to dementia was 70% for free recall and 48% for the MMSE (χ2=4.54; p=.033). Equating the sensitivity of both tests to 74%, which required a cut score of ≤28 for free recall and ≤26 for MMSE, specificity was 75% for free recall and 62% for the MMSE (χ2=4.07; p=.044).
The ROC curves for the MMSE and the pFCSRT+IR across the full range of cuts scores.
The optimal cut score according to Yoden’s index was ≤27 for free recall (70%sen/81%spec). Patients who scored ≤27 were 10 times more likely to have dementia than patients who scored above 27. The optimal cut score was ≤25 for the MMSE (61%sen/75%spec). Patients who scored ≤25 were 4.5 times more likely to have dementia than patients who scored above 25. The pFCSRT+IR outperformed the MMSE even though the MMSE was administered at the diagnostic assessment, 1 year after the pFCSRT+IR.
Discussion
The purpose of this study was to assess the discriminative validity of the Spanish language versions of pFCSRT+IR and the MMSE for unrecognized dementia in a primary care sample of older Latino adults. This was accomplished by testing Latino patients from an urban primary care clinic in the Bronx with both the pFCSRT+IR and the MMSE at their routinely scheduled out-patient visits. A total of 112 patients received a research diagnostic assessment for dementia used to assign a “gold standard” diagnosis of dementia through a case conference process by raters blind to pFCSRT+IR and MMSE scores. Among older Latino adults free of medically diagnosed dementia, 21% received a research diagnosis of dementia. Thus, among Latino adults there is a substantial burden of unrecognized dementia. This finding is consistent with previous reports in both Latino and non-Latino populations (Callahan, Hendrie, & Tierney, Reference Callahan, Hendrie and Tierney1995; Harris et al., Reference Harris, Ortiz, Adler, Yu, Maines, Barba and Vickrey2011; Knopman, Donohue, & Gutterman, Reference Knopman, Donohue and Gutterman2000).
Using the case conference diagnosis as the gold standard, accuracy of screening with the pFCSRT+IR and the MMSE was assessed across a range of cut scores. AUC was higher for free recall than for MMSE score. Free recall outperformed the MMSE in sensitivity (0.70 vs. 0.47) when specificity was set to.81 and specificity (0.75 vs. .62) when sensitivity was set at .74. For free recall, classification accuracy was highest for a cut score of 27 (70%sen, 81%spec). Patients who scored ≤27 were 10 times more likely to have dementia than patients who scored above 27. For the MMSE, classification accuracy was optimized at a cut score of 25 (61%sen,75%spec). Patients who scored ≤25 were 4.5 times more likely to have dementia than patients who scored above 25. MMSE scores were associated with education in patients without dementia whereas free recall performance was not.
Our results in a different primary care practice in the Bronx using the same protocol provide a useful comparison. The cohort was predominantly female and consisted of non-Latino patients evenly divided between Caucasians and African Americans who were tested with the English language version of the pFCSRT+IR (Grober, Sanders, Hall, & Lipton, Reference Grober, Sanders, Hall and Lipton2010). The cohort was 6 years older with 4 more years of education than the current cohort. Fifty-six patients had prevalent dementia and 262 were dementia free at baseline. Though mean free recall scores were similar for dementia free patients (30.1 vs. 31.6) and patients with dementia (19.1 vs. 20.2), cut scores differed. In the present study, the optimal cut score in Spanish was ≤27. In the previous study, the optimal cut score in English was ≤25. This comparison suggests that different cut scores may be needed in English and Spanish speaking primary care patients in the Bronx to achieve optimal levels of classification accuracy.
We found that the optimal cut score for dementia on the MMSE in the present sample was ≤25. In prior studies, the recommended cut score for identifying dementia was 23/24 for Spanish speakers (Mejia, Gutierrez, Villa, & Ostrosky-Solis, Reference Mejia, Gutierrez, Villa and Ostrosky-Solis2004; Ostrosky-Solis, Lopez-Arango, & Ardila, Reference Ostrosky-Solis, Lopez-Arango and Ardila2000;). It yielded a high sensitivity and specificity (83%/85%) for participants with 5 to 9 years of education. Applying the same cut score in the current sample yielded high specificity (88%) but low sensitivity (48%). Since the cohorts were comparable in age and educational level, the difference in sensitivity is likely due to the fact that dementia was much more severe in those studies and hence more easily distinguished from patients who had no cognitive impairment.
These comparisons demonstrate that results may differ by language of test administration, education and other aspects of the sample being tested. Emerging evidence suggests that there are differences in cognitive test performance among Latino subgroups based on country or region of origin (González, Tarraf, Gouskova, & Mosley, Reference González, Tarraf, Gouskova and Mosley2014). The present study is based on a relatively small number of patients largely of Caribbean descent. As Latinos are heterogeneous, it is not clear that either a single translation or a single set of cut scores on the pFCSRT+IR will be optimal. In this study, the examiners were native speakers and culturally congruent with the patient population, a feature that may have significant impact on the pFCSRT+IR’s operating characteristics.
Although the pFCSRT+IR outperforms the MMSE, the test can take 10 to15 minutes to administer which may be too long for many primary care settings. To optimize accuracy (sensitivity and specificity) and efficiency, we developed a two stage approach in which all patients aged 65 and older undergo a brief, high-sensitive dementia screen in the first stage and only the patients who fail undergo the more time consuming second stage testing using the pFCSRT+IR to diagnose memory impairment (Grober, Hall, McGinn, et al, Reference Grober, Hall, McGinn, Nicholls, Stanford, Ehrilich and Lipton2008; Grober, Hall, Lipton, Reference Grober, Hall, Sanders and Lipton2008). Patients screen positive for dementia when they fail both stages. This procedure, known as the Alzheimer’s Disease Screen for Primary Care (ADS-PC), has been validated in English but requires further study in Spanish.
There are several barriers to primary care based screening programs of older adults for dementia and the high-risk states which precede it. Those barriers include the absence of effective tools for screening, the absence of resources for screening and follow-up, the lack of available treatments, and the need for evidence that the aggregate benefits of treatment outweigh the costs (Boustani, Peterson, Hanson, Harris, & Lohr, Reference Boustani, Peterson, Hanson, Harris and Lohr2003; Brayne, Fox, & Boustani, Reference Brayne, Fox and Boustani2007; Lin, O’Connor, Rossom, Perdue, & Eckstrom, Reference Lin, O’Connor, Rossom, Perdue and Eckstrom2013; Petersen et al., Reference Petersen, Stevens, Ganguli, Tangalos, Cummings and DeKosky2001). We suggest that the pFCSRT+IR may be an effective tool for primary care screening. It identifies very mild dementia with high specificity in Latino, African American and Caucasian patients. It performs well in patients with both high and low education. The pFCSRT+IR could serve as an initial screen to select patients at high risk for more expensive, potentially invasive, biologically-based tests.
Acknowledgments
This work was supported by the National Institutes of Health (R21 AG036935) to PI: Ellen Grober and (NIA-AG03949) to PI: Richard Lipton. This study was presented in part at the Alzheimer Association’s International Conference in Boston, July, 2013. The FCSRT+IR is copyrighted by the Albert Einstein College of Medicine and is made freely available for non-commercial purposes. Ellen Grober receives a small percentage of any royalties on the FCSRT+IR when it is used for commercial purposes.
