Introduction
Several authors have encouraged the development of short cognitive instruments to avoid the use of comprehensive neuropsychological batteries for preliminary screening of patients potentially affected by dementia (Tombaugh and McIntyre, Reference Tombaugh and McIntyre1992; Feher et al., Reference Feher, Mahurin, Doody, Cooke, Sims and Pirozzolo1992). Although several shortened tests have been developed for diverse populations, tests developed for specific communities perform poorly in populations with a different cultural or linguistic background (García-Caballero et al., Reference García-Caballero2006).
Quantitative evaluations of aphasia are usually performed by using graded naming tests (Weintraub, Reference Weintraub and Mesulam2000). One of these instruments is the well-known Boston Naming Test (BNT) (Kaplan et al., Reference Kaplan, Goodglass and Weintraub1978), a 60-item battery originally devised for assessing aphasia, but widely used for evaluating anomia in several dementias (Goodglass et al. Reference Goodglass, Wingfeld and Hyde1998), and as a useful method to assess the evolution of the disease (Williams et al., Reference Williams, Mack and Henderson1989). A first 60-item Spanish version of the BNT was proposed as a translation into Spanish from the modified version published by Goodglass and Kaplan (Reference Goodglass and Kaplan1986). A second modified version was proposed as a new translation to Spanish from Goodglass (Reference Goodglass2005). This second version has replaced the first in most settings. Hereafter we will refer to these 60-item versions as BNT-1 and BNT-2, respectively.
Several reports have shown that the BNT efficiently discriminates healthy elderly from those with dementia (Martin and Fedio, Reference Martin and Fedio1983; LaBarge et al., Reference LaBarge, Edwards and Knesevish1986; Lansing et al., Reference Lansing, Ivnik, Cullum and Randolph1999; Welch et al., Reference Welch, Doineau, Johnson and King1996). Nevertheless, 60-item versions of the BNT require 15–30 minutes, time that is beyond the scope of a routine screening for dementia. In order to improve the efficiency of the test, shorter versions of the BNT have been the aim of different researchers. Williams et al. (Reference Williams, Mack and Henderson1989) demonstrated that the BNT could be split in half (30 items each) with similar performance as the complete version. Later, Mack et al. (Reference Mack, Freed, Williams and Henderson1992) observed similar performance for the four 15-item quarters of the BNT. However, the authors concluded that a reliable shortened version should be specially designed and validated for each target population. Later, Lansing et al. (Reference Lansing, Ivnik, Cullum and Randolph1999) demonstrated the discriminating ability of the four versions proposed by Mack et al. (Reference Mack, Freed, Williams and Henderson1992).
These short versions of BNT were devised for English-speaking communities, and therefore several items were not appropriate for populations with a different cultural background. Moreover, cultural and lexical characteristics are specific for each region or for specific groups like illiterates, bilinguals, and people living in rural environments or populations with high rates of migration. Following this rationale, Serrano et al. (Reference Serrano, Allegri, Drake, Butman, Harris, Nagle and Ranalli2001) developed a shortened version (12 items) adapted to Argentina by choosing items that specifically distinguished patients with dementia from those without dementia in their target area. Later, Calero et al. (Reference Calero, Arnedo, Navarro, Ruíz-Pedrosa and Carnero2002) tested the shortened version proposed by Mack et al. (Reference Mack, Freed, Williams and Henderson1992) in southern Spain with reasonable success.
The hypothesis we tested in this study was that a specially designed shortened version of the BNT could replace the original 60-item BNT as part of a battery for initial screening for dementia. The study objectives were therefore to design a shortened version for a rural population of low educational level in Galicia (Spain) and to compare its performance with other short versions devised for different Spanish-speaking populations in Argentina (Serrano et al., Reference Serrano, Allegri, Drake, Butman, Harris, Nagle and Ranalli2001) and Andalusia (Spain) (Calero et al., Reference Calero, Arnedo, Navarro, Ruíz-Pedrosa and Carnero2002).
Methods
A clinic group of 102 consecutive patients referred for cognitive evaluation to a psychogeriatric outpatient unit was recruited along with 78 healthy volunteers (Table 1). The clinic group included 43 patients affected by dementia following DSM-IVR criteria (American Psychiatric Association, 2000) and 59 patients who received other diagnoses, including mood disorders, anxiety disorders, and mild cognitive impairment not fulfilling DSM-IVR criteria for dementia.
Number of individuals in the sample of elderly rural population from Ourense (Spain) used for this research
Exclusion criteria were illiteracy, severe loss of visual acuity, severe hypoacusia, previous history of traumatic brain injury and active alcohol or drug dependence. Controls were excluded when MMSE score was less than 24. The clinic and control groups were assessed with the Spanish MMSE and BNT. In addition, the clinic group was examined using standard neuropsychological instruments. Furthermore, information was obtained from the relatives or people in charge regarding activities of daily living with several cognitive tasks. According to maternal language, the 180 individuals were divided into Galician speakers (114), Spanish speakers (34) and bilinguals (32). Bilingualism was assessed following a modification of the questionnaire of bilingualism included in the Spanish-Galician version of the Bilingual Aphasia Battery (BAT) (Paradis and Elias, Reference Paradis and Elias1989).
Both clinic and control group were studied using the following tests:
1. Spanish version of the Mini-mental State Examination (Spanish MMSE). This test was originally proposed by Folstein et al. (Reference Folstein, Folstein and McHugh1975) and adapted into Spanish by Lobo et al. (Reference Lobo1999).
2. Boston Naming Test (BNT). We used the first Spanish version of the BNT published by Goodglass and Kaplan (Reference Goodglass and Kaplan1986) and translated into Spanish, hereinafter called BNT-1, wherein 12 items from the former English version of the BNT had been altered. In addition, we used another Spanish version, the BNT-2, also translated into Spanish from the modification proposed by Goodglass (Reference Goodglass2005) in which seven items from the English version were replaced by seven items in the Spanish translation (Mushroom was replaced by “Zanahoria” (carrot), Pretzel by “Magdalena” (muffin), Wreath by “Corona” (crown), Knocker by “Chupete” (pacifier), Noose by “Aguja” (needle), Latch by “Cerradura” (locker) and Trellis by “Regadera” (watering-can). Altogether, the BNT-2 differed from the BNT-1 on five items; there were therefore 55 items common to both BNT versions and five items specific to each version. Consequently, participants were shown a battery of 65 items. Each subject was asked to identify the picture in around 20 seconds and the answer was considered positive if the name was correct. If the answer was incorrect owing to a misperception or a wrong identification, a previously established semantic cue was provided with an additional 20-second period to identify the object. If, after the semantic cue, the subject did not recognize the object or gave a wrong name, a phonetic cue was provided and the answer was registered for other studies. Following the procedure recommended in the Spanish translation of Goodglass (Reference Goodglass2005), for the purpose of this research the answer was considered positive if the person identified the item either spontaneously or after the semantic cue.
Specificity and sensitivity for all BNT items were calculated using SPSS software Version 17. Those items with specificity and sensitivity above 0.5 were selected to create a shortened battery of 11 pictures named BNTOu11. Analyses of variance were performed for the values of the Spanish MMSE and all BNT versions by using the GLM procedure of the Statistical Analysis Software (SAS Institute, 2007). The ANOVA included as fixed factors gender, language, education and diagnosis, along with their interactions. Means were compared by using least square means (Steel et al. Reference Steel, Torrie and Dickey1997). Pearson's correlations among scores of Spanish MMSE and versions of BNT were calculated with SAS Institute (2007). Receiver operating characteristics (ROC) curves, including their correspondent areas under the curve (AUC), and standard errors were calculated using SPSS for the Spanish MMSE, BNT-1, BNT-2, and the shortened versions BNT12 (Serrano et al., Reference Serrano, Allegri, Drake, Butman, Harris, Nagle and Ranalli2001), BNT15 (Calero et al., Reference Calero, Arnedo, Navarro, Ruíz-Pedrosa and Carnero2002) and BNTOu11. Curves were compared with confidence intervals, following the percentile bootstrap estimation method. Finally, internal consistency of the different BNT versions was measured by means of the Cronbach's α coefficient by using the SPSS program.
Results
The scores for BNT-1 and BNT-2 followed similar distributions (data not shown). The ANOVA and the subsequent mean comparisons clearly differentiated patients with dementia from patients without dementia and healthy individuals. As expected, patients with dementia had lower scores, while patients without dementia and the control group were not significantly different (Table 2). Differences between genders were significant for scores on BNT-2 and two of the abbreviated BNT versions (BNT12 and BNTOu11). Differences among language groups were significant for the Spanish MMSE, BNT-2 and the abbreviated version BNT12. Males scored significantly higher than females for all tests, although differences were not significant for half of them. Regarding language, Spanish speakers obtained higher scores than Galician monolinguals; bilinguals were not significantly different from the other monolingual groups except for BNT-2. None of the interactions was significant (data not shown).
Mean comparison of the whole sample of elderly from the rural population of Ourense (northwestern Spain)
Means followed by the same letter are not significantly different from the other means of the same category, according to the least square means at P = 0.05.
The items of the BNT with specificity and sensitivity above 0.5 were selected to create a shortened battery of 11 pictures named BNTOu11 (Table 3). Those items were: “Volcán” (Volcano), “Globo” (Globe), “Armónica” (Harmonica), “Bellota” (Acorn), “Zancos” (Stilts), “Hamaca” (Hammock), “Bozal” (Muzzle), “Compás” (Compass), “Pinzas” (Tongs), “Yugo” (Yoke), and “Pez Espada” (Swordfish). Among these 11 items, four were common with BNT12 (the Argentinian short version) and two with the BNT15 (the Andalusian version). These two items, Stilts and Muzzle, were common to the three abbreviated versions of BNT used with Spanish-speaking populations.
Sensitivity and specificity for the shortened versions of the Boston Naming Test from Argentina (underlined), Andalusia (italics) and BNTOu11 (bold)
The long BNT versions and the shortened ones had similar correlation coefficients to the Spanish MMSE, with values from 0.55 to 0.60. BNT-1 and BNT-2 were highly correlated (r2 = 0.99), and the shortened versions had high positive and similar correlation coefficients to the long versions and between themselves with values from 0.83 and 0.92.
ROC curves for Spanish MMSE and 60-item and shortened BNT versions are shown in Figure 1. Areas under ROC curves (AUC) did not show statistically significant differences based on confidence intervals (Table 4); therefore, the AUC of BNTOu11 was comparable to that of the abbreviated and the 60-item versions of BNT and to the Spanish version of the MMSE. Cronbach's α for BNTOu11 was 0.784 (>0.7 is broadly accepted as satisfactory), while for the Andalusian version this was 0.723 and for the Argentinian version 0.747. On the other hand, Cronbach's α for BNT-1 and BNT-2 were 0.909 and 0.912, respectively. Cronbach's α is expected to be higher for tests with more items.
ROC curve of Spanish MMSE, the 60-item BNT2nd and two shortened versions of BNT.
Area under the ROC curve (AUC), standard error and confidence interval for five tests: MEC, both complete and the three shortened versions of BNT
**Significant at p = 0.01.
Discussion
In our sample, one third of the healthy individuals have BNT scores below the critical value of 36 points, and even though the control group had higher values than the dementia group, the BNT values of the control individuals were low (38 out of 60). These low scores can be explained by the low cultural level of the target population and by the different cultural background of our population, which clearly differ from the North American population for which the test was originally devised. The distribution of sensitivity and specificity showed that in our population many items in the BNT 60-item versions are easily recognized by most participants irrespectively of diagnosis, whereas others are never recognized either by cases or by controls. This finding encourages the development of specific shortened versions of the BNT, a similar rationale to that followed by Calero et al. (Reference Calero, Arnedo, Navarro, Ruíz-Pedrosa and Carnero2002) and Serrano et al. (Reference Serrano, Allegri, Drake, Butman, Harris, Nagle and Ranalli2001) in Andalusia and Argentina, respectively.
Gender differences on the Spanish MMSE or BNT scores have been reported previously (Lansing et al. Reference Lansing, Ivnik, Cullum and Randolph1999; Calero et al. Reference Calero, Arnedo, Navarro, Ruíz-Pedrosa and Carnero2002) and could be explained in our sample by the informal schooling of males due to migration (García-Caballero et al., Reference García-Caballero2006) or simply because of their exposure to wealthier environments during work outside the family farms. It is a well-known fact that black and white silhouette drawings, like the BNT's pictures, show cultural effects (Reis et al., Reference Reis, Petersson, Castro-Caldas and Ingvar1998; Reference Reis, Faísca, Ingvar and Petersson2006), a fact that could penalize women more than men. This effect should be taken into account in order to avoid a false diagnosis of anomia. The same rationale can be argued regarding the superior scores of bilinguals over monolinguals. Traditionally, Galician was spoken in everyday life, while Spanish was limited to formal contexts. During Franco's regime Galician was banned and therefore formal education in Galician was prohibited until more recent decades (García-Caballero et al., Reference García-Caballero2007). Even though Galician and Spanish show a high degree of shared vocabulary (e.g. cow is “vaca” and comb is “peine” in both Spanish and Galician), many of the items on BNT (especially the least frequent) are accessible only in Spanish, the “school language” until 30 years ago. The intriguing question of whether bilingualism can provide a protective effect against dementia, as suggested by Bialystok et al. (Reference Bialystok, Craik and Freedman2007) and Kavé et al. (Reference Kavé, Eyal, Shorek and Cohen-Mansfield2008), cannot be addressed with our current data.
The strategy for defining a specific shortened version based on independent cutting levels for sensitivity and specificity has been efficient because the BNTOu11 is shorter than the previous short versions and shows significant differences between the dementia and control groups. The discriminative power of the short versions is not significantly different. The three short versions share two items – Stilts and Muzzle – but the two versions specifically designed for Spanish-speaking populations share four, while BNT15, which was designed (randomly) for an English population, only shares two items, suggesting that short versions should be designed specifically for the target population.
The areas under ROC curves (AUC) did not show statistically significant differences (Table 4), indicating that AUC (0.814) was comparable to that of the BNT 60-item versions (0.785 and 0.779, respectively) and also to BNT12 (0.772) and BNT15 (0.799). Also, these values were not significantly different from the Spanish version of the MMSE (0.866). Concerning the internal consistency, the 60-item versions of BNT had the highest values of Cronbach's α, as expected, because the number of items is higher. Although Cronbach's α depends on the correlation among items and the number of items, among the shortened versions, the BNTOu11 has higher consistency but fewer items. Furthermore, when checking the Cronbach's α of the three short versions after removing one item at random, the α of BNTOu11 produced lower values than the original for any item, indicating that none of the items should be removed. By contrast, for the other versions, BNT15 and BNT12, α increased by removing one item. This result confirms that BNTOu11 has higher internal consistency than the other short versions.
This research has some limitations. Even though we excluded the control participants with MMSE scores below 24, the control group could still include individuals with mild cognitive impairment or very mild dementia because the controls were not as thoroughly evaluated as the patients. The classes of sex and language were not similar among patients and controls, which could cause some bias in the analyses. In addition, our new shortened version BNTOu11 should be tested in an independent sample before replacing the complete versions of BNT for screening purposes.
In conclusion, our results support the idea that shortened versions of the BNT are at least as useful as the 60-item versions for screening for language impairment in dementia, with the advantage of saving considerable time. Shortened versions have been created by removing those items that were irrelevant for anomia screening because many elders have never been exposed to them. Previous authors have demonstrated the convenience of replacing the 60-item versions of the BNT with shortened ones specifically chosen for different target populations (Lansing et al., Reference Lansing, Ivnik, Cullum and Randolph1999; Serrano et al., Reference Serrano, Allegri, Drake, Butman, Harris, Nagle and Ranalli2001; Calero et al., Reference Calero, Arnedo, Navarro, Ruíz-Pedrosa and Carnero2002). The proposed short version BNTOu11 deserves some credit and further research to demonstrate its value as part of a battery for language screening in dementia among the rural community in Galicia.
Conflict of interest
None.
Description of authors' roles
M. C. Nebreda obtained most of the data and carried out the main part of the analysis and writing. A. García-Caballero, director of the project; designed and supervised the project and corrected the paper. E. Asensio assisted with data recording other than the BNT. P. Revilla carried out part of the statistical analysis and cooperated in the discussion and redaction of the text. M. Rodriguez-Girondo designed and performed some of the statistical analyses. R. Mateos supervised the project and corrected the paper.
Acknowledgments
The authors thank everyone who helped to carry out this research. We also thank the editor and reviewers for their useful suggestions for improving the paper.
This research was approved by the Ethics Committee of the Regional Government of Galicia.
