INTRODUCTION
Societies are aging and living longer, and consequently there is an unprecedented and rapidly increasing number of people with dementia, especially in low- and middle-income countries (LMICs). According to the World Alzheimer Report (Prince, Comas-Herrera, Knapp, Guerchet, & Karagiannidou, Reference Prince, Comas-Herrera, Knapp, Guerchet and Karagiannidou2016), 58% of all people with dementia live in LMICs and the proportion will increase to 63% in 2030 and 68% in 2050. Developing countries differ widely from developed countries in their profile of dementia with respect to demographic characteristics, risk and protective factor profiles, underlying aetiologies, genetic factors, and variability in care settings (Alladi & Hachinski, Reference Alladi and Hachinski2018).
Therefore, there is a compelling need to study dementia in LMICs, to identify effective preventive and management strategies to reduce the burden of dementia globally.
One of the main barriers to studying dementia in LMICs is the low rates of diagnosis (Alzheimer’s & Related Disorders Society of India, Reference Shaji, Jotheeswaran, Girish, Bharath, Dias, Pattabiraman and Varghese2010). Early diagnosis is crucial to institute effective therapeutic strategies and to delay progression. Lack of awareness about dementia, limited availability of appropriate diagnostic tools, and scarcity of skilled personnel specialised in dementia diagnosis contribute to this barrier to diagnosis in LMICs. Diagnosis of dementia and its precursor state of mild cognitive impairment (MCI) relies on the availability of neuropsychological tests, behavioural and functional measures that are culturally appropriate and validated for local populations. Several challenges arise in the use of neuropsychological tests for diagnosis of cognitive impairment in developing countries like India: firstly, most tests have been developed for educated, predominantly English-speaking Western populations and are not suitable for use in other languages and cultures (Ardila, Reference Ardila1995). Secondly, illiteracy remains widely prevalent in developing countries and complicates the adaptation of diagnostic tests.
It is recognised that 122 major languages and 1599 other languages are spoken in India. The 2011 census recorded Hindi as the mostly widely spoken language with 57.1% of the Indian population speaking Hindi; 10.6% speaking English, followed by Bengali (8.9%) and Telugu (7.8%), and 4.9% and 2.9% speaking Kannada and Malayalam languages, respectively (Census of India, 2011). Further, the Indian diaspora population is the largest in the world due to wide-scale international migration (United Nations Department of Economic and Social Affairs Population Division, 2017) and has resulted in a rising population of elderly Indians outside India who continue to speak Indian languages as their native language/tongue. These socio-demographic and cultural differences need to be accounted for while testing cognition in diverse populations. This will require culturally, educationally, and linguistically appropriate neuropsychological tests. Achieving this is a complex process and will entail different approaches such as translating existing standardised cognitive tests, adapting tests for cultural relevance, or developing innovative tests.
These demographic and socio-cultural factors also influence performance on neuropsychological tests. Apart from the recognised effect of age, educational status and gender are known to affect test performance (Ardila, Rosselli, & Ostrosky-Solis, Reference Ardila, Rosselli and Ostrosky-Solis1992; Ganguli et al., Reference Ganguli, Ratcliff, Huff, Belle, Kancel, Fischer, Seaberg and Kuller1991). There is also likely to be a language effect on cognitive test performance (Mungas, Reed, Marshall, & González, Reference Mungas, Reed, Marshall and González2000); however, this has not been as widely explored. Normative data and cut-off scores should therefore be available for tests across cognitive domains in various languages and for different levels of educational attainment for diagnosis of dementia in non-English-speaking communities. Another factor that may need to be considered is multilingualism, which is the norm in India due to its linguistic diversity. Cognitive effects of multi/bilingualism have been shown: bilinguals exhibit an advantage to executive functions with a concomitant disadvantage to fluency and naming (Bialystok, Reference Bialystok2010; Bialystok, Craik, & Ryan, Reference Bialystok, Craik and Ryan2006; Gollan, Montoya, Fennema-Notestine, & Morris, Reference Gollan, Montoya, Fennema-Notestine and Morris2005; Rosselli et al., Reference Rosselli, Ardila, Araujo, Weekes, Caracciolo, Padilla and Ostrosky-Solis2000; Singh & Mishra, Reference Singh and Mishra2013). It remains to be explored whether these differences lead to significant variability in test performance that necessitates different cut-offs for bilinguals and monolinguals.
Diagnostic criteria for dementia also require the demonstration of decline in functional activities. This can be quite different for elderly people living in LMICs that have varying levels of socio-economic status, family structure, and lifestyles of the elderly. Diagnosing dementia in regions characterised by linguistic, educational, and socio-cultural heterogeneity is, therefore, challenging.
In addition to developing and validating culturally appropriate tests to diagnose dementia in developing countries, there is a need to standardise these tests across different clinical cohorts, to ensure uniformity in diagnosis. Results generated in cohorts in under-studied LMICs will need to be compared with available results from developed countries, to make genuine comparisons between disease profiles possible. This will require the use of diagnostic tests that can potentially be harmonised with ongoing and future global studies. There have been a few efforts towards developing comprehensive neuropsychological test batteries for use in different languages and countries. The 10/66 global dementia studies (Prince et al., Reference Prince, Acosta, Acosta, Guerra, Huang, Jotheeswaran, Jimenez-Velazquez, Liu, Llibre Rodriguez, Salas, Sosa, Williams and Prince2016; Sosa et al., Reference Sosa, Albanese, Stephan, Dewey, Acosta, Ferri, Guerra, Huang, Jacob, Jiménez-Velázquez, Rodriguez, Salas, Williams, Acosta, González-Viruet, Hernandez, Shuran, Prince and Stewart2012) validated a cognitive test battery for use across seven LMICs (Cuba, Dominican Republic, Venezuela, Peru, Mexico, China, and India). The Spanish English Neuropsychological Assessment Scales (SENAS) (Mungas, Reed, Crane, Haan, & González, Reference Mungas, Reed, Crane, Haan and González2004) provide psychometrically matched cognitive tests that can be used in multilingual and multiethnic Spanish- and English-speaking populations. The international harmonisation standards proposed by National Institute of Neurological Disorders and Stroke (NINDS) and the Canadian Stroke Network (CSN) (Hachinski et al., Reference Hachinski, Iadecola, Petersen, Breteler, Nyenhuis, Black, Powers, DeCarli, Merino, Kalaria, Vinters, Holtzman, Rosenberg, Wallin, Dichgans, Marler and Leblanc2006) have been found to be feasible for use to diagnose vascular cognitive impairment (VCI) in several countries, including China and Korea (Je Cho, Kyu Kim, & Suh, Reference Je Cho, Kyu Kim and Suh2003; Zhang et al., Reference Zhang, Zahner, Román, Liu, Wu, Hong, Hong, Tang, Zhou, Qu, Zhang and Li2006). The Consortium to Establish a Registry for Alzheimer’s Disease (CERAD; Fillenbaum et al., Reference Fillenbaum, van Belle, Morris, Mohs, Mirra, Davis, Tariot, Silverman, Clark, Welsh-Bohmer and Heyman2008) developed standardised and validated clinical, neuropsychological, and neuropathologic measures for the assessment of Alzheimer’s disease (AD) at multiple centres, and the battery has been translated into many languages. More recently, the National Alzheimer’s Coordinating Centres (NACC) developed an updated version of Uniform Data Set that consists of standardised neuropsychological, clinical, and other instruments for data collection across multiple research institutions (Besser et al., Reference Besser, Kukull, Knopman, Chui, Galasko, Weintraub, Jicha, Carlsson, Burns, Quinn, Sweet, Rascovsky, Teylan, Beekly, Thomas, Bollenbeck, Monsell, Mock, Zhou, Thomas, Robichaud, Dean, Hubbard, Jacka, Schwabe-Fry, Wu, Phelps and Morris2018). With regard to cognitive screening instruments, several tests, including the Mini Mental State Examination (MMSE), Addenbrooke’s Cognitive Examination (ACE), Montreal Cognitive Assessment (MoCA) (Davis et al., Reference Davis, Creavin, Yip, Noel-Storr, Brayne and Cullum2015; Habib & Stott, Reference Habib and Stott2017; Rosli, Tan, Gray, Subramanian, & Chin, Reference Rosli, Tan, Gray, Subramanian and Chin2016; Steis & Schrauf, Reference Steis and Schrauf2009), among others, have been validated in many languages and cultures.
There have, however, been only a few validation studies of cognitive screening instruments and neuropsychological test batteries that have addressed linguistic, geographic, and educational diversity in populations like India. Among the cognitive screening tests, the MMSE and ACE have been developed for use in the Indian context in different languages and for the low-educated population (Ganguli et al., Reference Ganguli, Ratcliff, Chandra, Sharma, Gilby, Pandav, Belle, Ryan, Baker, Seaberg and Dekosky1995; Mathuranath et al., Reference Mathuranath, Hodges, Mathew, Cherian, George and Bak2004; Mekala, Alladi, Mridula, & Kaul, Reference Mekala, Alladi, Mridula and Kaul2008). Validation studies of neuropsychological test batteries in India are a few: the PGI battery of brain dysfunction (PGIBBD; Pershad & Verma, Reference Pershad and Verma1990), the Hindi cognitive test battery (Ganguli et al., 1995, Reference Ganguli, Chandra, Gilby, Ratcliff, Sharma, Pandav, Seaberg and Belle1996), NIMHANS neuropsychological battery (Rao, Subbakrishna, & Gopukumar, Reference Rao, Subbakrishna and Gopukumar2004), NIMHANS neuropsychological battery for elderly (Tripathi, Kumar, Bharath, Marimuthu, & Varghese, Reference Tripathi, Kumar, Bharath, Marimuthu and Varghese2013), and the Kolkata screening battery that has been validated for use in Bengali and Hindi (Das et al., Reference Das, Banerjee, Mukherjee, Bose, Biswas, Hazra, Dutt, Chaudhuri, Raut and Roy2006). While these were pioneering efforts, the batteries are limited in their generalisability, as they have been validated for one language only, and/or in one geographic region. Comparability across different languages and educational levels has not been established. In addition, most of the test batteries are applicable for literates and do not test cognitive domains in minimally educated or illiterate people. Furthermore, many tests in the batteries are not available in the public domain and are copyrighted, limiting their use in low-resource settings. Another limitation of existing cognitive test batteries is that majority have been developed for broader diagnosis of dementia or specifically for only AD and are not validated for other dementia subtypes such as vascular dementia, the second most common form of dementia in LMIC, and frontotemporal dementia that have cognitive profiles different from AD, or for milder conditions such as MCI, an important pre-clinical at-risk for dementia state. In order to study dementia across diverse populations, it is vital to standardise diagnostic methods to reduce variance related to socio-demographic and geographic factors. An essential step towards this process is to develop a common set of screening and diagnostic tools and protocols that are standardised for use in the context of linguistic diversity and educational and socio-cultural heterogeneity.
In this multicentric study from India, a systematic process of standardisation of a cognitive test battery was followed to create uniformity in dementia diagnosis across diverse cohorts. Towards achieving this, a multidisciplinary group of neurologists, neuropsychologists, speech and language pathologists, and experts from related fields worked together in a project supported by the Indian Council of Medical Research (ICMR) (http://icmr.nic.in). The aim of this project was to adapt and validate a standardised comprehensive cognitive and functional test protocol for diagnosing MCI and dementia due to neurodegenerative disease and stroke in five Indian languages and for different educational levels. In this study design paper, we describe the process of development of the test battery referred to as the ICMR-Neurocognitive Tool Box (ICMR-NCTB) and also present the plan for its validation in dementia and MCI due to degenerative disease and stroke. The results of data collected, analysis, and validation studies will be reported subsequently. The study is in compliance with the Helsinki Declaration and was approved by the Research Ethics Committee of Nizam’s Institute of Medical Sciences, Hyderabad, the coordinating centre of the project. Approvals were also obtained from the participating centres (All India Institute of Medical Sciences Ethics Committee, Delhi; Institutional Ethics Committee, Apollo Gleneagles Hospital, Kolkata; Ethics Committee of Manipal Hospital, Bengaluru; and Institutional Ethics Committee, Sree Chitra Tirunal Institute for Medical Sciences and Technology, Kerala).
METHODS
This study brought together a large group of experts and specialists, in a multicentric study to collaborate towards adapting and validating a common cognitive test battery in five Indian languages: Hindi, Bengali, Telugu, Kannada, and Malayalam, in literate and illiterates/low-literacy groups to diagnose dementia and MCI. The ICMR-NCTB aimed to be comprehensive in its scope and included tests that assess cognitive functions, functional activities, behavioural assessments, global cognitive screening tools, and quality of life measures. The overarching purpose of the ICMR-NCTB was to provide a standardised and validated test battery for MCI, dementia, and VCI that will be accessible for use in both clinical and research settings.
Towards this a collaboration between academic institutions in different linguistic zones of India was initiated by the ICMR as a national task force project. The collaborating institutions included All Indian Institute of Medical Sciences (AIIMS) New Delhi from the north; Apollo Gleneagles Kolkata from the east; Nizam’s Institute of Medical Sciences (NIMS, Hyderabad), Manipal Hospital (Bengaluru), Jawaharlal Nehru Medical College (Belgaum), and Sri Chitra Tirunal Institute of Medical Science and Technology (SCTIMST, Trivandrum) from three different states in southern India. The main languages spoken in these areas include Hindi in Delhi, Bengali in Kolkata, Hindi and Telugu in Hyderabad, Kannada in Bangalore and Belgaum, and Malayalam in Trivandrum.
The step-by-step process of development and adaptation of the ICMR-NCTB, as well as the methods employed to ensure a uniform and standardised testing protocol, is described in this report. The proposed plan for the recruitment of a diverse cohort across six centres and the common diagnostic protocol and the statistical plan that will be followed to develop normative data, cut-off scores and validate the battery are also described. Subsequent reports will present the results of the data collected, analysis of the performance of the heterogeneous cohort on the ICMR-NCTB, and validation studies.
Step 1: Review of Existing Batteries Used Worldwide for Relevance to Indian Context and Appropriate Modifications
The ICMR-NCTB was developed based on the review of existing standard cognitive test batteries used widely internationally, and also those already validated for use in the Indian context. Each battery was reviewed for the cognitive domains tested and the specific tests used. Each test from the battery was measured for content validity, their psychometric attributes and also examined for relevance for use in the Indian context and feasibility for adaptation. Availability of the test in the public domain and copyright policies were also considered. This was done over several rounds of discussions that included two workshops and multiple email communications, until a consensus was reached on tests to be used by the members of the group.
Step 2: Adaptation/Translation of Cognitive Tests
For adapting and translating the tests into the five Indian languages, the guidelines of cross-cultural adaptation proposed by Guillemin, Bombardier and Beaton (Reference Guillemin, Bombardier and Beaton1993) and others (Ardila, Reference Ardila1995; Beaton, Bombardier, Guillemin, & Ferraz, Reference Beaton, Bombardier, Guillemin and Ferraz2000) were used. The adaptation process was iterative, which involved several rounds of meetings/discussions, Skype sessions, and email correspondences. The process included a comprehensive literature review, and based on the information available, a clear development of standardised guidelines for adaptation was formulated, which included cultural modifications, translations and back translations, expert committee reviews, and pilot testing. To address cross-cultural bias, “decentring” was used: a method where words or concepts that are clearly specific to one particular language or culture is excluded through the simultaneous development of an instrument in different languages and cultures (Tanzer, Reference Tanzer, Hambleton, Merenda and Spielberger2004). Each test (stimuli and administration) was discussed in detail, and modifications were suggested to make the test relevant to the Indian context. Once a consensus was achieved, the tests were modified accordingly. Tests selected were grouped into the following categories: firstly, tests that could be translated directly without major changes; secondly, tests that required major modifications and adaptation; and finally, tests that need to be replaced completely. Translation was done for tests where their psychometric and linguistic properties that might affect validity of test were not altered (Kester & Peña, Reference Kester and Peña2002). Tests that were significantly culture dependent, requiring stimuli that derived from the cultural background of the population (George & Mathuranath, Reference George and Mathuranath2007), were adapted. In situations, especially when testing illiterates, when existing tests could not be used, an attempt was made to develop innovative tests. The first version of the battery was developed in Indian English, since it was the common language across centres. The Indian English version of the battery was then translated and back translated into the five languages: Hindi, Bengali, Telugu, Kannada, and Malayalam (stimuli, instructions, administration, and scoring). Expert consensus was again achieved for the final versions of the adapted tests.
Step 3: Establishing Common Test Administration Protocols
To increase the reliability of the data obtained and reduce the variability across the centres in administration, a common Case Report Form (CRF) was developed for all the centres. The CRF included a manual of operations that detailed the protocol for test administration with instructions for order of administration and scoring for each test in the battery. It contained subject demographic information, including educational status, occupation, socio-economic status, rural/urban dwelling, and a detailed language use questionnaire. Medical data based on a structured interview and physical examination were documented in all. All stroke patients also underwent NIH Stroke Scale (NIHSS) scale to assess the severity of stroke (Brott et al., Reference Brott, Adams, Olinger, Marler, Barsan, Biller, Spilker, Holleran, Eberle, Hertzberg, Rorick, Moomaw and Walker1989). Results of cognitive test performance were recorded uniformly. Test instructions were in English across centres.
Training workshops were held to train psychologists on test administration, and data collection methods. Inter-rater reliability was done among psychologists across centres to ensure data were collected in a standard manner.
Step 4: Recruitment of Study Participants and Diagnostic Process
Participants are recruited from out-patient services of neurology, geriatric, and internal medicine clinics of participating hospitals, as well as senior citizen associations and other community centres in the respective cities. The goal is to recruit individuals aged 40 years and above, with varying levels of education to include both literates and illiterates.
Participants with normal cognition, MCI, and dementia due to neurodegenerative disease and stroke, from both clinic and community, are included in the study.
Based on clinical evaluation, the individuals who fulfilled the following inclusionary criteria are recruited: persons who are ≥40 years and consented to participate; with no evidence of head injury, infections, and neurological disorders other than stroke and neurodegenerative disease that could cause cognitive impairment; with no history of major systemic medical or psychiatric conditions that could interfere with cognition; and with no significant hearing or visual impairment that could interfere with cognitive testing.
All study participants undergo cognitive assessment using tests that have been standardised to the local populations, referred to as “Gold standard battery” for which normative data are available in local languages, and have been in use for clinical diagnosis and research. This battery consists of the cognitive screening test Addenbrooke’s Cognitive Examination-III (ACE-III) and the Clinical Dementia Rating (CDR) which is administered in all subjects. In addition, in subjects with no dementia or questionable dementia (CDR 0 and 1), tests of episodic memory and executive functions: Rey Auditory Verbal Learning Test (RAVLT) and Color Trails Test (CTT) are performed to identify subjects with MCI. These tests have been validated and are widely used for diagnosis of MCI in the respective Indian languages (Alladi et al., 2011, Reference Alladi, Shailaja, Mridula, Haritha, Kavitha, Khan, Divyaraj and Kaul2014; Mathuranath et al., Reference Mathuranath, Cherian, Mathew, George, Alexander and Sarma2007; Nandi et al., Reference Nandi, Biswas, Pal, Basu, Senapati and Das2008; Rao et al., Reference Rao, Subbakrishna and Gopukumar2004). A subject is considered to be impaired on a test when his or her performance is below cut-off values in the gold standard battery. Clinical diagnosis is made by a neurologist experienced in diagnosis of MCI and dementia following a semi-structured interview, clinical examination, review of performance on gold standard tests, and other available investigations. A 1-day workshop was conducted for neurologists experienced in dementia diagnosis from each centre, to standardise clinical diagnosis across centres. Based on the uniform diagnostic process, participants are grouped as follows:
1. Healthy controls: All subjects who have no subjective cognitive complaints and scored normally on ACE-III, CDR, RAVLT, and CTT.
2. MCI: Subjects who fulfill modified Petersen’s criteria for MCI (Petersen, Reference Petersen2004).
3. Dementia: DSM-IV criteria for dementia.
Subtypes of dementia will be further diagnosed as follows:
Alzheimer’s disease (AD): (McKhann et al., Reference McKhann, Knopman, Chertkow, Hyman, Jack, Kawas, Klunk, Koroshetz, Manly, Mayeux, Mohs, Morris, Rossor, Scheltens, Carrillo, Thies, Weintraub and Phelps2011).
Vascular dementia (VaD): NINDS-AIREN criteria (Román et al., Reference Román, Tatemichi, Erkinjuntti, Cummings, Masdeu, Garcia, Amaducci, Orgogozo, Brun, Hofman, Moody, O'Brien, Yamaguchi, Grafman, Drayer, Bennett, Fisher, Ogata, Kokmen, Bermejo, Wolf, Gorelick, Bick, Pajeau, Bell, DeCarli, Culebras, Korczyn, Bogousslavsky, Hartmann and Scheinberg1993).
Frontotemporal dementia (FTD): (Rascovsky et al., Reference Rascovsky, Hodges, Knopman, Mendez, Kramer, Neuhaus, van Swieten, Seelaar, Dopper, Onyike, Hillis, Josephs, Boeve, Kertesz, Seeley, Rankin, Johnson, Gorno-Tempini, Rosen, Prioleau-Latham, Lee, Kipps, Lillo, Piguet, Rohrer, Rossor, Warren, Fox, Galasko, Salmon, Black, Mesulam, Weintraub, Dickerson, Diehl-Schmid, Pasquier, Deramecourt, Lebert, Pijnenburg, Chow, Manes, Grafman, Cappa, Freedman, Grossman and Miller2011).
Dementia with Lewy bodies (DLB): (McKeith et al., Reference McKeith, Dickson, Lowe, Emre, O’Brien, Feldman, Cummings, Duda, Lippa, Perry, Aarsland, Arai, Ballard, Boeve, Burn, Costa, Del Ser, Dubois, Galasko, Gauthier, Goetz, Gomez-Tortosa, Halliday, Hansen, Hardy, Iwatsubo, Kalaria, Kaufer, Kenny, Korczyn, Kosaka, Lee, Lees, Litvan, Londos, Lopez, Minoshima, Mizuno, Molina, Mukaetova-Ladinska, Pasquier, Perry, Schulz, Trojanowski and Yamada2005).
4 Strokes with normal cognition: All subjects are diagnosed to have stroke based on clinical evaluation and brain imaging (CT and/or MRI) and perform normally on ACE-III, CDR, RAVLT, and CTT.
5 Vascular MCI: Subjects with stroke who fulfil VASCOG criteria (Sachdev et al., Reference Sachdev, Kalaria, O’Brien, Skoog, Alladi, Black, Blacker, Blazer, Chen, Chui, Ganguli, Jellinger, Jeste, Pasquier, Paulsen, Prins, Rockwood, Roman and Scheltens2014).
All recruited subjects subsequently undergo the complete ICMR-NCTB by an independent team of clinicians and psychologists, blind to the diagnosis. All data are entered into a computerised data entry system that was developed for uniform online data entry across the six centres.
Step 5: Validation Process of ICMR-NCTB Battery and Generation of Cut-Off Scores
The plan for the validation process of the ICMR-NCTB includes the following: (a) establishing face validity: the expert committee members subjectively viewed the relevance of each test and certified its utility in assessing the intended cognitive domain; (b) internal and inter-rater reliability will be established using Chronbach’s alpha and Cohen’s kappa, respectively; (c) external validity will be assessed using receiver operating characteristics (ROC) curve analysis and/or by mean ± 2 SD method.
Another main objective of the study was to develop a reference neuropsychological test battery and cut-off values of tests for use in clinics and for research to diagnose MCI and dementia. Cut-off values will be determined at optimal sensitivity and specificity points which will be generated using ROC curve analysis. All the analysis will be performed using the statistical software SPSS version 23.0.
RESULTS
Step 1: Review of Existing Batteries for Relevance to Indian Context and Appropriate Modification to Develop the ICMR-NCTB
Based on a comprehensive literature review, the test batteries used for the diagnosis of dementia and MCI were identified: the CERAD (Fillenbaum et al., Reference Fillenbaum, van Belle, Morris, Mohs, Mirra, Davis, Tariot, Silverman, Clark, Welsh-Bohmer and Heyman2008), 10/66 (Prince et al., Reference Prince, Acosta, Chiu, Scazufca, Varghese and Dementia Research2003), and NINDS-CSN test battery (Hachinski et al., Reference Hachinski, Iadecola, Petersen, Breteler, Nyenhuis, Black, Powers, DeCarli, Merino, Kalaria, Vinters, Holtzman, Rosenberg, Wallin, Dichgans, Marler and Leblanc2006). The test batteries that have been validated for use in the Indian context were identified: the Hindi cognitive test battery (Pandav, Fillenbaum, Ratcliff, Dodge, & Ganguli, Reference Pandav, Fillenbaum, Ratcliff, Dodge and Ganguli2002), the NIMHANS battery (Rao et al., Reference Rao, Subbakrishna and Gopukumar2004), and Kolkata screening battery (Das et al., Reference Das, Banerjee, Mukherjee, Bose, Biswas, Hazra, Dutt, Chaudhuri, Raut and Roy2006). All test batteries were reviewed for the range of cognitive domains tested and the specific tests used. Majority of test batteries also incorporated a brief cognitive screening instrument. Furthermore, questionnaires that evaluated functional activities, behavioural changes, and quality of life were major components of the test batteries (Table 1). All tests and questionnaires were then reviewed for suitability of use in the Indian context, and for the availability of adapted versions in Indian languages. Based on the observations, the ICMR-NCTB was developed to include tests of attention/executive function, memory, language and visuospatial functions, and questionnaires for functional activities, behavioural changes, and quality of life. Since the ICMR-NCTB protocol was developed to diagnose dementia and MCI due to both degenerative disease and stroke, tests to detect focal cognitive deficits common in stroke, such as aphasia and unilateral spatial neglect, were specifically included.
Neuropsychological tests, behavioural and functional questionnaires used in existing cognitive test batteries in comparison to ICMR-NCTB
Note: Additional tests used in NINDS-CSN battery under executive/attention functions: Weschler Adult Intelligence Scale III (WAIS III) Digit Symbol Coding. Additional tests used in NIMHANS battery under executive/attention functions: Stroop Test, Tower of London, Triads, Wisconsin Card Sorting Test, and N Back Test (Verbal and Visual). Additional tests used in NIMHANS battery under visuospatial functions: Design Learning Test Copy.
The expert group also recommended the development of ICMR-NCTB for illiterates/low-literacy groups. This was based on research that suggested that illiterates differ in the use of cognitive strategies involved in memory, phonological processing, visuospatial, and other domains, which impacts their performance on tasks developed in predominantly literate settings (Castro-Caldas, Petersson, Reis, Stone-Elander, & Ingvar, Reference Castro-Caldas, Petersson, Reis, Stone-Elander and Ingvar1998; Folia & Kosmidis, Reference Folia and Kosmidis2003). Review of existing batteries suggested that apart from the Hindi cognitive test battery, majority of tests were applicable for use mainly in educated subjects. Therefore, there was a need to adapt cognitive tests or develop innovative tests for illiterates in the ICMR-NCTB. The expert group reached a consensus that the present project would focus on developing essential tests for illiterates, given its current scope. It was planned that, following the experience of testing illiterates in the current project, more extensive development of innovative tests will be done in the next phase of the project.
(A) Tests for Literates
1. Global cognitive function
Montreal Cognitive Assessment (MoCA; Nasreddine et al., Reference Nasreddine, Phillips, Bédirian, Charbonneau, Whitehead, Collin, Cummings and Chertkow2005) was used as a global cognitive screening measure because it is a widely used instrument validated across many languages to detect dementia (Hoops et al., Reference Hoops, Nazem, Siderowf, Duda, Xie, Stern and Weintraub2009).
2. Attention and executive functioning
Trail Making Test Black and White (TMT B & W; Kim, Baek, & Kim, Reference Kim, Baek and Kim2014) was selected as it is a widely used neuropsychological test of frontal lobe function that measures psychomotor speed, attention, sequencing, mental flexibility, and visual scanning (Oosterman et al., Reference Oosterman, Vogels, van Harten, Gouw, Poggesi, Scheltens, Kessels and Scherder2010). Category Fluency (Animals) (Lezak, Reference Lezak1995) was chosen because of its common use and the relative ease of cross-cultural application. Category fluency provides information on attention, set-shifting, and executive control. Furthermore, category fluency engages those anatomical structures accountable for retrieval and encoding of semantic and episodic memories (Venneri et al., Reference Venneri, McGeown, Hietanen, Guerrini, Ellis and Shanks2008). The expert committee decided to not include the letter fluency task, as there would be generalisability issue for non-phonetic languages (Hachinski et al., Reference Hachinski, Iadecola, Petersen, Breteler, Nyenhuis, Black, Powers, DeCarli, Merino, Kalaria, Vinters, Holtzman, Rosenberg, Wallin, Dichgans, Marler and Leblanc2006).
3. Episodic memory
Word List Memory Test (verbal learning test from Kolkata cognitive screening battery) (Das et al., Reference Das, Banerjee, Mukherjee, Bose, Biswas, Hazra, Dutt, Chaudhuri, Raut and Roy2006) was used to measure episodic memory. Word list-learning tests are recognised to be easier to develop in other languages and cultures (Hachinski et al., Reference Hachinski, Iadecola, Petersen, Breteler, Nyenhuis, Black, Powers, DeCarli, Merino, Kalaria, Vinters, Holtzman, Rosenberg, Wallin, Dichgans, Marler and Leblanc2006). In addition, list-learning tests are found to be most useful in identifying patients with MCI at risk for developing Alzheimer’s disease (Albert et al., Reference Albert, DeKosky, Dickson, Dubois, Feldman, Fox, Gamst, Holtzman, Jagust, Petersen, Snyder, Carrillo, Thies and Petersen2011), so the expert committee favoured its inclusion into the ICMR-NCTB.
Modified Taylor Complex Figure Test (MTCF; Hubley, Reference Hubley1999) was adopted as a test of visuospatial memory and construction, as an alternative form to Rey Osterrieth Complex Figure (ROCF). The MTCF test has demonstrated comparability to ROCF and has the advantage of being freely available for clinical practice and research (Hubley & Jassal, Reference Hubley and Jassal2006).
4. Language
Picture Naming Test (PNT) consists of 30 black-and-white line drawings that were largely derived from the naming test developed by George and Mathuranath (Reference George and Mathuranath2007) based on psycholinguistic properties such as naming agreement and familiarity in the Indian context. Line drawings of pictures are the standard stimuli to study language in healthy people and patients with cognitive impairment (Albert, Heller, & Milberg, Reference Albert, Heller and Milberg1988).
Frenchay Aphasia Screening Test (FAST; Enderby, Wood, Wade, & Hewer, Reference Enderby, Wood, Wade and Hewer1986) is a short, simple, and standardised test that identifies language deficits and aphasia, and has been used in India in clinical settings.
5. Visuospatial skills
The MTCF-Copy was used to measure visuospatial skills and copying (Hubley, Reference Hubley1999; Hubley & Jassal, Reference Hubley and Jassal2006).
6. Unilateral spatial neglect
Unilateral visual neglect was tested using the Line Bisection task (Heilman & Valenstein, Reference Heilman and Valenstein1979) which requires the participant to draw a vertical line through the centre of each of the 18 lines placed horizontally at the left, right, and centre of a paper.
(B) Tests for Illiterates
To test cognitive functions in illiterates, it was essential to adapt majority of tests used in the literate battery. Innovative tests also needed to be developed in situations where adaptations did not work for illiterates.
To mitigate the literacy bias while testing attention and executive functions, an innovative adaptation was made to the TMT-B & W (Kim et al., Reference Kim, Baek and Kim2014). The illiterate version called the Indian Trail Making Test is similar to the TMT B & W except for the use of pictures of hands with fingers indicating numbers instead of English numerals as used in TMT-B & W (Figure 1). The Indian adaptation of the TMT also required persons to point at the pictures with their finger, in serial sequence, instead of using a pen or pencil.
Showing the Indian Trail Making Test – Part-B sample version. The participants were instructed to point to the circles with fingers indicating numbers 1– to 5 in ascending order, alternating between black and white circles, without lifting the finger from the paper, as quickly as possible. The time taken to complete the test and the number of errors were recorded.
Episodic memory was tested using the verbal learning test from the Kolkata cognitive screening battery, similar to the test used in the Hindi cognitive test battery developed for largely low-literate population. In addition, Test des Neuf Images du-93 (TNI-93; Dessi et al., Reference Dessi, Maillet, Metivet, Michault, Le Clésiau, Ergis and Belin2009) was used as an additional episodic memory test in illiterates/low-literacy groups as it has been developed for the multicultural population of low socio-educational level and is rapid and easy to administer. The two language tests used in the illiterate/low-literacy population were Frenchay Aphasia Screening Test-Indian Version (FAST-Indian Version) without the reading and writing subsections; and Picture Naming Test (PNT). In both the tests, coloured pictures replaced the black-and-white line diagrams used in the literate versions, since line diagrams are difficult to name for low literates/illiterates (Reis, Faísca, Ingvar, & Petersson, Reference Reis, Faísca, Ingvar and Petersson2006).
The use of drawing tasks derives from graphomotor abilities, and hence is not appropriate to test visuospatial abilities in illiterates (Ardila & Rosselli, Reference Ardila, Rosselli, Uzzel, Pontón and Ardila2002; Nielsen & Jørgensen, Reference Nielsen and Jørgensen2013). These tasks were also unable to discriminate between low-educated healthy people and cognitively impaired individuals. A visuoconstructional test – the Stick Design test – developed for persons with low or no education in Nigeria was found to be significantly more sensitive to cognitive impairment and dementia than any other drawing test (Baiyewu et al., Reference Baiyewu, Unverzagt, Lane, Gureje, Ogunniyi, Musick, Gao, Hall and Hendrie2005). Based on this observation, the expert group developed the Indian Stick Figure Test (SFT) to evaluate visuoconstructional ability and visuospatial memory in illiterates and low-educated individuals. Following consensus of opinion of the expert group and several pilot trials, the Indian SFT was finalised to include common geometrical figures with five elements that could be constructed with 20 loose wooden sticks of different sizes (Figure 2). Similar to MTCF, the Indian SFT assesses copy, immediate, and delayed recall. Furthermore, the stimulus was presented in a 3D format and not in a printed paper format, because illiterates are unable to extract the features of 3D from a 2D image (Dansilio & Charamelo, Reference Dansilio and Charamelo2005).
Showing (a) the representative model of the Indian Stick Figure Test given to the participants during the copying task by instructing them to look at the design carefully and construct the same figure with the help of the (b) 20 loose wooden sticks given to them. Scoring was based on accuracy of placement, formation, and correct length of stick used for each of the (c) 5 units in the test.
(C) Questionnaires
The ICMR expert group recommended five questionnaires to assess behaviour, depression, activities of daily living, change in cognitive functioning, and quality of life, and these were common for both literates and illiterates/low-literacy groups. The Neuropsychiatric Inventory Questionnaire (NPI-Brief; Cummings et al., Reference Cummings, Mega, Gray, Rosenberg-Thompson, Carusi and Gornbein1994) was used to assess neuropsychiatric symptoms. Geriatric Depression Scale (GDS-30; Yesavage et al., Reference Yesavage, Brink, Rose, Lum, Huang, Adey and Leirer1982) was used as a screening measure for depressive symptoms as it is widely used in the Indian population (Ganguli et al., 1995, Reference Ganguli, Dube, Johnston, Pandav, Chandra and Dodge1999). Instrument of Daily Living-Elderly (IADL-E; Mathuranath, George, Cherian, Mathew, & Sarma, Reference Mathuranath, George, Cherian, Mathew and Sarma2005) was used to characterise and grade the performance of participants on domains of cognitive activities, social/recreational activities, community and household activities, and self-care. To obtain information provided by an informant to assess a person’s change in cognitive functioning, Informant Questionnaire for Dementia Diagnosis (IQ Code; Jorm & Korten, Reference Jorm and Korten1988) was used. To yield functional-health and well-being scores as well as psychometric-based physical and mental health measures, we used RAND Short Form Health Survey (RAND SF-36; Ware, Snow, Kosinski, & Gendek, Reference Ware, Snow, Kosinski and Gendek1993). All tests requiring permissions were obtained from the test developers.
Step 2: Adaptation/Translation
Subsequent to finalising the list of tests to be included in the ICMR-NCTB, appropriate modifications were done to the tests and stimuli. Table 2 provides the list of tests that could be translated directly into Indian languages without major changes and tests that needed major modifications and adaptation to Indian context. Significant adaptations were required for tests that used visual stimuli of objects and situations that differed significantly between Western and Indian cultures and included the PNT and FAST-Indian Version for language and TNI for visuospatial memory. Innovations were done mainly for tests that could not be used for illiteracy/low-literacy groups that included the Indian Trail Making Test to test executive functions and Indian SFT, to evaluate visuospatial functions.
Adaptation process of neuropsychological tests, behavioural and functional questionnaires in the ICMR-NCTB
Step 3: Standardising Test Administration and Interpretation
The 5 study psychologists administered and scored the ICMR-NCTB tests independent of each other, using the common CRF in 20 control volunteer participants. Inter-rater reliability of the test scores was calculated using Cohen’s kappa. The overall kappa coefficient value was 0.91 (p < .001) which shows good agreement between the raters for ICMR-NCTB administration and scoring. Inter-rater reliabilities for the individual tests ranged from 0.84 to 0.96. The discrepancies in the test administration and scoring were discussed, and consensus was reached among researchers. A uniform administration and scoring pattern between raters and centres was hence achieved.
Step 4: Subject Recruitment and Data Collection
Subjects older than 40 years will be randomly screened and those who fulfilled inclusion criteria will be recruited to form a cohort of people with varying levels of cognition ranging from normal to dementia. Based on their performance on clinical evaluation and the gold standard battery, participants will be diagnosed to have normal cognition, MCI, and dementia. Patients with stroke will be classified as stroke with normal cognition, vascular MCI, and VaD. All recruited subjects will undergo the ICMR-NCTB by a trained psychologist who is blind to clinical diagnosis.
A statistical power analysis was performed for sample size estimation. With an α = .05 and power = .87, the projected sample size was calculated using G*Power programme (Faul, Erdfelder, Lang, & Buchner, Reference Faul, Erdfelder, Lang and Buchner2007). About 300 subjects per centre and 1500 overall were estimated as the sample size.
A computerised data entry system was developed for uniform online data collection and analysis across six centres. The software programme called RECOLLECT (softwaRE tool for COLLECTion and consolidation of ICMR-NCTB data) was developed by Centre for Development of Advanced Computing (C-DAC), Hyderabad. The RECOLLECT programme was designed to capture demographic, clinical, medical details and test scores for study subjects, and ensured error-free and uniform data entry across the different centres by researchers.
Step 5: Validation Process of ICMR-NCTB Battery and Generation of Cut-Off Values
Face Validity: A group of experts reviewed and certified the appropriateness of each test and its items in tapping the targeted cognitive domain. Based on expert comments on each test in terms of feasibility and clarity of its contents, few items were revised. Expert consensus was then achieved with a series of meetings to establish face validity for ICMR-NCTB battery.
Reliability: Following data collection, the internal and inter-rater reliability of the neuropsychological test scores will be examined using Cronbach’s alpha and Cohen’s kappa coefficient, respectively.
External validity: External validity of the ICMR-NCTB will be established using ROC curve analysis of the collected data and/or by mean ± 2 SD method that is, 95% confidence interval (CI). As the first step in ROC curve analysis, age and education stratified mean and standard deviations of the control group will be used to compute z scores for each of the neuropsychological tests in the ICMR-NCTB. A composite score will be obtained from the average of z scores. Area under curve (AUC) will be obtained from ROC curve analysis across languages; this will indicate accuracy of the ICMR-NCTB in correctly diagnosing the subjects. Sensitivity and specificity of the ICMR-NCTB in diagnosing patients with dementia and MCI due to degenerative disease and stroke will be calculated with the help of ROC curve analysis or by mean ± 2SD method. ROC curve analysis of the individual tests will also produce optimum cut-off values of the test scores.
To identify the independent predictors of cognitive test scores, a univariate and multivariate generalised linear model analyses will be done. Firstly, a univariate linear model will be constructed to assess the bivariate association of the predictor variables (age, gender, education, and language) on the individual test score. Then, multivariate generalised linear model will be conducted after the adjustment for the variables, with a p-value of <.1 in the univariate analysis.
DISCUSSION
A key outcome of this study was the development of a comprehensive neuropsychological and behavioural test battery that can be used to assess cognitive functions in persons with dementia and MCI due to varied aetiologies across five different languages and different educational levels including illiterates/low literates for the first time. The ICMR-NCTB was developed by a systematic process of reviewing existing standardised test batteries available internationally and nationally, and through a rigorous process of adaptation and translation by a multidisciplinary group of experts for the Indian sociocultural and linguistic context. The uniform protocol developed for testing and diagnosis in a large multicentric cohort of individuals with normal cognition, MCI, and dementia will ensure an appropriate validation of the ICMR-NTB. Age- and education-appropriate normative data will also be available to diagnose cognitive impairment due to different aetiologies.
The advantages of the ICMR-NCTB are that it will meet different needs: a short cognitive screening instrument for the busy clinic, and a detailed version for more in-depth clinical testing or for research. Development of common methodology of testing across five Indian languages will reduce variability in clinical diagnosis in hospitals and clinics across India. The neurocognitive test battery will be made accessible by ICMR for free use across clinics in India and also for Indian diaspora.
Development and standardisation of a common instrument to diagnose cognitive impairment in a diverse population, such as the ICMR-NCTB, is complex. Educational status and linguistic and cultural backgrounds among other socio-demographic factors influence performance on a cognitive testing instrument. The attempt to develop culture-free uniform cognitive tests may, however, result in an instrument that is likely to be less sensitive compared to tests developed indigenously for a population. Based on the experience of this project, future research will be required to focus on the development of more innovative tests, especially for illiterates. However, since it is also vital to harmonise research methods across diverse populations, the ICMR-NCTB effort is an initial step towards understanding advantages and limitations of developing a common methodology to study heterogeneous populations. The experience of standardising cognitive and behavioural test performance across five languages, educational status, and cultures is unique and will contribute towards developing effective ways of conducting global collaborative research in cognitive disorders.
ACKNOWLEDGEMENTS
We would like thank the following individuals for their expertise extended towards the project: J.S. Chopra, S.K. Das, Prabhakar S., Narendra K. Arora, Anand Krishnan, M. Gourie-Devi (ICMR scientific experts), Prathiba Karanth, Sunil Kumar Ravi, and Annamma George. We would also like to thank the following people/organisations for providing us with permission to use and adapt the tests wherever required: John R Hodges for ACE III, SangYun Kim for Trail Making Test-B&W, Catherine Belin for TNI-93, Anitha M. Hubley for MTCF, P.S. Mathuranath for PN test and IADL-E, Pam Enderby and STASS Publications for FAST, Jeffery Cummings for NPI, Anthony Jorm for IQCODE, and Ziad Nasreddine for MoCA.
FINANCIAL SUPPORT
This work was supported by Indian Council for Medical Research under Grant [SWG/Neuro/32/2017-NCD-1].
CONFLICT OF INTEREST
The authors have nothing to disclose.
