Introduction
Investigations have documented the different prevalence rates for dementia in community surveys in India. Although the reported prevalence has been lower in India (1.36–3.50%) compared to the West (5.9–9.4%) (Chandra et al., Reference Chandra, Ganguli, Pandav, Johnston, Belle and DeKosky1998; Ferri et al., Reference Ferri2005), the size of India's population and the increasing proportion of its elderly mean that significant numbers of people will be affected by the condition.
Several instruments have been employed to screen patients for dementia (Cherbuin et al., Reference Cherbuin, Anstey and Lipnicki2008). However, despite the numerous available instruments, many challenges remain. These include: (i) many of the instruments, including the commonly used Mini-mental State Examination (Folstein et al., Reference Folstein, Folstein and McHugh1975), are influenced by education and by culture, making them inappropriate for use in societies with low literacy (Medical Research Council CFA, 1998); (ii) the education- and culture-free screening tests available do not test multiple cognitive domains systematically; and (iii) even screening instruments with good sensitivity and specificity, when used in situations of low prevalence (e.g. community and primary care), result in very high false positive rates (Jacob, Reference Jacob2003; Jacob et al., Reference Jacob, Kumar, Gayathri, Abraham and Prince2007; Cherbuin et al., Reference Cherbuin, Anstey and Lipnicki2008) as these rates are dependent on prevalence. We aimed to design brief instruments to screen patients for dementia for use in populations with low education that are adapted to the local culture, and to develop a strategy to improve predictive values of the screening procedure.
Methods
The aims of making the instrument culture- and education-fair were achieved by focusing on activities of daily living. We used simple assessments to test key cognitive domains employing day-to-day activities.
Development of instruments
Data from the literature on screening and diagnostic tests for dementia were reviewed by a panel of psychiatrists to select potential test items. Many instruments were analyzed. Each item and subtest was examined for relevance, adaptability, the conceptual basis for the test, the cognitive domain being evaluated and for their ability to be culture and education fair.
Each item was tested in elderly volunteers after obtaining consent. The objective was to examine the level of difficulty, acceptability, comprehensibility and relevance of the potential items and the distribution of scores on each subtest and item. A detailed operations manual with explicit instruction using each item was developed.
The patient version
The Vellore Screening Instrument for Dementia – Patient (VSID-P) version comprises 10 items; two items for each key cognitive domain affected in dementia (memory, aphasia, apraxia, agnosia and loss of executive function). The items focus on activities of daily living.
The cognitive domain tested and the items selected for the patient version (see Appendix 1) are briefly described as follows:
(i) Registration: registration and recall of three objects employed in the MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975) and Hindi version of the MMSE (HMSE) (Ganguli et al., Reference Ganguli1995) were modified for use in local Tamil-speaking populations.
(ii) Aphasia: two items were employed to test sensory and motor aphasia. These were modified from the HMSE (Ganguli et al., Reference Ganguli1995).
(iii) Apraxia: the ability to execute a voluntary motor movement in response to a verbal command, to imitate and to handle an object correctly was checked by asking the subject to demonstrate simple day-to-day activities.
(iv) Agnosia: the ability to recognize objects with eyes open (a key) and with eyes closed (a comb) and attach appropriate meanings were used to check for visual and tactile agnosia respectively. The objects chosen were selected for their suitability for use in the local population. The item was modified from MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975) and HMSE (Ganguli et al., Reference Ganguli1995).
(v) Disturbances in executive functioning: two tasks used to assess executive function were taken from the MMSE (Folstein et al., Reference Folstein, Folstein and McHugh1975).
(vi) Recall: the recall of the items employed for registration was employed to assess memory.
The informant version
The Vellore Screening Instrument for Dementia – Informant (VSID-I) version comprises 10 items; two items for each key cognitive domain affected in dementia (memory, aphasia, apraxia, agnosia and loss of executive function). The items also focused on activities of daily living. The cognitive domain tested and the items selected for the informant version (Appendix 2) are briefly described:
(i) Memory impairment: two items were used. The first item was taken from GPCOG (Brodaty et al., Reference Brodaty2002) and the second from Short IQCODE (Jorm and Jacomb, Reference Jorm and Jacomb1989).
(ii) Aphasia: two items were employed to test aphasia. The first was from GPCOG (Brodaty et al., Reference Brodaty2002) and the second from HMSE (Ganguli et al., Reference Ganguli1995).
(iii) Apraxia: this was checked by using two items from EASI (Fillenbaum et al., Reference Fillenbaum1999).
(iv) Agnosia: this was assessed using two items routinely employed in neurological examinations (Strub and Black, Reference Strub and Black1988).
(v) Disturbances in executive functioning: the first item was adapted from GPCOG (Brodaty et al., Reference Brodaty2002) and the second from Short IQCODE (Jorm and Jacomb, Reference Jorm and Jacomb1989).
The patient and informant versions of the VSID were translated (forward and back) into Tamil independently by two sets of bilingual health professionals. Consensus versions of the instruments were obtained. The assessment employed simple spoken Tamil.
The validation study
The study was conducted in the hospital and in the community and the details are briefly given here. Elderly patients attending the outpatient clinics of the Departments of Geriatric Medicine, Neurology and Psychiatry at the Christian Medical College, Vellore, India, during the period of the study formed the study population. The community sample consisted of the elderly population residing in Pennathur, Kaniyambadi Block, Vellore district.
Consecutive patients attending clinics at the hospital were contacted for possible recruitment into the study. The patients comprised those who were diagnosed clinically by the physician to have dementia using the Clinical Dementia Rating Scale; an equal number of patients without dementia were considered as controls.
The inclusion criteria for patients and for controls were: (i) age 65 years and above, (ii) conversant in Tamil, and (iii) accompanied by a reliable informant. The following were the criteria for exclusion: (i) presence of delirium, (ii) patients with hearing impairment, (iii) patients with visual impairment, and (iv) patients with wasting and weakness of hands. Patients had to meet DSM-IV criteria (American Psychiatric Association, 1994) for dementia, while controls did not meet such standards. Parkinson's disease and cerebrovascular accidents among controls were also criteria for exclusion.
In the hospital, patients and controls were referred to the investigator (RS) by their primary physicians based on the inclusion and exclusion criteria. Age and suitability for inclusion into the study were reconfirmed.
The details of the study were explained in Tamil to the patient and the accompanying relative. Written consent was obtained from all participants prior to inclusion in the study and sociodemographic details were recorded.
A physical examination, which included measurement of vital signs and a detailed neurological examination, was done to assess physical disabilities which could interfere with the performance on the tests. Hearing, vision and motor assessments were undertaken to ensure that the inability to perform the test items was not due to physical problems but due to the cognitive impairment. The informant was asked whether the cognitive impairment was of very recent acute onset. An affirmative answer was suggestive of probable delirium and hence such patients were excluded from the study.
The testing session began with a general conversation related to the subject's health. Sociodemographic data were also collected. This was followed by a “dummy test” (which was not scored) to help the subject comprehend the testing situation. The screening instruments were then administered, followed by the confirmatory interview, during both of which privacy was ensured. The required time for administration of the screening instruments was about 4–5 minutes each.
The following instruments were used to assess the elderly subjects and confirm the diagnosis:
(i) Geriatric Mental State (GMS) (Copeland et al., Reference Copeland, Dewey and Griffith-Jones1986). This is a standardized psychiatric interview, and its computerized diagnostic system, AGECAT, has been used and updated for over three decades. The Tamil version of the GMS has been employed in studies from the region (Rajkumar et al., Reference Rajkumar, Kumar and Thara1997).
(ii) Community Screening Instrument for Dementia (CSID) (Hall et al., Reference Hall1993). This was developed as a screening instrument for dementia for use in cross-cultural studies. It consists of two components: a cognitive test for non-literate and literate populations and an informant interview regarding performance in everyday living.
(iii) Modified CERAD 10 word list learning task (Ganguli et al., Reference Ganguli1996). This is the cognitive test proposed by the Consortium to Establish a Registry for Alzheimer's Disease (CERAD) and the Hindi version used in north India (Ganguli et al., Reference Ganguli1996). The Tamil word list consists of the Tamil words for butter, arm, letter, queen, ticket, grass, corner, stone, book, and stick, because these were deemed more appropriate in the local culture and language.
(iv) History and Aetiology Schedule Dementia Diagnosis and Subtype (HAS-DDS) (Copeland et al., Reference Copeland2002). This is part of the GMS-AGECAT package and is designed to clarify diagnosis into the sub-categories of AGECAT, ICD-10, and DSM-IV.
(v) The other assessments included: Neuropsychiatric Inventory (Cummings et al., Reference Cummings, Mega, Gray, Rosenberg-Thompson, Carusi and Gornbein1994), WHO Disability Assessment Scale II (WHODAS II) (World Health Organization, 2001), sociodemographic data, general physical, anthropometric and neurological examinations.
Research workers were trained in the use of instruments, interview techniques, data collection, data entry and analysis procedures. The diagnostic standards employed were based on computerized algorithms developed by the 10/66 Dementia Research Group (Prince et al., Reference Prince, Acosta, Chiu, Scazufca and Varghese2003; Reference Prince, Acosta, Chiu, Scazufca and Varghese2004).
The primary researcher who carried out the screening test for dementia (RS) was blind to the case/control status of the participants. The standard diagnostic interviews were administered by PSK and KG.
The computerized database of the Community Health and Development Program of the Christian Medical College, Vellore, was employed to obtain a census list of the residents of Pennathur village for the community sample. In addition, a door-to-door survey of the village was done to obtain a comprehensive list of eligible participants. The inclusion and exclusion criteria were similar to those used in the hospital population. All eligible subjects were administered the screening instruments followed by the diagnostic interview.
The details of the study were explained to patients/subjects and their relatives and consent for participation obtained. The study and procedures for obtaining informed consent were approved by the Institutional Review Board of the Christian Medical College, Vellore.
Analysis
One-step items of the patient screen (numbers 2, 3, 4, 5, 6, 7, 9) were scored 0/1 while 3-step items (numbers 1, 8, 10) were scored 0–3 depending on the steps completed. Hence the values ranged from 0 to 16.
All items of the informant version were scored 0/1. However, items 9 and 10 may not have been applicable if the patient was housebound. A detailed analysis of the data suggested minimal differences in the instrument's indices (sensitivity, specificity, area under the curve, Cronbach's α, split-half reliability, and predictive values) if such patients were given scores of 0, ½ or 1 for these items. Hence if these items were not applicable to the individual the scoring in the final model was 0.
The new screening instruments were validated against the standard of the confirmatory interviews. Sensitivity, specificity, positive and negative predictive values were calculated for the screening instruments. Receiver operator characteristic (ROC) analysis and the area under the curve were used to assess the optimal threshold for screening. Psychometric properties of the scales were assessed using Cronbach's α and Guttmans' split-half reliability. The indices of efficacy of tests in series and in parallel were calculated using standard formulae (Weinstein et al., Reference Weinstein, Obuchowski and Lieber2005). The statistical software SPSS for Windows Release 16 was employed for the analysis of data.
Results
Table 1 presents the sociodemographic and clinical characteristics of the patients and their informants in the hospital and community samples. The majority of patients and informants were poor and a significant proportion were illiterate. The number of subjects who refused consent for the study in hospital and in the community was 5 and 0, respectively.
Sociodemographic and clinical characteristics of the hospital and community sample of patients
*US$1 = INR 45.00.
Table 2 shows the psychometric properties and the indices of efficacy for the patient screen. The inter-rater reliability was good (intra-class correlation coefficient = 0.982; 95% CI, 0.929–0.996). The patient screen performed well in the hospital setting with its high prevalence of dementia and less well in the community context. The instrument performed better against the DSM-IV standard in the hospital setting while it did better against the education-adjusted 10/66 dementia standard in the community. However, the positive predictive values resulted in unacceptably high false positive rates especially in the community. The relationship between patient test scores and education was not statistically significant.
Psychometric properties and indices of efficacy of the VSID-P version in hospital and community samples using the DSM-IV and the education-adjusted 10/66 criteria for diagnosis of dementia
VSID-P = Vellore Screening Instrument for Dementia patient version.
Table 3 shows the psychometric properties and indices of efficacy for the informant screen. The informant version also performed well in the hospital setting and less well in the community context. The instrument performed better against the DSM-IV standard in the hospital setting, while it did better against the education-adjusted 10/66 dementia standard in the community. However, the positive predictive values resulted in unacceptably high false positive rates especially in the community. The inter-rater reliability was good (intra-class correlation coefficient = 0.995; 95% CI, 0.981–0.998). The relationship between informant version scores and education was not statistically significant.
Psychometric properties and indices of efficacy of VSID-I version in hospital and community samples using the DSM-IV and the education-adjusted 10/66 criteria for the diagnosis of dementia
VSID-I = Vellore Screening Instrument for Dementia informant version.
Table 4 presents the indices of efficacy when the patient screen and the informant versions were combined. Similar values were obtained when combined in parallel and in series using the “AND rule” (Weinstein et al., Reference Weinstein, Obuchowski and Lieber2005). The combination resulted in very good indices of efficacy with a significant improvement in the positive predictive values in both the samples and against both the gold standards used for comparison. The combination correctly predicted four out of five cases in the hospital setting. In the community, it could correctly predict one out three DSM-IV cases and half of the subjects diagnosed to have dementia using the education-adjusted 10/66 criteria.
Indices of efficacy using a combination of patient and informant screening tests with both screens positivea
a Both screen positive in parallel using the AND rule.
Discussion
This study is an attempt to develop a culture- and education-fair instrument for use in low- and middle-income countries, like India, which have low rates of literacy. Its strengths include the fact that it employed an elaborate procedure as gold standard to confirm the diagnosis of dementia, it used hospital and community samples with a range of cognitive abilities, and the fact that the person doing the screening was blind to the results of the detailed assessment.
The demands of screening pose formidable challenges and are briefly mentioned here. Screening instruments need to have increased sensitivity. However, such increase in sensitivity without high specificity results in high false positive rates and poor prediction. Similarly, high specificity with lower sensitivity also results in poor positive predictive values. The emphasis on high sensitivity of screening instruments implies that while all cases will be identified there will be a cost in terms of resources required to confirm the diagnosis during the second step of the two-step screen-and-confirm procedure. This is particularly true for screening for disorders like dementia in the community where the confirmatory procedure requires qualified psychiatric/medical professionals or very elaborate instruments when used by non-specialists. Consequently, screening instruments need to have both high sensitivity and high specificity. This is particularly true for low- and middle-income countries and for screening in the community with its low prevalence of dementia. While high specificity and sensitivity are essential for instruments to be useful in low prevalence settings, the need for brief and simple screens for use in the community places a ceiling on obtaining instruments which are both highly sensitive and specific.
The sensitivity and specificity of screening instruments are usually considered constant. However, changes in population characteristics (e.g. age, gender, socioeconomic status, severity and spectrum of cognitive function, literacy) can influence the sensitivity and specificity of tests. Some differences between settings are inevitable due to the variation in the composition of hospital and community samples.
The dependent nature of predictive values on prevalence makes screening in low prevalence setting particularly problematic. The low prevalence of dementia in the community means that screening everyone results in high false positive rates, placing enormous demands on the confirmatory procedure and on personnel doing the second step test. The mathematics means that such single-step screening is not really practical as it results in very high false positive rates (Jacob, Reference Jacob2003; Jacob et al., Reference Jacob, Kumar, Gayathri, Abraham and Prince2007). Ideally, screening in the community should involve a series of steps which raises the prevalence of the disorder in the subsequent stages, making them increasingly efficient and thus reducing false positives. In practice, this would mean setting up a referral system (i.e. health worker, public health nurse, general practitioner and specialist), as happens in most high-income countries, making the final results accurate. However, the absence of such a functioning tiered system has contributed to the failure of community psychiatry programs in low-income countries. The lack of resources therefore requires an increase in the efficiency of diagnostic screens by sequential testing, the use of multiple sources of information, and screening of individuals who report symptoms or whose informants complain of cognitive decline.
The many screening instruments currently available have a variety of problems, especially when used in societies with low rates of literacy, due to the fact that most tests employ items which are dependent on a certain level of education. The elderly, in particular, have low rates of literacy and these instruments are not suited for screening such individuals. Similarly, the diverse cultures have a major impact on the familiarity of the test contents and procedures, making them inappropriate for use in these settings. While many instruments designed in the west have attempted to reduce the impact of items on education levels, ethnicity and across cultures, and are said to be useful in diverse settings (Milne et al., Reference Milne, Culverwell, Guss, Tuppen and Whelton2008), the use of items which are region and culture specific (e.g. recall of information including dates, paper-and-pencil tests, complex verbal tasks, details related to finance and travel) makes it difficult for people without memory impairment living in less literate and non-western cultures to fare well on these tests. The patient and the informant versions developed at Vellore employed activities of daily living in order to be education- and culture-fair. These instruments are brief and yet systematically assess the multiple cognitive domains involved in dementia.
The VSID-P version performed well in the hospital setting with good sensitivity, specificity, area under ROC curve, Cronbach's α and Guttman's split-half reliability. Its false negative rate was very low and in the hospital sample tested it could correctly predict three out of five cases. However, in the community setting, with a prevalence of 10%, it performed poorly on all indices (except negative predictive value) with a very high false positive rate making it less than useful as a sole instrument for screening dementia in settings of low prevalence.
Similarly, the VSID-I version also performed reasonably in the hospital setting with adequate sensitivity, specificity, Cronbach's α, split-half reliability and the area under ROC curve. It performed well in the hospital sample, but its positive predictive values in the community sample was 43% against the 10/66 standard.
The positive predictive values were improved markedly by using the screening instruments in combination. The combination could predict one in two true positives against the 10/66 standard and one in three for the DSM-IV criteria in the community. The improvement occurred in both the hospital and the community setting but its impact in reducing the false positive rates will mainly be in the community and in low prevalence settings where the predictive ability of the individual screening instruments is low.
Most screening instruments have been validated in situations of high prevalence (i.e. hospital populations or selected community samples) and seem to perform reasonably well in such settings. However, their subsequent use in the community and low prevalence settings makes them inefficient in their ability to predict false positives, despite their reasonable sensitivity and specificity, as the positive predictive ability of screening tests are based on prevalence of the condition in the population tested (Jacob, Reference Jacob2003; Jacob et al., Reference Jacob, Kumar, Gayathri, Abraham and Prince2007). Nevertheless, the issues related to prevalence and its impact on predictive values are rarely highlighted when such instruments are used in low prevalence settings, such as the general population and in primary care. In fact, evaluations of the screening instrument have not considered the positive predictive values as they are low and dependent on prevalence (Brodaty et al., Reference Brodaty, Low, Gibson and Burns2006; Milne et al., Reference Milne, Culverwell, Guss, Tuppen and Whelton2008).
The dependent nature of predictive values on prevalence argues that improvements in diagnostic accuracy is more often achieved by artificially increasing the prevalence in the population tested (e.g. clinic or hospital-based testing, and in those who complain about or are brought in with significant clinical symptoms) (Jacob, Reference Jacob2003; Jacob et al., Reference Jacob, Kumar, Gayathri, Abraham and Prince2007) and much less because of improved indices of efficacy of screening tests or due to the skill of the person making the diagnosis.
The current practice of employing single instruments to screen for dementia in low prevalence settings results in high false positive rates. The two-step screening strategy described – using and combining data from informants and from subjects, and employing the “AND rule” (where only subjects positive on both tests are considered screen positive) (Weinstein et al., Reference Weinstein, Obuchowski and Lieber2005) – results in increased specificity (compared to the individual tests) and a consequent reduction of false positive cases. The one in two true positives against the 10/66 standard and one in three for the DSM-IV criteria are a substantial improvement when used to screen consecutive community samples with low prevalence of dementia. A further increase in prediction can be achieved by screening subjects only if they or their relatives complain of symptoms and/or are brought to clinics and primary care settings. Screening such subjects, where the prevalence of dementia is higher, will result in better prediction with a further reduction of false positive. These issues also apply to the screening and identification of other disorders in low prevalence settings.
Conclusion
The instruments developed are brief, systematically screen for dementia, are education- and culture-fair, and have good psychometric properties. The use of a combination of the patient screen and the informant version results in reducing the problem of high false positive rates in situations of low prevalence while the use of activities of daily living to test cognitive domains makes the instruments education and culture fair. This study, to the best of our knowledge, is the only one to rely solely on activities of daily living to assess subjects and informants in the community and describes their use in combination to increase the positive predictive values. The high sensitivity and the improved specificity contributed to a lowering of the false positive rate. Further reductions can be achieved by only screening populations with symptoms of cognitive impairment, which will thereby artificially increase the prevalence of the condition in the populations tested.
The results of the study will need to be replicated in larger and more diverse samples. There is also a need to reassess the usefulness of screening instruments and strategies for rare diseases in low prevalence settings.
Conflict of interest
None.
Description of authors' roles
Rena Stanley was involved in designing the study and in analysis. Rena Stanley, Senthil Kumar and K. Gayathri collected data. Anju Kuruvilla was involved in designing the study, supervised data collection and helped with analysis. Prasad Mathews and Vinod Abraham supervised data collection. Anto Rajkumar helped with analysis. K. S. Jacob designed the study, analyzed the data and wrote the paper. The original and revised manuscripts were reviewed by all the authors.
Acknowledgments
The authors would like to thank the subjects and their relatives for participating in the study. They would also like to thank Professor M. Alexander for permission to include his patients in this study. This study was funded by a grant (No. 22 X 118) from the Christian Medical College, Vellore.
Appendix 1 The Vellore Screening Instrument for Dementia – Patient version
1. I went to Chennai and brought back a mango, a chair, and a coin. Can you tell me the three things I brought from Chennai?
I want you to remember the three items I brought from Chennai. I will ask you to recall them after sometime.
2. Look at my face and do exactly as I do. (The instructor closes his/her eyes for 2 seconds and then opens them.)
3. Can you tell me a sentence about your house?
4. Can you show me how you light a lamp?
5. Can you show me how you comb your hair?
6. Show the subject a KEY and ask “Can you tell me what this is?”
7. Close your eyes. I will put an object in your hand. Can you tell me what is in your hand?
8. Can you take this paper with your right hand, fold it into two and put it down on the left side of the table?
9. Can you open this lock?
10. Can you recall the three things that I bought from Chennai?
Appendix 2 The Vellore Screening Instrument for Dementia – Informant version
1. Does he/she regularly forget events that have happened recently?
2. Does he/she have trouble remembering where he/she has kept her belongings?
3. Does he/she regularly have difficulty in finding the right words or does he/she use the wrong words in conversation?
4. Does he/she regularly have difficulty understanding what is said to him/her?
5. Does he/she regularly have difficulty in dressing appropriately?
6. Does he/she urinate in the appropriate place?
7. Does he/she have difficulty recognizing familiar faces?
8. Does he/she have difficulty recognizing familiar objects like a key, comb and spoon?
9. Is he/she able to go to the market to buying things as before?
10. Has he/she ever got lost in the village/town?
