Introduction
China's aging society and increasing population with dementia (Flaherty et al., Reference Flaherty2007) imposes a heavy burden on families and society (Sousa et al., Reference Sousa2009; Ken et al., Reference Ken, Okuno, Fukasaku, Tomura and Yanagi2010). The early detection of cognitive decline can help patients and families prepare for the future, and assist clinicians to identify patients at risks for delirium or driving mishaps (Molnar et al., Reference Molnar, Patel, Marshall, Man-Son-Hing and Wilson2006; Alagiakrishnan et al., Reference Alagiakrishnan2007). Early detection also provides an opportunity for treatment to improve function, behavior, and cognition (Aranda-Abreu et al., Reference Aranda-Abreu, Hernandez-Aguilar, Manzo, Garcia and Herrera2011). In this context, cognitive screening represents an initial step in the process of further assessment for dementia and can help in identifying potential cases for management. General practitioners (GPs), as an initial and main source of medical care for the elderly, assume an important role in this evaluation process.
Evidence from western countries suggests that the majority of dementia patients often remain undetected in general practice (Brodaty et al., Reference Brodaty, Howarth, Mant and Kurrle1994, Connolly et al., Reference Connolly, Gaehl, Martin, Morris and Purandare2011). One study found that GPs missed up to 91% of mild dementia patients (Valcour et al., Reference Valcour2000). Underdetection may be even greater in China, as professional training and continuous education for clinicians are less well organized (Xiao et al., Reference Xiao2010). Widely used screening tests, such as the Mini-Mental Sate Examination (MMSE; Folstein et al., Reference Folstein, Folstein and McHugh1975) and the Hasegawa's Dementia Scale (HDS; Hasegawa, Reference Hasegawa1979), fail to satisfy busy GPs, as they require training and the tests are time-consuming (Brodaty et al., Reference Brodaty, Howarth, Mant and Kurrle1994; Reference Brodaty2002; Xiao et al., Reference Xiao2010). In addition, some factors such as the patient's education can influence test scores, which compromise diagnostic accuracy (Malloy et al., Reference Malloy1997; Kim et al., Reference Kim2011). Thus, an easy and reliable screening instrument for GPs is needed.
The General Practitioner Assessment of Cognition (GPCOG) was developed to assist GPs in detecting dementia (Brodaty et al., Reference Brodaty2002). It was recommended as one of the best dementia screening instruments for GPs to use for its sound psychometric properties, quick administration time, easy use, and acceptance by GPs and their patients (Brodaty et al., Reference Brodaty, Low, Gibson and Burns2006). A further review concluded that the GPCOG was a high-quality and time saving tool for primary care, which was free of many biases common in other screening tests (Milne et al., Reference Milne, Culverwell, Guss, Tuppen and Whelton2008). The GPCOG, however, has not been used in Mainland China.
This paper aims to evaluate the reliability and validity of the Chinese version of the GPCOG (GPCOG-C) in community-dwellers and hospital outpatients. We further aimed to explore its diagnostic utility and advance value by comparing it to other commonly used screening scales (MMSE and HDS).
Methods
Setting
Participants for this research were either patients of Waitan Hospital, the psychogeriatric department of the Shanghai Mental Health Center, or members of the Waitan community.
Waitan Hospital is a large primary care facility in the central Shanghai district of Waitan. Its catchment area has an older population (i.e. 60 years or older) of about 27,000 people, which is about a quarter of the district's total population (Shanghai Civil Affairs Bureau, 2011). Waitan Hospital currently employs 62 GPs and provides general medical inpatient and outpatient care to the residents of the Waitan area. GPs who participated as raters in this research were based at Waitan Hospital. Approximately 80% of their patients are aged 60 years or over.
The psychogeriatric department at the Shanghai Mental Health Center lies within a five-mile radius from Waitan Hospital and currently employs eight geriatric psychiatrists or psychogeriatricians. Together they examine 20–30 outpatients every day. Between 50% and 60% of outpatients present with and seek treatment for memory problems. No referrals are needed for consultations and assessments.
Recruitment process
The study was advertised through notice boards at the Shanghai Mental Health Center; Waitan Hospital; and the Waitan community centre. Invitations and calls for participants who were concerned about their memory were also placed on the Waitan community bulletin board, which is frequently checked and read by elderly community. Individuals who responded to advertisement were invited for assessment at one of the sites.
Inclusion/exclusion criteria
In order to be eligible, participants needed to be aged between 50 and 90 years and in good physical health. We excluded individuals with acute or unstable psychiatric disorders (such as major depression) as well as patients suffering from delirium. We further excluded individuals with vision or hearing impairment if their disability prevented them from completing parts of the study. A history of stroke or other neurological condition (e.g. Parkinson's disease) did not exclude individuals from the study as long as physical or sensory impairment did not prevent them from completing study components, nor did chronic conditions such as hypertension or diabetes.
Participants
With each participant, we recruited a family member (i.e. spouse or adult child) as his/her informant. Of the 414 initially eligible participants, 58 were excluded due to poor vision or hearing upon testing (11 participants), severe anxiety (6), or lack of a suitable informant (41). The remaining 356 individuals were enrolled into the study; 253 of whom were community-dwelling individuals with memory concerns who attended Waitan Hospital or responded to advertisements in the community; and 103 were outpatients of the Psychogeriatrics Department at the Shanghai Mental Health Centre, seeking treatment for their memory problems.
Raters and inter-rater consistency
Four GPs and six junior psychogeriatricians acted as raters in this research. They had worked as physicians for an average of 9.9 years (SD = 6.7). Their mean age was 34.2 years (SD: 7.7; range: 25–48). Prior to the commencement of study, all raters received thorough training in administering the MMSE, HDS, and GPCOG-C. Their inter-rater reliability (IRR) was assessed through simultaneous scoring of participants by at least two raters. The overall consistency between raters as measured by Kendall's W coefficient was 0.96 (p < 0.01).
Study procedure
Study participants were administered the GPCOG-C, the MMSE, and the HDS by one of the ten raters (see above). Subsequently, each participant and his/her informant were interviewed by one of the four senior psychogeriatricians (50% were females) who were qualified as doctors for an average of 13.8 years (SD = 4.2) and were blinded to the results of the GPCOG-C, the MMSE and the HDS. During this interview the senior psychogeriatrician obtained the following information from the participant and/or the informant: a detailed history of memory problem, a detailed medical and psychiatric history, current medications, family's medical history, physical examination, and brain CT or MRI (if available). During this interview, the senior psychogeriatrician also assessed the participant's cognition and functional abilities. Additional information was gathered, if needed, to obtain a diagnosis. Dementia diagnoses were based on the psychogeriatrician's clinical judgment according to the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) criteria. In the case of diagnostic uncertainty, patients were reviewed by a second senior psychogeriatrician and ultimately referred for review to the director of the psychogeriatric department of the Shanghai Mental Health Centre.
Participants and informants received no monetary reimbursement for their participation. All participants and/or their legal guardians gave written informed consent. Approval to conduct this study was obtained from the Ethical Review Board of the Shanghai Mental Health Center.
Measurements
The GPCOG-C, like its original English version by Brodaty et al. (Reference Brodaty2002), consists of two sections: a patient examination (GPCOG-patient) with four cognitive test items (maximum score = 9), and an informant interview with six items (maximum score = 6). The patient examination contains the following items: time orientation (1 point), clock drawing (2 points), reporting of a recent event (1 point), and a word recall task (5 points). Each correct answer scores one point, and higher scores reflect less impairment. The informant interview (GPCOG-informant) assesses the patient's function by asking six questions, such as “Does the patient have more difficulty in remembering recent issues than he/she used to?” Each question, which is negated, scores one point. According to the sequential two-stage scoring method (Brodaty et al., Reference Brodaty2002), the informant interview is only administered if the patient scores 5–8 on the cognitive testing. Using this approach, scores of 3 or less on the informant section indicate cognitive impairment. The informant interview can be omitted if the patient scores 9 (i.e. no cognitive impairment) or 4 or less on the patient assessment (i.e. indicates cognitive impairment).
The process of translating the original English GPCOG to Chinese is shown Figure 1.
Flowchart of translation process.
The MMSE, a 19-item screening instrument for detecting cognitive impairment (Folstein et al., Reference Folstein, Folstein and McHugh1975), is widely used in clinical and scientific research. In a recent meta-analysis (Mitchell, Reference Mitchell2009), the MMSE's pooled sensitivity was 80% across memory clinic settings and 83% in primary care/community settings. The specificity was slightly higher with 81% and 87% for the respective settings. Due to its widespread use, the MMSE is most useful when communicating disease severity to other clinicians (Holsinger et al., Reference Holsinger2007), but has been criticized for demonstrating demographic biases (i.e. education) or floor and ceiling effects toward well-educated, high functioning individuals and severe impairment respectively (Mitchell, Reference Mitchell2009).
The HDS, developed by Hasegawa in Japan in 1974 (Hasegawa, Reference Hasegawa1979), was introduced in China in the 1980s (Tsai and Gao, Reference Tsai and Gao1989). Due to cultural similarities between China and Japan and its brief administration time, it has since been widely used in China. The correct classification rate between the HDS cut-off score of 20 and a clinically derived DSM dementia diagnosis reached 96% in the initial publication (Tsai and Gao, Reference Tsai and Gao1989). A comparison between the MMSE and the HDS in a Korean study led the authors to conclude that the HDS was significantly superior to the MMSE in detecting Alzheimer's disease (Kim et al., Reference Kim2005).
Statistical analyses
The statistical package SPSS for Windows (release 11.0; SPSS, Chicago, IL) was used for data analyses. The sensitivity, specificity, and areas under receiver operating characteristic (ROC) curves were calculated. ROC analyses were applied to assess the GPCOG patient section, informant section, total score, and sequential two-stage administration in their ability to detect DSM-IV-defined dementia. The ROC curve is constructed by plotting the true-positive ratio against the false-positive ratio for each possible cut-off point of the test. The area under this curve (AUC) therefore represents the probability that dementia patients and patients not having dementia are correctly classified according to the DSM-IV dementia diagnoses. χ 2 tests were used to compare AUCs for the GPCOG-patient and informant subscales, the GPCOG total score and two-stage test, and the HDS and MMSE.
Results
Demographics
Demographic details of the 356 participants who completed the study are shown in Table 1. A diagnosis of DSM-IV-defined dementia was made in 78 out of 356 participants (22%), of whom 31 (40%) were males (Table 1). Compared with those without dementia, patients with dementia had significantly lower GPCOG-patient scores (SD = 7.68 ± 1.72 and 2.05 ± 1.63 respectively; t = −25.48, p < 0.01) as well as lower GPCOG-informant scores (SD = 5.18 ± 1.57 and 0.28 ± 1.00 respectively, t = −32.96, p < 0.01). The dementia group was older than the cognitively healthy group (SD = 75.22 ± 8.23 vs. 71.78 ± 8.93, t = −3.03, p < 0.01). No significant differences were found regarding education and gender.
Demographic information of participants (n = 356)
Notes: M: mean, SD: standard deviation, n: sample size, GPCOG-C: General Practitioner Assessment of Cognition-Chinese Version; MMSE: Mini-Mental State Examination; HDS: Hasegawa's Dementia Scale.
Reliability
For the GPCOG patient section, internal consistency (Cronbach's α) was 0.68, the correlation coefficient between every item and the total score of the scale ranged from 0.50 to 0.94, and the correlation coefficient among items ranged from 0.41 to 0.61 (Table 2). Thirty-one participants were again administered GPCOG-C after two weeks. The test–retest correlation coefficients ranged from 0.86 to 0.98 (Table 3).
Correlation coefficient among items and between every item and total score
Test–retest correlation coefficient
Validity
The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and misclassification rates were examined for the GPCOG total, the sequential two-stage method, the patient and the informant sections separately, the MMSE, and the HDS (Table 4). For the purpose of this validation study, the GPCOG informant section was completed for every participant, thus the GPCOG total score included the actual patient and informant scores. However, in order to analyze the validity of the sequential two-stage method (i.e. where informant data are only obtained for patients scoring between 5 and 8 on cognitive assessment), informant scores were imputed to be the maximum of 6 or the minimum of 0 for patients whose cognitive scores were more than 8 or less than 5 respectively. For those who scored between 5 and 8, the informant score was the actual score obtained.
Sensitivity, specificity, and AUC for GPCOG-C patient and informant sections, GPCOG-C total score, GPCOG-C “two-stage” method, and MMSE against DSM-IV dementia diagnosis
The sensitivity of the GPCOG patient section was high (97%), while its specificity was lower (66%). Addition of informant interview increases specificity from 86–89% (Table 4). Compared with the informant section interpreted on its own, the total score and the two-stage method each resulted in reduced misclassification rates and increased specificity without influencing the sensitivity (Table 4).
Table 4 also lists the PPV and NPV for GPCOG components, the MMSE, and the HDS. While the NPVs for all GPCOG scores were slightly higher than those of MMSE and HDS scores, the PPVs were lower, particularly if the GPCOG-patient score was interpreted in isolation.
PPVs were particularly affected by the prevalence of the disease/symptom in the studied population. As can be seen in Table 1, prevalence of dementia among community-dwelling participants (10%) was substantially lower than in outpatients with memory complaints (51%). For 253 community-dwelling participants, the PPV was 53% and the NPV was 99.5%, while for 103 outpatients, PPV and NPV were 88% and 98% respectively.
The ROC curves showed very similar slopes (Figure 2). ROC analysis showed no significant differences between AUCs, suggesting similar diagnostic power of the GPCOG-C total and two-stage method compared with the MMSE and the HDS (Table 4).
The receiver operating characteristic (ROC) curves for each test as screening for dementia.
(GPCOG-C: General Practitioner Assessment of Cognition – Chinese Version).
Administration time
The time required to administer the GPCOG-C and the MMSE was measured for 20 participants. The administration time of the GPCOG (two-stage method) was significantly shorter than that for the MMSE (4.3 ± 2.4 min vs. 6.5 ± 2.1 min, p < 0.05). The individual administration time for the GPCOG-patient and GPCOG-informant interviews were 3.5 min (±1.5) and 2.2 min (±1) respectively.
Discussion
The GPCOG is a screening test for dementia designed for general practice (Brodaty et al., Reference Brodaty2002). The data obtained in this study strongly indicate that the Chinese version of the GPCOG maintains the advantageous characteristics of the original English version, proven to be an efficient, reliable, simple, and time-saving instrument for dementia screening (Brodaty et al., Reference Brodaty2002; Reference Brodaty, Kemp and Low2004).
Our study showed that Cronbach's α was 0.68. Considering that the GPCOG is so brief and yet consists of four different tasks that broadly assess cognitive abilities, the correlation coefficients between every item and the total score as well as the correlation coefficients among items of the scale were satisfactory. The high test–retest reliability demonstrated that the GPCOG-C was quite stable and probably less affected by intra-personal fluctuations, which is important for clinical practitioners.
The diagnostic accuracy of the Chinese GPCOG was similar to that reported originally (Brodaty et al., Reference Brodaty2002). The sensitivity and specificity of the English GPCOG, established for a sample of 283 community-dwelling elderly people, was 85% and 86% respectively (Brodaty et al., Reference Brodaty2002). A study of the French GPCOG, which included 280 participants, showed a sensitivity of 96% and a specificity of 62% (Thomas et al., Reference Thomas, Hazif-Thomas, Vieban, Faugeron, Peix and Clement2006). Lastly, Pirani and colleagues (Reference Pirani, Brodaty, Martini, Zaccherini, Neviani and Neri2010) demonstrated a sensitivity of 82% and a specificity of 92% for the Italian GPCOG. The sensitivity and specificity values in our study involving 356 participants were 97% and 89% respectively, slightly higher than for the aforementioned studies. This could be due to differences in raters or sampling. For example, unlike other studies with 100% GP raters, six out of ten raters in our research were junior psychogeriatricians, who were well trained and experienced in dementia screening. At the same time, prevalence of dementia among the outpatients with memory complaints was quiet high (51%), which would have affected the diagnostic accuracy of any screening instrument. Also, those who volunteered in response to the advertisements may well have had memory complaints. However, to counteract this limiting fact, we analyzed the diagnostic accuracy for the total sample, which had a combined dementia prevalence (i.e. 22%) similar or lower to that of the English or Italian version (Brodaty et al., Reference Brodaty2002; Pirani et al., Reference Pirani, Brodaty, Martini, Zaccherini, Neviani and Neri2010).
Our results also highlighted that the diagnostic performance of the GPCOG-C was very good (AUC = 0.97) and did not differ significantly from the MMSE and HDS. The ROC curves were very similar and were consistent with the results of the original (Brodaty et al., Reference Brodaty2002) and Italian GPCOG (Pirani et al., Reference Pirani, Brodaty, Martini, Zaccherini, Neviani and Neri2010). We were thus able to confirm our hypothesis that the GPCOG-C is an efficient screening tool capable of differentiating between dementia patients and patients without dementia, and possesses the same diagnostic value as other widely used tests.
Only 1% of the patients who were identified by the GPCOG-C as being not impaired met the DSM-IV diagnostic criteria for dementia (NPV = 0.99). By comparison, both MMSE and HDS provide less certainty (NPVs = 96% and 94% respectively). This should reassure clinicians, as it means that with 99% chance a negative GPCOG result (i.e. no impairment detected) is a true negative result. At the same time, 28% of the patients who the GPCOG-C identified as having cognitive impairment did not meet the diagnostic criteria for dementia (PPV = 0.72). This rate was even higher when the GPCOG patient score was interpreted in isolation (PPV = 0.44). While this is consistent with previous GPCOG data (Brodaty et al., Reference Brodaty2002; Thomas et al., Reference Thomas, Hazif-Thomas, Vieban, Faugeron, Peix and Clement2006; Pirani et al., Reference Pirani, Brodaty, Martini, Zaccherini, Neviani and Neri2010), it also highlights the importance of following up any positive GPCOG results with a more thorough assessment. Thus, the strength of the GPCOG-C is to rule out dementia patients while flagging those who may require further investigations and follow-up.
We also note that the misclassification could in parts be due to participants with mild cognitive impairment (MCI) who showed some impairment on the GPCOG-C but failed to meet the DSM-IV dementia criteria (15 participants). However, establishing the diagnostic accuracy of the GPCOG-C against a diagnosis of MCI was beyond the scope of this study. We encourage future research to evaluate this though.
Another advantage of the GPCOG is the incorporation of informant interview, which represents a unique feature compared to other common screening tools. The combined or sequential administration of the two components increases the specificity and positive predictive value compared with the GPCOG-C patient assessment performed on its own. This means for clinicians that by conducting the informant interview they are decreasing the rate of false positives and are obtaining a more reliable snapshot of their patient's abilities.
Furthermore, by applying the sequential two-stage method, the administration time of the GPCOG-C could be reduced considerably. Compared with the MMSE, GPCOG-C's two-stage procedure was more time-efficient. The GPCOG administration is significantly shorter while demonstrating the same level of diagnostic accuracy than longer tests. This might mean that it is more acceptable among patients and GPs. In addition, as per the two-stage method, only 64% of all GPCOG-C administrations required the informant section to be completed because a number of patients either scored below 5 or above 8, which deemed informant questioning unnecessary.
Limitations
While this research certainly has its strengths, it is important to also highlight its limitations. As outlined above, prevalence of dementia in our participants, who were outpatients with memory complaints and actively sought treatment for their memory problem (n = 103), was substantially higher than for the overall sample (i.e. 51% vs. 22%). We combined all participants in an effort to increase statistical power, acknowledging that high dementia prevalence in this group would have affected the diagnostic performance of the GPCOG-C (and any other screening test). Psychometric properties are likely to be less impressive in sequential attendees in general practice.
We further note that senior psychogeriatricians who diagnosed dementia according to DSM-IV criteria used their clinical judgment rather than a pre-specified diagnostic algorithm or neuropsychological test results. However, they were all very experienced psychogeriatric specialists (having between 8 and 20 years experience) and obtained the same set of information for every participant. Although certain test results (i.e. CT/MRI of the brain) were only available for a subset of participants, we argue in our defense that senior psychogeriatricians would have particularly requested these tests when the standard information obtained was insufficient to diagnose dementia. At the same time, we note that this represents inconsistency of our data collection process.
In conclusion, our study demonstrated that the GPCOG-C is brief, efficient, reliable, and valid. Our results are in line with other research on the GPCOG. As not all raters in our study were GPs, further investigations on the GPCOG-C will be carried out involving more GPs to explore its characteristics.
Conflict of interest
None
Description of authors’ roles
Xia Li supervised the data collection and wrote the paper; Shifu Xiao and Henry Brodaty formulated the research questions; Xia Li, Shifu Xiao, Henry Brodaty, and Katrin Seeher designed the study; Minjie Zhu and Tao Wang collected the data; Xia Li and Yuan Fang performed the statistical analyses; and Katrin Seeher and Yuan Fang assisted with writing the paper.
Acknowledgments
This work was supported by grants from National Major Project for IND (No. 2012ZX09303-303), National Natural Science Foundation of China (No. 81101007), and the Shanghai Science & Technology Committee (No. 09411965600).
