Introduction
General practitioners (GPs) are frequently the first health professionals contacted by older people and/or their family/carer(s) when symptoms of dementia cause concern (Speechly et al., Reference Speechly, Bridges-Webb and Passmore2008). Dementia is a complex and variable condition (Hansen et al., Reference Hansen, Hughes, Routley and Robinson2008; Palmer et al., Reference Palmer2008a), which makes recognition of it particularly difficult in a low prevalence primary care setting (Pentzek et al., Reference Pentzek2009a). In the early stages, GPs need to make the often challenging distinction between normal aging, mild cognitive impairment, early dementia, and/or cognitive impairment associated with depression, delirium, or drugs (Pond and Brodaty, Reference Pond and Brodaty2004). Timely detection and earlier recognition of dementia is desirable to optimize patient care and carer well-being (Phillips et al., Reference Phillips, Pond and Goode2011). Conversely, a false-positive diagnosis of dementia may result in medical over-investigation and over-treatment, lack of attention to manageable conditions such as depression, and unnecessary distress to patients and families.
About 50% of patients older than 65 years with dementia are not diagnosed by GPs (Iliffe et al., Reference Iliffe2009b) for various reasons which include system characteristics and GP-related and patient-related factors (Koch and Iliffe, Reference Koch and Iliffe2010). GPs’ recognition of dementia appears to be associated with dementia severity (as measured by cognitive function tests and Instrumental Activities of Daily Living functioning; Wind et al., Reference Wind, van Staveren, Schellevis, Jonker and van Eijk1994; Pentzek et al., Reference Pentzek2009b) regardless of potential confounders such as depression, other illness, and recent events (Wind et al., Reference Wind, van Staveren, Schellevis, Jonker and van Eijk1994; van Hout et al., Reference van Hout, Vernooij-Dassen and Stalman2007). People with mild dementia and those who live alone are less likely to be diagnosed by their GP (Pentzek et al., Reference Pentzek2009b). Presumably those living alone lack informant reports which have been shown to be valuable in recognizing patients’ cognitive deficits (Jorm, Reference Jorm2004).
GPs also need to be cautious not to over-diagnose dementia. Factors associated with false-positive judgments of dementia include higher age, patients’ mention of memory complaints, GP-documented depression, lower education levels, and physical problems (e.g. mobility and hearing; Pentzek et al., Reference Pentzek2009a; Reference Pentzek2009b). Patient gender can also influence GP judgment of dementia, although findings are inconsistent and the effects are generally not marked (Wind et al., Reference Wind, van Staveren, Schellevis, Jonker and van Eijk1994; van Hout et al., Reference van Hout, Vernooij-Dassen and Stalman2007).
Patient mention of subjective memory complaints (SMC) can influence GPs’ judgments, contributing to correct identification of dementia (Wind et al., Reference Wind, van Staveren, Schellevis, Jonker and van Eijk1994) but also to over-diagnosis (Pentzek et al., Reference Pentzek2009b). Whilst patients’ mention of SMC may place them at higher risk of being falsely judged as having dementia, patients with cognitive problems may not necessarily raise these as an issue with their GP (Waldorff et al., Reference Waldorff, Rishoj and Waldemar2008). People with early dementia can lack insight into their decline or may attribute symptoms such as memory problems to “normal aging” (Clare, Reference Clare2003; Waldemar et al., Reference Waldemar2007; Campbell et al., Reference Campbell2008; Iliffe et al., Reference Iliffe, Jain and Wilcock2009a). Controversy exists in the literature about the merit of patients’ reports of SMC as a predictor or precursor of dementia (Reid and MacLullich, Reference Reid and MacLullich2006; Coley et al., Reference Coley, Ousset, Andrieu, Matheix Fortunet and Vellas2008; Mitchell, Reference Mitchell2008; Palmer et al., Reference Palmer2008a; Reference Palmer2008b; Pentzek et al., Reference Pentzek2009b), partly due to the range of non-standardized methods used to measure SMC, and their consistent association with depression and some personality traits (Reid and MacLullich, Reference Reid and MacLullich2006). Although SMC appear to be predictive of future cognitive decline and/or dementia (Reid and MacLullich, Reference Reid and MacLullich2006), these have modest diagnostic value and should not be relied upon for case-finding (Mitchell, Reference Mitchell2008; Palmer et al., Reference Palmer2008b). However, as a brief method of excluding healthy people in low prevalence settings, SMC seem to work reasonably well, that is, they exclude healthy elderly people from those with dementia about 19 times out of every 20 non-cases (Mitchell, Reference Mitchell2008).
It has been suggested recently that the process of dementia diagnosis should be split into four diagnostic steps (the trigger phase, disease-oriented diagnosis, care-oriented diagnosis, and carer assessment) followed by a monitoring phase (Buntix et al., Reference Buntix, De Lepeleire, Paquay, Iliffe and Schoenmakers2011). In this case the first stage of dementia diagnosis is not testing, but considering the possibility that dementia may be emerging. GPs’ suspicion and recognition of dementia rests on their clinical judgment based on their objective and subjective impressions from personal observation along with information from the patient and/or the patient's family/carer(s). Making the correct diagnosis is not always easy; many factors may interplay and can confound decisions.
We aimed to use data from a larger study to examine predictors (in terms of GP and patient factors) of GP agreement with an objective assessment of cognitive impairment, i.e. the revised Cambridge Cognitive Assessment (CAMCOG-R).
Methods
Participants
This study utilized baseline data collected from GPs and their patients as part of the “Ageing in General Practice” study (Pond et al., Reference Pond2012). Briefly, GPs and their patients were recruited from one rural and four metropolitan Australian sites: Sydney (NSW), Melbourne (Victoria), Adelaide (South Australia), Newcastle (NSW), and Bendigo (Victoria). GP practices within each study location were allocated a random approach order, and invited to participate in the study via phone. GPs who expressed an interest in participation were visited in order for the study to be explained. Basic demographics were collected at this visit from those GPs who agreed to participate.
Patients were recruited by a mail-out from each consenting GP to all community-dwelling, English speaking patients aged 75 or older on their database. Patients with Parkinson's disease, multiple sclerosis, motor neuron disease, central nervous system inflammation, pre-existing psychotic symptoms, developmental disability, a history of substance abuse, or a progressive malignancy were excluded.
GP audit
GPs were sent a list of their participating patients by fax or email. They were asked to confirm each patient's eligibility for the study and to provide their clinical judgment in relation to each patient's dementia status using one of four options: no dementia, possible dementia, probable dementia, or definite dementia. GPs returned their completed audit forms to the local project officer via mail, fax, or email. For any patients with a clinical judgment of dementia, GPs were asked whether the patient had completed a formal cognitive function test or had been referred to a specialist. In Australia, specialist attendance is not reimbursed unless the patient has been referred by their GP.
Patient assessment
After receipt of the completed audit from the GP, patients were visited at home by a research nurse who collected demographic data from all consenting patients and assessed their cognitive function and depression status using the instruments described below. Patients were asked the following questions: “Do you have any complaints about your memory? Have you mentioned these to your GP?”
Revised cambridge cognitive examination
Cognitive function was assessed using the CAMCOG-R subsection of the revised Cambridge Examination for Mental Disorders of the Elderly (CAMDEX-R; Roth et al., Reference Roth, Huppert, Mountjoy and Tym1998). The CAMCOG-R consists of 68 questions covering seven domains: orientation (10 items), language (17 items), memory (13 items), attention/calculation (4 items), praxis (8 items), abstract thinking (4 items), and perception (3 items). The highest possible score is 104, with a cut-off point of 79/80 indicative of dementia in lower scorers with 93% sensitivity and 87% specificity (Huppert et al., Reference Huppert1996). The authors recognize that interpretation of CAMCOG-R scores around the cut-off is problematic. Although CAMCOG-R score is a significant predictor of dementia, it is insufficient for a clinical diagnosis of dementia (van Hout et al., Reference van Hout2001). For the purposes of this study, a CAMCOG-R score of 79 or less was used as a sensitive and reasonably specific indicator of dementia.
Geriatric depression scale
Depression was measured using the short form 15-item GDS developed by Sheikh and Yesavage Reference Sheikh, Yesavage and Brink(1986), that is widely used in primary healthcare settings for assessment of depression in the elderly people (de Craen et al., Reference de Craen, Heeren and Gussekloo2003). Scores above 5 on the short form of the GDS are considered to indicate depression, with 6 to 10 indicative of mild to moderate depression, and 11 to 15 indicative of severe depression (Friedman et al., Reference Friedman, Heisel and Delavan2005). A cut-off score of 5/6 was used in this study to distinguish those without depression from those participants with mild to moderate or severe depression.
GP agreement with CAMCOG-R
The clinical judgment of the GP was classified as being in agreement with the CAMCOG-R if their dementia audit indicated (i) “no” for patients with a CAMCOG-R score greater than 79, or (ii) “yes,” “possible,” or “probable” for their patients that scored less than 80 on the CAMCOG-R. The rationale for this categorization was that it was the most clinically relevant option: a judgment of “possible” dementia indicated that the patient had been flagged by the GP for surveillance (albeit informal) of worsening condition – the “trigger phase” of a dementia diagnosis (Buntix et al., Reference Buntix, De Lepeleire, Paquay, Iliffe and Schoenmakers2011) had occurred.
Statistical analyses
Continuous data were summarized using the mean and standard deviation (SD). Categorical data were summarized as counts and percentage of agreement within each category. Consistency of agreement between the CAMCOG-R classification of dementia and GP suspicion or recognition of dementia was tested using Cohen's κ. Logistic regression was used to examine the predictors of agreement of GPs’ diagnoses and CAMCOG-R classifications separately in participants with and without CAMCOG-R dementia. The logistic regression model was fitted within a generalized estimating equation (GEE) framework to adjust for the clustering of patients within GPs.
Ethics approval
Ethics approval was sought and granted initially from the Newcastle University Human Research Ethics Committee (Approval No. H-151-1205), and following this, from the appropriate Ethics Committees at each site.
Results
A total of 2,028 community-dwelling patients aged 75 years or older were recruited via 169 GPs. The response rate was 6% for GPs and 19% for patients. Home assessment of cognitive and depression status (using CAMCOG-R and GDS) was completed for 2,024 patients. The dementia prevalence based on a CAMCOG-R cut-off score of 79/80 was 8.2%. Mild depression was present in 6.7% of patients (GDS score 6–10) and severe depression in 0.5% of patients (GDS score 11–15). Approximately 20% of patients indicated that they had expressed concerns about their memory to their GP (Table 1).
Characteristics of patient and GP participants
GP = General practitioner; CAMOG-R = revised Cambridge Cognitive Assessment; GDS = Geriatric Depression Scale.
aIndex of relative socioeconomic advantage and disadvantage (scale 1–10, with 10 being most advantaged).
bExtra qualifications in mental health or geriatrics.
The participating GPs (n = 165) had between 1 and 56 patients in the study. Four GPs were not included in the study, as none of their patients consented or completed the home assessment. A GP audit of dementia status was completed for 1,974 (97.5%) patients (Table 2). Of the 164 patients with CAMCOG-R dementia, 44.5% were identified by GPs as having dementia: 15.9% as “definite,” 9.1% as “probable,” and 19.5% as “possible” dementia. GPs reported using a pencil and paper test of cognitive function for 63 of the 254 (24.8%) patients whom they judged to have dementia, and had referred 23 patients (9%) to a memory specialist.
GP judgment of dementia status compared to CAMCOG-R result
aPatients who scored < 80 on the CAMCOG-R were classified as having dementia.
CAMCOG-R = revised Cambridge Cognitive Assessment.
GP audit for dementia
There was fair agreement between the CAMCOG-R and the clinical judgment of GPs in relation to the dementia status of patients, with GPs correctly identifying 73 (44.5%) patients with and 1,629 (90%) patients without CAMCOG-R dementia (κ = 0.276). Using our definition, (definite, probable, and possible dementia) GPs also identified dementia in 181 people (11%) without CAMCOG-R dementia. This resulted in a positive predictive value (PPV) of 73/254 (28.7%) for definite, probable, or possible dementia. That is, only 28.7% of patients the GPs identified as having dementia were identified as such on the CAMCOG-R assessment. The sensitivity and specificity of GP judgment in relation to patient dementia status were 0.45 and 0.90, respectively.
The PPV of a GP judgment of “definite” dementia was 46%, and dropped to 31% and 21% for judgments of “probable” and “possible,” respectively, substantiating lower levels of certainty amongst the GPs.
Patients with CAMCOG-R dementia
Two patient-dependent factors were most important for predicting agreement between the CAMCOG-R and GP judgment in patients with dementia: the CAMCOG-R score (p = 0.006) and patient mention of SMC to the GP (p = 0.040; Table 3). For dementia patients, the CAMCOG-R score was significantly lower for those with a GP clinical judgment of dementia (67.0 ± 12.2), compared to those judged to be cognitively intact by their GP (73.3 ± 7.5). Agreement between the GP and CAMCOG-R was approximately 20% greater in those patients with dementia that had mentioned a memory complaint to the GP (Table 3). Depression scores (GDS) were not significantly associated with GP diagnosis of dementia (p = 0.085; Table 3).
Predictors of agreement between GP clinical judgment and the CAMCOG-R assessment of dementia
CAMCOG-R = revised Cambridge Cognitive Assessment; GEE = generalized estimating equation; GP = General practitioner.
Patients without CAMCOG-R dementia
Patient factors
A higher CAMCOG-R score was significantly associated with GP agreement that the patient did not have dementia (p < 0.001; Table 3). Two other patient factors were significant: gender, with the GP more likely to correctly identify the patient as not having dementia if they were female (p = 0.005); and patient complaint about their memory (p < 0.001). GPs were more likely to incorrectly identify the patient as having dementia if they complained about their memory. Thus, for patients without CAMCOG-R dementia, GP agreement with the CAMCOG-R was greatest when the patient was female, did not mention a memory complaint, and scored higher on the CAMCOG-R (Table 3).
GP factors
Several GP factors were associated with correct identification of patients who did not have dementia. GPs from larger practices were more likely to agree with the CAMCOG-R in this group (p < 0.001; Table 3). GPs who visited nursing home patients, were more likely to see dementia where it was not present (p = 0.036) as were solo GPs (p < 0.001). That is, GP agreement with the CAMCOG-R was greatest where GPs were from a larger practice and did not visit nursing home patients.
The characteristics of GP practices were examined by practice size (Table 4) in order to further examine the association of improved GP clinical judgment in larger practices. Larger practices were more likely to have a practice nurse (p < 0.001); their GPs in this study were younger (p < 0.001), and had spent less time as a practicing GP (p < 0.001). Almost 80% of solo GPs were male, but there was no significant relationship overall between GP gender and practice size (p = 0.082).
Characteristics of general practices based on practice size
GP = General practitioner.
Subjective memory complaints
A total of 337 patients mentioned a concern about memory problems to their GP; 61 (18%) of these patients had CAMCOG-R dementia (Table 5). The sensitivity and specificity for diagnosing dementia on the basis of SMC expressed by the patient to their GP was 0.37 and 0.85, respectively. The positive and negative predictive values of SMC for CAMCOG-R dementia were 0.18 and 0.94, respectively. Calculation of a positive likelihood ratio from these values indicated that patients in this study who expressed SMC to their GP were 2.46 times more likely than the general population to have CAMCOG-R dementia.
Relationship between subjective memory complaint and CAMCOG-R result
aBased on participants report that they had discussed memory complaint with their GP.
GP = General practitioner; CAMOG-R = revised Cambridge Cognitive Assessment.
Discussion
The current study investigated GP and patient factors that predicted GP agreement with an objective patient assessment for dementia (CAMCOG-R). GP agreement with the CAMCOG-R was fair (κ = 0.276), in large part due to agreement between the GPs and the CAMCOG-R in the absence of dementia. GPs correctly identified 90% of patients without CAMCOG-R dementia, though they were correct in less than half of their patients (44.5%) who had dementia, resulting in a relatively low PPV (28.7%). This is similar to the results of other studies of GPs or family physicians (Valcour et al., Reference Valcour, Masaki, Curb and Blanchette2000; Boustani et al., Reference Boustani, Peterson, Hanson, Harris and Lohr2003; Pentzek et al., Reference Pentzek2009a).
GPs were more likely to correctly identify dementia in those patients with a lower (i.e. poorer) score on the CAMCOG-R and more likely to correctly judge absence of dementia in those that scored well on the CAMCOG-R. Other studies have also reported that GPs’ recognition of dementia was better in moderate to severe cases (Wind et al., Reference Wind, van Staveren, Schellevis, Jonker and van Eijk1994; van Hout et al., Reference van Hout, Vernooij-Dassen and Stalman2007; Pentzek et al., Reference Pentzek2009a; Reference Pentzek2009b). Relatively few GPs relied on specialist advice or the use of an objective assessment tool (e.g. Mini-Mental State Examination) to assess patient cognition (used for 24.8% of patients), suggesting that their judgments were based on predominately subjective evidence. Impaired cognitive function and/or everyday functioning have previously been found to be associated with GP's subjective judgment on dementia (Wind et al., Reference Wind, van Staveren, Schellevis, Jonker and van Eijk1994). Clearly, GPs are on the right track in agreeing with dimensions measured by the CAMCOG-R, though for earlier detection of dementia, improvements in sensitivity are required.
Patients with SMC were 20% more likely to be diagnosed correctly as having dementia, though almost 20% less likely to be correctly identified as not having dementia. The positive and negative predictive values of SMC for CAMCOG-R dementia were 0.18 and 0.94, respectively, which is the same as previously reported for a low prevalence community sample (Mitchell, Reference Mitchell2008). Memory complaints increased the likelihood of dementia by 2.46 times, from 8.2% to around 20%. Although SMC were reported by 56% of people with CAMCOG-R dementia in this study, only 37% had mentioned it to their GP. These GPs clearly responded to SMC cues from their patients to identify dementia, however they also overrated the presence of dementia more frequently in patients with SMC, a finding in accord with previous research (Wind et al., Reference Wind, van Staveren, Schellevis, Jonker and van Eijk1994; Pentzek et al., Reference Pentzek2009b). Thus, while helpful in making the diagnosis for those patients who have dementia, SMC also contribute to over-diagnosis in those who do not have dementia. In at least some cases though, SMC may be predictive of future cognitive decline (Geerlings et al., Reference Geerlings, Jonker, Bouter, Ader and Schmand1999; Reid and MacLullich, Reference Reid and MacLullich2006).
In a community setting, even when patients report SMC to their GP, there is only a 20% chance that dementia is present, but these people may have up to a threefold increased risk of developing Alzheimer's disease in the future (Geerlings et al., Reference Geerlings, Jonker, Bouter, Ader and Schmand1999). A population-based study in Sweden reported that only 9–15% of participants with memory or multiple impairments did not have SMC (Palmer et al., Reference Palmer2008b). It may be that GPs are sensitive to this possibility. Although SMC alone have modest diagnostic value for dementia, in combination with an objective assessment they may offer advantage over either method alone, and provide a mechanism for improvement of GP diagnosis of dementia.
In addition to lower CAMCOG-R score and SMC, the other patient characteristic related to a false-positive diagnosis of dementia was gender. There were fewer false-positive GP judgments of dementia for female patients (9%) compared to male patients (11%).
This small but significant effect may be related to differences in the length and type of GP consultation between female and male patients. GPs spend longer on average with female patients (Britt et al., Reference Britt, Valenti and Miller2005), and spend a higher percentage of time on physical examination, screening, patient questions, and emotional counseling, whereas visits by men involve a higher percentage of time spent on procedures and health behavior counseling (Tabenkin et al., Reference Tabenkin, Goodwin, Zyzanski, Stange and Medalie2004). The dynamics of the GP consultation with female patients may be more conducive to a correct ruling-out of dementia.
GP factors that were associated with an increased false-positive diagnosis of dementia were practice size, and regular visits to nursing home patients. GPs from larger practices were more likely to correctly rule-out dementia. Interestingly, Connolly et al. Reference Connolly, Gaehl, Martin, Morris and Purandare(2011) reported lower rates of diagnosis in practices with one GP as opposed to several. They considered that one of the reasons for this may have been lack of time for the solo GP. Lack of time may also be a factor in the over-diagnosis of dementia by solo GPs in our study, as larger practices were more likely to have a practice nurse who may assist with dementia assessments. Larger practices were also more likely to employ younger GPs who may have received more dementia education. In the Australian context, GPs in solo practice and those who do nursing home visits tend to be older (Charles et al., Reference Charles, Britt, Valenti, Charles, Britt and Valenti2006; Gadzhanova et al., Reference Gadzhanova, Reed, Gadzhanova and Reed2007).
GPs that did nursing home visits were also more likely to over-diagnose dementia, an interesting finding that is difficult to explain. Possibly, their exposure to dementia in nursing homes may have influenced their perception of the incidence of dementia in the aged population. That is, GPs with regular exposure to the nursing home environment may perceive dementia in the aged to be a common and likely occurrence, as dementia is the most common problem managed in Australian nursing homes by GPs – 33 times the usual management rate in everyday practice (O’Halloran et al., Reference O’Halloran, Britt, Valenti, O’Halloran, Britt and Valenti2007).
Limitations of this study include the self-selected nature of the GP and patient populations and the use of a single cut-off point on the CAMCOG-R as the only benchmark for a correct diagnosis of dementia. Although the CAMCOG-R is a significant predictor for the clinical diagnosis of dementia (van Hout et al., Reference van Hout2001), most cases require additional clinical judgment for a definitive diagnosis. GPs may have been sensitive to patient presentation that suggested deficits relating to factors such as activities of daily living and the mention of memory problems which may not be reflected in a cognitive function test, as only 28.7% of patients with a GP judgment of dementia had CAMCOG-R dementia. This may have inflated the false-positive diagnoses found in this study.
While timely recognition of dementia is advocated for optimal dementia management, at present the early recognition of a possible dementia syndrome needs to be balanced with awareness of the likelihood of false positives in detection. Clearly, GPs are on the right track in agreeing with dimensions measured by the CAMCOG-R, though for earlier detection of dementia improvements in sensitivity are required. Enquiring about memory issues and monitoring patients voicing SMC is a consideration, though using this enquiry for diagnosis raises the risk of increasing the false-positive rate. Screening tests have been advocated but are also associated with a poor positive predictive value (Brodaty et al., Reference Brodaty, Low, Gibson and Burns2006). A more complex approach is needed, perhaps including the use of pretest probabilities such as that provided by memory complaints.
Conflict of interest
CDP has been on advisory boards for dementia for a number of companies, in the last 3 years including Pfizer, Lundbeck, and Nutricia, all of which have been paid positions; received sponsorship to attend conferences from Pfizer; been a speaker for the Dementia Study Training Centres and for Alzheimer's Australia and received honoraria for these activities; received grants from the NHMRC and the Department of Health and Ageing in relation to dementia; has sat on advisory boards for Pfizer, Novartis, Janssen, and Lundbeck.
HB received grants from pharmaceutical companies that are a participating center in multicenter international drug trials for Alzheimer's disease (currently Sanofi, Servier, and Roche); has received honoraria from Pfizer, Janssen, Lundbeck, and Novartis for giving presentations or chairing meetings; has been a sponsored speaker by Pfizer, Janssen, and Novartis; and has been a consultant and/or sat on advisory boards for Pfizer, Janssen, Lundbeck, Novartis, Merck. Nutricia, and Lilly.
All other authors have no competing interests to declare.
Description of authors’ roles
C. Dimity Pond conceived and developed the study and contributed to writing the paper. Karen E. Mate assisted with design of the study, supervised the data collection, and contributed to writing the paper. Jill Phillips assisted with writing the paper. Natasha Weaver did the statistical analyses. Parker J. Magin, Nigel P. Stocks, and Henry Brodaty assisted in study design and writing of the paper.
Acknowledgments
This study was supported by the Australian Government's National Health and Medical Research Council (Grant ID #351220).
