Introduction
With an increasing prevalence worldwide (Ferri et al., Reference Ferri2005), dementia is emerging as a major health concern. As important primary care providers in the health-care systems, general practitioners (GPs) play an essential role in dementia recognition and management, and assume a key position in dementia care (Downs, Reference Downs1996). They see a significant percentage of the elderly population on a regular basis and have long-term relationships with patients and their relatives (Linden et al., Reference Linden, Horgas, Gilberg and Steinhagen-Thiessen1997; Schers et al., Reference Schers, van den Hoogen, Bor, Grol and van den Bosch2005). Efforts are being made to enable GPs to carry out this function competently, including educational approaches to enhance GPs' knowledge of dementia (Downs et al., Reference Downs2006; Vollmar et al., Reference Vollmar, Butzlaff, Lefering and Rieger2007; Perry et al., Reference Perry2008). Thus, the measurement of dementia knowledge is an essential part of evaluation. Existing knowledge tests on dementia are not practicable to evaluate GPs' knowledge. Some are simply out-dated, requiring knowledge that is now obsolete (Palmore, Reference Palmore1977; Dieckmann et al., Reference Dieckmann, Zarit, Zarit and Gatz1988); some are not designed for application in general practice (Gilleard and Groom, Reference Gilleard and Groom1994; Graham et al., Reference Graham, Ballard and Sham1997); and some include only a few questions about dementia as a small part of geriatric knowledge (Reuben et al., Reference Reuben1997; Lee et al., Reference Lee, Wilkerson, Reuben and Ferrell2004). The only, and most recent, test specifically constructed for health professionals is now over a decade old (Barrett et al., Reference Barrett, Haley, Harrell and Powers1997).
The current dearth of up-to-date, reliable and valid questionnaires has led to the use of self-made ad hoc questions and abbreviated or modified versions of existing questionnaires to measure dementia knowledge in recent observational studies (Chodosh et al., Reference Chodosh2004; Meuser et al., Reference Meuser, Boise and Morris2004; Turner et al., Reference Turner2004) and interventional studies (Bryans et al., Reference Bryans, Keady, Turner, Wilcock, Downs and Iliffe2003; Vollmar et al., Reference Vollmar, Butzlaff, Lefering and Rieger2007; Perry et al., Reference Perry2008). The use of such questionnaires can generate false or distorted results, including an underestimation of the relationship between GP characteristics and dementia knowledge.
The aim of the study presented here is to develop a new, reliable and valid dementia-knowledge test for GPs, based on previously selected items which have proved to test unambiguous knowledge and to be relevant to GPs' clinical practice (Pentzek et al., Reference Pentzek, Wagner and Abholz2006). We applied a cross-validation technique with psychometric item analysis and regression analyses conducted separately on two samples.
Methods
Subjects
Because of the homogeneous nature of the target group (GPs), a sample size of 300 GPs (150 for item analysis, 150 for validation) was thought to be sufficient (Nevo, Reference Nevo1980; Lienert and Raatz, Reference Lienert and Raatz1994). We used a simple, cost-effective and incentive-free recruitment method. An initial enquiry including a fax form for returning the reply (yes/no/reason for non-participation) was sent out to a random sample of 2,000 GPs in three German federal states (North Rhine-Westphalia, Saxony-Anhalt, Thuringia). Non-respondents received a reminder letter with a fax reply form after four weeks. Questionnaires were sent out immediately upon receipt of an affirmative reply; a reminder to participants was sent out four weeks after the questionnaire was dispatched. In view of the low priority with which dementia is regarded by GPs (Ryynänen et al., Reference Ryynänen, Myllykangas, Kinnunen, Halonen and Takala2000), we anticipated a low participation rate of about 15%. Because the response rate fell below this expectation (see Results section below for details), we decided to supplement the sample by recruiting GPs who were involved in teaching at German university departments or teaching units for general practice. We asked 29 departments/units to distribute the questionnaire among their teaching GPs. We provided 30 questionnaires per unit, but did not follow up on the process of questionnaire dispersal any further, so that response rates among units and GPs could not be calculated. If not stated otherwise, we counted the two sub-samples as one sample in our analyses.
Measures
The 37 questions had been tested for their unambiguousness and relevance to general practice in a previous study (Pentzek et al., Reference Pentzek, Wagner and Abholz2006). In brief, we sent out a 59-item questionnaire to nine dementia experts (leading members of memory clinics) and seven experienced GPs holding academic positions. These experts answered each question and rated its relevance to general practice. From the 59 questions, we chose 37 which were answered most consistently by the experts (reflecting unambiguousness) and were rated “relevant to General Practice”. All questions are in multiple-choice form with one correct answer, three distractors, and a “don't know” option. The initial instruction was: “Please check the one answer which is most appropriate in your opinion.” In our analyses, we scored the correct answer 1 point; all other possibilities scored 0. Some questions were taken from existing questionnaires (modified and translated into German) (Diekmann et al., Reference Dieckmann, Zarit, Zarit and Gatz1988; Barrett et al., Reference Barrett, Haley, Harrell and Powers1997; Turner et al., Reference Turner2004) but most were developed specially for the new questionnaire. Items covered the areas of epidemiology, genetics, diagnosis, dementia symptoms, therapy, management, and support.
GPs were required to specify their age, gender, experience in surgery, the location of their surgery (rural vs. urban), and whether they had an additional qualification in “clinical geriatrics”. They also had to estimate the number of their patients over the age of 70 living at home and living in care facilities, and the number of patients with dementia over 70 living at home and living in care facilities. On the basis of those last four estimations we calculated the estimated prevalence of dementia for each GP as a measure of the perceived dementia workload.
Eleven questions concerned GPs' own personal views on dementia (see Table 1). With these questions we intended to measure three specific components: general attitudes towards early recognition and drug therapy, confidence in the management of dementia, and the practice of disclosing a diagnosis to patients and their relatives.
GPs' personal views – varimax-rotated factor matrix
1Factor “attitude”: response options were: strongly agree – mainly agree – rather disagree – strongly disagree.
2Factor “confidence”: response options were: absolutely true – mainly true – rather false – absolutely false
3Factor “disclosure”: response options were: always – mostly – seldom – never.
4Omitted due to low factor loadings.
*Percentage of first two response options (strongly/mainly agree; absolutely/mainly true; always/mostly).
In the sub-sample of teaching GPs, we additionally asked if there were care facilities for dementia patients and their caregivers in the surgery's local area (items are taken from Turner et al., Reference Turner2004): support groups for caregivers, support groups for patients, day-care facilities, and memory clinics. Response options were “yes”, “no”, and “don't know”. If the response was “yes”, the name of the providing institution had to be given. Only when a name was provided did we consider this as an indication of having adequate knowledge and assumed there was cooperation with dementia-specific care facilities. We added up the answers of the four questions (1 point per named institution) and obtained a “care facility knowledge score” ranging from 0 to 4.
Analyses
We divided our sample into two sub-samples (n = 146 each) in order to analyze the psychometric properties and the proposed associations with the knowledge test separately. The division was carried out randomly, retaining an equal distribution of GPs from the initial sample and from the teaching GP sample. The two random samples were comparable with regard to all important variables: number of correct answers to the 37 knowledge test items (M1 = 23.1, SD1 = 4.6 vs. M2 = 22.2, SD2 = 5.2; t (290) = 1.70; p = 0.09; η2 = 0.01), age (M1 = 50.7, SD1 = 7.6 vs. M2 = 51.6, SD2 = 8.5; t (279) = −0.91; p = .36; η2 = 0.002), gender (sub-sample 1: 62.5% male vs. sub-sample 2: 62.1% male; χ2 (1) = 0.006; p = 0.94), years of practicing in surgery (M1 = 14.6, SD1 = 7.9 vs. M2 = 14.9, SD2 = 7.4; t (278) = −0.38; p = 0.71; η2 = 0.001), location of surgery (sub-sample 1: 47.6% rural, 52.4% urban vs. sub-sample 2: 52.4% rural, 47.6% urban; χ2 (1) = 0.68; p = 0.41), the estimated prevalence of dementia in patients over 70 living at home (M1 = 15.0, SD1 = 13.7 vs. M2 = 13.8, SD2 = 13.0; t (273) = 0.79; p = 0.43; η2 = 0.002) and living in care facilities (M1 = 66.5, SD1 = 24.1 vs. M2 = 62.6, SD2 = 24.1; t (261) = 1.30; p = 0.19; η2 = 0.006). All analyses were carried out using version 16 of the SPSS program (SPSS Inc., Chicago, IL, U.S.A.).
Psychometric evaluation
To analyze the structure of the 11 questions pertaining to personal views, we performed a principal component analysis (PCA) in subsample 1.
Because we included questions on a number of different areas of dementia care, we also conducted a PCA with the 37 items of the knowledge test.
We calculated the selection index (Lienert and Raatz, Reference Lienert and Raatz1994) for each question as the basis for item selection. The selection index takes into account both the question discrimination index and question difficulty; it starts at 0 and could approximate 5 in questions with extreme values.
We eliminated all items with a selection index below 0.25. For the selected questions, we calculated internal consistency (Cronbach's α) as a reliability index.
Impact of GP characteristics
In order to determine the variables associated with dementia knowledge, in sub-sample 2 we performed a stepwise linear regression analysis with the knowledge test score as a criterion variable and several possible predictor variables: age, gender, years of surgery practice, location of surgery (rural vs. urban), estimated prevalence of dementia, and three factors of personal views (as a result of the first factor analysis, see below).
A separate regression analysis was performed in the teaching GP sub-sample, using the same variables as in sub-sample 2, but with the additional predictor variable of “care facility knowledge score” (see Measures above).
Descriptive results from across the whole sample
To give an impression of further findings, we provide the descriptive results (frequencies, group comparisons, and correlations) found across the whole sample. We have to bear in mind that our sample does not aim to be representative, as we concentrate on the psychometric evaluation and cross-validation of the knowledge test.
Results
Sample
Of the 2,000 initially mailed requests, 11 were undeliverable and 20 potential participants answered that they no longer practiced as a GP. Of the remaining 1,969, no response was received from 1,626 (82.6%). A further 141 (7.2%) GPs refused to participate via fax: in 59.4% of these cases no reason was given, 28.0% claimed to have no time to participate, and 12.6% stated either no interest in the project or regarded dementia as an irrelevant issue in their surgery (affecting only a small percentage of elderly patients, e.g. due to a specific surgery focus). A total of 202 GPs (10.3%) agreed to take part, of which 137 (7.0%) returned the completed questionnaire.
Of the 870 dispatched questionnaires to the 29 teaching units/departments of General Practice, we received 155 (17.8%) back. However, this cannot be equated with a response rate because we did not know the actual number of questionnaires dispatched by each department.
The resulting total sample of 292 GPs had a mean age of 51.13 years (SD 8.05), 62.3% were men, and 50.0% had their surgeries in an urban, 50.0% in a rural location.
Psychometric analyses (n1 = 146)
We conducted a principal components analysis (PCA) with the 11 questions pertaining to GPs' personal views. The suitability of the data was validated with the Kaiser-Meyer-Olkin (KMO) value (0.68) and the Bartlett's Test (χ2 = 316.24, p < 0.001). PCA revealed the presence of three components. The varimax-rotated solution (presented in Table 1) reveals a simple structure, with several strong factor loadings, and all variables (except one) loading substantially onto only one factor. The 3-factor solution explained a total of 53.1% of the variance, with component 1 contributing 23.0%, component 2 contributing 17.7%, and component 3 contributing 12.4%. The interpretation of the resulting components confirmed our a priori assumed structure statistically (attitude, confidence, and disclosure).
Further analyses were based on the factor-score coefficient matrix (not shown), and we calculated 3-factor scores for each GP, based on the individual scores relating to attitude, confidence, and disclosure (see Table 1).
A principal components analysis (KMO = 0.56; Bartlett's χ2 = 911.31, p < 0.001) of the 37 knowledge test questions revealed one overriding factor, which explained 10.87% of variance. We selected 20 items based on the selection index (see Table 2) and calculated the knowledge factor score based on the factor score coefficient matrix (not shown) for each GP, which was used in the regression analyses (see below). The resulting 20-item test shows a satisfactory reliability of 0.733 (Cronbach's α).
The 20 selected knowledge items
*Higher values reflect better psychometric properties; we selected items with a selection index ≥0.25.
#We suggest to modify this item as follows: “Which statement is true concerning the pharmacological treatment of Alzheimer's patients with severe depressive symptoms?”.
Associations with GP characteristics (n2 = 146)
In sub-sample 2 we carried out a stepwise linear regression analysis using the knowledge factor score as a criterion variable and a number of predictors (including the three-factor scores on “attitude”, “confidence”, and “disclosure”). There were 126 questionnaires with complete data available for this analysis. The resulting model (F (2) = 7.7; p = 0.001; r2 = 0.11) indicates a significant influence of age and attitude on dementia knowledge (Table 3): the higher a GP's age, the worse his/her knowledge of dementia. Furthermore, we found that a more positive attitude (towards early recognition and dementia therapy) is related to a lower knowledge score.
Stepwise linear regression analysis with dementia knowledge as criterion variable
In the sub-sample of teaching GPs, we also included the “care facility knowledge score” as a further predictor. 110 questionnaires were available for analysis. The resulting model (F (4) = 10.1; p < 0.001; r2 = 0.28) includes knowledge of care facilities (beta = 0.387; t = 4.606; p < 0.001), age (beta = −0.246; t = −2.882; p = 0.005), location of surgery (beta = −0.242; t = −2.807; p = 0.006), and attitude (beta = −0.187; t = −2.199; p = 0.030) as significant predictors of dementia knowledge (no table shown). Age and attitude display their influence in the same direction as in sub-sample 2 (see above). Greater familiarity with local dementia facilities is linked to a higher score in the knowledge test. Those practicing in a rural location displayed better dementia knowledge in this sub-sample.
Descriptive results from across the whole sample (n = 292)
We analyzed the distribution of knowledge test scores (see Figure 1) across the whole sample. Visual inspection and the basic statistical values suggest only minor deviations from a normal distribution of knowledge scores: mean = 11.26, median = 11.5, modus = 12.0, skewness = −0.168, kurtosis = −0.115. We further calculated z-scores for skewness (zsk = −1.172) and kurtosis (zku = −0.401) according to Hair et al. (Reference Hair, Tatham, Anderson and Black1998). These z-scores do not exceed the proposed critical value for a 0.05 error level, and therefore the distribution of knowledge scores does not deviate significantly from a normal distribution (neither skewed nor peaked/flattened).
Distribution of knowledge test scores.
Figure 2 shows the GPs' estimates regarding the prevalence of dementia among patients aged over 70 years and living at home. Estimates for prevalence in patients over 70 living in care facilities (no figure shown) average out at 64.6% (SD 24.1).
GPs' estimation of the prevalence of dementia among their patients aged over 70 living at home.
Group comparisons
We calculated t-tests to compare male and female GPs, as well as rural and urban GPs. Female GPs demonstrated a more positive overall attitude (women: M = 3.07, SD = 0.57 vs. men: M = 2.77, SD = 0.57; t (283) = −4.32; p < 0.001; η2 = 0.062). In more detail, women scored slightly higher (more positive) on the question concerned with the advantages of early recognition for the patients' relatives (women: M = 3.40, SD = 0.65 vs. men: M = 3.22, SD = 0.73; t (286) = −2.11; p = 0.036; η2 = 0.015) and on the two questions regarding the positive effects of anti-dementia drugs on the progress of the disease (women: M = 2.72, SD = 0.80 vs. men: M = 2.22, SD = 0.75; t (287) = −5.44; p < 0.001; η2 = 0.093) and on the welfare of the patient (women: M = 2.79, SD = 0.80 vs. men: M = 2.44, SD = 0.81; t (286) = −3.49; p = 0.001; η2 = 0.041). No differences were found in the answers pertaining to the advantages of early recognition for the patient (see Table 1 for item wording).
Slight differences could be detected between rural and urban GPs in their estimates for the prevalence of dementia among their patients. Rural GPs showed higher estimates in both patients living at home (rural: M = 16.20%, SD = 16.31 vs. urban: M = 12.66%, SD = 9.22; t (273) = 2.22; p = 0.027; η2 = 0.018) and patients in care facilities (rural: M = 67.93%, SD = 24.01 vs. urban: M = 61.57%, SD = 23.86; t (261) = 2.15; p = 0.032; η2 = 0.017).
GPs who realistically estimated the prevalence of dementia among their patients (5–20%, n = 163) were compared with GPs who under-estimated (0–5%) or over-estimated (>20%) the prevalence of dementia (n = 112). There was a tendency toward better test scores among GPs who gave realistic estimates of dementia prevalence (M = 11.58, SD = 3.43 vs. M = 10.76, SD = 3.41; t (273) = −1.96; p = 0.051; η2 = 0.014). GPs with the more realistic prevalence estimates also displayed a more negative overall attitude toward dementia (M = 2.82, SD = 0.58 vs. M = 2.97, SD = 0.62; t (270) = 1.97; p = 0.050; η2 = 0.014), specifically in the questions (see Table 1 for wording) on the advantages of early dementia recognition for patients (M = 3.19, SD = 0.77 vs. M = 3.37, SD = 0.74; t (272) = 1.95; p = 0.052; η2 = 0.014) and on the positive effects of anti-dementia drugs on patients' welfare (M = 2.48, SD = 0.79 vs. M = 2.71, SD = 0.88; t (272) = 2.27; p = 0.024; η2 = 0.019).
As stated in the Methods section above, we also asked if the GPs held an additional qualification in “clinical geriatrics”. Only nine GPs claimed to hold this certificate, so we conducted Mann-Whitney-U tests due to unequal sample sizes. GPs with the certificate scored higher on the dementia knowledge test than GPs without the qualification (12.9 vs. 11.2; U = 771.5; p = 0.049). Among the teaching GPs (n = 155), those with a qualification (n = 4) also showed a tendency towards better knowledge of dementia-specific care facilities (care facility knowledge score: 3.00 vs. 1.73; U = 120.0; p = 0.061).
Correlations
We found a correlation between experience (measured by years in surgery) and the “confidence” factor. More experienced GPs displayed a greater professional confidence (Pearson's r = 0.24; p < 0.001), which was also true for the four single questions relating to confidence with provision of care (r = 0.16; p = 0.007), with diagnosis (r = 0.15; p = 0.015), with treatment (r = 0.23; p < 0.001), and with social support (r = 0.16; p = 0.007) (see Table 1 for item wording). No correlation was found between knowledge and confidence.
Possible answers to the question on dementia frequency among 100 patients older than 85 years (see question 11 in Table 2) were “5 to 15”, “25 to 35” (correct), “40 to 50”, and “60 to 70”. We correlated the answer with the GPs' own estimations of the prevalence of dementia among their patients living at home (see Methods section). Spearman's rank correlation coefficient ρ was 0.29 (p< 0.001) indicating an association between knowledge item and subjectively perceived prevalence in clinical practice.
One further result needs highlighting because it illustrates an interesting phenomenon. One of the questions not included in the final version of the test (owing to weak psychometric properties) was: “It is supposed that current anti-dementia drugs – given early in the course of Alzheimer's disease – can: delay cognitive decline for 1–2 years [right answer], delay cognitive decline for 5–6 years, halt cognitive decline, or not influence cognitive decline”. The right response was provided by 75% of the participants, while 20% opted for second or third option. Only 3% stated that anti-dementia drugs are supposed to have no influence on cognitive decline. In contrast, responses to the attitudinal questions (see Table 1) show that only 40% and 50% respectively believe anti-dementia drugs can slow down the progress of the disease and can have a positive effect on patients' welfare. One possible interpretation of these differences of opinion is that most GPs are aware of the expectations surrounding anti-dementia drugs (“are supposed to have. . .”), but that for many GPs these expectations are optimistic and are not in accord with their own experiences (Riedel-Heller et al., Reference Riedel-Heller, Schork, Fromm and Angermeyer2000; Pentzek et al., Reference Pentzek, Fuchs and Abholz2005).
Discussion
Our results provide evidence of the reliability and validity of the 20-question test and highlight its excellent psychometric properties. Factor analysis supports the test to assess a one-dimensional construct of “factual dementia knowledge”.
An association of continuing education in geriatrics and dementia knowledge was only shown in a small sample, but this is in line with previous results that found geriatric education to be related to better dementia recognition (Chodosh et al., Reference Chodosh2004). The association of the knowledge test score with GPs' age may be of special importance, as this is in line with previous studies on dementia knowledge (Brown et al., Reference Brown, Mutran, Sloane and Long1998; Turner et al., Reference Turner2004). Furthermore, younger GPs have been shown to report better diagnostic, therapeutic, and management approaches in dementia (Downs et al., Reference Downs, Cook, Rae and Collins2000; Kaduszkiewicz et al., Reference Kaduszkiewicz, Wiese and van den Bussche2008). The association of knowledge with GPs' attitudinal statements on early recognition and drug therapy is also relevant as it illustrates proximity to the participant's personal views and concepts of disease.
We are aware of the complexity and diversity of knowledge and the learning process. Our test was developed to cover one small part of GPs' dementia knowledge: the declarative theoretical (factual) knowledge. Other dimensions of competence, such as communication skills, problem-solving capacity, intuitive decision-making, practical and instrumental knowledge, procedural and tacit knowledge, or reflected observation cannot be measured with such an instrument. Even though we have checked our items for relevance in general practice in our previous work, the requested knowledge remains abstract and factual. Nevertheless, this kind of knowledge is part of the whole knowledge construct and it is a cognitive part of an individual's attitude towards objects and issues. It is clear that there is a plethora of possible questions to measure factual knowledge. We have concentrated on items from existing questionnaires and we tried to broaden the scope to various aspects of dementia management.
The construction of a knowledge test makes it essential to be geared to the best available evidence and the most unequivocal expertise. In a field like dementia it is clear that nearly every aspect is debatable because there is more opinion than evidence. To solve this problem our items had to meet two requirements: first, the question had to be answered concordantly by the majority of experts in dementia and general practice. This has been assured in a previous study (Pentzek et al., Reference Pentzek, Wagner and Abholz2006). Second, the answer had to be in agreement with current dementia guidelines and expert opinion. These constraints may have resulted in more or less abstract items which can only cover a small part of dementia knowledge. In order to keep the test up-to-date, the items have to be revised at regular intervals so as to reflect changing evidence.
We chose a time- and cost-effective recruitment method, and conveniently supplemented the sample with teaching GPs. In view of this simple strategy, the length of the questionnaire, and other barriers to high response rates (Kaner et al., Reference Kaner, Haighton and McAvoy1998; Edwards et al., Reference Edwards2002; Stocks et al., Reference Stocks, Braunack-Mayer, Somerset and Gunell2004), participation was low. Representative conclusions cannot therefore be made, but these are beyond the scope of this study anyway.
Our study did not support the use of the test in interventional studies, as we did not assess test-retest reliability or sensitivity to change (e.g. after training).
For reasons of practicability and feasibility we designed a questionnaire comprising multiple-choice (MC) questions. Another item form, such as multiple-select or essay questions, may have yielded more information, but mostly require longer time to complete. Given the limited time available in general practice and that factual knowledge can only be one part of a meaningful survey (including further questions on personal views and practice), a MC questionnaire best lends itself to widespread use.
The data presented here demonstrate the reliability and validity of the test with respect to its associations with GPs' age, attitudes, cooperation with dementia-related care facilities, perceived workload, and training in geriatrics. This instrument may be the best basis for a brief and simple measure of factual dementia knowledge in observational studies, in which a detailed assessment of knowledge is not practical. An individual test score cannot display knowledge gaps in specific aspects but it can serve as a rough knowledge estimate.
Conflict of interest
None.
Description of authors' roles
Michael Pentzek designed the questionnaire, planned and conducted the study, analyzed the data, and wrote the paper. Heinz-Harald Abholz designed the questionnaire, planned and conducted the study. Martin Ostapczuk analyzed the data and revised the manuscript. Attila Altiner and Anja Wollny designed the questionnaire and revised the manuscript. Angela Fuchs designed the questionnaire, planned the study, analyzed the data, and revised the manuscript.
Acknowledgments
We thank all participating general practitioners for their efforts. This study was supported by the German Alzheimer Society. MP, AW, and AF received funds from the German Federal Ministry of Education and Research (German Competence Network Dementia (KND): grant 01GI0423 and German Competence Network Degenerative Dementias (KNDD): grant 01GI0713).
