Introduction
The issue of capacity to consent has received much attention lately. In the U.K., the Mental Capacity Act (Department of Health, 2005) became part of legislation in 2007, affording increased protection to patients and clearer guidelines for clinicians. Researchers have recognized the need for valid and efficient capacity assessments for some time. The difficulty in gaining assent from relatives where capacity is lacking has led to under-representation of certain groups in clinical trials, such as those with delirium (Adamis et al., Reference Adamis, Martin, Treloar and Macdonald2005).
This poses a practical problem for research in the elderly. The advent of computerized notes systems means that large samples of clinical data are potentially available for research; and there is a legitimate need to conduct research in people without capacity to consent – for example, evaluating the differential response to anticholinesterase inhibitors in different sub-populations with dementia. A valid and commonly used instrument for assessing capacity would be of enormous benefit, especially if it was already routinely used. For example, it could inform researchers if they need to approach the patient directly for consent or whether they need to speak to a relative to gain proxy consent to participate in research, thus saving much time. Whilst such instruments exist, such as the MacArthur Competence Assessment Tool for Clinical Research (MacCAT-CR) (Appelbaum and Grisso, Reference Appelbaum and Grisso2001), they are not used routinely in clinical practice, partly because they are specific to U.S. jurisdictions, and partly because of their complexity and length.
The Mini-mental State Examination (MMSE) (Folstein et al., Reference Folstein, Folstein and McHugh1975) is the most widely used instrument for assessing cognitive impairment in older people. Gregory et al. (Reference Gregory, Roked, Jones and Patel2007) found it a useful screening tool to help inform assessments of capacity to give enduring power of attorney in 74 patients with Alzheimer's disease, with a MMSE cut-off score of 18 giving optimal sensitivity (86.6%) and sensitivity (82.2%) in predicting capacity status. Kim and Caine (Reference Kim and Caine2002) assessed the utility of the MMSE in evaluating capacity to consent to research in a U.S. population with established Alzheimer's disease and found that a range of MMSE scores of 21 to 25 did not discriminate capacity status well. However, they used a relatively high range of MMSE scores and there were only 37 subjects in the study.
In a small U.K. study by Ripley et al. (Reference Ripley, Jones and Macdonald2008) examining capacity to consent to a major life-change (admission to a care home), the MMSE was strongly predictive of capacity. The best cut-point on the MMSE was 16/17; and of the 15 scoring below 17 only one had capacity (7%), whereas of the 23 scoring 17 or above 19 (83%) had capacity. Using this cut-point in this population the MMSE had a 93% positive predictive value and an 83% negative predictive value against the formal capacity algorithm, with an overall misclassification rate of 13%.
A number of other studies have assessed the utility of the MMSE for guiding capacity assessments in residents living in care home settings (see the review by Kim et al., Reference Kim, Karlawish and Caine2002), but none of these addresses the issue of capacity to consent to research. Our primary aim was to see if the findings of Gregory et al. and Ripley et al. could be replicated in a larger sample with regard to capacity to consent to research. We used a sample from a randomized controlled trial (RCT) conducted in an elderly care home setting and assessing the utility of a greater range of MMSE scores than those used by Kim and Caine (Reference Kim and Caine2002).
In the U.K., the three main tenets of capacity are that (a) the person understands the decision; (b) can retain the relevant information; and (c) can weigh up the pros and cons before making a decision. Tenets (a) and (c) require global cognitive ability, including executive function. The MMSE is a global assessment of cognitive function but does not specifically measure executive function. However, tenet (b) is directly tested by the delayed recall item of the MMSE.
Palmer et al. (Reference Palmer2005) examined capacity to consent to research in three diagnostic groups of older patients (Alzheimer's disease, schizophrenia and type 2 diabetes) by providing them with the details of a hypothetical study and then administering a three-item questionnaire specifically targeting the main elements of research disclosure and the MacCAT-CR. The MMSE was also conducted as a measure of cognitive impairment. The three questions posed in the questionnaire – “What is the purpose of the study?” “What are the risks of the research?” and “What are the benefits?” – assess the same areas as the tenets (a), (b) and (c), given that the participants were required to retain information about the research project when completing the hypothetical consent form. When compared to the MacCAT-CR, the three-item questionnaire generated a receiver operator characteristic (ROC) curve with an area under the curve of 0.92 (95% CI 0.86–0.97). The authors noticed that degree of cognitive impairment was closely related to capacity status and therefore decided to also compare the MMSE with the MacCAT-CR. The area under the ROC curve for the MMSE was 0.88 (0.79–0.96).
As a secondary aim we decided to adapt the idea of Palmer et al. (Reference Palmer2005) to determine which individual item of the MMSE best predicts capacity status. We hypothesized that delayed recall would be the item most associated with capacity status.
The objectives of the study were: (1) to assess the utility of the MMSE in predicting capacity to consent to research in older U.K. care home residents and determine the optimal cut-point score; and (2) to determine which, if any, items of the MMSE best predict capacity to consent to research.
Methods
The FEVER trial was a RCT of the efficacy of testing elderly care home residents' antibody response to a routine influenza vaccine and administering a booster dose if response was inadequate, versus standard practice. The sampling procedure and the trial methods have been described previously (Gaughran et al., Reference Gaughran2007). We included people who were over 60 years old and living in a nursing home, or in residential care in the same facility, in south London and who were in receipt of the annual influenza vaccine.
Research nurses were trained at the start of the trial in the MMSE, Cornell Scale for Depression in Dementia (CSDD) (Alexopoulos et al., Reference Alexopoulous, Abrams, Yound and Shamoian1988) and in capacity assessments using criteria proposed by the Law Commission and subsequently embodied in the Mental Capacity Act (Department of Health, 2005). This latter training took two hours and was conducted by AM using an algorithm as described in Macdonald et al. (Reference Macdonald, Roberts and Carpenter2004) and Ripley et al. (Reference Ripley, Jones and Macdonald2008). MMSE and capacity assessments were conducted within one month of each other. Although not formally blinded to the outcome of these, the research nurses were blind to the hypothesis being tested, and the practicalities of collecting considerable amounts of data for a large RCT probably meant that they were unlikely to be biased by the result of previous assessments.
Two hundred and seventy-seven subjects were randomized in the trial; all of whom were assessed for capacity to consent (157 had capacity and consented). Consent was sought from proxies (relatives or residential home staff) in the remaining 120. No MMSE data were available for participants who refused or for whom assent could not be obtained. The MMSE and CSDD were administered after the assessment of capacity had been made. If three or fewer items in the MMSE were missing – for instance due to physical or sensory deficits – then the total MMSE score was calculated as a proportion of a maximum of thirty.
Statistical analysis
A ROC curve was generated and the maximal MMSE cut-point that best predicted capacity status determined. Logistic regression analyses were conducted to determine the effects of gender, ethnicity, care home type, dementia (as defined by MMSE ≤ 24) and depression on capacity status. Further logistic regression analyses were conducted to generate odds ratios (ORs) to determine which items of the MMSE best predicted capacity status.
Results
Characteristics of the sample
Of the 277 residents randomized in the FEVER trial, 227 residents had a valid MMSE score and were therefore eligible for the present study. Table 1 describes the characteristics of the sample. The median MMSE score was 16 (range 0–30, IQR 9–22). Eighty-one per cent (n = 184) of the residents had MMSE scores ≤ 24. Forty-one per cent (n = 94) of the sample lived in an elderly mentally infirm (EMI) home, a type of residential home for elderly people with dementia with behavioral problems. One-fifth of the sample was depressed as determined by a CSDD score of greater than 8 (Whelan et al., Reference Whelan, Gaughran, Walwyn, Chatterton and Macdonald2008).
Characteristics of the sample (n = 227)
EMI – Elderly Mentally Infirm; MMSE – Mini-mental State Examination; CSDD – Cornell Scale for Depression in Dementia
*Out of 216 with a valid CSDD score
Missing data
The trial protocol determined that if four or more items were missing from the MMSE then the score was deemed invalid. There were no significant differences between the residents for whom there were complete data and those with missing data in terms of age, gender and ethnicity. However, residents who had missing/invalid MMSE scores were less likely to be able to give consent and more likely to live in an EMI home.
Receiver operator characteristic curve
Figure 1 shows the ROC curve of MMSE score against the assessment of capacity to consent to the study. The area under the curve was 0.86 (p < 0.001, 95% CI 0.80–0.91, SE 0.026). Table 2 shows the coordinates of the curve. The optimal MMSE threshold for determining capacity in this group was 13/14. This gave a sensitivity of 79.4 % (95% CI 74.5–83.6%) and specificity of 75.6% (95% CI 67.4–82.4%). The OR of a MMSE score ≥ 14 predicting having capacity was 11.95 (95% CI 6.03–23.94). Table 3 shows the various predictive values of the “diagnostic test” based on a MMSE cut-point of 13/14.
Receiver operator characteristic curve
Coordinates of ROC curve
ROC – Receiver operator Characteristic; MMSE – Mini-mental State Examination.
Predictive values based on MMSE cut-point 13/14
MMSE = Mini-mental State Examination.
Individual MMSE items
Table 4 shows the ORs of the individual items of the MMSE associated with capacity status. ORs were generated using logistic regression with capacity status as the dependent variable and the individual items of the MMSE as the independent variables. The ORs were calculated based on the subject answering all the components of the item correctly or not, i.e. for this part of the analysis the responses were considered as categorical rather than continuous data. For example, being able to register only two, rather than three, objects was deemed a response failure for that item.
Odds ratios for association between individual items of the MMSE and capacity status*
*Note: Odds ratio if all components of the item are answered correctly.
MMSE = Mini-mental State Examination; SE = standard error.
All individual items of the MMSE apart from following a written command (“close your eyes”) had ORs that were significantly associated with capacity status. The OR for being able to recall three items correctly was 2.5 (95% CI 1.6–3.8). However, contrary to our hypothesis this was not the item most associated with capacity status; this was being able to verbally repeat a sentence (“no ifs, ands, or buts”) correctly (OR 5.6, 95% CI 3.0–10.5). The per-point MMSE OR was 1.22 (95% CI 1.16–1.29), i.e. every one point increase in MMSE score increased the odds of having capacity by 22%.
Regression analyses
Initially, a logistic regression analysis was conducted with capacity status as the dependent variable and gender, ethnicity, care home type (EMI or non-EMI), dementia and depression the independent variables. Gender and ethnicity did not affect capacity status but depression reduced the odds of having capacity (OR 0.32, 95% CI 0.17–0.62, p = 0.001). Care home type did not affect capacity status in this analysis (OR 0.832, 95% CI 0.48–1.43, p = 0.51). Unsurprisingly, having dementia reduced the odds of having capacity (OR 0.039, 95% CI 0.01–0.29, p = 0.002).
A multiple logistic regression analysis was then conducted including the variables found to influence capacity status in the logistic regression analyses. As both our measure of dementia and the predictor of capacity status were based on the MMSE score we did not include dementia in the multiple regression equation, leaving depression as the only variable requiring testing. However, we did include care home type in the analysis despite this variable not affecting capacity status in the logistic regression analysis. This is because residents from EMI homes were under-represented and our subsequent work (as yet unpublished) suggests that type of care home may indeed be associated with exclusion from research because of incapacity. The OR of a positive “diagnostic test” (MMSE ≥14) predicting capacity when the effects of depression and care home type were controlled for was 9.55 (95% CI 4.93–18.5, p < 0.001).
Discussion
The finding of a MMSE score of 14 or greater predicting capacity to give or withhold informed consent to participate in a research project was somewhat lower than the cut-point of 18 found by Gregory et al.(Reference Gregory, Roked, Jones and Patel2007) to predict capacity to appoint enduring power of attorney, and also than that found in the study by Ripley et al. (Reference Ripley, Jones and Macdonald2008) of capacity to enter a care home. This may be a reflection of the gravity of decisions concerning the power of attorney and care home decisions. Participation in the FEVER project was agreed to be in a patient's best interest, in that if they had not responded to their first dose of influenza vaccine they were offered a booster dose, the hypothesis being that this would improve their chances of staying healthy over the winter. However, if, for example, the participants were being asked to consent to a drug trial where clinical equipoise pertained, then there would be no obvious benefit for the individual in participating in the research. Despite the clear population benefit, it is possible that a higher cut-point MMSE score would correlate better with capacity to consent in such a study. As far as we are aware, this has not actually been addressed in the literature in relation to the MMSE. However, Kim et al. (Reference Kim, Caine, Swan and Appelbaum2006) found that clinicians require more stringent evidence of capacity as the risk/benefit profile of a research subject's decision-making situation worsens, e.g. a RCT for a new dementia treatment compared to a clinical trial involving a neurosurgical intervention.
All except one of the individual items of the MMSE were predictive of capacity status. The finding that delayed recall was not the item most associated with capacity status was interesting but not so surprising. Although delayed recall is the item that most directly relates to the second tenet of capacity (being able to remember the information relevant to the decision at hand), the MMSE is a global assessment of cognitive function and therefore all items should broadly be associated with capacity status. The sentence repetition item was most predictive of competency. A possible explanation for this could be that being unable to repeat a sentence is indicative of advanced dementia.
There are a number of limitations that pertain to this study. First, we trained the research nurses in capacity assessments using criteria proposed by the Law Commission rather than a using a research capacity instrument such as the MacCAT-CR. However, as has already been discussed, this tool is specific to U.S. jurisdictions and the Law Commission criteria have been embodied in the Mental Capacity Act in the U.K.
Issues relating to the representativeness of the sample have already been discussed. In short, difficulty in gaining assent from a proxy where capacity was lacking meant that residents lacking capacity and those in EMI homes were under-represented. Furthermore, the sample population comprised residents in south London care homes with high rates of dementia (81% with MMSE ≤ 24). Whilst this is in keeping with the local prevalence of dementia in residential homes the findings may not be generalizable to other elderly populations who may be target samples for research, such as community-dwelling elderly people without dementia.
There was a gap of up to four weeks between MMSE and capacity assessments. It is possible that MMSE score or capacity status could have fluctuated during that time. However, as the majority of the sample had dementia it is unlikely that there would have been significant variance had the assessments been conducted more contiguously. Furthermore, delirium was an exclusion criterion from the FEVER trial. In addition to this limitation, no interrater reliability data were generated for the research nurses' capacity assessments.
Although there was a strong association between MMSE score and capacity status even the optimal cut-point in terms of sensitivity/specificity (13/14) mislabeled 24% of capable subjects as lacking capacity, and 21% of incapable subjects as having capacity. There is a particular issue relating to false negatives, as it would be undesirable to have individuals being enrolled in a study who lacked capacity.
The analyses of the individual items of the MMSE predicting capacity status need to be interpreted with some caution as each component was coded as a dichotomous ‘correct/not correct’ score to facilitate statistical analysis rather than as a score on a continuous scale (although this method was not used for scoring the overall MMSE score). Nonetheless, all items except one were significantly associated with capacity status. However, the items with the largest ORs tended to have the largest standard error and/or widest confidence intervals.
Despite these limitations the findings, in particular the use of the total MMSE as a predictor of capacity status based around a threshold of 13/14, are robust: we used a large sample; the OR of the “diagnostic test” was high (11.95) and statistically significant; the area under the ROC curve was large (0.86); and the findings were broadly in keeping with similar research. However, we do not suggest using the MMSE in lieu of a formal clinico-legal assessment of capacity to consent to research. Instead it may have a role in guiding researchers, at an early stage of a study and where a database already exists with MMSE scores recorded, as to which participants may lack capacity and for whom family members need to be approached to give proxy assent before they can enter into the research program.
Conflict of interest
None.
Description of authors' roles
P. Whelan formulated the research question, carried out the study, analyzed data and wrote the paper. J. Oleszek conducted a background literature search, and assisted with data analysis and writing the paper. A. Macdonald formulated the research question, supervised data collection and assisted with writing the paper. F. Gaughran designed and carried out the study, supervised and collected data and assisted with writing the paper.
Acknowledgments
Preliminary data from this paper were presented as a poster at the International Psychogeriatric Association European Meeting, Dublin, Ireland, in April 2008. The study was funded by the Charitable Trustees of Guys and St. Thomas' Hospitals, London, U.K. The sponsor of the study was the South London and Maudsley NHS Foundation Trust.
We thank Robert Blizard, South London and Maudsley NHS Foundation Trust, Mental Health of Older Adults, Ladywell House, Lewisham, London, U.K., for his advice on the statistical analysis, and also the FEVER Trial team: Elizabeth Atere-Roberts, Olubukola Awodiya, Orla Barnewell, Lynne Batty, Nas Belazka, Joanne Brooke, Kate Chatterton, Elizabeth Francis, Fiona Gaughran, Colin Gelder, Veronica Ibizugbe, Rob Lambkin-Williams, Niamh Keane, Alastair Macdonald, Pat Meeking, John Oxford, Jonathan Riley, Rebecca Walwyn, and Paul Whelan.
