Introduction
Alzheimer's disease (AD) is an ever increasing public health burden. Every 66 sec someone develops AD in the USA. So far in 2016, 5.4 million individuals in the USA have been diagnosed with AD, 5.2 million of whom are over the age of 65. In 2016, the estimated cost for people with AD and dementias to utilize hospice, healthcare and long-term care is $236 billion (Alzheimer's Association, 2016). As more people with AD are admitted to hospital services, there will be an increase in demand for physician–patient contact time. Healthcare providers face a challenging task to efficiently and effectively evaluate cognition in patients with severe dementia in time-limited inpatient settings. Patients with severe dementia have marked cognitive impairment in language fluency, episodic memory, and semantic memory, although short-term memory is impaired but not to the level of impairment of episodic and semantic memory. Visuospatial skills, executive function, and praxis are additionally severely impaired (Boller et al., Reference Boller, Verny, Hugonot-Diener and Saxton2002).
One of the most utilized brief cognitive assessments is the Mini Mental State Examination (MMSE) (Folstein et al., Reference Folstein, Folstein and McHugh1975). The MMSE is in part widely-used because it can be administered to a wide range of adults from 18 to 100 years old. The MMSE also assesses performance on six cognitive domains. While the MMSE is quick to administer, it is limited by a floor effect among individuals with severe dementia (Auer et al., Reference Auer, Sclan, Yaffee and Reisberg1994; Wajman and Bertolucci, Reference Wajman and Bertolucci2006). Another cognitive assessment, the Severe Impairment Battery ((SIB); Saxton et al., Reference Saxton, McGonigle-Gibson, Swihart, Miller and Boller1990), targets severe cognitive impairment. The SIB assesses cognitive skills across seven cognitive domains. The SIB was designed to evaluate individuals with severe cognitive impairment that are difficult to test with other cognitive assessments such as the MMSE. The SIB is also an assessment measure that is more sensitive to detect cognitive changes at the severe stage of dementia (Wajman and Bertolucci, Reference Wajman and Bertolucci2006). Although it is considered to be the gold standard in measuring severe cognitive impairment, the SIB can take considerable time to administer, well beyond the usual timeframe that is practical in many clinical settings (Schmitt et al., Reference Schmitt1997; Wajman and Bertolucci, Reference Wajman and Bertolucci2006).
In addition to the MMSE, other brief cognitive assessments include the Severe Impairment Battery-8 (SIB-8), Severe Mini Mental State Examination (sMMSE), and the Brief Interview of Mental Status (BIMS) (Harrell et al., Reference Harrell, Marson, Chatterjee and Parrish2000; Chodosh et al., Reference Chodosh2008; Schmitt et al., Reference Schmitt2009). Prior studies have compared the performance of the MMSE and the sMMSE in patients with severe dementia, as well as the performance of the SIB and the SIB-8 (Sales et al., Reference Sales, Suemoto, Nitrini, Jacob-Filho and Morillo2012; Schmitt et al., Reference Schmitt, Saxton, Ferris, Mackell and Sun2013). The SIB appears to have sensitivity to change even in individuals with severe cognitive impairment (Wild and Kaye, Reference Wild and Kaye1998).
The SIB, SIB-8 (a subset of the SIB), and sMMSE are specifically designed cognitive assessment measures for individuals with severe cognitive impairment, including severe dementia. The MMSE is considered the gold-standard of memory/cognitive evaluation in a clinical setting. The BIMS is a brief cognitive exam of only five questions that could be more conducive to the population setting as it presumably requires a shorter length of attention than the longer cognitive assessment measures which have more questions. We sought to assess the validity, inter-correlations, and feasibility of shorter cognitive assessments in a cohort of patients with severe dementia by comparing the SIB-8, sMMSE, and the BIMS to the more widely-used MMSE and the gold standard SIB.
Methods
Participants
This performance improvement project occurred during 2013 and 2014 on a subacute dementia inpatient unit, located on the campus of Johns Hopkins Bayview Medical Center. In total, 49 patients (N = 49) ranging in age from 62 to 97 years were studied. Two patients were tested twice due to a second hospital stay occurring during data collection, resulting in 51 testing sessions. In this performance improvement project, some patients had been re-admitted to the unit several times during this time period and were reassessed. This was a convenience sample, as the patients were already residing on the unit prior to the start of the project and were deemed by clinical staff to be appropriate to have additional cognitive assessments administered during their daily activities. Since, this was a performance improvement project there were no formal inclusion and exclusion criteria other than residing as an impatient on the Medical Behavioral Unit (MBU), which is a long-term acute care unit. All residing patients in the MBU, who nurses said would be agreeable to additional testing, were included. This study was approved as exempt from consent by the Johns Hopkins University Institutional Review Board (IRB). The Johns Hopkins University IRB deemed that the patients in this performance improvement project were exempt from providing consent.
Design
Over the course of 2–3 sessions, five cognitive assessments were administered to each patient: SIB, SIB-8, MMSE, sMMSE, and BIMS. The assessments were administered in random order, although the SIB and SIB-8 were not given in the same testing session to avoid practice effects, as the SIB-8 items are a subset of the SIB. The order of the other assessments was not standardized as the participants’ patience for testing was not known in advance, and therefore, the number to be administered in one session is not standardized but roughly balanced between the sessions.
Instruments
The SIB is a 40-item cognitive measure targeting persons with severe impairment (Saxton et al., Reference Saxton, McGonigle-Gibson, Swihart, Miller and Boller1990). The domains assessed are attention, language, praxis, social interaction, orientation, and visuospatial ability. The SIB is designed to be administered to individuals between 51 and 91 years of age, and the range of scores is 0–100. The SIB-8 is an 8-item test, which is a subset of SIB items (Schmitt et al., Reference Schmitt2009). The SIB-8 assesses praxis, attention, orientation, memory, and language; the range of scores is 0–16.
The MMSE is one of the most widely used cognitive assessments in clinical practice (Folstein et al., Reference Folstein, Folstein and McHugh1975). With an average administration time of 5–10 minutes, the test contains 12 items, with scores ranging from 0 to 30, with higher scores indicating better performance. The cognitive domains evaluated are: orientation, registration, attention, recall, language, and praxis. The sMMSE is a 10-item cognitive measure derived from the MMSE, but is specifically geared toward individuals who are severely cognitively impaired (Harrell et al., Reference Harrell, Marson, Chatterjee and Parrish2000). However, similar to the MMSE, the range of scores is 0–30. Language, praxis, autobiographical knowledge, recall, attention, and orientation are the cognitive domains evaluated in the sMMSE. Comprising only five questions, the BIMS is the briefest among these cognitive measures. It assesses registration, recall, and orientation, with possible scores ranging from 0 to 15 (Chodosh et al., Reference Chodosh2008).
For all cognitive assessment measures, lower scores indicate cognitive impairment and higher scores indicate better cognitive abilities/skills. Particularly for the MMSE, scores below 10 are indicative of severe dementia.
Patient barriers
As the majority of participants had advanced dementia, the researchers sought to assess potential barriers to optimal patient performance on cognitive tests. Patient barriers are cognitive, mood or behavioral characteristics that have the ability to interfere with a patient's performance quality on cognitive assessment measures. These barriers can hinder performance in a few ways. A patient can perform worse on an assessment measure, cannot finish specific tasks on an assessment measure due to a barrier or cannot complete an assessment measure at all. For this performance improvement project, specifically four patient barriers were rated: difficulty comprehending instructions, distractibility, apparent fatigue, and frustration. These barriers have been noted to be present in some patients with severe dementia. The perception of the barriers was based on subjective judgment of the staff performing the tests. Difficulty comprehending instructions includes: staff having to repeat questions, the patient appearing confused when the questions were asked, or the patient verbalizing that they did not understand what was being asked of them. Distractibility includes: physically or mentally wandering from task or appearance of confusion due to lack of attention to current tasks. Apparent fatigue includes: appearance of drowsiness or patient vocalizing that they are tired. Frustration includes: appearance of agitation or patient vocalizing that they are frustrated. A 4-point scale was utilized as follows: 0 – no issue, 1 – mild issue, 2 – moderate issue, and 3 – severe issue. The amount of time required to administer each test was recorded.
Statistical analysis
Spearman's rank correlation coefficient was used for data analysis, with p < 0.05 used as the threshold for statistical significance. Statistical analysis was performed with Stata 13 (Corp, 2013). Linear regression was also used to compare SIB total scores and SIB-8 total scores from 49 patients and 51 test scores for each assessment measure. The range of scores vary from a minimum score of 0 on both assessment measures to the maximum score of 16 on the SIB-8 and a maximum score of 100 on the SIB. The best-fit linear regression line is superimposed on the scatterplot (p < 0.05). Linear regression was also used to compare SIB total scores and sMMSE total scores from 49 patients and 51 test scores for each assessment measure. The range of scores varies from a minimum score of 0 on both assessment measures to the maximum score of 30 on the sMMSE, and a maximum score of 100 on the SIB. The best-fit linear regression line is superimposed on the scatterplot (p < 0.05). Researcher-determined ratings comparing frustration, difficulty comprehending, fatigue, and distractibility across the five cognitive assessments for 49 patients and 51 ratings per assessment measure. Scores measured on a 4-point scale: 0 – no issue, 1– mild issue, 2 – moderate issue, and 3– severe issue.
Results
Data were analyzed for 49 patients from the inpatient dementia unit with a total of 51 samples. Of the samples, 25 were female and 21 were male. Five samples had no gender identified and that demographic information cannot be obtained retrospectively due to institutional medical record abstraction restrictions. The age range of patients was 62–97 years old with the mean of 80.2 (standard deviation of 8.5). Mean scores, standard deviation, and mean administration time are highlighted (Table 1). All of the inter-correlations between the five cognitive assessments had Spearman coefficients of (rs) > 0.7 and were significant with p < 0.001 (Table 2). The SIB-8 and sMMSE were positively associated with the SIB (Figures 1 and 2).
Mean total score, standard deviation (SD), and mean administration time (minutes) of SIB, SIB-8, BIMS, MMSE, and sMMSE in 49 participants with 51 observations
Spearman's rank correlation coefficients between SIB, SIB-8, MMSE, sMMSe, and BIMS in 49 participants with 51 observations. All p values were <0.001
Linear regression comparing SIB total scores and SIB-8 total scores from 49 patients and 51 test scores for each assessment measure. The range of scores vary from a minimum score of 0 on both assessment measures to the maximum score of 16 on the SIB-8 and a maximum score of 100 on the SIB. The best-fit linear regression line is superimposed on the scatterplot (p < 0.05).
Linear regression comparing SIB total scores and sMMSE total scores from 49 patients and 51 test scores for each assessment measure. The range of scores varies from a minimum score of 0 on both assessment measures to the maximum score of 30 on the sMMSE and a maximum score of 100 on the SIB. The best-fit linear regression line is superimposed on the scatterplot (p < 0.05).
The patient barrier scores ranged from 0 (no issue) to 1 (mild issue) on all five cognitive assessments (Figure 3). Across all five instruments, the SIB had the highest ratings for difficulty comprehending, fatigue, frustration, and distraction. The SIB-8 had low ratings on frustration and distraction, and high ratings on difficulty comprehending and fatigue.
Researcher-determined ratings comparing frustration, difficulty comprehending, fatigue, and distractibility across the five cognitive assessments for 49 patients and 51 ratings per assessment measure. Scores measured on a 4-point scale: 0 – no issue, 1– mild issue, 2 – moderate issue, and 3 – severe issue. SIB rated higher than other assessment measures on all four patient barriers. On frustration, BIMS and SIB-8 had comparable low ratings while the MMSE had the second highest rating. On both difficulty comprehending and distractibility, the SIB-8, BIMS, MMSE, and sMMSE had comparable ratings. On fatigue, the BIMS had the lowest rating followed by SIB-8, MMSE, and sMMSE, respectively.
Discussion
We sought to examine 5 cognitive assessments targeting severe cognitive impairment, and studied the feasibility of administration, duration of administration, and correlations between instruments for validity. As expected, there were relatively high correlations between the shorter and longer assessments, with the shorter assessments better tolerated by participants with fewer apparent barriers.
The SIB is widely considered to be the gold standard for assessing cognition in patients with advanced dementia. Its mean administration time of over 20 minutes is, however, impractical for use in typical clinical settings (Schmitt et al., Reference Schmitt1997). The BIMS had the shortest mean administration time of less than 3 minutes, but a relatively low correlation with SIB, most likely due to the fact that orientation is the only domain shared by both assessments. The BIMS also had relatively low correlations with SIB-8 and sMMSE. The SIB-8 appears preferable with mean administration time less than 5 minutes, high correlation with SIB, and overlap of five of seven cognitive domains from the SIB (memory, orientation, language, attention, and praxis). SIB-8 also had the highest correlation with MMSE with overlap in 4 cognitive domains (orientation, language, attention, and praxis). The sMMSE had similar characteristics to SIB-8, with mean administration time less than 6 minutes and high correlation with SIB. The MMSE and sMMSE assess similar cognitive domains: praxis, attention, language, orientation, and registration; however, the MMSE includes recall whereas the sMMSE assesses autobiographical knowledge. The SIB-8 and the sMMSE total scores were also highly correlated. Thus, the sMMSE and SIB-8 both appear to have relatively short administration times and total scores were highly correlated with widely-used cognitive assessments with longer administration times (SIB and MMSE). For these reasons, we believe that these tests are practical and validated tests for clinical assessment of cognition in advanced dementia, and they deserve more widespread application.
Administration time and breadth of cognitive domains are not the only aspects to consider when assessing cognition in patients with severe cognitive impairment. Patient barriers that may prohibit effective administration and test performance must also be considered. Not surprisingly, researchers rated the patients as having more frustration during the 40-item SIB than during administration of any of the other cognitive assessments. The SIB-8 also had one of the lowest ratings for frustration, adding to its effectiveness given the high correlation between the two assessments. The SIB was ranked highest for fatigue; and notably, administration of the sMMSE earned the second highest examiner ratings of patient fatigue. Regarding patient distractibility, the SIB, MMSE, and sMMSE were rated similarly. The sMMSE was also judged to cause the fewest difficulties with comprehension. In contrast, administration of the SIB-8 produced the second highest ratings for comprehension difficulties.
There are a number of limitations to the generalizability of these results, including (1) recruitment of a convenience sample of patients with advanced dementia on a long-term acute care unit. The sample may be biased towards enrollment of participants who were relatively cooperative according to the clinical staff, which might bias the results toward shorter administration. (2) the perception of social barriers was based on subjective judgment of the staff performing the tests. Preconceived notions of how a patient may feel during a certain cognitive assessment may bias these findings, since the staff member was aware that certain tests had longer administration times and might be biased to perceive greater barriers on these tests. (3) other limitations of results include lack of information on potential confounding factors, including specific diagnoses of dementia, education, and medical/psychiatric co-morbidities.
Finding an efficient and expedient cognitive instrument is crucial in an inpatient dementia care setting where physician time is limited. We sought to demonstrate the effectiveness of using brief cognitive instruments for patients with severe dementia. Shorter cognitive tests designed for severe dementia such as the SIB-8 and sMMSE have been evaluated in this project to be shorter in administration duration and highly correlated with gold standard instruments: the SIB and MMSE. Evaluating a variety of cognitive domains, using fewer questions, illustrating minor patient barriers, and having shorter administration times are favorable aspects of the SIB-8 and the sMMSE. Improving the efficiency with which physicians can assess cognition in patients with severe dementia by using shorter cognitive assessments would improve quality of care in several respects. Specifically, reductions in administration time would allow for serial cognitive assessment such that the ability to detect acute cognitive decline in a timely manner will be improved. Furthermore, reducing the likelihood that patients will become distracted or frustrated during cognitive test administration will optimize the validity of the test results.
Conflict of interest
None.
Description of authors’ roles
J. Dixon conducted literature search and wrote the paper. D. Saddington collected the data, performed statistical analysis, and assisted in writing the paper. C. Shiles collected the data. K. Sreevalsan collected the data and performed statistical analysis. C. Munro assisted in writing the paper. P. Rosenberg designed the study, supervised data collection performed statistical analysis, and assisted in writing the paper.
Acknowledgments
D. Saddington and K. Sreevalsan were sponsored during the time of data collection by Medical Student Training in Aging Research (MSTAR) program funded by American Federation for Aging Research (AFAR) Grant Number: T35 AG 026758. C. Shiles was sponsored during the time of data collection by UK Royal College of Psychiatrists Medical Student Elective Bursary funded by Royal College of Psychiatrists.
