Introduction
Falls are a major health problem among older people, particularly in nursing homes (Dijcks et al., Reference Dijcks, Neyens, Schols, van Haastrecht, Crebolder and de Witte2005). Drug use and gait and mobility impairments are strong predictors for falling (Tinetti et al., Reference Tinetti, Speechley and Gibster1988; Studenski et al., Reference Studenski1994; American Geriatrics Society, British Geriatrics Society and American Academy of Orthopaedic Surgeons Panel on Falls Prevention (hereafter AGS Panel on Falls Prevention), 2001; CBO, 2004; Sterke et al., Reference Sterke, Verhagen, van Beeck and van der Cammen2008). Studies examining the relationship between falls and gait function in nursing home residents with dementia indicate that the number of falls and gait impairments increases with advancing severity of dementia (Nakamura et al., Reference Nakamura, Meguro and Sasaki1996; O'Keeffe et al., Reference O'Keeffe, Kazeem, Phillpott, Playfer, Gosney and Lye1996).
The Tinetti Performance Oriented Mobility Assessment (POMA) is a widely used clinical measure that provides an insight into the abnormalities in balance and gait, and which has the ability to predict fall risk. Clinimetric properties like reliability and validity of the POMA have been well demonstrated (Tinetti, Reference Tinetti1986; Tinetti et al., Reference Tinetti, Baker, Garrett, Gottschalk, Koch and Horwitz1993; Raîche et al., Reference Raîche, Hebert, Prince and Corriveau2000; Faber et al., Reference Faber, Bosscher and van Wieringen2006).
The original test was developed and used in an institutionalized population of intermediate care residents with chronic diseases, who were independent or required minimal assistance in activities of daily living (Tinetti et al., Reference Tinetti, Williams and Mayewski1986). A recent study of the validity and reliability of the POMA revealed that this test is also suitable for measuring gait and mobility in patients with mild to moderate dementia (van Iersel et al., Reference van Iersel, Benraad and Olde Rikkert2007). However, the POMA has not yet been tested in populations with moderate to severe dementia. It is therefore not clear if the POMA can be used in a population of nursing home residents with moderate to severe dementia. This type of population presents special problems when testing physical performance. Dementia is a process of progressive deterioration in memory, judgment, attention and executive functions, and may be accompanied by aphasia and apraxia (DSM-IV-TR; American Psychiatric Association, 2000). The POMA items require the ability to perform executive functions, so this could make the POMA less suitable for persons with severe cognitive impairment. Because of the magnitude of the problem of falls in nursing home residents with dementia, it is important that a test is available which is feasible and reliable and has predictive abilities to identify those at increased risk of a fall in the short term. This will allow early recognition and intervention in this specific population, in order to prevent falls in a practical way. It is important to evaluate systematically the fall risk profile of each individual resident so that tailor-made preventive measures (e.g. physical therapy, drug withdrawal, hip protectors) can be readily applied to each resident. A systematic evaluation of fall risk should include an assessment of all major contributing components, such as a history of previous falls, use of drugs known to increase the risk of falls, existing medical problems (e.g. visual impairment, incontinence) and impaired balance and mobility. Each single risk factor should be assessed with the help of specific instruments, which have proven feasibility, reliability and validity in the target population. In populations with moderate to severe dementia, impaired balance and gait are a major component with respect to increased fall-risk. However, assessment methods for this component, such as the POMA, have not yet been tested in these populations. Nevertheless, the POMA has been recommended as part of a multidisciplinary evaluation tool for nursing home patients with dementia in the Netherlands (CBO, 2004; Neyens et al., Reference Neyens2006). Internationally, there seems to be a need for evidence-based advice on the best method to assess balance and gait in populations with moderate to severe dementia as part of a multifactorial evaluation.
We therefore evaluated the feasibility and examined the inter-rater reliability and the predictive ability of the POMA to predict fall risk in a population of nursing home residents with moderate to severe dementia. We focused on the prediction of fall risk in the short term (i.e. three months) because we tested the performance of the POMA in a frail population where early recognition is of paramount importance. Identifying those patients who have increased fall risk in the short term could allow the timely application of preventive measures.
Methods
Design and setting
This design was a prospective cohort study with a three-month follow-up. The Medical Ethics Committee of the Erasmus University Medical Center approved the study.
Population and study period
We included nursing home residents with a diagnosis of moderate to severe dementia, living in the psychogeriatric nursing home Smeetsland from the De StromenOpmaatGroep, Rotterdam, the Netherlands.
Residents living in the chronic care psychogeriatric department because of a diagnosis of dementia (DSM-IV-TR; American Psychiatric Association, 2000), and who were able to walk independently, were asked to participate in the study after written informed consent had been obtained from their legal guardians.
Residents who were not able to stand without the aid of a person or walk independently, or who had other types of cognitive impairment such as Korsakoff's syndrome, as well as those without written informed consent from their legal guardians, were excluded.
Of the 220 residents with dementia who were resident at the start of the inclusion period (1 May until 31 July 2008), 109 were eligible to participate in the study; 111 residents were not eligible because of the exclusion criteria. The legal guardians of 87 residents gave their informed consent. Before the test took place, six residents were no longer ambulatory and hence not able to participate further, four residents died and two residents refused to participate. Finally, 75 residents participated in the study (Figure 1).
Flow diagram of participants included and excluded in the study.
Baseline data abstracted from medical records and nursing home charts were: age, gender, medication use and comorbid conditions considered potentially causative of falls. These comorbid conditions included: visual impairment, urinary incontinence, Parkinson's disease, arthritis and other joint diseases, depression and cardiovascular diseases (AGS Panel on Falls Prevention, 2001; Tromp et al., Reference Tromp, Pluijm, Smit, Deeg, Bouter and Lips2001). Severity of dementia was defined as stage 5 or 6 on the Global Deterioration Scale (Reisberg et al., Reference Reisberg, Ferris, de Leon and Crook1982), and was based on the regular multidisciplinary team assessment by the nursing home staff, including the nursing home physician. This multidisciplinary team assessment takes place six weeks after admission to the nursing home and thereafter twice a year. The most recent assessment before the test date was taken into account.
Falls were recorded on a standardized incidence registration form (Arcares, 2002), which is part of the incidence registration system. This standard procedure is a national instrument to monitor the quality of care in nursing homes in the Netherlands (Maastricht University, Department of Health Care and Nursing Sciences, 2008). The staff are trained to complete the forms immediately that a fall has been witnessed or after a resident is found on the floor or any lower surface (Kellogg International Work Group, 1987). A committee collects the forms and makes incidence reports. Falls data were gathered from these reports after a three-month follow-up period.
POMA
The total POMA scale (POMA-T) consists of a balance scale (POMA-B) and a gait scale (POMA-G), and was administered between 1 May and 31 July 2008. The balance subtest includes eight positions and position changes that stress stability such as sitting balance, rising up from a chair, immediate and prolonged standing balance, withstanding a nudge on the sternum, balance with eyes closed, turning balance and sitting down. The gait subtest reflects gait maneuvers used during normal daily activities. The eight gait items are initiation, step length and height, step continuity, symmetry, path deviation, trunk sway, walking stance and turning while walking. Some items are graded dichotomously (can/cannot perform) whereas other items are scored 0, 1 or 2 points to denote quality of performance. The total score can range from 0 to 28 points, with lower scores indicating increased fall risk (Tinetti, Reference Tinetti1986).
To assess the inter-rater reliability, one investigator (SH), who received eight hours of training on scoring the POMA, and one investigator (CS), with 17 years’ experience in physical testing of nursing home residents with dementia, conducted the test on 75 participants. Both investigators (CS and SH) recorded the findings simultaneously without discussion.
First, each participant was invited to perform the eight maneuvers of the POMA-B as described in the original protocol (Tinetti, Reference Tinetti1986). If a participant had difficulty in understanding and following verbal instructions, we allowed the examiner to use any combination of verbal persuasion, physical cueing, and physical assistance required to get the participant to stand up from the chair, to turn 360° and to sit down. The participants were encouraged to rest between the maneuvers as necessary. If a participant was frightened, uncertain or unable to perform certain maneuvers, he or she could proceed to the next item and was scored 0 for that maneuver.
Second, we invited each participant to walk 6 meters at his or her natural walking speed, turn and walk back as described in the original protocol (Tinetti, Reference Tinetti1986). The gait subtest was performed without a walking aid since, in our experience, many older people with dementia usually forget their walking aid. To test with a walking aid would not be a true reflection of the daily practice in the nursing home. We allowed the examiners to use any combination of verbal persuasion, physical cueing, and physical assistance required to perform the gait subtest.
A note was made of movements or gestures that suggested that verbal instructions of the POMA-B or the POMA-G were not understood; for example, when a participant began walking immediately after rising, and didn't follow the instruction to stand for 3–5 seconds. In that case it was not possible to score the item “immediate standing balance.” In the gait subtest it was not possible to test the item “turning while walking” if a participant walked straight ahead and failed to follow the instruction to turn and walk back.
In order to gain insight into the effect of our assessment method, an independent physical therapist with 18 years’ experience in physical testing of nursing home residents with dementia, but who did not know this specific study group, conducted additional tests on a random subsample of 11 participants after one month. The two investigators (CS and SH) recorded their findings without discussion during the assessment. The independent physical therapist recorded her findings without discussion after performance.
Statistical analysis
Frequencies were calculated for items in which participants had difficulty in following instructions, due to cognitive impairment.
For the POMA-T, as well as for the subscales, intraclass correlation coefficients (ICCs) (2,1) were calculated to examine agreement between raters (Shrout and Fleiss, Reference Shrout and Fleiss1979).
The inter-observer agreement of the individual items of the POMA-B and the POMA-G was derived by kappa statistics because of categorical data. An inter-observer agreement of κ = 0.41 to 0.60 represents moderate agreement. An inter-observer agreement of κ = 0.61 to 0.80 represents a good agreement, while κ = 0.81 to 1.00 represents a very good agreement (Landis and Koch, Reference Landis and Koch1977).
The predictive validity was expressed in terms of sensitivity and specificity. The sensitivity was defined as the percentage of fallers within three months who were correctly identified. Specificity was defined as the percentage of non-fallers within three months who were correctly identified. Optimal cut-off values were determined by plotting Receiver Operating Characteristic (ROC) curves for the POMA-T, and for the subtests, to determine the point that provided the best trade-off between sensitivity and specificity. The area under the ROC curves (AUCs) for the POMA-T, as well as for the subtests, were also computed, with larger AUC indicating better predictive ability. It is a general rule that a test does not discriminate if ROC = 0.5; is acceptable if 0.7 ≤ ROC < 0.8; is excellent if 0.8 ≤ ROC < 0.9; and is outstanding if ROC ≥ 0.9 (Hosmer and Lemeshow, Reference Hosmer and Lemeshow2000). Positive and negative predictive values were calculated for each cut-off score for the POMA and the subtests.
Loglinear regression analysis was used to examine the relationship between each participant's POMA-T, POMA-B and POMA-G scores and the occurrence of a fall within three months. Because a considerable proportion of the participants (n = 14) did not complete the follow-up at three months we had to correct for time-at-risk by way of an offset-parameter. The time at risk ended at the moment of the first fall or at the end of the observation period (i.e. three months or earlier). Hazard ratios (HRs) and 95% confidence intervals (CIs) were calculated for POMA-scores, falls history, comorbidities and medication use. POMA-T, POMA-B and POMA-G scores were included in multivariate models, with adjustments for variables which were significantly associated with a fall in the univariate analysis.
To examine if the prediction of a fall differs between participants who had no difficulty in understanding verbal instructions and those who did have difficulty in understanding the instructions of the test, the interaction between test scores and understanding was added to the regression model. All statistical analyses were performed using SPSS software (version 15.0, SPSS INC., Chicago, IL, U.S.A.).
Results
The mean age ± standard deviation (SD) of the participants was 81 ± 8 years, and 64% were female.
The scoring of the POMA-total ranged from 6 to 28 points (mean = 18.7, SD = 5.9). The scoring of the balance subtest ranged from 0 to 16 points (mean = 9.2, SD = 4.1). The scoring of the gait subtest ranged from 3 to 12 points (mean = 8.7, SD = 2.5). Twenty residents (26.7%) experienced at least one fall during the follow-up period. Of the 75 residents who participated in the study, 14 were lost during follow-up. Three participants died, two were no longer ambulatory, and nine left the facility.
Feasibility
Despite the fact that we allowed the examiner to use verbal persuasion, physical cueing and physical assistance whenever a participant had difficulty understanding the instructions, we encountered several problems with regard to the feasibility of the POMA in this population with dementia. Thirty-one participants (41%) had difficulty following one or more instructions of the POMA due to cognitive impairment. For example, when testing the item “immediate standing balance” of the POMA-B, some participants did not stand for 3–5 seconds as required but began walking immediately after rising, or were holding the chair and it was not clear if they could stand without holding the chair. When testing the item “turning while walking” of the POMA-G, some participants did not turn but walked straight ahead. Hence it was not possible to score these items.
Inter-rater reliability
The inter-rater reliability expressed as ICCs and κ coefficients is shown in Table 1. The ICC of the POMA-T scored 0.97, the POMA-B showed an ICC of 0.97, and the POMA-G an ICC of 0.88.
Intra-class correlation coefficients of the POMA-T, the POMA-B, and the POMA-G and kappa coefficients of the individual POMA-items rated by two raters
aNo kappa (κ) statistic was computed because sitting balance was a constant.
bNo κ statistic was computed because initiation was a constant.
Kappa coefficients between the two raters varied for the POMA-B between κ = 0.65 (withstanding a nudge on the sternum) and κ = 0.92 (standing balance with eyes closed). Kappa coefficients for the POMA-G varied between κ = 0.47 (step height), κ = 0.50 (walking stance), κ = 0.54 (symmetry) and κ = 0.83 (step length). No κ statistic could be computed for the items “sitting balance” and “initiation of gait”, since these items were scored 2 and 1 respectively for all participants, and thus were treated as a constant.
In the additional testing with the independent assessor, the ICC for the POMA-T scored 0.80, the POMA-B showed an ICC of 0.75, and the POMA-G scored an ICC of 0.83.
Predictive validity
For the prediction of a fall in the short term, i.e. within three months, the optimal cut-off points for each assessment are presented in Table 2. The point with the best predictive value for the POMA-T was the score of ≤ 21 with a sensitivity of 85% and a specificity of 56%. The positive predictive value for a score of ≤ 21 was 38%, which means that the probability that a resident with a POMA-T score of ≤ 21 will experience a fall within three months is 38%. For the POMA-B, the best predictive value was a score of ≤ 11 points with 70% sensitivity and 51% specificity. The positive predictive value for a score of ≤ 11 was 35%. For the POMA-G, the best predictive value was a score of ≤ 9 points with a sensitivity of 70% and a specificity of 61%. The positive predictive value for a score of ≤ 9 was 37%.
The predictive validity of the POMA-T, the POMA-B and the POMA-G
*Cut-off score with the best predictive value.
PPV = positive predictive value; NPV = negative predictive value.
The ROC curves for the POMA-T, the POMA-B and the POMA-G are plotted in Figure 2. The AUC for the POMA-T was 0.70 (95% CI 0.53–0.81, p = 0.01). The AUC for the POMA-B and POMA-G was 0.67 (95% CI 0.52—0.81, p = 0.04) and 0.67 (95% CI 0.53 –0.81, p = 0.03) respectively.
Receiver operating characteristic curves for POMA-T, POMA-B and POMA-G.
In the univariate analysis, falls history (HR = 2.32; 95% CI 1.00–5.46), POMA-T (HR = 1.09; 95% CI 1.01–1.17) and POMA-B (HR = 1.11; 95% CI 1.01–1.23) scores were significantly associated with a fall within three months. For participants who had no problem understanding the verbal instructions (n = 44) each point lower on the POMA-T was associated with a 10% increase in fall risk. For participants who did have problems understanding at least one item of the verbal instructions (n = 31) each point lower on the POMA-T was associated with a 7% increase in fall risk. However, the interaction term was not found to be significant (p = 0.73). There was no difference in the prediction of a fall between those who had difficulties understanding instructions and those who did not. POMA-G scores, comorbidities and medication use were not found to be significant in the univariate analysis.
In the multivariate model with POMA-T, falls history, age and gender, the POMA-T remained significantly associated with a fall within three months (Table 3). Each point lower on the POMA-T was associated with a 7.9% increase in fall risk. In the multivariate model with POMA-B, falls history, age and gender, the POMA-B subtest did not remain significant. In the multivariate model with POMA-G, falls history, age and gender, the POMA-G subtest was not significant.
Associations between POMA-scores and the occurrence of a fall
*Adjusted HR for age (continuous variable), gender and falls history.
aPOMA-T scores for those without problems in understanding verbal instructions.
bPOMA-T scores for those with problems in understanding at least one item of the verbal instructions. Interaction term p-value = 0.73.
Discussion
In this study population of nursing home residents with a diagnosis of moderate to severe dementia, we found a POMA-T score ranging from 6 to 28 (mean = 18.7, SD = 5.9), a POMA-B score ranging from 0 to 16 (mean = 9.2, SD = 4.1), and a POMA-G score ranging from 3 to 12 points (mean = 8.7, SD = 2.5). We found good inter-rater reliability and an acceptable validity to predict a fall within three months. Nevertheless, the POMA cannot be recommended yet as an assessment method for balance and gait in populations with moderate to severe dementia, because several feasibility problems were observed. We found that 41% of the participants had some degree of difficulty in following the instructions of the POMA due to cognitive impairment. The performance of the test is dependent on how well the participant can understand instructions. In this regard, the items of the balance subtest were particularly difficult to perform because they contain dual tasks. If a participant was unable to perform a certain item, he or she scored 0. When the score of a specific item is zero because of major cognitive problems, severity of dementia could have been the predictor of a fall rather than the balance or gait impairment. This stresses the need for further studies, focusing on the differences in performance of the POMA between persons who do or do not understand the instructions.
In our study, we did not find that the association between the test scores and fall risk was different in those who had difficulty following verbal instructions and those who did not, but this could be due to lack of power. We found that participants with no problems in understanding verbal instructions had an increased fall risk of 10% per point lower on the POMA-T, whereas participants with problems in understanding verbal instructions showed a 7% increase of fall risk for each point lower. However, these differences were not significant. With larger samples, perhaps, significant differences in results between participants who understand verbal instructions and those who do not would be identified.
Because of the low feasibility of the POMA in severe dementia, the performance of assessment methods that are less dependent on cognition should be evaluated, such as, for example, electronic walkway systems (Hausdorff et al., Reference Hausdorff, Rios and Edelberg2001).
The POMA has the advantage of ease of administration and low cost and was therefore chosen for initial testing, but our results show that its application is hampered by feasibility problems and alternatives should be considered after careful evaluation.
Other potential limitations of our study are related to our method of assessing inter-rater reliability. First, two raters working in the same room with a patient might provide opportunities for mutual influences. However, in the additional testing we observed that the inter-rater reliability between three raters remained good. Second, the reliability outcomes of the study might be overestimated, since reliability is likely to be influenced by variability in the tester's instructions, tone of voice, encouragement, and so on.
Our results support previous findings that, in general, the inter-rater reliability of the POMA is good to very good. In a study of cognitively intact community-dwelling older people, an inter-rater reliability for the POMA-T, as well as for the subscales, of over 0.95 was reported (Tinetti et al., Reference Tinetti, Baker, Garrett, Gottschalk, Koch and Horwitz1993). In another study of 30 residents living in either self-care or nursing-care residences, with a mean age of 84.9 years, inter-rater reliabilities for the POMA-T and the subscales ranged from 0.80 to 0.93. In this study, none of the participants had dementia, or they had only mild cognitive impairment. Residents with moderate dementia were excluded (Faber et al., Reference Faber, Bosscher and van Wieringen2006). In one study of 29 hospital inpatients and nursing home residents focusing on the inter-rater reliability of scores on the eight individual items of the POMA-B, κ coefficients ranging from 0.40 to 1.00 were reported, indicating fair to good reliability (Cipriany-Dacko et al., Reference Cipriany-Dacko, Innerst, Johannsen and Rude1997). None of the aforementioned studies on the inter-rater reliability of the POMA was carried out in a population of residents with dementia. One study on the reliability of a physiological test battery designed to assess fall risk in a population of 21 community-dwelling older people with mild to moderate Alzheimer's disease, reported an ICC of 0.90 for the balance range test (Lorbach et al., Reference Lorbach, Webster, Menz, Wittwer and Merory2007).
Our results on predictive validity are also comparable to previous studies. With regard to the predictive validity observed in our study, given optimal cut-off criteria, sensitivity ranged from 70% to 85% and the specificity ranged from 51% to 65%, with better predictive validity for the POMA-T than for the subtests only. In our opinion, the high sensitivity we found is more important than the lower specificity in our study. The identification of a large percentage of fallers within three months creates the opportunity for efficient intervention. For example, physical exercise has been shown to improve physical performance and reduce fall risk in a population of ambulatory nursing home residents with mild to severe Alzheimer's disease (n = 134) (Rolland et al., Reference Rolland2007). The consequences for the false positives, i.e. non-fallers who were not correctly identified, would not be harmful. They will probably experience a fall during a longer follow-up period, and the interventions (e.g. physical exercise) are not likely to have any damaging side effects. A cut-off value of ≤ 21 for POMA-T therefore seems excellent.
In a study in which the same version of the POMA scale was used, a sensitivity of 64% for the POMA-T, as well as for the subscales, a specificity ranging from 62.5% to 66.1 %, and cut-off values of 19 for POMA-T, 10 for POMA-B, and 9 for POMA-G were reported. In this study, a follow-up of 10 months was chosen (Faber et al., Reference Faber, Bosscher and van Wieringen2006). In another study, a sensitivity of 70%, and a specificity of 52% were found for a group of 225 community-dwelling older people with a mean age of 80.0 years. In this study, the 40-point version and a one-year follow-up were chosen (Raîche et al., Reference Raîche, Hebert, Prince and Corriveau2000).
With regard to the predictive accuracy, our study also has limitations. First, all residents were tested without their usual walking aid, which could have lowered the predictive accuracy. However, there are no studies known which show that walking with a walking aid can reduce falls (CBO, 2004). Second, sampling error could have influenced our results on predictive validity. Although we obtained a high consent rate (87 of 109 eligible patients), consent may perhaps have been lower among legal guardians of residents in more severe stages of dementia and the predictive validity of the POMA may have been overestimated. Finally, a three-month follow-up is inconsistent with studies of other high risk populations, such as older people presenting to Emergency Departments after a fall, in which a one-year follow-up was chosen. However, the results of such studies cannot be extrapolated to people in residential care (Russell et al., Reference Russell, Hill, Day, Blackberry, Gurrin and Dharmage2009).
Despite the described limitations, our results regarding the predictive validity are comparable with other studies and other populations, where longer follow-up times have been chosen. We expected that during a longer follow-up period almost the whole population would experience at least one fall or would be lost to follow-up. A three-month follow-up period was chosen, in the first place, to test the performance of the POMA for the timely recognition of fall risk in a frail population. In addition, a 12-month follow-up period, as recommended for community-dwelling older persons (Lamb et al., Reference Lamb, Jørstad-Stein, Hauer and Becker2005), would have caused much difficulty in interpretation, since in our study population many participants would have shown changes in medication and physical status over such a period.
We conclude that application of the POMA in populations with moderate to severe dementia is hampered by feasibility problems. Its routine implementation in clinical practice cannot therefore be recommended, despite good reliability and an acceptable predictive validity.
To the best of our knowledge, this is the first study to assess the POMA in nursing home residents with moderate to severe dementia. However, because of the possible lack of sufficient power in our study, we do not know in detail if and how the prediction of fall risk differs between those who understood the instructions and those who did not. To refine our findings, a large prospective study on the predictive validity of the POMA in a population with moderate tot severe dementia is needed, and should focus on the differences in understanding of instructions between those with mild, moderate and severe dementia. In addition, the performance of mobility assessment methods that are less dependent on cognition, and might therefore be more feasible, should be evaluated in nursing home residents with dementia.
Conflict of interest
None.
Description of authors’ roles
Carolyn Sterke and Sawadi Huisman designed the study, performed the examinations, analyzed the data, and wrote the paper. Ed van Beeck advised on the study and participated in writing the paper. Caspar Looman advised on the statistical analysis. Tischa van der Cammen initiated and developed the study, and participated in and supervised the writing of the paper.
