Introduction
Many very old people (those aged ≥85 years), who have become dependent on assistance in their basic activities of daily living because of long-term physical or/and mental disability, need long-term care services like nursing homes. The number of nursing home beds per 1000 people aged 65 and older varies from 11.6 in Japan, 47.8 in Germany, to 95.4 in Canada, while the Organisation for Economic Co-operation and Development (OECD) average is 40 (OECD, 2008).
Nursing home residents are known to suffer from several chronic illnesses resulting in substantial physical dependence and/or cognitive impairment (Nygaard et al., Reference Nygaard, Naik and Ruths2000). The main medical diagnosis in nursing home residents is dementia. This is a common condition among the elderly, which affects an estimated 6.4% of all persons aged 65 years and over, increases with age, and doubles every five years. Today 35.6 million people have dementia worldwide (World Alzheimer Report, 2010).
Dementia is an acquired syndrome of decline in memory and other cognitive functions, sufficient to affect the daily life of the patients concerned (Cobbs et al., Reference Cobbs, Duthie and Murphy2002). The cognitive status can be measured with the Mini-Mental State Examination (MMSE) (Folstein et al., Reference Folstein, Folstein and Mchugh1975).
As dementia progresses, the person concerned increasingly requires care due to decreasing cognitive performance, which also has a measurable influence on the health-related quality of life (HRQOL) (Dreer et al., Reference Dreer2007; Owsley et al., Reference Owsley, McGwin, Schilley, Meek, Sker and Dyer2007). As a key endpoint and a patient-relevant outcome in health service research, the maintenance of HRQOL is important (Gunzelmann et al., Reference Gunzelmann, Hinz and Brähler2006) since complete recovery is either impossible or unlikely (Moons et al., Reference Moons, Budts and de Geest2006).
The term HRQOL has been defined as an individual's subjective experience of the impact that illnesses and their treatments have on the individual's functioning in a variety of domains such as physical, psychological, and social functioning as well as the impact of illnesses on the ability to engage in activities of daily living (Cella, Reference Cella1996). According to Kane et al. (Reference Kane2003), 60% of nursing home residents are able to report reliably on their HRQOL. In order to obtain response rates of this magnitude and to avoid information loss, the questionnaires for settings with cognitively impaired persons should be as simple as possible (Kane et al., Reference Kane2003).
According to Post et al. (Reference Post, Gerritsen, van Leusen, Paping and Prevo2001), one of the commonly used HRQOL questionnaires that meets these requirements is the Nottingham Health Profile (NHP). The NHP is designed to measure perceived emotional, social, and physical health problems and the extent to which such problems affect daily activities.
So far, the NHP has been used in a variety of settings – for example, among people with severe physical disabilities (Post et al., Reference Post, Gerritsen, van Leusen, Paping and Prevo2001), in the general population (Hinz et al., Reference Hinz, Klaiberg, Schumacher and Brähler2003), and among dementia patients (Boyer et al., Reference Boyer, Novella, Morrone, Jolly and Blanchard2004). The NHP was found to work best with chronic, disabling conditions, with elderly populations, and with cognitively impaired persons (Wann-Hansson et al., Reference Wann-Hansson, Klevsgard and Hagell2008), but so far the German version has not been used and tested in nursing homes. Earlier studies have shown different rates of administration in patients with severe cognitive impairment (MMSE score ≤ 10) ranging from 5.9% to 80.8% (Bureau-Chalot et al., Reference Bureau-Chalot, Novella, Jolly, Ankri, Guillemin and Blanchard2002; Baro et al., Reference Baro2006), which is a huge variance. The administration rate is, among others, an important issue for choosing a questionnaire, as researchers need comprehensive statements regarding the quality of life and conditions of settings. To our knowledge, there have been no analyses so far examining the degree of cognitive impairment for which the NHP can be used appropriately. Among other parameters, the administration rate depends on missing items. Even when there are missing items, the questionnaire will be still analyzable up to a certain threshold and therefore missing items do not influence the administration rate in general. The possible number of missing items, however, varies from questionnaire to questionnaire; too many will reduce its feasibility. A short duration of administration can be advantageous for residents with a limited ability to concentrate.
We wanted to find out, in particular, whether the NHP is a feasible instrument to use with nursing home residents. Therefore, we operationalized this aim in the following four steps:
1. What is the administration rate regarding the cognitive status for the NHP?
2. Up to what grade of cognitive impairment can the NHP be administered?
3. How many missing items appear regarding the cognitive status?
4. How long does it take to complete the questionnaire and what type of administration (self-completion or administered by interview) is required to complete the NHP?
In addition, we analyzed the internal consistency and test–retest reliability of the NHP in this setting.
Methods
Study design
The present study was designed as a prospective multicenter observational study incorporated in the project “Health-related quality of life of residents in nursing homes in Germany,” which aims to evaluate the HRQOL with regard to pressure ulcers, falls, incontinence, care dependency, and structural factors of the nursing homes, such as staff qualifications and activities provided for the residents. Ethical approval was given by the local Ethical Committee.
Setting and participants
Eleven out of 288 nursing homes in Berlin and Brandenburg, each with a minimum of 50 beds, were randomly selected for this study. Permanent residents newly admitted to the nursing homes, or their legal representatives, were informed of the project. One of the inclusion criteria was that written informed consent of nursing home residents or their legal representatives be obtained within the first two weeks of admission. Nursing home residents with a probable remaining life expectancy of less than four weeks were excluded from our study. Short-time residents with a planned stay of less than four weeks were also excluded.
Questionnaire
The NHP was developed during the 1970s in the UK. The questionnaire was designed to be a standardized and simple measurement of the subjective health status in physical, social, and emotional domains. It was validated in different settings and to date has been translated into 22 languages (PROQOLID, 2010).
The German validation was performed by Kohlmann et al. in the 1990s in ten different samples of more than 1000 healthy and ill persons, such as medical students, citizens of a town, rehabilitation patients with different illnesses, and outpatients (Kohlmann et al., Reference Kohlmann, Bullinger and Kirchberger-Blumstein1997).
The German-validated version of the NHP consists of 38 items in six domains: “Energy” (three items), “Pain” (eight items), “Emotional reactions” (nine items), “Sleep” (five items), “Social isolation” (five items), and “Physical mobility” (eight items). The items are answered either by “yes” if the statement adequately reflects the individual's current status or feelings or by “no” if it does not. Positive responses were weighted based on Thurstone's Method of Paired Comparisons. Negative responses were not weighted. The values of all statements of one domain were added to produce the domain total. Domain scores ranged from zero (good subjective health status) to 100 (poor subjective health status) (McKenna et al., Reference McKenna, Hunt and McEwen1981). The questionnaire was regarded as being analyzable if not more than one item per domain was missing, except for the three-item domain “Energy,” where all items had to be answered (Kohlmann et al., Reference Kohlmann, Bullinger and Kirchberger-Blumstein1997).
A pilot project was conducted to refine the data collection procedures. In this pilot phase, several residents believed that the NHP statements were actually made by the research assistants rather than part of the instrument. Therefore, for our purpose and in agreement with the German author Kohlmann, the NHP was modified from statements into questions. For instance, the original statement “Everything is an effort” in our study was changed to “Is everything an effort for you?” No other changes were made.
Data collection
The recruitment period ran from April 2008 to December 2009. The modified NHP was administered to each nursing home resident in the second week following admission and a second time in the fourth week after admission to determine the test–retest reliability. The time interval of two weeks was considered appropriate according to Streiner and Norman (Reference Streiner and Norman2008), because no substantial changes were expected in the HRQOL in these settings.
Each interview was carried out in the resident's room. Where the resident was unable to complete the NHP independently due to sight or other physical impairments, NHP interviews were conducted by research assistants. Each resident was interviewed by the same research assistant on both occasions. If the resident was unable to answer the questions – for example, if he/she could not communicate with the research assistant or was disoriented and talked about things other than those asked, the NHP was not collected.
The resident's age, sex, marital status, cognitive status, level of care dependency, medical diagnoses, and all currently taken drugs were documented by the research assistants using a machine-readable data form.
Each resident's cognitive status was evaluated using the MMSE, which has scores ranging from 0 (very severe cognitive impairment) to 30 (no cognitive impairment). The MMSE is widely used and has been validated in long-term care populations, like nursing homes (Folstein et al., Reference Folstein, Folstein and Mchugh1975; Kempen et al., Reference Kempen, Brilman and Ormel1995). The MMSE scores were categorized as follows:
(a) normal cognitive function (MMSE ≥ 21);
(b) moderate cognitive impairment (MMSE 10–20); and
(c) severe cognitive impairment (MMSE <10).
Completed forms were scanned, evaluated, verified, and exported to a database.
Data analysis
Demographic variables were analyzed using descriptive statistics. Normally distributed variables were summarized using the mean and standard deviation (SD). Where data were not normally distributed, medians and inter-quartile ranges (IQR) were reported. Comparisons between groups – such as age, gender, cognitive status, and dementia (yes/no) – were made using the χ2 test, median test, and t-test (as appropriate). A difference was considered statistically significant when P <0.05.
The feasibility of the NHP was determined using the following indicators:
1. The administration rate was calculated for all six domains together and for each individual domain. The condition was that not more than one item per domain should be missed except for the three-item domain “Energy,” where all items had to be answered (Kohlmann et al., Reference Kohlmann, Bullinger and Kirchberger-Blumstein1997).
2. To determine the level of cognitive status at which the NHP was inadequate, we calculated a cut-off point for the MMSE score using the receiver operating characteristics curve (ROC) and calculating the area under the curve (AUC). An ROC curve is a plot of a measurement's sensitivity (true positive rate) versus one minus specificity (false positive rate) at each cut-off score. A theoretical, optimal prediction can achieve 100% sensitivity and 100% specificity. The higher the sensitivity and specificity of an instrument, the better its diagnostic accuracy. An AUC of 0.5 indicates random performance, whereas an AUC of 1.0 denotes perfect performance (Zweig and Campbell, Reference Zweig and Campbell1993).
3. An important aspect for the feasibility of a questionnaire is the number of missing items. If more than one item per domain (except for the domain “Energy”) was missing, we considered that the questionnaire could not be analyzed. Therefore, the percentages of missing items and any frequently missing items in each domain were calculated. For the calculation, at least one item per domain had to be completed.
4. The time and type of administration (self-completion or administered by interview) were analyzed.
Because the items in each domain should address different aspects of the same attribute, the internal consistency was tested using Cronbach's α coefficient. Cronbach's α ≥ 0.7 was considered to indicate acceptable internal consistency for each domain (Cronbach and Warrington, Reference Cronbach and Warrington1951; Nunally and Bernstein, Reference Nunnally and Bernstein1994).
Test–retest reliability of each domain score was calculated using the intraclass correlation coefficient (ICC) (Shrout and Fleiss, Reference Shrout and Fleiss1979) at 95% confidence intervals (CIs). The ICC was estimated using a two-way mixed variance analysis model, with values of 0.40–0.75 indicating moderate to good agreement, and values greater than 0.75 indicating excellent agreement (Nunally and Bernstein, Reference Nunnally and Bernstein1994).
The complete statistical analysis was performed using SPSS Statistics 19.0 for Windows.
Results
Sample
A total of 553 residents were newly admitted in the selected nursing homes. Of those, we excluded 37 (6.7%), because informed consent had not been signed within two weeks due to language problems or other reasons. The remaining 516 (93.3%) residents were informed about the project, of whom 209 (40.5%) refused to participate. The remaining 307 (59.5%) residents agreed to participate in the project and fulfilled the inclusion criteria. However, a further 21 residents could not be included in the final NHP analyses for the following reasons:
• the resident was in hospital at the time the NHP was administered (n = 7)
• the resident refused to participate in the NHP questionnaire (n = 10)
• The resident died before the first NHP was administered (n = 4)
Thus, the final sample size was 286 residents, 69.2% (n = 198) of whom were women. The mean age of the residents was 83.6 years (SD ± 8.8 years).
Completion of the MMSE was refused in 11 cases and missing in 35. However, the groups did not differ according to age (refused: 80.3 years, SD ± 10.4, p = 0.222; missing: 81.6 years, SD, 8.7, p = 0.157) and sex (refused: women, n = 6, p = 0.282) except for the missing group which contained significantly fewer women (missing: women, n = 18, p = 0.015).
The MMSE score could be determined for 240 residents: the median MMSE score was 17 (IQR: 4, 23). The group with normal cognitive functions (MMSE score >21) comprised 88 residents (36.7%); the group with moderate cognitive impairments (MMSE score 10–20) comprised 74 residents (30.8%), and the group with severe cognitive impairments (MMSE score <10) comprised 78 residents (32.5%) (n = 78).
Overall, dementia was documented in 50.7% (n = 145) of the residents.
Feasibility
Administration rate
It was possible to administer the NHP in all six domains to 44.4% (n = 127) of the residents.
The administration rate for each domain varied between 51.0% (n = 146) (domain “Pain”) and 60.1% (n = 172) (domain “Sleep”) (Table 1). Nearly a third of the residents (30.7%; n = 88) answered all 38 items of the NHP.
Administration rate of the Nottingham Health Profile and frequently missing items (n = 286)
* A maximum of one item per domain can be missing; only in the domain “Energy” must all items be answered.
It was possible to administer the NHP to 76.1% (n = 66) of the residents with normal cognitive function, to 48.6% (n = 36) of those with moderate cognitive impairment, and to 9.0% (n = 7) of those with severe cognitive impairment.
Table 2 shows the comparison of the residents’ demographic and cognitive status as regards the NHP administration.
Characteristics of residents according to Nottingham Health Profile (NHP) administration (n = 286)
a t-test.
b χ2 test.
c Median test.
Determination of the MMSE cut-off point
The accuracy of the MMSE for predicting the completion of the NHP was calculated using the area under the ROC curve (Figure 1) for all 240 residents where the MMSE score was determined. With a value of 0.80, the AUC showed that the MMSE is a good predictor for NHP completion. The most suitable cut-off point of the MMSE score for administering the NHP was given with a score of 16 and a sensitivity of 80% and specificity of 76%.
Receiver operating characteristic: Nottingham Health Profile vs. MMSE (n = 240).
Missing items and frequently missing items
The most frequently missing item was detected in the domain “pain when climbing up and down stairs” with 47.2% (n = 69) (Table 1).
The highest percentage of missing items was found in the group of severely cognitively impaired residents, with statistically significant differences in the group of moderately impaired residents in the four domains of “Pain,” “Emotional Reaction,” “Sleep,” and “Physical Mobility” (p<0.001). Almost equal rates of missing items were observed for the domain “Energy” (Figure 2).
Percentage of missing items of the Nottingham Health Profile domains according to cognitive status.
Time and type of administration (self-completion or interview)
The average time to complete the questionnaire was 12.6 (SD ± 6.0) minutes.
The administration of the NHP (n = 127) was performed by interview in 92.1% (n = 117) of cases, while 4.7% (n = 6) of the residents completed the NHP independently; in 3.1% (n = 4) of cases, the form of administration was not documented.
Internal consistency and test–retest reliability
Four of the six domains had a Cronbach's α coefficient ≥ 0.7; (“Pain” α = 0.8, “Emotional Reaction” α = 0.7, “Sleep” α = 0.7, “Physical Mobility” α = 0.7). In the domains “Energy” and “Social Isolation,” the Cronbach's α was 0.6.
In all domains, the ICC values were above 0.70 (Table 3).
Intra-rater reliability of the Nottingham Health Profile
Discussion
This study analyzed the feasibility, internal consistency, and test–retest reliability of the modified NHP questionnaire in nursing home residents with respect to their cognitive status. To our knowledge, the NHP has so far not been used and tested in residents of nursing homes in Germany. In our study, the NHP showed an acceptable internal consistency and excellent test–retest reliability.
We also found that the NHP is a feasible questionnaire for nursing home residents, but the rate of administration decreased with increasing cognitive impairment. The administration rate in severely cognitively impaired residents was 9.0% (n = 7). A study in Spain of older adult inpatients with varying cognitive functions found similar results, with an administration rate of 5.9% (Baro et al., Reference Baro2006). A completely different result was found in a study in France which used the NHP in dementia patients, with a completion rate of 80.8% in patients with severe cognitive impairment (Bureau-Chalot et al., Reference Bureau-Chalot, Novella, Jolly, Ankri, Guillemin and Blanchard2002). A reason for this huge difference might be the small sample size (n = 17) compared to our study (n = 78). A further reason could be that the interview was conducted in an environment familiar to the patients, which can increase the administration rate because the interviewer knows the residents well and therefore also his/her current condition and the best time to administer the questionnaire.
Compared to the study by Kane et al. (Reference Kane2003), where 60% of nursing home residents were able to report reliably on their quality of life, our administration rate was lower than previously expected. Overall, the rate of administration of 44.4% was lower than that of 70.1% established by Baro et al. (Reference Baro2006), although we chose a questionnaire with a dichotomized response scale and asked the patients for their current state of HRQOL (“at the moment”), which should have supported the administration rate (Gerritsen et al., Reference Gerritsen, Steverink, Ooms, de Vet and Ribbe2007).
It was conspicuous that the group of moderately impaired residents showed similar rates of administered and non-administered NHPs, which is different from the Spanish study. Baro et al. (Reference Baro2006) found an administration rate in moderately impaired patients of 86.3%, which is higher than that of our study (59.0%) even though both studies were administered by interview. It might be possible that residents with a normal cognitive status understood the necessity of the study in contrast to the group of moderately impaired residents. Taking the results of Gerritsen et al. (Reference Gerritsen, Steverink, Ooms, de Vet and Ribbe2007) into account, the administration rates of the questionnaires depend significantly on both the resident's willingness to answer questions and on his/her cognitive status.
Generally, a comparison with other studies was sometimes difficult. In most of the studies, there was a lack of definition of what is meant by administered/completed questionnaires. In addition, scales were calculated differently across studies; for example, the study by Boyer et al. (Reference Boyer, Novella, Morrone, Jolly and Blanchard2004) did not compute the domain scores when one item was missing, while other studies calculated the domain scores even if half of the items were missing (Hinz et al., Reference Hinz, Klaiberg, Schumacher and Brähler2003). The different definitions and calculation procedures can therefore invalidate comparisons based on rates of administration, although sample sizes and types of administration are similar. Thus, study comparison is limited and comparative results should be handled with care.
Our study also found correlations between the level of cognitive impairment (measured with the MMSE) and the NHP administration rate. We found that best results were achieved with an MMSE score ≥ 16. This, however, does not mean that the NHP cannot be administered to residents with a lower MMSE score. The score under the curve (0.80) and the number of residents included (n = 240) indicated that the MMSE cut-off point of 16 is a good reference point for similar settings (Heinz, Reference Heinz2006).
The number of missing items appear to increase with decreasing cognitive function. Furthermore, some items seem to be inapplicable to nursing home residents. The most frequently missing item was “pain when climbing up and down stairs or steps” in the domain “Pain” (47.2%, n = 69). The Spanish study by Baro et al. (Reference Baro2006) also tested the applicability of items and found that three of the eight items of the domain “Pain” were not applicable, including “pain when climbing up and down stairs”. Unfortunately, our study did not collect data about the applicability of the items.
The duration of administration can be regarded as excellent (12.6 ± 6.0 min) and, when compared with studies undertaken in dementia settings (13.0 ± 10.0 min; 13.5 min (2–60) (Bureau-Chalot et al., Reference Bureau-Chalot, Novella, Jolly, Ankri, Guillemin and Blanchard2002; Boyer et al., Reference Boyer, Novella, Morrone, Jolly and Blanchard2004), it is quite low.
In the present study, almost all questionnaires were administered via interview (92.1%) which was similar to the French study of dementia patients (77.0%) (Boyer et al., Reference Boyer, Novella, Morrone, Jolly and Blanchard2004).
The analysis of internal consistency was found to be acceptable in four of the six domains: “Pain,” “Emotional reaction,” “Physical mobility,” and “Sleep.” Residents associated the items as homogeneous so it was possible to assign them to one domain. The domains “Social isolation” and “Energy” showed a weak internal consistency (Cronbach's α = 0.6) (Cronbach and Warrington, Reference Cronbach and Warrington1951; Nunally and Bernstein, Reference Nunnally and Bernstein1994). This low internal consistency was also reported in some other studies (Kohlmann et al., Reference Kohlmann, Bullinger and Kirchberger-Blumstein1997; Bureau-Chalot et al., Reference Bureau-Chalot, Novella, Jolly, Ankri, Guillemin and Blanchard2002). One explanation for the lack of sufficiently high internal consistency might be the fact that some domains only included a small number of items leading to a lack of homogeneity of the items and finally to a low reliability.
The ICC revealed excellent test–retest reliability in all domains (values > 0.76), with similar results to those reported previously (Bureau-Chalot et al., Reference Bureau-Chalot, Novella, Jolly, Ankri, Guillemin and Blanchard2002).
Limitations
One limitation lies in the process of data collection itself. Before NHP administration, the resident was questioned and examined with respect to falls, incontinence, pressure ulcers, malnutrition, and cognitive status. It therefore might be possible that tests of cognitive status were undertaken when the residents were already tired and exhausted because of the physical and mental activity that occurred before NHP administration. A further limitation was that no information was collected about the reasons for not answering the questions. It must also be recognized that the low administration rate might also have a negative impact on the generalizability of the results found in our study.
Conclusion
Our study results show that the NHP and therefore the HRQOL can be administered in nursing homes depending on the residents’ cognitive status. The NHP is feasible for residents with normal cognitive function and for those with moderate cognitive impairment.
Conflict of interest
None.
Description of authors' roles
Manuela Tabali participated in the design of the study, helped perform the statistical analysis and also drafted the paper. Elke Jeschke made substantial contributions to the interpretation of data and statistical analysis. Thomas Ostermann helped with the interpretation of the data, and with the drafting and critical revision of the paper. Theo Dassen conceived of the study and participated in its design. Cornelia Heinze conceived the study and participated in its design and coordination, and in critical revision of the paper. All authors read and approved the final version of the paper.
Acknowledgment
The authors thank Professor T. Kohlmann for his valuable advice and support.
