Introduction
Caregivers of relatives with dementia represent over 10 million people in the USA alone, and this population is growing due to increased survival to older age (Schulz and Martire, Reference Schulz and Martire2004). Dementia caregivers experience chronic stress associated with serious health-related problems in this population (Schulz and Sherwood, Reference Schulz and Sherwood2008; Mills et al., Reference Mills2009; von Kanel et al., Reference von Kanel, Ancoli-Israel and Dimsdale2010; Fonareva et al., Reference Fonareva, Amen, Zajdel, Ellingson and Oken2011), and their stress impacts on the quality of care they provide. Every caregiver is an asset for the family and society, and ensuring their well-being is an important goal both for primary care providers as well as for researchers evaluating interventions to combat effects of caregiver stress and improving coping with the caregiving burden (Schulz and Martire, Reference Schulz and Martire2004).
Accurate assessment of the physical and psychological symptoms experienced by caregivers of people with dementia is critical for both primary care providers and investigators involved in caregiver research. Some of the most important aspects of function affecting caregivers’ health include perceived stress and stress-related measures such as fatigue, attitudes to caregiving, and coping with the caregiving burden. These measures are usually assessed either in a clinic or research center by self-reports that summarize symptoms retrospectively over a certain period of time, e.g. the degree to which people felt “stressed” in the previous week. This type of retrospective recall is affected by bias such as current mood state or recall biases (e.g. current events more easily recalled) (Yoshiuchi et al., Reference Yoshiuchi, Yamamoto and Akabayashi2008). Furthermore, the source of caregiver stress originates at home, and asking caregivers how stressed they feel in a clinic or research center might not be optimal when evaluating perceived stress due to caregiving. When using typical assessment methods outside the home, clinicians might underestimate the severity of the caregiver's home situation. In fact, reviews of several studies testing dementia caregivers’ interventions indicated modest effect sizes on stress-related outcomes and also suggested there might be a room for improvement in the quality of interventions as well as in the quality of evaluations (Acton and Kang, Reference Acton and Kang2001, Brodaty et al., Reference Brodaty, Green and Koschera2003, Pinquart and Sorensen, Reference Pinquart and Sorensen2006, Thompson et al., Reference Thompson2007, Smits et al., Reference Smits2007). Some of these results might be due to suboptimal evaluation methodology, including using retrospective recall measures in artificial environments when assessing main outcomes. Exploring alternative assessment methods that avoid recall biases and permit testing in natural environments is an important step in furthering research in caregiver population interventions.
Ecological momentary assessment (EMA) is a method of evaluating “phenomena at the moment they occur in natural settings, thus maximizing ecological validity while avoiding retrospective recall” (Yoshiuchi et al., Reference Yoshiuchi, Yamamoto and Akabayashi2008). Crucial EMA features include collecting data in subjects’ natural environments, assessing current feelings or states rather than recall of past experiences, and performing multiple evaluations (Shiffman et al., Reference Shiffman, Stone and Hufford2008). EMA can be implemented as questionnaires accessible via palm-size computers or touch-phones that can be programmed to alert people during the day to complete brief assessments of their current situation (Shiffman et al., Reference Shiffman, Stone and Hufford2008). Numerous studies have successfully used EMA for assessment of various psychosocial and physiological outcomes, ranging from engagement in physical activity to craving abused drugs (Shiffman et al., Reference Shiffman, Stone and Hufford2008). Our first aim was to assess the feasibility and utility of EMA for evaluating stress-related outcomes in a sample of elderly dementia caregivers and non-caregivers. Our second aim was to compare results of stress-related assessments conducted in the research center and in the home environment between dementia caregivers and non-caregivers using EMA. We hypothesized that ratings of stress-related measures are differentially affected by the assessment environment in caregivers and non-caregivers, with caregivers reporting greater stress at home than in the research center. To the best of our knowledge this is the first study utilizing EMA for evaluating stress-related outcomes in caregivers and non-caregivers with results compared between research center and home environments.
Methods
Participants
Participants included 41 adults (mean age, 66.4 years; 97.6 % white) from the metropolitan area of Portland, Oregon, USA. Consent and preliminary screening were completed prior to participating in study procedures. Study inclusion criteria were: (i) being aged between 50 and 85 years; and (ii) being in good physical and cognitive health. Exclusion criteria, evaluated by a self-report during a screening interview, included serious medical conditions (e.g. cancer), significant current mental health problems (e.g. untreated major depression), diagnosis of neurological or sleep disorders, use of medications affecting cognitive function, starting new medications less than two months ago, recent use of illicit drugs, and excessive alcohol consumption.
Participants in the caregiver group had to be primary caregivers for a family member with dementia. They had to be involved in the care of their relative with dementia for at least 12 hours a week. Enrolled caregivers participated in a mind-body intervention study to reduce stress (Oken et al., Reference Oken2010), but the assessments for the current study were completed at the baseline visit prior to randomization and any interventions. Caregivers were recruited by posting study flyers at the university neurology clinic and in the community, as well as by advertising in local caregiver support groups.
The participants for the non-caregiver group had to be free of any caregiving duties and of any major stress (e.g. recent death in the family) at the time of the study. Non-caregivers were recruited from the community by posting study flyers at adult community centers and other public locations. To control for seasonal effects, subjects for both groups were recruited in several interleaved cohorts. The study protocol was approved by the Oregon Health & Science University Institutional Review Board.
Measures
Baseline self-report measures
Prior to completing EMAs all participants completed the Perceived Stress Scale (PSS), a commonly used measure assessing perceptions of stress during the previous week (Cohen et al., Reference Cohen, Kamarck and Mermelstein1983). The PSS was only used at baseline to evaluate group differences in perceived stress at intake. The caregivers also filled out the Revised Memory and Behavior Problem Checklist (RMBPC) that assesses behaviors of care recipients and caregivers’ reactions to the reported behaviors (Teri et al., Reference Teri1992).
Ecological Momentary Assessment questions
EMA testing included five questions assessing several stress-related measures. Though some questions (e.g. current fatigue) were adapted from widely used instruments (Hoddes et al., Reference Hoddes, Zarcone, Smythe, Phillips and Dement1973), these five questions do not comprise a validated instrument but rather were chosen based on our previous findings indicating aspects of functioning affected by caregiving and remediable by a stress-reducing intervention (Oken et al., Reference Oken2010). Our goal was to have a brief assessment that can be completed in less than 5 minutes to minimize the burden on our participants. The questions with the rating scales that were used for EMA assessment at home and in the research center are presented in Appendix 1.
Ecological Momentary Assessment implementation and protocol
EMA was implemented using a personal digital assistant (PDA), Dell® Axim x51v running Microsoft® Windows Mobile 6.1. The assessment included a brief questionnaire consisting of the five questions described in Appendix 1. Each question was displayed separately on the screen followed by the prompt asking participants to rate each of the measures (current perceived stress, ability to cope with the present situation, mindfulness, demand of the present situation, and fatigue).
Research center assessment: During the research center visit, participants received training on using the PDA and completed a single 5-minute assessment identical to the EMA presented in their home. At the end of the visit, a research assistant inquired about the participant's bedtime and wake times and plans for the rest of the day. This information was used to program the PDA to initiate the assessments at three or four semi-random time points during participants’ wakeful hours. Participants were not told the times or number of scheduled EMAs.
EMA in home environment. In the home environment an alarm signaled the beginning of semi-randomly scheduled assessments. If the alarm sounded at an inconvenient time participants could postpone the assessment for 10 minutes. If after postponing the EMA twice the participant was not able to complete it, that particular assessment was automatically abandoned. The software stored the data and the information (including time) of all scheduled, incomplete, and completed assessments.
The following day a research assistant collected the PDA from the participant's home and inquired about their experience including any problems associated with using the device.
Statistical analyses
Statistical analyses were conducted using IBM SPSS Statistics Version 19 for Windows. Demographic data for the two groups were compared using t-tests or χ2 tests.
For each assessment measure for each participant, we used a single score obtained during the research center visit for the research center assessments and an average score calculated from all the completed at-home assessments for at-home EMAs.
Preliminary data screening indicated that some of the discrete data for the stress-related outcome variables had skewed distributions that could not be normalized, thus not meeting the assumptions for the use of parametric statistical analyses. Therefore, non-parametric statistical analyses were utilized.
To evaluate our first hypothesis that stress-related measures are differentially affected by the assessment environment among caregivers and non-caregivers, we used Pearson χ2 tests to compare distributions of participants in caregiver and non-caregiver groups based on their ratings in different environments coded the following way: (1) participants with ratings higher at home vs. research center, (2) participants with ratings lower at home vs. research center, and (3) participants with the same ratings in both testing environments. The significance level for these series of analyses was adjusted to p ≤ 0.01 to account for the five stress-related variables assessed.
We followed these analyses with Mann-Whitney U tests to determine whether any group differences in ratings of the stress-related measures with significant χ2 values exist in either research center or home environments. Secondary analyses using Mann-Whitney U tests to determine any group differences in ratings were also conducted for the rest of the stress-related measures used in the study. Owing to a sample size below 42 in our study, exact significance values are reported for all Mann-Whitney U tests.
To evaluate our second hypothesis – that caregivers report greater stress at home than in the research center – we used the sign test to evaluate within-subject differences between testing environments for ratings of stress-related measures that were differentially affected by testing environment as determined by χ2 analyses described above. We also conducted similar secondary analyses with non-caregivers. The significance level for these analyses was adjusted to p ≤ 0.025 to account for two stress-related variables that were significantly affected by the testing environment.
Results
Group characteristics
Caregiver and non-caregiver groups were similar in age, education, and gender distribution (all p > 0.05). As expected, compared to non-caregivers, caregivers reported significantly greater perceived stress in the previous week on PSS at baseline, t (39) = 3.20, p = 0.001. The participants in both groups were similar in terms of their health status. The most common health problem reported by participants in both groups was hypertension controlled with medication. Other conditions reported by participants included noninsulin dependent diabetes, mild asthma, thyroid deficiency, and acid reflux, all of which were controlled with medication. Groups also had a similar frequency of history of depression controlled by medication. All health-related measures were non-significant at p > 0.05. Table 1 presents group demographics and baseline characteristics.
Group baseline characteristics
Caregivers were caring for a parent (22.2%) or spouse (77.8%) with Alzheimer's disease or fronto-temporal dementia. The RMBPC subscale scores (mean ± SD) suggested moderate severity of problems in care recipients for depression (10.4 ± 5.7), disruptive behavior (9.5 ± 6.6), memory problems (20.4 ± 7.6), and overall (40.3 ± 16.1). The RMBPC scores, indicating caregiver reaction to care recipients’ behaviors, were in the low to medium range on average for depression (10.4 ± 8.3), disruptive behavior (10.3 ± 7.0), memory problems (9.1 ± 5.3), and overall (29.7 ± 17.2).
Difference between research center assessment and EMA at home
Subjects from both groups reported no difficulties completing assessments using the PDA. The EMA compliance rate of over 85% in our sample was comparable to previous EMA studies (Cain et al., Reference Cain, Depp and Jeste2009) and was similar among caregivers and non-caregivers. The reasons for not completing EMA at home included skipping assessment due to caregiving duties, missing the alarm in noisy environments, and EMA device scheduling errors. There were no systematic differences in missed EMAs between the groups.
Caregivers and non-caregivers rate stress-related measures differently depending on assessment environment (Figure 1). There were significant differences between caregiver and non-caregiver groups in distributions of participants based on whether they had higher ratings for stress-related measures in the research center or at home for perceived stress, χ2 (2, N = 35) = 10.48, p < 0.01 and situational demand, χ2 (2, N = 35) = 8.90, p = 0.01. No significant group differences in distributions based on differences in ratings completed in different assessment environments emerged for ratings of fatigue, ability to cope with the situation, or mindfulness (p > 0.10 for all ratings).
Distributions of ratings for stress-related measures based on differences between ratings obtained in the research center and home.
The patterns of scores in the two testing environments are different between caregivers and non-caregivers for ratings of perceived stress and situational demand. All data points represent the percentage of participants from a specific group with a certain difference pattern: (1) lower rating at home than in the research center, (2) equal ratings at home and research center, (3) higher rating at home than in the research center. Higher scores indicate greater severity of the problem for all measures except coping with the current situation.
We followed these analyses with Mann-Whitney U tests to determine whether any group differences in ratings of the stress-related measures with significant χ2 values existed in either research center or home environments. Secondary analyses using Mann-Whitney U tests to determine any group differences in ratings were also conducted for the rest of the stress-related measures used in the study.
When tested in the research center, the groups were similar on ratings for all stress-related variables (p > 0.10). Only for the rating of current situational demand was there a non-significant trend suggesting that when tested in the research center, non-caregivers rated the situational demand higher (mean rank of 21.03) than did caregivers (mean rank of 14.79), Z = −1.93, p = 0.07, r = 0.33. When tested at home, caregivers reported greater perceived stress (mean rank of 25.47) than did non-caregivers (mean rank of 16.83), Z = −2.35, p = 0.02, r = 0.37. There was also a non-significant trend suggesting greater fatigue in caregivers (mean rank of 24.59) compared to non-caregivers (mean rank of 17.48), Z = −1.91, p = 0.058, r = 0.30. For the other measures, coping with the current situation and mindfulness, there were no significant differences between caregivers and non-caregivers in any testing environment, p > 0.10. Information about mean values and standard deviations for all measures for both groups is reported in Table 2.
Within and between group differences in ratings for stress-related measures in the research center and home environments
Assessment environment differentially affects ratings for stress-related measures in caregivers and non-caregivers.
Data are presented as mean ± SD; higher ratings indicate greater severity of the problem except for the coping with situation measure.
* p ≤ 0.025, T0.10 > p > 0.05; significant differences are in bold font.
a Significance level for within-group difference in ratings obtained in different testing environments using the sign test.
b Significance level for between-group difference in ratings obtained in same testing environment using the Mann-Whitney test.
Caregivers report greater stress at home than in the research center. As expected, caregivers rated their perceived stress level higher when tested at home compared to their rating for the same measure in the research center, as indicated by the sign test, p = 0.02. There were no significant differences between ratings obtained in different testing environments for any other measures for the caregiver group.
Non-caregivers report greater situational demand in the research center than at home. Non-caregivers rated situational demand in the research center significantly greater than that at home, as indicated by the sign test, p < 0.01. There was also a non-significant trend for non-caregivers to have greater ratings for fatigue when tested in the research center than the ratings for the measure obtained at home as indicated by the sign test, p = 0.08. The effect sizes based on the sign tests for within-subjects ratings in different locations were small (r ≤ 0.08) for the majority of measures for both caregivers and non-caregivers. The only exceptions were fatigue (r = 0.17) and situational demand (r = 0.32) for non-caregivers, which had medium and large effect sizes, respectively.
Discussion
Our results comparing self-rated stress measures among dementia caregivers and non-caregivers in two different environments confirmed our hypothesis that assessment environment differentially affects ratings for stress-related measures in caregivers and non-caregivers. We found significant interactions between a group and an assessment environment related to the fact that caregivers and non-caregivers provided similar ratings of all assessed measures when tested in the research setting, but caregivers provided higher ratings of perceived stress than non-caregivers when tested at home. Moreover, for all stress-related measures when tested in the research center, caregivers indicated similar or lower severity ratings compared to their ratings for the same measures when tested at home. This might potentially suggest that in the research center caregivers might function at a level similar or higher to their level of functioning at home. Conversely, non-caregivers on average provided lower ratings for all stress-related measures when tested at home compared to their research center ratings, potentially suggesting their overall better functioning at home.
There are several implications of our findings for clinicians and researchers working with caregiver populations. First, when caregivers’ current perceived stress is assessed in the clinic or research center, as is often done, it may not accurately reflect their typical level of ongoing stress. The ongoing stress becomes more evident when the assessments are completed at home where the source of caregiver stress originates and persists. Therefore, assessing caregivers at home might provide a more sensitive measure of functioning. Furthermore, our results indicate that compared to being at home, a clinic or research center visit might be viewed as a more stressful experience for non-caregivers, while for caregivers a research center visit might be associated with similar or less stress than being at home. Therefore, when assessing stress-related outcomes in cross-sectional studies comparing caregivers and non-caregivers, testing in the home rather than a research center environment might provide a better indication of experiences that caregivers and non-caregivers have on a daily basis.
Next, our participants reported no difficulties completing EMA using a PDA after a brief training session conducted during a research center visit. Despite their busy schedules caregivers in our sample demonstrated good EMA compliance during the 24-hour assessment period, suggesting that at least short-term EMA is feasible in this burdened population. Another study using EMA methods with older caregivers of ill spouses suggested that even longer assessment periods might be used (Poulin et al., Reference Poulin2010).
Although only a few published studies reported utilizing EMA in caregiver populations, EMA methodology has been effectively used as an alternative to retrospective recall in different populations to evaluate mood, anxiety, drug and medication use, and diet (Shiffman et al., Reference Shiffman, Stone and Hufford2008). EMA may provide a more valid way of capturing dynamic momentary experiences like stress, thoughts, and emotions that change over time than retrospective recall affected by biases (Trull and Ebner-Priemer, Reference Trull and Ebner-Priemer2009). Retrospective recall biases might be more pronounced in older adults who have increased incidence of memory deficits (Cain et al., Reference Cain, Depp and Jeste2009), which makes EMA an attractive methodology for researching aging populations. Moreover, being conducted in the participants’ natural environment, EMA has greater ecological validity than assessments conducted in the controlled research center setting (Cain et al., Reference Cain, Depp and Jeste2009).
To our knowledge this is the first published study that compared differences in ratings on the same stress-related measures between the research center and home environment using EMA in caregivers and non-caregivers. However, findings indicating differences between results of psychophysiological assessments conducted at home and in the research setting have been previously reported in studies assessing depressed participants and people with bipolar disorder (Ebner-Priemer and Trull, Reference Ebner-Priemer and Trull2009).
Although EMA appears to be a useful methodology for reporting subjective experiences in the moment; there is a paucity of research assessing how this methodology compares to objective measures of stress obtained in the research center. Owing to dynamic changes in many markers of stress because of circadian variations or in response to stressful situations, EMA might be an attractive way of evaluating such objective biomarkers. As suggested by several recent investigations, known stress biomarkers such as salivary hormone cortisol or salivary enzyme alpha amylase can be evaluated at multiple time points in participants’ natural environment using EMA (Nater et al., Reference Nater, Rohleder, Schlotz, Ehlert and Kirschbaum2007; Matias et al., Reference Matias, Nicolson and Freire2011; Robles et al., Reference Robles2011). Further, physiological measures such as blood pressure have been successfully collected using EMA, and they were shown to be influenced by the environment where the readings are taken (in the clinic or via ambulatory blood pressure monitoring in the natural environment) (Ebner-Priemer and Trull, Reference Ebner-Priemer and Trull2009). Additionally, current technology is also capable of evaluating cognitive performance and evoked potentials related to completing a cognitive task utilizing EMA in participant's natural environments (Ellingson and Oken, Reference Ellingson and Oken2010; Fonareva et al., Reference Fonareva, Ellingson, Zajdel, Amen and Oken2010). Further research on this topic is warranted to address the utility of EMA for diverse types of measures and to further evaluate how EMA compares to obtaining similar measures in the research center environment.
Although using EMA might require substantial initial investment to pay for devices and software programming expertise, advancing technology provides powerful, versatile platforms for implementing EMA which directly supports automated data acquisition, therefore ultimately reducing personnel and other related costs while improving accuracy and reliability. In addition, studies utilizing EMA, including the current study, demonstrate that with proper training, presented with a user-friendly assessment interface, participants of different backgrounds and ages can complete EMA successfully.
Several study limitations must be taken into consideration when interpreting our findings. First, while EMA have been used for various assessment periods ranging from one day to several weeks, we collected EMA at home over a single 24-hour period. It is possible that more than one day might be advantageous because the situation in caregivers’ homes might change as a result of fluctuations in the care recipient's behavior. However, because dementia caregivers are a population already pressured by many external demands, we believe using EMA for a single 24-hour period is a suitable compromise between collecting real-time data in caregivers’ natural environment and minimizing their participation burden.
Second, the stress-related measures used for EMA in our study do not comprise a validated instrument but rather were chosen based on our previous findings indicating aspects of functioning affected by caregiving and remediable by a stress-reducing intervention (Oken et al., Reference Oken2010). Our goal was also to reduce the time spent completing a single assessment to less than 5 minutes. Therefore, although we were able to show that caregivers’ ratings of perceived stress and non-caregivers’ ratings of situational demand differ significantly between research center and home environments, the study might have had insufficient power to detect significant differences in other stress-related measures of interest as indicated in the results.
Third, our study sample was relatively small, and included cognitively intact, high-functioning, well-educated, mostly female, participants who were motivated to participate in research. It is unknown how our results might generalize to populations more heterogeneous in terms of gender, cognitive health, overall functioning, and education.
Despite these limitations we believe that our findings have important implications for assessing stress in caregivers and non-caregivers. Specifically, when evaluating stress and related measures in dementia caregivers and when comparing functioning in caregivers and non-caregivers, the assessment environment appears critical, and EMA represents a feasible way of evaluating stress in natural environments for these groups.
Future studies assessing stress in caregivers and non-caregivers using EMA are needed to replicate our findings. Using longer testing periods and recruiting more heterogeneous samples might be of value. Further, in addition to collecting psychological data, it might be beneficial to evaluate EMA utility for assessing physiological data (especially measures affected by stress level) and cognitive function in natural environments for dementia caregivers and non-caregivers.
In conclusion, for clinicians and researchers working with dementia caregivers, the assessment location is critical when evaluating stress-related measures in this population. EMA offers a more valid way of assessing stress-related outcomes in this population by allowing the conduct of certain assessments at home – the environment in which the source of caregivers’ stress is rooted.
Conflict of interest
None.
Description of the authors' roles
Irina Fonareva collected non-caregiver data, conducted statistical analyses, and wrote the paper. Roger Ellingson implemented the EMA on the PDA, provided technical support throughout data collection period, and assisted with writing the paper. Alexandra Amen collected caregiver data and carried out statistical analyses. Barry Oken designed the study, supervised the data collection, and assisted with writing the paper.
Acknowledgments
This research was supported by National Institutes of Health (NIH) grants U19AT002656, T32AT002688, K24AT005121, P30AG008017, and UL1RR024140. We thank Daniel Zajdel and Helane Wahbeh for help in conducting the study, and Andy Fish for excellent administrative support.
Questions used for the research center assessment and at home EMA and comparisons
