Introduction
The negative emotional, psychological, and health effects of caring for a relative with Alzheimer's disease or a related dementia (ADRD) and the need to facilitate the rewards and meaningfulness of this role has resulted in a range of clinical interventions to support family caregivers (Gaugler et al., Reference Gaugler, Potter and Pruinelli2014). The number of empirical evaluations of dementia caregiver interventions has grown considerably, and there now exist many meta-analyses and systematic reviews (SRs) that consider a range of family caregiver (e.g. stress, depression) and person with ADRD (e.g. behavior problems, time to institutionalization) outcomes (Gitlin and Hodgson, Reference Gitlin, Hodgson, Gaugler and Kane2015). A potential concern, however, when synthesizing or comparing the efficacy of dementia caregiver interventions is inconsistency in how they are classified (Czaja et al., Reference Czaja, Schulz, Lee and Belle2003; Schulz et al., Reference Schulz, Czaja, McKay, Ory and Belle2010). This can result in confusion when ascertaining whether certain types of interventions are more efficacious than others for given caregiver and care-recipient outcomes, and may also inhibit the ability to implement these programs into routine clinical care settings (Gitlin and Hodgson, Reference Gitlin, Hodgson, Gaugler and Kane2015; Gaugler and Burgio, Reference Gaugler, Burgio, Burgio, Gaugler and Hilgeman2016). The actual extent of this problem, however, is unknown. The objective of this study was to systematically describe the classification of individual interventions in existing SRs to examine possible inconsistencies, with the aim of advancing the literature to facilitate more effective categorization.
Prior classifications of dementia caregiver interventions
Syntheses of the dementia caregiver intervention literature suggest that an ad hoc classification approach is utilized that is based largely on the general function of interventions (Pinquart and Sörensen, Reference Pinquart and Sörensen2006). Organizations that serve as clearinghouses for dementia caregiver intervention information, such as the Rosalynn Carter Institute for Caregiving or the Alzheimer's Disease Supportive Services Program in the USA, do not categorize dementia caregiver interventions by type (Maslow, Reference Maslow2012). However, some researchers have advocated for a more detailed approach when describing intervention components. In the ADRD caregiving context, Czaja et al. (Reference Czaja, Schulz, Lee and Belle2003) “decomposed” the various multi-component interventions of the Resources for Enhancing Alzheimer's Caregiver Health (REACH) program into the following dimensions: the primary entity (e.g. caregiver, person with ADRD, environment); the functional domain (e.g. skills, behavior of the person with ADRD); and delivery characteristics including mode of delivery, frequency and intensity, group or individual delivery, and similar aspects. Results across the six decomposed REACH interventions for 1,222 dementia caregivers indicated that targeting caregiver behavior (e.g. ensuring and validating caregivers’ abilities to implement effective behavioral approaches) reduced caregiver depression (Czaja et al., Reference Czaja, Schulz, Lee and Belle2003).
Schulz and colleagues have built on this earlier work to create a taxonomy that categorizes health-related behavior change interventions. Their taxonomy focuses on delivery characteristics including method of contact between intervention provider and recipient; materials used in the delivery of the intervention; location of intervention delivery; duration and intensity; extent of intervention “scripting;” sensitivity of intervention to participant background, skills, and abilities; interventionist training; adaptability; and treatment implementation (Schulz et al., Reference Schulz, Czaja, McKay, Ory and Belle2010). The content and goals of the intervention, such as specific strategies to improve outcomes of participants and mechanisms that operate to achieve targeted outcomes, are additional domains of the taxonomy.
Advancing syntheses of evidence for dementia caregiver interventions
It is not apparent whether the recommendations of Schulz and others have influenced the reporting of individual dementia caregiver interventions or the classification strategies of existing SRs. More specifically, an ongoing issue when conducting SRs of dementia caregiver interventions is that many individual evaluations do not report extensive details about their context, structure, or process. For these reasons, authors of SRs must often rely on subjective approaches and definitions to categorize interventions. This can lead to wide disparities in how dementia caregiver interventions are classified across SRs, and such ambiguity may continue to hinder efforts to synthesize the effectiveness of various interventions and translate these programs into practice settings.
As early as 2001, a comparison of two SRs of dementia caregiver interventions that included largely similar review questions, study inclusion criteria, and years of review found that only 44% of controlled trials appeared in both reviews (Charlesworth, Reference Charlesworth2001). To our knowledge, no study to date has systematically described the extent of misclassification of individual dementia caregiver interventions across SRs. Until the discipline better understands how individual interventions are categorized, conclusions of efficacy of different types of interventions will continue to vary considerably. For these reasons, the current study employed visual data method and network analysis (Kolaczyk and Csárdi, Reference Kolaczyk and Csárdi2014), qualitative methods, and empirical description to examine how specific interventions are classified across SRs.
Methods
Search strategy
We searched Ovid Medline®, Ovid PsycINFO®, Ovid Embase®, and the Cochrane Library to identify previous SRs published and indexed in bibliographic databases through January 2015. Our search strategy (Online Supplemental Material) included relevant medical subject headings and natural language terms for concepts related to dementia and caregivers. These terms were combined with filters to select SRs. Bibliographic database searches were supplemented with backward citation searches of relevant SRs. Two independent investigators (EJ and MB) reviewed titles and abstracts of search results. Citations deemed eligible by either investigator underwent full-text screening. EJ and MB independently screened full texts to determine if the inclusion criteria (Table 1) were met. Discrepancies in screening decisions were resolved by consultation between investigators and, if necessary, consultation with a third investigator (TPS). We documented the exclusion reason for reviews at the full-text screening stage (Online Supplemental Material). EJ and MB assessed the quality of eligible SRs using AMSTAR (A Measurement Tool to Assess Systematic Reviews) (Shea et al., Reference Shea2009). Quality assessment of SRs included items such as a priori design, dual review, and individual study risk-of-bias assessment. Results of previous SRs used in lieu of de novo extraction were updated with new data when additional relevant studies were identified (Viswanathan et al., Reference Viswanathan2012).
Study inclusion criteria
Note: SR = systematic review; RCT = randomized controlled trial; CG = caregiver; CR = care-recipient.
We extracted information from SRs that were relevant to address the taxonomy of CG interventions and were assessed as fair or high quality. Study quality reflects the belief that the study methods yield unbiased results. To that effect, and consistent with Agency for Healthcare Research and Quality Evidence Based Practice Center methodology, we excluded low-quality reports from the analysis to report the best available evidence (Viswanathan et al., Reference Viswanathan2012). EJ identified relevant SR characteristics (search dates, inclusion criteria, number and types of studies included, intervention categorization, and outcomes reported) and entered them into an evidence table. Definitions of intervention types as defined by the SR authors were then extracted. A database using bibliographic management software that contained the SRs and randomized controlled trials (RCTs) of dementia caregiver interventions included in each SR was then created.
Evaluation and extraction of intervention classifications
We initially identified the original grouping categories used in each SR and the individual studies included in each group. Using a network analysis framework, we graphically plotted results by showing connections (i.e. common studies) between the original SR grouping labels. The first and third authors (JEG and TPS) then conducted open-coding of intervention grouping definitions in SRs that matched the inclusion criteria. Both authors compared their individual codes for inter-rater reliability. We used an inductive (data-driven) approach for the analysis. As a result, we developed codes and identified categories and themes based on language used in the SRs, rather than fitting the data into preconceived categories. Coding and analyses proceeded in a manner largely similar to that described by Charmaz (Reference Charmaz, Nagy Hesse-Biber and Leavy2004). Our approach is similar to grounded theory, which focuses on developing emergent codes entirely from the data, but does not fully adhere to grounded theory's all-inductive and iterative approach. Specifically, to develop codes, JEG and TPS read definitions and used “line-by-line” open coding, which entails closely examining each line of text and assigning codes to reflect the meaning(s) contained in it (Charmaz, Reference Charmaz, Nagy Hesse-Biber and Leavy2004). Codes were arranged into categories, or groups of similar codes under one label. We then grouped categories into themes. To help organize the data, we used the NVivo 10 software package (QSR International Pty Ltd., 2012).
We then empirically described the consistency of intervention classification across SRs. Using the themes identified in open coding, we compared the individual studies included in common themes across reviews and noted discrepancies between SRs. Where possible, we identified the reason for the discrepancy (e.g. differences in search dates or differences in study inclusion criteria). Discrepancies that could not be identified nor explained were considered an inconsistency and are reported here.
Results
Search results
Our search identified 220 citations, of which 80 required full-text review after title and abstract screening. Of the 80 full-text SRs screened, we identified 26 eligible SRs. Three of these were assessed as poor quality. We extracted information from the remaining 23 SRs (Online Supplemental Material). On average, reviews included 20.21 (SD = 21.44) studies and categorized studies into 2.74 (SD = 2.13) groupings (n = 63). The average grouping included 7.17 (SD = 7.07) studies. Table 2 presents the intervention grouping categories in each review and the number of studies included in each category.
Intervention grouping categories and studies included in each category (n = 23)
Note: See Online Supplemental Material for full citation of each review.
Descriptive network analysis of studies and groupings
Of the 232 individual studies identified across the 23 SRs, each individual study was included in 2.01 SRs on average (SD = 1.36). Often the same individual study was classified differently in separate reviews. In Figure 1, the nodes indicate the groupings used to categorize studies in a SR. To minimize “noise,” Figure 1 only shows groupings that had at least 30 different connections. A line connecting two groupings indicates that the groups included at a minimum a single study (in some cases, there was multiple overlap of individual studies which is indicated by thicker weighted lines). For example, Parker et al. (Reference Parker, Mills and Abbey2008) (grouping label = multi-component) and Olazarán et al. (Reference Olazarán2010) (grouping label = education) both included the same study in their reviews but categorized the study differently. Figure 1 reveals a significant amount of overlap in the studies included across SRs, but also noticeable graphical differences in how SRs labeled or categorized studies.
Network analysis of dementia caregiver intervention classification across systematic reviews.
Qualitative content analysis of intervention classifications
Our qualitative content analysis identified the following seven themes: (1) content of the intervention (the clinical content and focus of ADRD caregiver interventions; this included coded categories such as case management, education, cognitive stimulation, psychosocial support, skills building, and similar areas that reflected the content of reviewed interventions); (2) delivery method (whether dementia caregiver interventions were delivered face-to-face, through computer or telephone, video, or web-based platforms); (3) intended audience (individual, dyadic, or group); (4) standardized versus tailored content; (5) structure/type (multi-domain or singular content); (6) intensity (fewer than six or more than six sessions); and (7) source of delivery (professional- or peer-led). Table 3 provides additional detail of the categories identified within each theme.
Categories and themes of existing dementia caregiver intervention classifications
Analysis of inconsistent classification of interventions across reviews
Focusing on the content theme identified in our qualitative content analysis (as this domain was most commonly used to group intervention types) in SRs, we selected and compared reviews that: (i) utilized similar definitions of intervention content; (ii) identified intervention studies from similar yearly intervals (e.g. 1990–2010); and (iii) used similar inclusion criteria (e.g. a sole focus on RCTs) to determine the degree of overlap or misclassification across reviews. Due to space considerations, citations of the individual interventions categorized across SRs are not included here; they are available from the authors.
The first intervention content area considered was respite (a service designed to give caregivers “time off” from responsibilities either at-home or at another site). Three SRs utilized similar definitions when grouping and synthesizing results of respite for dementia caregivers (Cooper et al., Reference Cooper, Balamurali, Selwood and Livingston2007; Olazarán et al., Reference Olazarán2010; Maayan et al., Reference Maayan, Soares-Weiser and Lee2014). Olazarán and colleagues included two studies of respite, while Maayan et al. included four and Cooper and colleagues included three. Only one study was consistent across the three reviews. In contrast, four studies were included in at least one review but not the others. Furthermore, one study was included in two of the reviews, but was classified as respite in one and not the other.
Case management (e.g. “interventions that involve a case manager providing advocacy over time, support, information about caregiving services, and financial and legal advice to a caregiver;” Pimouguet et al., Reference Pimouguet, Lavaud, Dartigues and Helmer2010) was the next grouping examined. Olazarán and colleagues included eight trials and Pimouguet et al. included 12 (Olazarán et al., Reference Olazarán2010; Pimouguet et al., Reference Pimouguet, Lavaud, Dartigues and Helmer2010). Nine studies were included in Pimouguet et al. that were not included in Olazarán and colleagues, even when accounting for non-overlapping search dates. Two studies in Olazarán et al. and one in Pimouguet et al. were unique to those respective reviews and not included in another SR, and an additional 12 studies were included in either Olazarán et al. or Pimouguet et al. but not in any other review.
The next content category examined was physical activity, which classified interventions that promote exercise or other physical activities for dementia caregivers. Two SRs considered physical activity interventions (Cooper et al., Reference Cooper, Balamurali, Selwood and Livingston2007; Orgeta and Miranda-Castillo, Reference Orgeta and Miranda-Castillo2014). Orgeta and colleagues included four interventions in this particular category while Cooper et al. included two. Two studies were consistent between the two reviews. Relaxation and yoga was another content classification but was only included in one review (Cooper et al., Reference Cooper, Balamurali, Selwood and Livingston2007). This SR only included two studies in this category, with one not grouped in any other SR.
Skills building was a common classification across reviews. Eight SRs included skills building classifications, although the content definitions varied (Selwood et al., Reference Selwood, Johnston, Katona, Lyketsos and Livingston2007; Olazarán et al., Reference Olazarán2010; Eggenberger et al., Reference Eggenberger, Heimerl and Bennett2013; Elvish et al., Reference Elvish, Lever, Johnstone, Cawley and Keady2013; Griffin et al., Reference Griffin2013; Li et al., Reference Li, Cooper, Austin and Livingston2013; Brasure et al., Reference Brasure2015; Jensen et al., Reference Jensen, Agbata, Canavan and McCarthy2015). Nonetheless, no intervention studies were consistent across all seven reviews. Sixty-one studies were reported in only one of the SRs that considered skills building interventions. Nine interventions were reported in two reviews, and eight were reported in three SRs. In contrast, 45 interventions were inconsistently classified in a skills-based grouping in one review and a different category in another.
Education (e.g. information provision) was included as a category in eight SRs (Pusey and Richards, Reference Pusey and Richards2001; Selwood et al., Reference Selwood, Johnston, Katona, Lyketsos and Livingston2007; Parker et al., Reference Parker, Mills and Abbey2008; Elvish et al., Reference Elvish, Lever, Johnstone, Cawley and Keady2013; Brasure et al., Reference Brasure2015; Jensen et al., Reference Jensen, Agbata, Canavan and McCarthy2015). Although there was less overlap of inclusion dates reported across these reviews, no individual caregiver intervention study was consistent between all SRs. There were 70 individual studies reported in only one review, 15 in two, four in three, and three studies reported across four of the SRs. In contrast, there were a total of 42 studies that were included in an education-based grouping in one review and in a non-education grouping in another SR.
Four SRs were identified that classified cognitive behavioral interventions, and as with psycho-educational interventions, there were no studies that were consistent across all four reviews (Cooper et al., Reference Cooper, Balamurali, Selwood and Livingston2007; Vernooij-Dassen et al., Reference Vernooij-Dassen, Draskovic, McCleery and Downs2011; Griffin et al., Reference Griffin2013; Li et al., Reference Li, Cooper, Austin and Livingston2013). Twenty studies were included in only one review, three were reported in two reviews, and one was reported in three reviews. Another 22 individual intervention studies were classified in a cognitive-behavioral therapy category in one of four SRs but were classified as another type in a different review. Eight studies that were published after 2005 (i.e. outside of the Cooper et al. search date range) were included in only one of the other three SRs.
Among the six reviews that included a psychosocial category, there were no studies that were consistent across reviews (although it is important to note that dates of inclusion varied widely; Cooper et al., Reference Cooper, Balamurali, Selwood and Livingston2007; Parker et al., Reference Parker, Mills and Abbey2008; Olazarán et al., Reference Olazarán2010; Elvish et al., Reference Elvish, Lever, Johnstone, Cawley and Keady2013; Griffin et al., Reference Griffin2013; Van't Leven et al., Reference Van't Leven2013). Fifty-four studies were only reported in one review, whereas ten studies were reported in two reviews. Thirty-nine studies were classified as “psychosocial” in one of six reviews but were then classified in an altogether different category in other SRs.
Discussion
Even when considering time frames and inclusion criteria, the inconsistency of dementia caregiver intervention classification across SRs was striking. A general pattern seemed to emerge where the more complex the clinical content of a given intervention (e.g. skills building approaches), the more irregular the classification across reviews. In many instances, an intervention was categorized in one review as one type, but then classified as a different type in another SR (even if the same category was utilized in the latter review); in others, many individual interventions were included in one SR but not another. The extent of discrepancy is likely due to a number of factors, including the range of rigor adopted when conducting SRs (e.g. varying search strategies), a more narrow focus in some reviews than others leading to alternative classifications of individual interventions, and the highly subjective criteria utilized when categorizing interventions.
Some inconsistency is understandable; for example, SRs may be guided by questions that require alternative grouping of interventions. However, even these explanations are unlikely to account for the amount of discrepancy that currently exists. This inconsistency looms as a major concern: There are a number of efforts throughout the USA and elsewhere to translate evidence-based interventions for ADRD caregivers to advance the quality of existing long-term services and supports for family caregivers and persons with dementia. The current state of the evidence, however, hinders overall assessments of “what works for whom” when determining the efficacy of interventions and whether there are certain interventions or intervention types that result in more positive outcomes for caregivers or persons with ADRD than others (Gitlin and Hodgson, Reference Gitlin, Hodgson, Gaugler and Kane2015).
In defense of current and past SRs, the reporting standards of individual dementia caregiver interventions could be considered less than optimal. Although process data may exist in the grey literature (i.e. unpublished reports or manuscripts), even the most careful synthesis of existing dementia caregiving interventions is unlikely to fully characterize the administration method, clinical content, delivery approach, and other key characteristics of selected programs. The quality of reporting continues to hamper attempts to determine not only whether a given type of dementia caregiver intervention is efficacious, but also how it achieves or does not achieve its stated benefits. As the themes identified in our qualitative content analysis suggest, there are a number of important components (content of the intervention; delivery method; source of delivery; standardized vs. tailored content; structure/type; intensity; and intended audience) that should serve as a reporting framework of individual dementia caregiver interventions. Reporting such information in individual studies will likely result in more consistent classification of single studies in subsequent SRs.
There also exist clear, compelling recommendations that to date have not been adopted consistently: those of Schulz and colleagues outlining frameworks to decompose, identify, and report core components and processes of dementia caregiver interventions (Schulz, Reference Schulz2001; Czaja et al., Reference Czaja, Schulz, Lee and Belle2003; Schulz et al., Reference Schulz, Czaja, McKay, Ory and Belle2010). Our content analysis of existing intervention definitions used in SRs confirmed these recommendations. It is not immediately clear why the reporting recommendations of Schulz and colleagues have not been uniformly adopted. The evolution of dementia caregiver interventions can be traced from the evaluation of brief, circumscribed, singular component interventions that demonstrated modest to minimal efficacy for key caregiver and care-recipient outcomes to those that include multiple treatment components that are delivered over longer periods of time (Gitlin and Hodgson, Reference Gitlin, Hodgson, Gaugler and Kane2015). Although multi-component protocols tended to demonstrate more consistent significant effects on key outcomes, the drive to establish efficacy outpaced process-oriented evaluations or component analyses to better understand mechanisms of benefit (which may be less palatable to some peer-reviewed journals). The empirical results reported here offer further support for the adoption of comprehensive reporting standards (via extramural grant mechanisms and/or peer-reviewed journal requirements) to yield greater understanding for why certain dementia caregiver interventions do or do not exert benefits for families. They would also facilitate improved syntheses of the literature to provide consistent conclusions of efficacy and opportunities for comparative effectiveness research. Expert consensus panels (that include the original principal investigators of interventions), use of more rigorous SR methods, or similar strategies could also be employed to result in more consistent intervention classifications.
In addition to the qualifications described above, there are several limitations to our evidence-based synthesis. An open coding methodology was used to identify themes across reviews. While there was strong consensus between coders, others may have interpreted the coding results differently. The primary conclusions of our analysis advocate for use of reporting standards. However, the existing reporting standards are not without limitations themselves (e.g. they may be time consuming or require additional information that do not fit within journal word limits). Although existing criteria were used to ensure comparisons of intervention classifications across SRs, there may be other unobserved reasons that explain why inconsistent categorization occurred.
The implications of our findings extend beyond the performance of meta-analyses or SRs of the literature to clinical settings. The evidence-base of dementia caregiver interventions has advanced to the point where many of these programs are now in translation in various community and clinical settings (Gitlin et al., Reference Gitlin, Marx, Stanley and Hodgson2015; Wethington and Burgio, Reference Wethington, Burgio, Gaugler and Kane2015). However, there exist a number of barriers that have hindered efficacious dementia caregiver interventions from reaching the number of families that they should. For example, many clinical and community-based service providers who are ideal candidates to translate dementia caregiver interventions require improved information to guide their decision-making when selecting a given intervention for their setting and clients (Maslow, Reference Maslow2012; Burgio and Gaugler, Reference Burgio, Gaugler, Burgio, Gaugler and Hilgeman2016). In particular, key information that could guide such decision-making includes more effective categorization of intervention types that better informs whether certain approaches are more appropriate than others for a given set of outcomes. Improved reporting and analysis of interventions could also help organizations identify components that are most essential to improving outcomes for persons with dementia and their family caregivers. To date, there is a distinct lack of analysis of intervention components among established, evidence-based interventions to ascertain what aspects drive positive outcomes (Burgio and Gaugler, Reference Burgio, Gaugler, Burgio, Gaugler and Hilgeman2016). As emphasized in our findings, improved classification of and rigorous reporting of intervention components could not only help to facilitate an overall understanding of dementia caregiver intervention efficacy, but may also help to inform more robust translational efforts.
Conclusion
Moving forward, reporting standards of dementia caregiving interventions and SRs require substantial enhancements. Comprehensive and transparent reporting of various intervention domains would benefit future research by facilitating consistent classification of individual interventions in SRs, as would greater rigor in the conduct of SRs themselves. Moreover, improvements in reporting would likely have important clinical ramifications: Many practitioners, when seeking to translate or implement an ADRD caregiver intervention in their community or healthcare setting, could use improved reporting of intervention content, design, and delivery to inform their decisions as to whether a given program is feasible and appropriate. A fuller accounting of a given intervention's clinical components could help organizations avoid the adoption of programs that require more extensive resources and expertise than anticipated. Scientific outlets that publish and disseminate the results of dementia caregiver interventions could also feature process evaluations or similar reports that could better guide implementation efforts beyond traditional outcome reporting.
A considerable gap between evidence and clinical practice continues to persist in dementia (Chodosh et al., Reference Chodosh2007), and generating higher quality evidence regarding the efficacy of caregiver interventions could help to bridge this gap and facilitate the translation of these programs in clinical contexts. Due to inconsistency in how dementia caregiver interventions are categorized across SRs, whether certain interventions are more or less effective for various caregivers remains opaque. This lack of clarity has hindered the science and practice of dementia caregiver interventions from achieving their considerable potential.
Conflict of interest
None.
Description of authors’ roles
J. E. Gaugler was responsible for all aspects of the study, including the design, qualitative analysis, interpretation of findings, and the writing of this paper. E. Jutkowitz conducted the evidence-based synthesis, the graphical network analysis, and the descriptive analysis of intervention classification across SRs. T. P. Shippee provided assistance in the evidence-based review of SRs, conducted the qualitative content analysis with J. E. Gaugler, provided input into the design of the paper and its interpretation, and assisted with reviewing the paper. M. Brasure provided considerable assistance in conducting the evidence-based synthesis and also reviewed the paper.
Acknowledgments
The research reported here was supported by grant K18HS022445-02 to Dr. Gaugler from the Agency for Healthcare Research & Quality.
Supplementary Material
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/S1041610216001514.
