Introduction
Acceptance and commitment therapy (ACT) is a ‘3rd wave’ behaviour therapy that promotes the acceptance of unwanted and distressing psychological/emotional experiences in the service of consistently living in accordance with personal values (Hayes et al., Reference Hayes, Strosahl and Wilson2012). Early meta-analytic evidence highlighted that many randomized and controlled clinical trials (RCTs) of ACT were poor quality and so concluded that ACT (at that time) was not an empirically supported treatment (Öst, Reference Öst2008). However, more recent meta-analytic evidence has consistently reported moderate to large effect sizes for ACT interventions (when compared with waitlist or treatment as usual) for anxiety, depression, addictions and somatic health complaints (A-Tjak et al., Reference A-Tjak, Davis, Morina, Powers, Smits and Emmelkamp2015; Hacker et al., Reference Hacker, Stone and Macbeth2015; Öst, Reference Öst2014; Powers et al., Reference Powers, Zum and Emmelkamp2009; Smout et al., Reference Smout, Hayes, Atkins, Klausen and Duguid2012) to conclude that ACT is comparable to other extant evidence-based therapies (i.e. CBT) for these diagnoses. Despite this progress made in creating an evidence base regarding the effectiveness and efficacy of ACT, researchers have attempted to provide evidence that ACT produces change through defining the mechanism(s) through which ACT operates therapeutically (Villate et al., Reference Villate, Vilardaga, Villate, Viladaga, Atkinds and Hayes2016).
Kazdin (Reference Kazdin2007) has usefully provided definitional clarity of four key concepts (causes, mediators, mechanisms and moderators of change) to help with this scientific endeavour, and these can be explained in an ACT context. Cause concerns when ACT would be observed responsible for outcome, mediation is an intervening variable that accounts (statistically) for the relationship between ACT and its outcome, mechanisms are the ACT processes responsible for therapeutic change, the reasons why change occurred or how change came about and moderators are characteristics that influence the direction or magnitude of the relationship between ACT and its outcome. With respect to moderation, if the relationship between ACT and outcome was statistically significantly different for male patients for example, then gender would be a moderator of the relationship between ACT and outcome. Moderators are related to mediators and mechanisms because they suggest that different processes might be involved for those patient groups (Kazdin, Reference Kazdin2007).
An advantage of the ACT approach has been its well-defined conceptual development and associated clear statement of the proposed mechanisms through which ACT enables change. These processes are united under the conceptual umbrella of the ‘psychological flexibility’ model (Hayes et al., Reference Hayes, Strosahl and Wilson2012) consisting of six core aspects: diffusion (i.e. stepping back and observation of cognition and an evaluation of cognition as representing general thought processes), acceptance (i.e. choosing to adopt an open/curious/receptive and non-avoidant stance on pleasant and unpleasant thoughts, feelings, memories and impulses), contact with the present moment (i.e. being psychologically present through consciously engaging in any moment, through flexibly bringing awareness to inner or environmental context), values (i.e. defining what truly motivates and matters to a person, to shape the desired qualities of any proposed or ongoing action), committed action (i.e. taking effective action guided by values to enable a rich, full and meaningful life) and finally self-as-context (i.e. development of an non-evaluative, observing self). Hayes et al. (Reference Hayes, Strosahl and Wilson2012) provide full clarity of the core components of the psychological flexibility model and various research methods have evidenced the clinical utility of the six core processes of ACT including laboratory-based component analyses (Ruiz, Reference Ruiz2010) and mediation studies (Hayes et al., Reference Hayes, Luoma, Bond, Masuda and Lillis2006a). This conceptual development has also facilitated the associated clinical competencies to be defined based on the functional assessment of psychological flexibility (Luoma et al., Reference Luoma, Hayes and Walser2007).
Mediation studies are important as they test the underlying theoretical model of ACT through demonstrating that conceptually important processes play a role in observable improvements in particular outcomes. A variable that mediates an outcome may not necessarily explain the processes of how that outcome was achieved. The mediator might be a proxy for one or more other processes or be a general psychotherapeutic approach which is not necessarily intended to explain the mechanisms of change. A mediator may therefore be a guide that highlights possible mechanisms of change, but is not necessarily a bone fide mechanism of change. For example, if there were changes over time in acceptance, one of the six core aspects of psychological flexibility and a proposed mechanism of change for ACT, and changes in acceptance statistically explained changes in the treatment outcome, then changes in acceptance could be viewed as both a mediator and a mechanism of change. However, if other mechanisms were also involved, such as concurrent changes in defusion, then acceptance would be considered a mediator, but not necessarily fully explain the mechanisms of change between ACT and its outcomes. In its simplest form therefore, a mediation model consists of a chain of relations amongst three variables, such that an antecedent variable influences a mediator variable, which in turn affects a dependent variable (MacKinnon and Fairchild, Reference MacKinnon and Fairchild2009). Importantly, what distinguishes a mediation model is that the influence of an independent variable on the dependent (i.e. outcome) variable passes through the mediator (i.e. the a/b path); this is an indirect effect (see Fig. 1).
Basic mediation model. IV, independent variable; DV, dependent variable.
In a previous review of mediation studies in ACT, Hayes et al. (Reference Hayes, Luoma, Bond, Masuda and Lillis2006a) summarized eight mediation analysis studies (as well as eight studies examining changes in process variables without conducting formal mediation analysis). Whilst there was some evidence of mediation found within each study (e.g. the believability of stigmatizing attitudes functioned as a mediator of ACT's impact on stigma and burnout during the training of substance-misuse counsellors illustrating the mediating role of diffusion; Hayes et al., Reference Hayes, Bissett, Roget, Padilla, Kohlenberg and Fisher2004), some studies failed to show consistent mediation [e.g. Gregg (Reference Gregg2004) found that committed action did not mediate changes in HbA1c scores in type 1 diabetic patients]. The mediating processes examined were also limited within each study (e.g. assessment of the role of defining values were neglected). Furthermore, a number of consistent methodological weaknesses were found across the studies including use of unvalidated measures, over-reliance on self-report, process measures being taken after outcomes have significantly improved, low power and overly focusing on a limited number of putative processes (i.e. cognitive diffusion). The subsequent review of Hayes et al. (Reference Hayes, Villatte, Levin and Hildebradt2011b) found that ACT treatment effects were partially or fully mediated by changes in overall psychological flexibility. About 50% of the between-group differences in follow-up outcomes could be explained by the mediating role of differential post-treatment levels of overall psychological flexibility and its components. Due to an increase in the volume of subsequent ACT studies and also recent recommendations to improve the quality of mediational studies in psychotherapy (Wilt, Reference Wilt2012), it is timely to undertake a systematic approach to reviewing the literature investigating the mechanisms through which ACT is proposed to enable/facilitate change. The aims of the present review were therefore (1) to systematically locate and report (in brief) clinically and methodologically relevant contemporary ACT meditation studies, (2) to synthesize the results of each study by considering the evidence provided for each putative process, (3) to assess whether appropriate modern statistical mediation methods were being used, and (4) to assess the methodological quality of the contemporary mediation evidence base to consider whether the improvements suggested by Hayes et al. (Reference Hayes, Luoma, Bond, Masuda and Lillis2006a) have been acted upon.
Before presenting the methods and results of this systematic review, the main methodological approaches employed to test for mediation are introduced and contemporary topics in mediation analysis are identified. In particular, complex mediation models including intervening variables (e.g. moderated mediation models) and multi-level mediation models for longitudinal designs are outlined. These two preliminary sections therefore serve to delineate the basic statistical models and criteria which are then used to assess the contemporary ACT mediation studies.
Approaches for assessing mediation models
The main methods for assessing the strength of a mediated effect in a single mediator model are: (1) the causal step approach, (2) partial correlation strategies, (3) product of coefficient strategies, (4) distribution of the products strategy, and (5) bootstrapping (MacKinnon et al., Reference MacKinnon, Fairchild and Fritz2007; MacKinnon and Fairchild, Reference MacKinnon and Fairchild2009; Preacher and Hayes, Reference Preacher and Hayes2008). The causal step strategy has historically been one of the most commonly used methods to probe mediation, in which three conditions need to be satisfied; (1) antecedent variable X should be related to the dependent variable Y, (2) each variable affects the following variable in the causal chain, and (3) the relation between X and Y becomes non-significant when controlling for the mediator variable (Baron and Kenny, Reference Baron and Kenny1986; Judd and Kenny, Reference Judd and Kenny1981). However, the causal step approach suffers from serious limitations compared with more modern statistical methods. MacKinnon et al. (Reference MacKinnon, Lockwood, Hoffman, West and Sheets2002) conducted a Monte Carlo simulation study that compared 14 different methods to test the statistical significance of mediation models. The causal step approach had low statistical power to detect small and medium indirect effects and to highlight Type-I errors.
Developments in the causal step approach have included formal tests to estimate the indirect effect. The most common estimator used is the Sobel test. In the Sobel test, the product of the estimates relating X-M (α) and M-Y (β), are used to infer the indirect effect α×β divided by its standard error, and then this ratio is compared against a normal distribution test (z) for statistical significance. Problems with this approach include: (1) measurement error in variables (MacKinnon et al., Reference MacKinnon, Lockwood and Williams2004) in that sampling variance may not necessarily converge with the distribution of the parameter, (2) the formula assumes that α and β are independent and that there is no interaction between the antecedent variable X and the moderator M, and (3) it is assumed that sampling distribution of the indirect effect is normal, which implies that confidence intervals may be inaccurate if this criterion is not satisfied (Bollen and Stine, Reference Bollen and Stine1990).
In contrast, simulation studies have shown that methods based on the distribution of the products and resampling techniques (bootstrapping) have statistically better performance (MacKinnon et al., Reference MacKinnon, Lockwood, Hoffman, West and Sheets2002, Reference MacKinnon, Lockwood and Williams2004). The distribution of the products approach relies on a non-normal distribution of the product of two normally distributed variables. This is a complex method that basically transforms the indirect effect α×β to a different metric, which in turn serves to define the confidence intervals, and then converting these estimates back into the original metric (Hayes, Reference Hayes2013).
The bootstrapping technique treats the original sample of observations as the basis for estimating multiple (usually thousands) of other distributions. Repeating numerous times, this replacement and resampling procedure produces a sampling representation which is closer to population parameters. In contrast to the Sobel test, no assumption is made about the shape of the sampling distribution of the indirect effect (Hayes, Reference Hayes2013). Furthermore, the original distribution of the sampling distribution is maintained, therefore allowing inferences that are more accurate compared with using the normal theory approach (Hayes, Reference Hayes2013). Whilst this approach is useful in mediation studies with small sample sizes, sampling distributions that are non-realistic may result in implausible estimates (Hayes, Reference Hayes2013).
Overall, comparing the three main aforementioned approaches, it is clear that the Sobel test demands stronger assumptions to be held in order to estimate precise indirect effects. Nevertheless, the Sobel test provides conservative estimates that may prevent conclusions being drawn that support that an indirect effect has occurred, when it actually has not (i.e. a Type-I error). Conversely, methods such as bootstrapping and distribution of the products provide greater confidence to detect an indirect effect that is real (i.e. a Type-II error), a common problem in mediation models that lack statistical power (MacKinnon et al., Reference MacKinnon, Fairchild and Fritz2007).
Recent approaches to mediation analysis
In addition to analyses that focus on how treatment processes mediate outcome differences between different psychotherapies, recent studies have investigated the differential effects of psychotherapies on mediation effects. This has been assessed by testing whether type of psychotherapy interacts with different parts of the mediation pathway. This form of analysis allows identification of mediated moderation or moderated mediation. There are many different methods of testing for moderating interactions of mediating variables (see MacKinnon et al., Reference MacKinnon, Fairchild and Fritz2007). A common model used in clinical outcomes research (MacKinnon et al., Reference MacKinnon, Fairchild and Fritz2007) involves testing whether the changes (over time) of a treatment process mediates the effects of a particular psychotherapy on the treatment outcome (e.g. the ability to challenge thoughts), and then testing whether treatment type (e.g. whether ACT or CBT is used) moderates the α and/or the β paths (see Fig. 2). If type of psychotherapy were to interact significantly with the α path, or both the α and β path, then this would be indicative of mediated moderation (as the initially occurring moderation effect at the α path is mediated by the target process). If type of psychotherapy were only to interact with the β path, then this would be indicative of moderated mediation (as the moderation occurs after the initial interaction between the IV and the mediator).
Conceptual model for mediated moderation of the effects of treatment on outcomes. Note that the mediator reflects changes in treatment mechanisms over time. Outcomes may be measured at a single or multiple time points.
Studies including treatment processes mediating the association between different psychotherapies and therapy outcomes, may consider the case where the same psychotherapist treats multiple patients participating in the same study or service. In this situation, patients are nested under psychotherapists (i.e. the ‘caseload’), resulting in individual observations that are dependent on the clustering unit (i.e. the treating psychotherapist). If this non-independent data structure is ignored, it typically results in Type-I error (Krull and MacKinnon, Reference Krull and MacKinnon2001). Modern mediation studies commonly treat α and β paths as representing random effects (Kenny et al., Reference Kenny, Korchmaros and Bolger2003), such that the indirect effects vary freely across psychotherapists. Considering cluster variables in the analysis prevents conflation of direct and indirect effects and more precise estimates can be computed (MacKinnon et al., Reference MacKinnon, Fairchild and Fritz2007).
Finally, mediation studies considering the relationship between type of psychotherapy and outcome can also consider changes over time. Alongside controlling for psychotherapist-cluster variables, studies can account for the effect of time on outcomes resulting in a longitudinal mediation model (Cole and Maxwell, Reference Cole and Maxwell2003; MacKinnon et al., Reference MacKinnon, Fairchild and Fritz2007). When the study design involves testing changes in outcomes over time as a result of type of psychotherapy, and mediating for psychotherapy-specific processes and/or changes in these processes, it is possible to test a teleological form of causality that assumes temporal precedence, where the antecedent variable and mediator(s) are measured before the outcome and the independent variable varies randomly across conditions (i.e. type of psychotherapy; MacKinnon et al., Reference MacKinnon, Fairchild and Fritz2007). This represents the gold standard approach to assessing mediated changes in psychotherapy outcomes, as both time and cluster variables are accounted for in the same model. Three common methods test longitudinal mediation models (MacKinnon and Fairchild, Reference MacKinnon and Fairchild2009): (1) autoregressive models, (2) latent growth curve models, and (3) latent difference-score models. Autoregressive models consider a variance-covariance matrix controlling for autocorrelation errors, whilst latent growth curve and latent difference-score models are commonly analysed using Structural Equation Modelling (SEM).
To summarize, state of the art methods for investigating mediation account for common problems observed in the single mediation model (e.g. such as not accounting for the effects of time or cluster variables) and can incorporate the effect of intervening variables in the form of moderated mediation models. Therefore, studies that want to reliably assess the mechanisms through which ACT enable/facilitate change should incorporate such methods in order to appropriately assess the impact of ACT processes on therapy outcomes.
Method
Study identification
In January 2015, a comprehensive search of three scientific and medical journal databases (PsychINFO, Medline and Web of Science) was conducted using key search terms (see Table 1). Search terms were applied to ‘abstract’ only. Duplicates were removed and abstracts were checked for adherence to inclusion and exclusion criteria. Inclusion criteria are as follows: (1) clinical trials comparing ACT with a different form of active treatment [e.g. cognitive behavioural therapy (CBT) or psychoeducation], (2) data analysis included exploration of interaction of intervention type on mediator variables, (3) studies that applied mediation analysis procedures, (4) papers from 2006 to 2015. Exclusion criteria were: (1) trials comparing ACT with waitlist control (WLC) group, (2) trials in which both study arms were not completely distinct from one another (e.g. comparing ACT with ACT + psychoeducation, or comparing psychoeducation with psychoeducation + ACT), (3) studies utilizing interventions including components from multiple therapeutic approaches (e.g. intervention with components from ACT and compassion focused therapy), (4) theses not published in a peer-reviewed journal, and finally (5) articles published in languages other than English. Studies that did not fulfil both inclusion criteria and/or fulfilled one or more of the exclusion criteria were excluded. Full text reviews were then conducted on qualifying studies with inclusion/exclusion criteria being re-applied. It is worth noting that the studies identified in this review adopted the strategy of either investigating the psychological flexibility model as a whole, or isolating one of more of the six components of the model. The preferred reporting items for systematic reviews and meta-analyses (PRISMA; Liberati et al., Reference Liberati, Altman, Tetzlaff, Mulrow, Gøtzsche and Ioannidis2009) were utilized as a guide in structuring this review. A flow diagram depicting the study review process is given in Fig. 3.
Search terms
Flow diagram of study identification
Study quality assessment
Study quality was assessed according to the methodological and statistical approaches used. Methodological study quality was assessed using the Rhodes and Pfaeffli (Reference Rhodes and Pfaeffli2010) mediation study quality checklist. The checklist was developed specifically for mediator studies and consists of 11 questions answered yes (scoring 1) or no (scoring 0). Studies are given a summary index score of low (0–4), moderate (5–8) and high quality (9–11). In order to examine inter-rater reliability of quality ratings, an independent reviewer scored five studies (chosen at random) in order to compare against ratings of all studies (D.S.). The number of comparator studies included for inter-rater reliability testing was determined using Cantor's (Reference Cantor1996) sample size calculation for Cohen's kappa. Results were analysed using Cohen's kappa (Cohen, Reference Cohen1968), before resolving any disagreements. Observed agreement was 80% with a ‘moderate’ level of inter-rater reliability (ҡ = .59).
Disagreements between reviewers were resolved by deferring to a third independent reviewer. The statistical approach to determining study quality used three criteria: (1) the type of analytical technique implemented where longitudinal mediation and bootstrapping mediation (i.e. either using ordinary least squares regression or multilevel modelling) would be state of the art, (2) whether the study reported a specific indirect effect (i.e. Sobel test, cross-product test or any other computation procedure), and (3) whether the study accounted for the effects of time and psychotherapist (i.e. assuming that better estimates are produced when these two factors are taken into account).
Results
The results are divided into three sections: ACT mediation study methods, ACT mediation study quality and a synthesis section. Table 2 provides a summary of the methods and quality assessments for each of the n = 12 studies.
Summary of methods
LSAS-SR, Liebowitz Social Anxiety Self-Scale Report; SAIS, Social Interaction Anxiety Scale; SPS, Social Phobia Scale; QoLI, Quality of Life Index; MASQ, Mood and Anxiety Symptom Questionnaire; SSPS, Self-Statements During Public Speaking Questionnaire; AAQ, Acceptance and Action Questionnaire; PBI, Brief Pain Inventory Short Form; AIS, Avoidance and Inflexibility Scale; THI, Tinnitus Handicap Inventory; TAQ, Tinnitus Acceptance Questionnaire; ADIS-IV-R, Anxiety Disorders Interview Schedule-IV-Revised; PSWQ, Penn State Worry Questionnaire; FQ, Fear Questionnaire; ASI, Anxiety Severity Index; BAFT, Believability of Anxious Feelings and Thoughts questionnaire; BSQ, Before Session Questionnaire; FACT-G, Functional Assessment of Cancer Therapy; POMS, Profile of Mood States; PII, Pain Interference Index; CAS-DC, Centre for Epidemiological Studies Depression Scale for Children; PAIRS, Pain Impairment and Relationships Scale; TSK, Tampa Scale of Kinesiophobia; PCQ, Pain Coping Questionnaire; BDI, Beck Depression Inventory; ATQ(-B), Automatic Thoughts Questionnaire (-Believability); DAS, Dysfunctional Attitude Scale; GHQ12, General Health Questionnaire; WHOQOL, World Health Organisation Quality of Life measure; SWLS, Subjective Wellbeing Life Scale; PWI, Personal Wellbeing Index; AAEpQ, Acceptance and Action Epilepsy Questionnaire; BAI, Beck Anxiety Inventory; OQ, Outcome Questionnaire; GAF, Global Assessment of Functioning Scale; CGI, Clinical Global Impression Scale; SLS, Satisfaction with Life Scale; KIMS, Kentucky Inventory of Mindfulness Skills; RCT, Randomized and Controlled Clinical Trial; TAU, Treatment as Usual.
Study methods
The mediating variables investigated were limited to five of the six ACT processes – acceptance, cognitive defusion, contact with the present moment, values and committed action. Therefore, self as context has been consistently overlooked as a potential mediator in study designs. Studies were from four countries: USA, Sweden, United Kingdom and South Africa. Sample sizes of studies ranged from 27 to 222 participants. Six of the twelve studies utilized mediation analysis procedures; the remaining studies tested for interaction effects (moderation) of treatment type on the mediation pathwayFootnote 1 (i.e. moderated mediation or mediated moderation). Various analytic procedures were used to test mediator effects, with n = 4 studies using the cross products of the coefficients.
Study quality
All studies were classified as ‘moderate quality’. Common methodological flaws were (a) only a single study (Lundgren et al., Reference *Lundgren, Dahl and Hayes2008) was adequately powered to detect mediation with all the remaining studies failing to report power, (b) only a single pilot study tested mediation (Bricker et al., Reference *Bricker, Wyszynski, Comstock and Heffner2013) with all remaining studies failing to report on whether pilot studies had been conducted to test mediation, (c) inconsistent use of objective measures or use of reliable outcome and/or mediator measures, (d) only three studies (Arch et al., Reference *Arch, Wolitzky-Taylor, Eifert and Craske2012; Hesser et al., Reference *Hesser, Westin and Andersson2013; Rost et al., Reference *Rost, Wilson, Buchanan, Hildebrandt and Mutch2012) ascertained whether changes in mediator variables preceded changes in outcome. With regard to the additional statistical quality criteria, (a) three studies (Arch et al., Reference *Arch, Wolitzky-Taylor, Eifert and Craske2012; Bicker et al., Reference *Bricker, Wyszynski, Comstock and Heffner2013; Forman et al., Reference *Forman, Herbert, Moitra, Yeomans and Geller2007) did not use a state of the art technique to test mediation and so failed to report a computation of the indirect effect, (b) only two studies (Forman et al., Reference *Forman, Herbert, Moitra, Yeomans and Geller2007, Reference *Forman, Chapman, Herbert, Goetter, Yuen and Moitra2012) accounted for time and therapist, and (c) only four studies (Hesser et al., Reference *Hesser, Westin and Andersson2013; Niles et al., Reference *Niles, Burklund, Arch, Lieberman, Saxbe and Craske2014; Rost et al., Reference *Rost, Wilson, Buchanan, Hildebrandt and Mutch2012; Zettle et al., Reference *Zettle, Rains and Hayes2011) accounted for time only with the remaining studies failing to account for either psychotherapist or time. The omission of reporting of reliability analyses occurred for n = 5 mediator variables and for n = 7 outcome measures. A summary of study quality scores can be found in Appendix 1.
Synthesis
Tables 3 and 4 provide an overview of contemporary ACT mediation studies and these studies have been synthesized and summarized at two levels. Firstly, mediators under investigation have been organized by the treatment-specific process they represent. These are primarily ACT processes (such as acceptance and cognitive defusion, presented in Table 3), but other non-ACT processes are also presented (such as changes in dysfunctional cognitions, presented in Table 4). Secondly, mediators have been tested on a range of outcome types, which are reported for each process variable. Outcomes are organized into four categories: (1) quality of life/wellbeing, (2) mental health symptomology (e.g. symptoms of depression and/or anxiety), (3) observable behavioural changes (e.g. smoking cessation, rehospitalization rates and frequency of physical symptoms), and (4) functioning (e.g. goal progress, pain interference and subjective accounts of functioning). A summary is provided for each process.
ACT putative processes
*The AAQ was used as a process measure in four studies (Flaxman and Bond, Reference *Flaxman and Bond2010; Forman et al., Reference *Forman, Herbert, Moitra, Yeomans and Geller2007; Lundgren et al., Reference *Lundgren, Dahl and Hayes2008; Niles et al., Reference *Niles, Burklund, Arch, Lieberman, Saxbe and Craske2014). Two studies cited the measure as a measure of psychological flexibility, and two studies cited the measure as a measure of acceptance/experiential avoidance (the counter-process to acceptance; Hayes et al., Reference Hayes, Strosahl, Wilson, Bissett, Pistorello and Toarmino2006b). Although initially developed as a measure of experiential avoidance, the AAQ is now considered as a measure of psychological flexibility (Hayes et al., Reference Hayes, Strosahl, Wilson, Bissett, Pistorello and Toarmino2006b). Therefore, for the purpose of this review, the AAQ shall be considered a measure of psychological flexibility in the studies that utilized it.
Non-ACT putative processes
Of the five studies investigating ‘psychological flexibility’ as a mediator, four studies indicated that psychological flexibility was a mechanism of change for mental health outcomes (Flaxman and Bond, Reference *Flaxman and Bond2010; Forman et al., Reference *Forman, Herbert, Moitra, Yeomans and Geller2007; Niles et al., Reference *Niles, Burklund, Arch, Lieberman, Saxbe and Craske2014; Rost et al., Reference *Rost, Wilson, Buchanan, Hildebrandt and Mutch2012). However, there was limited evidence available to determine whether psychological flexibility is a mechanism of change in physical health (Lundgren et al., Reference *Lundgren, Dahl and Hayes2008) or patient functioning (Forman et al., Reference *Forman, Herbert, Moitra, Yeomans and Geller2007).
Evidence is mixed as to whether improved quality of life is mediated by changes in psychological flexibility (Forman et al., Reference *Forman, Herbert, Moitra, Yeomans and Geller2007; Lundgren et al., Reference *Lundgren, Dahl and Hayes2008; Niles et al., Reference *Niles, Burklund, Arch, Lieberman, Saxbe and Craske2014; Rost et al., Reference *Rost, Wilson, Buchanan, Hildebrandt and Mutch2012). Of the five studies investigating ‘acceptance’ as a mediator, four studies indicated that acceptance was a mechanism of change regarding mental health (Forman et al., Reference *Forman, Chapman, Herbert, Goetter, Yuen and Moitra2012; Hesser et al., Reference *Hesser, Westin and Andersson2013; Rost et al., Reference *Rost, Wilson, Buchanan, Hildebrandt and Mutch2012; Wicksell et al., Reference *Wicksell, Olsson and Hayes2011) and two studies indicated that patient functioning was mediated by acceptance (Forman et al., Reference *Forman, Chapman, Herbert, Goetter, Yuen and Moitra2012; Wicksell et al., Reference *Wicksell, Olsson and Hayes2011). Albeit limited, there was some evidence of acceptance acting as a mechanism of change regarding quality of life (Arch et al., Reference *Arch, Wolitzky-Taylor, Eifert and Craske2012) and behavioural outcomes (Bicker et al., Reference *Bricker, Wyszynski, Comstock and Heffner2013).
Of the four studies investigating the role of ‘cognitive defusion’, three found that this aspect of the psychological flexibility model did not mediate or moderate mediate the relationship between type of psychotherapy and mental health outcomes (Arch et al., Reference *Arch, Wolitzky-Taylor, Eifert and Craske2012; Forman et al., Reference *Forman, Chapman, Herbert, Goetter, Yuen and Moitra2012; Zettle et al., Reference *Zettle, Rains and Hayes2011). Cognitive defusion failed to show moderated mediation effects on quality of life (Arch et al., Reference *Arch, Wolitzky-Taylor, Eifert and Craske2012) and functioning (Forman et al., Reference *Forman, Chapman, Herbert, Goetter, Yuen and Moitra2012). This indicates that this process is not entirely unique to ACT, but this evidence is limited. There was limited evidence that cognitive defusion could act as a mechanism of change regarding rehospitalization during psychosis (Bach et al., Reference *Bach, Gaudiano, Hayes and Herbert2013). In terms of ‘contact with the present moment’, only a single study examined the mediating effects of this process on outcomes (Forman et al., Reference *Forman, Herbert, Moitra, Yeomans and Geller2007) and therefore no overall conclusions can be drawn. Similarly, only a single study examined the mediating effects of values on outcomes (Lundgren et al., Reference *Lundgren, Dahl and Hayes2008) and therefore no overall conclusions can be drawn with regard to this process. Two studies investigated the mediating effects of committed action (Forman et al., Reference *Forman, Chapman, Herbert, Goetter, Yuen and Moitra2012; Hesser et al., Reference *Hesser, Westin and Andersson2013). However, both studies found that this process was a mechanism of change during both ACT and cognitive therapy.
As well as the ACT processes investigated (as described above), many of the studies investigated the mediating effects of non-ACT processes. Seven studies investigated whether challenging negative/dysfunctional cognitions mediated outcome and failed to consistently demonstrate mediation or moderated mediation (Arch et al., Reference *Arch, Wolitzky-Taylor, Eifert and Craske2012; Flaxman and Bond, Reference *Flaxman and Bond2010; Forman et al., Reference *Forman, Herbert, Moitra, Yeomans and Geller2007, Reference *Forman, Chapman, Herbert, Goetter, Yuen and Moitra2012; Niles et al., Reference *Niles, Burklund, Arch, Lieberman, Saxbe and Craske2014; Wicksell et al., Reference *Wicksell, Olsson and Hayes2011; Zettle et al., Reference *Wicksell, Olsson and Hayes2011). Five of these studies that examined this process compared ACT with CT or CBT (Arch et al., Reference *Arch, Wolitzky-Taylor, Eifert and Craske2012; Forman et al., Reference *Forman, Herbert, Moitra, Yeomans and Geller2007, Reference *Forman, Chapman, Herbert, Goetter, Yuen and Moitra2012; Niles et al., Reference *Niles, Burklund, Arch, Lieberman, Saxbe and Craske2014; Zettle et al., Reference *Zettle, Rains and Hayes2011). The findings suggest that this process may not be a mechanism of change during cognitive therapies. Other non-ACT specific mediators examined [symptom frequency (Bach et al., Reference *Bach, Gaudiano, Hayes and Herbert2013; Zettle et al., Reference *Zettle, Rains and Hayes2011), symptom distress (Bach et al., Reference *Bach, Gaudiano, Hayes and Herbert2013), self-efficacy (Wicksell et al., Reference *Wicksell, Olsson and Hayes2011) and pain intensity (Wicksell et al., Reference *Wicksell, Olsson and Hayes2011)] failed to demonstrate any mediation effect. This provides further evidence that the processes of change in ACT are linked to the various components of the psychological flexibility model.
Discussion
As mediation studies evidence the mechanisms by which interventions produce clinical outcomes, they are a vital aspect of the evidence base for any psychotherapy (MacKinnon et al., Reference MacKinnon, Fairchild and Fritz2007). Defining core differences between the mechanisms of change during psychotherapies provides evidence of important theoretical distinctions (and associated indicated content) between the panoply of talking treatments. Hayes et al. (Reference Hayes, Luoma, Bond, Masuda and Lillis2006a) previously found consistent positive mediation results for putative ACT processes, but in the context of studies with poor methodological quality and limited process scope. The current study sought to update the evidence base concerning mediators of outcome during ACT through employing a formal method of study quality assessment and to see whether indicated methodological lessons have been learned. Twelve studies satisfied criteria for inclusion. In general, mediation results were found to be consistent with the psychological flexibility model (Hayes et al., Reference Hayes, Strosahl and Wilson2012). Disappointingly, the evidence base of mediation during ACT (a) continues to be stymied by studies with poor internal reliability, and (b) fails to consistently investigate all six processes of the psychological flexibility model. Perhaps the exception to this was the ‘acceptance’ aspect of the model, which has perhaps been over-studied in comparison. The ‘hexaflex’ ACT model defining psychological flexibility denotes equal weight to each of the six core concepts (Hayes et al., Reference Hayes, Strosahl and Wilson2011a), but this theoretical equipoise has not been reflected in the design of associated mediation studies.
Of the primary processes examined (psychological flexibility, cognitive defusion and acceptance), ‘acceptance’ was the only process for which mediation/moderated mediation was found across type of outcome. These results were therefore consistent with a meta-analysis of laboratory-based component studies (Levin et al., Reference Levin, Hildebrandt, Lillis and Hayes2012), finding a significant and large effect size for acceptance-based interventions compared with inactive conditions. Acceptance appears to be a distinct component within the psychological flexibility model of ACT when compared with other psychotherapies (predominantly CBT in the studies) and so can be considered a primary mechanism of change during ACT. ‘Cognitive defusion’ did not consistently mediate the association between type of psychotherapy and outcome. Of the eight mediation analyses (conducted across four studies) examining cognitive diffusion, only one failed to find a significant mediation result. However, four out of the five moderated mediation analyses failed to show that type of psychotherapy moderated mediation effects. This suggests that whilst the ability to be able to engage in cognitive defusion appears to result in positive outcomes, this process may not be entirely theoretically distinct (and so unique) to ACT. Those studies that failed to find a significant mediation/moderated mediation effect for cognitive defusion (Arch et al., Reference *Arch, Wolitzky-Taylor, Eifert and Craske2012; Forman et al., Reference *Forman, Chapman, Herbert, Goetter, Yuen and Moitra2012) used non-standardized measures. In comparison, the Bach et al. (Reference *Bach, Gaudiano, Hayes and Herbert2013) study used an objective behavioural measure (e.g. rehospitalization rates) and the Zettle et al. (Reference *Zettle, Rains and Hayes2011) study used a validated measure of cognitive defusion – and both reported positive mediation results. It is possible, therefore, that the mixed findings for cognitive defusion may be due to measurement issues.
‘Psychological flexibility’ strongly mediated the association between type of psychotherapy and mental health outcomes, patient functioning and physical symptom reduction, but yielded mixed results for quality of life outcomes. Quality of life is often deemed to be a more appropriate measure of therapeutic outcomes during ACT, due to the focus on values-based living over symptom reduction (Hayes et al., Reference Hayes, Strosahl and Wilson2012). Therefore, these findings were surprising and intriguing. Whilst these findings may be due to methodological limitations, research is needed in order to explore this finding in more detail. Although limited to two studies, ‘committed action’ was a consistent mediator that was not moderated by type of psychotherapy. This indicates that although committed action appears to be a mechanism of change during ACT, it is not solely theoretically distinct from the psychological flexibility model. This is understandable given that this process shares similarities with components of other behavioural models, such as behavioural activation (Martell et al., Reference Martell, Addis and Jacobson2001).
‘Present momentness’ and ‘values’ were only examined in a single study each and there were no studies that examined the ‘self-as-context’ aspect of the psychological flexibility model. The lack of studies examining self-as-context may be due to difficulties with measuring this process (Gootzeit, Reference Gootzeit2014) and the self-experiences questionnaire (SEQ; Yu et al., Reference Yu, Norton and McCracken2016) therefore shows promise as a self-report measure of self-as-context in future ACT mediation studies. Present momentness and values do have extant validated measures and researchers therefore have the means by which to conduct mediation studies. It is crucial, therefore, that further mediation-based research is conducted on values and present momentness, with self-as-context mediation studies pending valid and reliable measurement development. With regard to the non-ACT specific processes investigated, no processes were associated with outcome change during ACT. This provides further evidence that the processes of change during ACT are linked to the components of psychological flexibility.
Quality issues
This review considered the methodological quality of papers subsequent to the call of Hayes et al. (Reference Hayes, Luoma, Bond, Masuda and Lillis2006a) for mediation studies to be conducted with a sound internal validity. Use of a methodological quality assessment tool is an advance on the original Hayes et al. (Reference Hayes, Luoma, Bond, Masuda and Lillis2006a) review. The use of the checklist identified a number of limitations with regard to quality and scope of contemporary ACT mediation studies. The original Hayes et al. (Reference Hayes, Luoma, Bond, Masuda and Lillis2006a) criticism of mediation studies over-focusing on particular ACT processes still stands. In the present review, the evidence base was found to be overly focused on testing psychological flexibility, acceptance and cognitive defusion. Evidence for the mediating effects of the other components of psychological flexibility remains limited or completely untested. All studies were either underpowered or did not report power calculations (with the exception of Lundgren et al., Reference *Lundgren, Dahl and Hayes2008), thus increasing the possibility of Type-I errors. Studies were over-dependent on self-report measures, and the reliability of key mediator and outcome measures was not consistently reported. Two studies examined the mediator at a single time point when testing mediation, and examined this static mediator in relation to change in outcomes (Lundgren et al., Reference *Lundgren, Dahl and Hayes2008; Zettle et al., Reference *Zettle, Rains and Hayes2011). Thus, the assumption of changes in treatment processes predicting changes in outcome illustrated in Fig. 2 was not met. However, all other studies included in this review did include a consideration of time when assessing moderated mediation by testing the treatment process at multiple time points and using this change over time as the mediator. This provides a more rigorous test of the proposal that treatment processes mediate treatment-driven changes in outcomes.
In addition, only a small proportion of the studies met the three statistical quality criteria in full. Some studies implemented an appropriate technique and estimated an indirect effect, but omitted controlling for therapist-cluster or time effects. Some studies did not even report an estimate of an indirect effect, which make the mediation difficult to probe. Finally, about half of the studies did not achieve a minimum of 80% of statistical power, creating a risk that studies testing more complex mediation models may not find a significant effect due to insufficient sample size. It is also worth noting that the methodological quality assessment tool itself (Rhodes and Pfaeffli, Reference Rhodes and Pfaeffli2010) may have had poor reliability for the statistical methods used. This is because the ‘statistical appropriate/acceptable methods’ item (question 10) is possibly too broad given the advancement in state of the art mediation methods. The quality checklist therefore needs updating in line with the statistical criteria used here.
Limitations
The scope of the review was limited to studies conducting mediation analyses and so did not consider studies using other means of correlation/regression to test changes in process measures between treatments. While the clear rationale for this was provided due to the ability of mediation analysis to infer underlying mechanisms through examining indirect effects and interactions between variables (Baron and Kenny, Reference Baron and Kenny1986), it is accepted that the inclusion of other studies would have widened the scope of the review.
Clinical implications
There is strong evidence from this review to suggest that acceptance is an inimitable mechanism of change during ACT and therefore that acceptance is a theoretically unique treatment component. Increasing acceptance abilities through ACT has been shown to improve mental health (Forman et al., Reference *Forman, Chapman, Herbert, Goetter, Yuen and Moitra2012; Rost et al., Reference *Rost, Wilson, Buchanan, Hildebrandt and Mutch2012; Wicksell et al., Reference *Wicksell, Olsson and Hayes2011), quality of life (Rost et al., Reference *Rost, Wilson, Buchanan, Hildebrandt and Mutch2012), health-related behaviours (Bricker et al., Reference *Bricker, Wyszynski, Comstock and Heffner2013) and patient functioning (Forman et al., Reference *Forman, Chapman, Herbert, Goetter, Yuen and Moitra2012; Wicksell et al., Reference *Wicksell, Olsson and Hayes2011). ACT requires therapists to recognize and respond to any presenting inflexibility process during sessions with a corresponding flexibility process (e.g. responding to cognitive fusion with defusion; Hayes et al., Reference Hayes, Strosahl, Wilson, Bissett, Pistorello and Toarmino2006b). This review supports that a primary process in ACT should be helping patients to shift towards acceptance and away from experiential avoidance (i.e. the corresponding inflexibility process). This is in keeping with the ‘experiential avoidance disorder’ approach of ACT to formulating psychopathology (Boulanger et al., Reference Boulanger, Hayes and Pistorello2010; Hayes et al., Reference Hayes, Wilson, Gifford, Follette and Strosahl1996). Whilst cognitive diffusion and committed action failed to consistently evidence a mediation effect that was moderated by treatment type, there was still strong evidence suggesting that these represent core mechanisms of change during ACT (albeit being non-distinct from processes occurring within CBT). Clearly, many psychotherapies share some components of change with their theoretical cousins.
Research implications
An important finding from this review is that despite apparent theoretical equipoise regarding the components of the psychological flexibility model, the mediation evidence base has failed to respond in a coordinated and coherent manner, despite previous prompting (Hayes et al., Reference Hayes, Luoma, Bond, Masuda and Lillis2006a). Future ACT mediation research should reflect the underpinning theoretical model more consistently. Future studies also need to consider testing more complex mediation models (i.e. parallel mediation models) in order to more appropriately and accurately assess ACT mechanisms of change. Also, isolating components of the psychological flexibility model during the design of mediation studies would make a greater theoretical contribution rather than studying the entire model at once, e.g. via use of the multidimensional psychological flexibility inventory (MPFI; Rolffs et al., Reference Rolffs, Rogge and Wilson2016).
Conclusion and future directions
High-quality research is needed in order to address the identified gaps in the ACT mediation literature via sustained improvements to the internal validity of mediation studies and also the expanding the scope of the research across the psychological flexibility model. Guidance from Hayes et al. (Reference Hayes, Luoma, Bond, Masuda and Lillis2006a) has been worryingly neglected and ignored, and lessons should be learned so that future reviews do not arrive at the same conclusion. Firstly, this review again highlighted the continued lack of the use of reliable process measures. Therefore, further research needs to employ (and, if necessary, develop) psychometrically robust process measures, enabling putative processes to be tested in a reliable fashion. Secondly, this review found that a number of processes within the psychological flexibility model remain under-investigated (i.e. values, committed action and contact with the present moment). Future mediation studies need to broaden their scope to focus on these under-investigated processes. Finally, processes occurring within ACT that do not lend themselves particularly well to self-report (e.g. self-as-context) should be investigated via methods such as dismantling studies (Ahn and Wampold, Reference Ahn and Wampold2001) or component analyses (Ruiz, Reference Ruiz2010). Whilst behaviour change is a common goal across behavioural therapies, models widely diverge on their explanation of the key mechanisms/processes that enable outcome (McCracken and Voles, Reference McCracken and Vowles2014).
Future studies should also consider more complex mediator designs by testing several ACT mediators in parallel in order to evaluate their relative strength in the psychological flexibility model. Parallel multiple mediator models test a relationship between mediators in which, although several mediators are involved, none of the mediators affects each other. In theory, any number of mediators are possible to model and that would be in keeping with the psychological flexibility model. Ding et al. (Reference Ding, Ng and Li2015) listed the advantages of such parallel multiple mediator models as (a) the chance of parameter bias due to omitted variables is reduced in the multiple putative mediators, (b) the sum of the indirect effects calculated in simple mediation analyses may not equal to the total indirect effect, as the mediators in a multiple mediator model may be inter-correlated, and (c) a multiple mediator model enables the definition of the relative magnitudes of specific indirect effects which then enables effective comparison of competing theories of change. Specifying which ACT variables prove to be stronger mediators may help the psychological flexibility model to further evolve and provide evidence as to its main contributory therapeutic components. Twelve studies satisfied criteria for inclusion in the present review and mediation results were generally found to be consistent with the psychological flexibility model. However, due to identified methodological limitations and narrowness of scope, any conclusions drawn are done so cautiously. Only further high quality research can confidently unearth the theoretically independent mechanisms of change within ACT's psychological flexibility model.
Acknowledgements:
This article is based on research conducted as part of a DClinPsy thesis entitled ‘Exploring the Underlying Components of Acceptance and Commitment Therapy’ by D Stockton at The University of Sheffield, submitted in 2015.
Conflicts of interest: The authors have no conflicts of interest with respect to this publication.
Ethical statement: The authors have abided by the Ethical Principles of Psychologists and Code of Conduct as set out by the APA. No ethical approval was required as this is a review of published literature.
Financial support: This research was partially supported by Proyecto Basal Usach USA1498_BR011127.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1352465818000553
Appendix 1. Summary of study quality
Appendix 1: Statistical re-analysis of data from Flaxman and Bond's (Reference *Flaxman and Bond2010) study
Statistical model of moderated mediation used (Hayes, Reference Hayes2013)
Statistical output
Comments
No Comments have been published for this article.