Overview of Multisystemic Therapy (MST)

This contribution to the Special Issue focuses on clinical supervision in the context of a continuous quality assurance and improvement system used to support the large-scale implementation of one empirically supported treatment, Multisystemic Therapy (MST; Henggeler et al. Reference Henggeler, Schoenwald, Borduin, Rowland and Cunningham2009b ). A brief overview of MST is provided first, because the defining features of the treatment model permeate all aspects of the quality assurance and improvement system, including clinical supervision.

The MST MST QA/QI system

Nearly 15 years ago, pre-eminent health services researchers in the UK and United States posited the use of Continuous Quality Improvement (CQI) at four levels of the practice context could catalyse industry-wide improvement of healthcare outcomes: individual practitioner, group or team, overall organization, and larger system (Ferlie & Shortell, Reference Ferlie and Shortell2001). CQI had been described as a ‘philosophy of continual improvement of the processes associated with providing a good or service that meets or exceeds customer expectations’ (Shortell et al. Reference Shortell, Bennet and Byck1998, p. 594). These processes involve a variety of individuals within an organization who are trained in basic statistical techniques and tools and are empowered to make decisions based on their analysis of data. In the US, the application of CQI in mental healthcare had not been empirically evaluated. Quality assurance consisted chiefly of professional licensure, accreditation, and clinical supervision of unknown substance – the effects of which on the quality and outcomes of care were largely unknown (Bickman, Reference Bickman1999, Reference Bickman2002).

The first published descriptions and depictions of the comprehensive quality assurance and improvement system designed to support the implementation of MST appeared in the late 1990s (Henggeler & Schoenwald, Reference Schoenwald1998, Reference Henggeler and Schoenwald1999), when empirical evaluations of linkages among its components were underway. The system was designed to replicate procedures and resources provided therapists in randomized trials of MST. Since 1996, the MST QA/QI system has been deployed through MST Services, LLC (MSTS) and its network partners, collectively known as the ‘purveyors’ of MST (Fixsen et al. Reference Fixsen, Naoom, Blase, Friedman and Wallace2005). MSTS is licensed by the Medical University of South Carolina to transfer MST-related technology to interested communities. Network partners deploy the MST QA/QI system with the majority of MST programmes, which operate in 34 states and 15 nations.

The MST QA/QI system, depicted in Figure 1, contains six elements: (a) pre-adoption site assessment; (b) 5-day orientation for therapists and clinical supervisors; (c) clinical supervision on site guided by a supervision protocol (Henggeler & Schoenwald, Reference Henggeler and Schoenwald1998); (d) consultation with an MST expert trained in a consultation protocol (Schoenwald, Reference Schoenwald1998) and employed by a network partner or MSTS; (e) quarterly booster training; and (f) valid and reliable measures of therapist adherence [Therapist Adherence Measure – Revised (TAM-R); Henggeler et al. Reference Henggeler, Borduin, Schoenwald, Huey and Chapman2006], supervisor adherence [Supervisor Adherence Measure (SAM); Schoenwald et al. Reference Schoenwald, Henggeler and Edwards1998] and expert consultant adherence [Consultant Adherence Measure (CAM); Schoenwald, Reference Schoenwald2001]. These elements are integrated into a feedback loop that includes data about MST implementation at the level of the family, therapist, supervisor, expert consultant, and organization operating the MST programme. In addition, an organizational manual, organizational consultation, and semi-annual formal programme reviews focus on cultivating and sustaining organizational, service system, and purveyor policies and practices that support the implementation and outcomes of MST. MST supervisors and expert consultants collaborate in assessing and addressing these organizational and service system aspects of implementation. Measurement and reporting of MST implementation at all levels of the practice context is a function of the MST Institute (MSTI; www.mstinstitute.org).

Fig. 1.

MST Quality Assurance and Quality Improvement (MST QA/QI) system. From Henggeler et al. (Reference Henggeler, Schoenwald, Borduin, Rowland and Cunningham2009b ) Multisystemic Therapy for Antisocial Behavior in Children and Adolescents, 2nd edn. © Guilford Press. (Reprinted with the permission of Guilford Press.)

Findings from a prospective multisite study of the transportability and implementation of MST provided empirical support for linkages among adherence to MST at multiple levels of the practice context and youth outcomes. Funded by the National Institute of Mental Health, with subsequent follow-up funded by the National Institute on Drug Abuse, participants in this study, hereinafter referred to as the MST Transportability Study, were 1979 youth and families treated by 429 therapists across provider organizations in 45 sites. Caregiver ratings of therapist adherence predicted reductions in youth behaviour problems at the end of treatment and through a 1-year post-treatment follow-up and criminal charges through 4 years post-treatment (Schoenwald et al. Reference Schoenwald, Sheidow, Letourneau and Liao2003, Reference Schoenwald, Carter, Chapman and Sheidow2008, Reference Schoenwald, Chapman, Sheidow and Carter2009a ). Supervisor adherence predicted therapist adherence and reductions in youth behaviour problems through 1 year post-treatment (Schoenwald et al. Reference Schoenwald, Sheidow and Chapman2009b ), and long-term criminal charges (Schoenwald et al. Reference Schoenwald, Chapman, Sheidow, Schoenwald and Reid2006). The effects of consultant adherence on therapist adherence and youth outcomes were assessed in two samples of therapists, consultants, and families, one of which was drawn from the MST Transportability Study. Linkages were found among consultant competence, focus on MST procedures, therapist adherence, and youth outcomes (Schoenwald et al. Reference Schoenwald, Sheidow and Letourneau2004).

MST supervision

With the exception of early treatment development and feasibility studies and a randomized trial in which doctoral students served as therapists, the individuals implementing MST in randomized trials have been master’s-level professionals with previous experience treating clients in community practice settings. Treatment model developers or study co-investigators conducted weekly supervision with these professionals. Accordingly, MST supervision is designed to support the implementation of MST among practising professionals rather than for pre-professional training. Today, most MST supervisors are recruited from the ranks of effective MST therapists. Therapists and supervisors are employed by the same organization, typically one that operates a variety of mental health services. The supervisor provides clinical and administrative leadership for the team.

Assumptions underlying MST supervision

The MST Supervisory Manual (Henggeler & Schoenwald, Reference Henggeler and Schoenwald1998) articulates four assumptions underlying MST supervision, paraphrased here. (1) The purpose of clinical supervision is to enable clinicians to adhere to the MST principles in all aspects of treatment – engagement, case conceptualization, intervention design and implementation, and evaluation of outcomes. (2) Clinicians are hard-working, competent professionals with unique strengths and experiences; yet, to treat youth with serious clinical problems effectively, will benefit from ongoing support. (3) The process of clinical supervision should mirror the process of MST. That is, supervision is present-focused, action-oriented, and targets specific problems clinicians appear to be having in each aspect of the treatment process. (4) Therapists, supervisors, and the agency operating the MST programme are accountable for youth outcomes. This final assumption reflects the ‘acid test’ paradigm of supervision (Ellis & Ladany, Reference Ellis, Ladany and Watkins1997), namely that its effectiveness is ultimately measured by its effects on client outcomes.

Responsibilities of MST supervisors

The responsibilities of MST supervisors fall into four domains: (1) development and maintenance of MST-relevant knowledge, clinical competencies, and skills; (2) group supervision; (3) clinician development; and, (4) management of the CQI process for the MST team. Like MST therapists, supervisors are available 24 h a day, 7 days a week. Supervisors of one team may also carry a reduced caseload of families, whereas supervisors of two or more teams typically do not. Training and support of MST supervisors occurs in several ways, including: guided review of the MST Supervisory Manual (Henggeler & Schoenwald, Reference Henggeler and Schoenwald1998); initial supervisor orientation training provided by the MST expert consultant prior to or during the initial 5-day orientation training for the entire team; and, periodic conjoint review with the expert of supervisor work samples, including at least one audio-tape of group supervision monthly. In addition, the expert and supervisor together identify supervisory strengths and needs and devise, implement, and monitor strategies to address the identified needs. Finally, booster training sessions designed specifically for MST supervisors are made available several times annually. The sessions are designed to address the opportunities and challenges awaiting supervisors with different levels of MST experience.

Towards empirically supported supervision of empirically supported treatment

The nature of effective supervision and its role in supporting the larger scale implementation of empirically supported psychosocial treatment is attracting increased attention in implementation research. Surveys and systematic reviews of supervisory practices correlated with select aspects of supervisee learning and behaviour, if not also client outcomes, have yielded compendia of methods that might form the basis for empirically supported supervision. The Supervisor Adherence and Guidance Evaluation (SAGE; Milne & Reiser, Reference Milne, Reiser, Watkins and Milne2014) for example, identifies 23 elements of supervision that are organized into three domains – common factors, supervision cycle, supervisee cycle. There is considerable overlap among the SAGE elements and components of MST group supervision.

As explicated elsewhere (Schoenwald et al. Reference Schoenwald, Sheidow and Chapman2009b ) the conceptualization, planning, intervention discussion, practice, and functions of MST supervision are reminiscent of Kolb's (Reference Kolb1984) learning cycle, but interleaved within the same group supervision session. For example, the completion of weekly case summaries requires therapists to enact elements of the Supervisee Learning Cycle such as conceptualizing planning, and experimenting before the supervision session; as well as elements identified as part of the Supervision Cycle, such as formulation and questioning. During supervision, therapists and the supervisor conjointly execute these activities. MST group supervision also entails many activities subsumed under the Supervision Cycle (e.g. supervisor giving and receiving feedback, case formulation, listening, observation, prompting, questioning, teaching, training, experimenting).

Some supervisory elements categorized within distinct domains of the SAGE are found empirically to cluster on the same domain of MST supervision. For example, the SAM Structure and Process subscale includes items indexing the SAGE common factors activities of collaborating, managing, and facilitating as well as the SAGE supervision cycle activities of agenda setting, giving feedback, receiving feedback. Conversely, some activities grouped within a single domain of the SAGE lie on distinct dimensions of MST supervision; as examples, collaboration, management, facilitating, relating, and agenda setting are all on the SAM Structure and Process subscale.

Another approach to enhancing the effectiveness of clinical supervision may be to adapt a clinical supervision protocol demonstrated to support the implementation of one empirically supported treatment for the purposes of supporting the implementation of another. An adaptation of the MST supervision model to support the implementation of an innovative, multi-component, school-and-community mental health service called Links To Learning (R01 MH073749, Atkins, PI; hereinafter referred to as Links) provided an initial foray into this line of inquiry (Schoenwald et al. Reference Schoenwald, Mehta, Frazier and Shernoff2013). Under the auspices of a randomized trial evaluating the feasibility and effects of Links on disruptive behaviour and learning in children in grades K–3, an observational study evaluated the implementation of a clinical supervision protocol adapted from MST. Data obtained from structured field notes of supervision were coded by members of the investigative team. Results suggested the structure, process, and some content of supervision conformed reasonably well to design. However, one feature of Links and the supervision thereof, conjoint intervention planning across the team of professionals employed in different contexts (schools, mental health agencies) and conjoint supervision across that team, occurred much less frequently than planned. The coded field notes suggested aspects of the organizational contexts of the mental health agencies and schools affected the planned frequency and some content of supervision. School district and state level policies – aspects of the ‘outer context’ of implementation (Aarons et al. Reference Aarons, Hurlburt and Horwitz2011) – also played a more prominent role in service delivery and supervision than had been anticipated.

These challenges raise questions regarding the extent to which the scope of clinical supervision, and responsibility of the clinical supervisor, extends to the proactive cultivation and maintenance of organization-intervention fit and inter-agency collaboration. In the case of MST, that scope explicitly extends to these domains. However, as described elsewhere (Schoenwald & Hoagwood, Reference Schoenwald and Hoagwood2001) the extent to which organizational and inter-organizational changes are needed to implement an effective treatment may vary in accordance with the demand characteristics of the treatment extent to which these contrasts with current practice. The same may be true regarding the extent to which the effective implementation of a treatment requires expansion of the scope of clinical supervision.

Emerging developments in MST clinical supervision

To date, the feasibility of measurement in practice (i.e. low burden, low cost) and establishment of measurement validity and reliability have been equally prioritized in the transport of MST. Accordingly, observational adherence measurement methods, such as coding of audio-recorded treatment sessions, have not been incorporated into the measurement and reporting functions of the MST QA/QI system. The advancement of digital and internet-based technologies, and accrual of evidence of the effectiveness of audit-and-feedback systems on physician adherence to evidence-based guidelines (Jamtvedt et al. Reference Jamtvedt, Young, Kristoffersen, O’Brien and Oxman2006), inspired a pair of studies to develop and evaluate the effects of an observationally based audit and feedback system for clinical supervisors. In the first, a measurement development study, digitally recorded and uploaded weekly group supervision sessions of 30 MST supervisors were coded by individuals who were not familiar with MST, but trained to code supervisor adherence and competence to the group supervision process (R21 MH097000; J. Chapman & S. Schoenwald, co-PIs). The second study, a randomized trial currently underway (R33 MH097000–03; J. Chapman, PI), is evaluating the effects on group supervision, therapist adherence scores, and youth outcomes at case closure of providing to supervisors monthly a brief web-based feedback report based on the observational adherence and competence instrument. In both studies, consideration is being taken to identify aspects of supervision, and of the web-based feedback system, potentially applicable to supervision of other empirically supported treatments.

Potential implications for clinical supervision and quality improvement

Relatively little is known empirically regarding aspects of clinical supervision that are both effective (i.e. affect clinician behaviour and client outcomes) and likely to generalize across diverse treatments and the target problems they are designed to address. Accordingly, circumspection is warranted regarding the potential utility of MST supervision experience and research to the clinical supervision of other evidence-treatments, or more generally. The following propositions are offered as ‘food for thought.’

First, the assumptions about practising professionals and the potential value of clinical supervision to them that underlay MST supervision (reprised in this article) may, with minor modifications, be applicable to the behavioural workforce treating client populations whose serious clinical problems render treatment failure a high-stakes event (e.g. hospitalization, residential treatment, job loss or school expulsion, jail or prison term, injury of self or others, etc.). To the extent that effective treatments for such populations require tailoring on a case by case basis of the nature, combination, and sequencing of treatment techniques, ongoing supervisorial support of the sort provided in MST may be particularly critical. Conversely, for treatment approaches characterized by a relatively circumscribed set of assessment and treatment techniques, delivered in a standard sequence or in an algorithmically determined manner, it is possible that clinician implementation with fidelity at the point of performance (i.e. in sessions with clients) could be sustained with less frequent or comprehensive ongoing clinical supervision. However, recent findings from an observational study of supervision conducted during a hybrid effectiveness/implementation trial of standard and modular delivery of manualized treatments for children show that even in such scenarios practice in supervision is needed to support clinician use of techniques in sessions (Bearman et al. Reference Bearman, Weisz, Chorpita, Hoagwood, Ward, Ugueto and Bernstein2013). The findings of studies such as this, and of randomized trials in which supervision components are manipulated and evaluated will shed light on the extent to which aspects of supervision such as case review, practice, and fidelity monitoring affect clinicians and clients (Dorsey et al. Reference Dorsey, Pullman, Deblinger, Berliner, Kerns, Thompson and Garland2013).

Second, the MST Analytical Process that guides both treatment and supervision is simply the scientific method applied to clinical work, using the same rules of hypothesis development and testing (gathering evidence to support and to refute hypotheses) found in scientific experiments. The use of such a systematic, stepwise, and iterative process to guide clinician treatment of clients, and clinical supervision, establishes a parsimonious and predictable structure and process for learning and executing case conceptualization and intervention skills. Third, the collaborative, strength-focused, and action-oriented and present-focused nature of MST supervision, although prescribed by the application to supervision of MST treatment principles, may be pertinent to clinical supervision more broadly. Fourth, the time and cost efficiencies, team-building, and mutual learning of team members from one another afforded by the group supervision model may also be replicable for other treatments. Such efficiencies are facilitated at least in part, by the weekly advanced preparation of the clinicians, and the supervisor, for supervisory discussion, preparation that includes documentation in specific and measurable terms of case progress and challenges, and outcomes.

In addition, two of the four domains of MST supervisor responsibility seem consistent with responsibilities ascribed to clinical supervision more generally, namely maintaining mastery of the knowledge, competencies, and skills in the treatment being implemented; and, supporting clinician development with respect to the knowledge, competencies, and skills needed to execute at the point of performance (in sessions with clients). The potential utility of the third domain, group supervision, has already been considered. The fourth domain, management of the CQI process for a clinical team, may become increasingly pertinent to the supervision of clinicians implementing treatments other than MST. This is because interest is increasing in research on the effects of distinct quality assurance components (i.e. training, clinical supervision, fidelity monitoring, routine outcomes monitoring, audit-and-feedback interventions) and combinations thereof on the larger-scale implementation and outcomes effective psychosocial treatments. In addition, efforts to define and measure indicators of the quality of mental health treatments (including treatment fidelity) and to integrate such measurement into clinical feedback systems are gaining traction (Wandersman et al. Reference Wandersman, Chien and Katz2012; McLeod et al. Reference McLeod, Southam-Gerow, Tully, Rodriguez and Smith2013). Wandersman and colleagues have proposed, for example, an evidence-based system for innovation support for implementing innovations with quality (Wandersman et al. Reference Wandersman, Chien and Katz2012) to guide prevention research and practice. The major components of the system are tools, training, technical assistance, and quality assurance/quality improvement – the latter designed so as to reinforce the former. Clinical supervision is one element of quality assurance. The extent to which CQI processes can improve the nature and effects of clinical supervision; and, conversely, the effects of supervisor management of CQI processes, are potentially important, but as yet largely unanswered, empirical questions.

Summary

CQI systems designed to affect multiple aspects of the practice context can be used to support the implementation with fidelity and attainment of positive outcomes of comprehensive evidence-based treatments such as MST in diverse community contexts. The MST QA/QI system embodies training, support, measurement, and feedback for therapists, supervisors, expert consultants, and organizations implementing MST programmes. The scope of clinical supervision in a system such as this extends the support of treatment-specific conceptual and skills and competencies among therapists and includes shared accountability for client outcomes, cultivation and maintenance of team cohesion and performance, management of select CQI processes, and navigation of organizational and system policies that can affect treatment implementation and outcomes.