Software for ROM

One of the new technologies that is becoming commonly used in clinical supervision is software that records session-by-session clinical outcome measures, also called ROM or continuous assessment (CA) (Lambert, Reference Lambert, Duncan, Miller, Wampold and Hubble2010). CBT supervisors will be familiar with standardized supervision instruments [e.g. Beck Anxiety Inventory (BAI); Beck & Steer, Reference Beck and Steer1993; and the Cognitive Therapy Adherence and Competence Scale; Liese et al. Reference Liese, Barber and Beck1995); ROM software wraps these instruments into a software package that provides a visual graph showing clients’ trajectory of change in treatment, and thus facilitates use in supervision and clinical practice. ROM software lets clients complete outcome assessment forms on a variety of devices (e.g. desktop computers, laptops, tablets, or smartphones), in the clinician's office or waiting room. ROM programs typically take 2–5 minutes to complete. The programs automatically graph clients’ symptom progress, flag clients at risk for deterioration, and highlight notable risk factors, such as risk for suicide. Controlled studies have repeatedly demonstrated ROM programs help clinicians achieve to better clinical outcomes by reducing the likelihood of client deterioration and decreasing drop-out rates (Duncan & Reese, Reference Duncan, Reese, Rousmaniere and Renfro-Michel2015; Miller et al. Reference Miller, Hubble, Chow and Seidel2013). Two feedback systems have such a powerful impact on client outcome that they are now considered an ‘evidence-based practice’ by the Substance Abuse and Mental Health Services Administration [the Partners for Change Outcome Management System (PCOMS) and OQ-Analyst; http://nrepp.samhsa.gov/].

Like professionals in many fields, clinicians have been shown to exhibit a positive bias about their own clinical skills, which can form a blind-spot that interferes with their ability to assess their own clients’ risk of clinical deterioration (Lambert, Reference Lambert, Duncan, Miller, Wampold and Hubble2010). For example, in a survey of 129 mental health professionals, the average participant rated their own work performance in the 80th percentile, no participants rated themselves below average, and 25% of participants rated themselves in the 90th percentile (Walfish et al. Reference Walfish, McAlister, O’Donnell and Lambert2012). ROM programs can provide an evidence-based perspective that can address this blind-spot and thus may help improve clinical outcomes. Since the blind-spot is common to pre- and post-licensure clinicians (Lambert, Reference Lambert, Duncan, Miller, Wampold and Hubble2010), ROM programs are useful for both training of pre-licensure clinicians and helping licensed clinicians monitor their own work performance. ROM software reduces the time and effort required for outcome assessment, when compared to paper measures.

The OQ Analyst ‘clinical support tools’, a program that provides session-by-session feedback to clinicians on clients that are at risk for deterioration, was one of the first examples of internet-based ROM software (Whipple et al. Reference Whipple, Lambert, Vermeersch, Smart, Nielsen and Hawkins2003). The PCOMS, developed by Miller et al. (Reference Miller, Duncan, Sorrell and Brown2005), is another widely used ROM program that uses the Outcome Rating Scale (ORS) and Session Rating Scale (SRS). The ORS and SRS are very brief (four item) measures of overall life functioning and the working alliance in psychotherapy. PCOMS has been demonstrated to help clinicians improve their outcomes and reduce drop-outs by multiple controlled studies (Anker et al. Reference Anker, Duncan and Sparks2009). The Counseling Center Assessment of Psychological Symptoms (CCAPS) is another ROM program that was designed to be used at University Counselling Centers (UCCs) and is in use in over 200 locations (Locke et al. Reference Locke, Bieschke, Castonguay and Hayes2012). One benefit of CCAPS is that it is easily integrated into common electronic records management (ERM) software. Another ROM program is the Contextualized Feedback System, which uses variety of internet-based measures for family and marriage therapy (Bickman et al. Reference Bickman, Kelley and Athay2012).

There have recently been calls for organizations to provide systemic support for the application of EBCS (Milne & Reiser, Reference Milne and Reiser2016). In line with this goal, some outcome tracking software packages have been designed to be integrated into an organization's infrastructure. One example is OwlOutcomes, an internet cloud-based measurement feedback system (MFS) that provides access to an extensive digital library of evidence-based assessment measures (www.owloutcomes.com; originally called Evidence-Based Assessment System for Clinicians; Smith et al. Reference Smith, Fagan, Wilson, Chen, Corona, Nguyen, Racz and Shoda2011). OwlOutcomes is used by all clinical staff at the Centre, throughout all contact with the patient. At first contact with the patient, an intake coordinator or screener uses a structured screening tool within OwlOutcomes to assess patients’ initial clinical presentation. Next, the Centre director uses that data (accessible securely online) to assign cases to specific trainees or staff, as well as supervisors. OwlOutcomes is then used throughout psychotherapy treatment to collect session-by-session clinical assessment data, which is used by clinical supervisors to inform and guide supervision. This data allows supervisors and the Centre director to easily monitor patients’ progress, including alerts for risk factors, such as suicidality and child abuse. The software features clinical decision-making support based on Best Practice guidelines, or the Centre's own internal guidelines. OwlOutcomes is compliant with the Health Insurance Portability and Accountability Act (HIPAA), a United States federal law passed in 1996 that set standards for the use, storage, and disclosure of patients’ information (e.g. notes, charts, etc.)

See Table 1 for a list of ROM software. All of these programs include features that allow Centre directors view and export aggregated data.

Table 1.

Web-based software for routine outcome assessment (ROM)

Remote live supervision: videoconference for internet-based live one-way-mirror supervision

Live supervision supports evidence-based supervision by offering an unedited view of the therapy session, allowing supervisors to assess therapist competence, adherence, and treatment outcome in real time (Jakob et al. Reference Jakob, Weck and Bohus2013). Having an unedited view of trainees’ work is valuable, given that 97% of supervisees reported non-disclosure in supervision (Ladany et al. Reference Ladany, Hill, Corbett and Nutt1996). Live supervision offers several other advantages to delayed-report supervision, including the ability to make more timely feedback and corrections than delayed-report supervision (Champe & Kleist, Reference Champe and Kleist2003; Bartle-Haring et al. Reference Bartle-Haring, Silverthorn, Meyer and Toviessi2009). In live supervision the client more directly benefits from the experience and skills of the supervisor, leading to a greater likelihood of positive therapeutic outcomes and enhanced supervisee learning (Bernard & Goodyear, Reference Bernard and Goodyear2014; Moody et al. Reference Moody, Kostohryz and Vereen2014). Live supervision has been described as ‘economical’, because ‘in the role of both the therapist and the observer, trainees got so much out of live supervision in such a short period of time’ (Wong, Reference Wong1997, p. 151). Silverthorn and colleagues (2009) found that a single session of live supervision could significantly improve the supervisee's perceptions of clinical progress.

Until recently, live supervision has been limited by the need for expensive one-way mirrors and the requirement for both members of the supervisory dyad to be in the same location. This can be challenging in rural areas, when supervisees may work several hours (or more) away from their supervisors. The limited availability of supervision in remote areas has been found to be a risk factor for clinician dissatisfaction, turnover, and burnout (Mitchell et al. Reference Mitchell, Destefano, Martinez and Potter2009). One potential solution to these limitation is to use internet videoconference software for live supervision, termed Remote Live Supervision (RLS; Rousmaniere & Frederickson, Reference Rousmaniere and Frederickson2013). With RLS, a supervisor uses videoconference software to provide internet-based live one-way-mirror supervision, at any geographical distance, without the need for expensive one-way mirrors (Jakob et al. Reference Jakob, Weck and Bohus2013; Rousmaniere & Frederickson, Reference Rousmaniere and Frederickson2013).

Physical set-up for RLS

During RLS, the client and therapist (supervisee) sit across from each other, as in a traditional counselling room. A webcam focused on the client transmits live video during the counselling session. The supervisor then provides immediate feedback for the therapist, enabling them to immediately improve therapy effectiveness. In visual-format RLS, the supervisor types text interventions into a chat window in videoconference software. A computer monitor or laptop computer sitting next to the client displays the supervisor's interventions in large type, like a teleprompter for the supervisee, but out of the client's sight (a method referred to as bug-in-the-eye; Klitzke & Lombardo, Reference Klitzke and Lombardo1991). Alternately, in audio-format RLS, the supervisor speaks interventions into the computer's microphone, which are heard by the supervisee via an earpiece (the bug-in-the-ear method of live supervision; Klitzke & Lombardo, Reference Klitzke and Lombardo1991). The supervisee decides which interventions to integrate into therapy. Note that the videoconference software simply transmits the video of the therapy session to the supervisor's computer, like a television transmission. If the supervisor wishes to save a video of the session, a separate device may be used for this purpose (Rousmaniere & Frederickson, Reference Rousmaniere and Frederickson2013). Similarly, Angela Yu recently developed ISup, an innovative ‘app’ that utilizes the IPad for live one-way-mirror supervision (www.isupelive.com).

RLS technical set-up

RLS requires that both the supervisor and supervisee have a computer (desktop or laptop), and that the supervisee has a webcam. A USB cable extender may be helpful to connect the webcam to the laptop computer. An external microphone may optionally be used to provide better sound quality. If audio-format RLS is used, the supervisee can use a Bluetooth wireless earbud to hear the supervisor's comments. As of July 2015, all of this equipment except for the computers could be purchased for less than US$250 (total) from major electronic retail stores. RLS can be performed on any of a wide range of videoconference software packages.

RLS benefits

Traditional delayed-report supervision occurs after psychotherapy sessions end, and thus rely on the use of memory aids to recall what happened in the session (e.g. progress notes, videotapes). Delayed-report supervision uses a cognitive recall manner of learning, which can help supervisees learn about psychotherapy theory, processes, and models. In contrast, live supervision is an experiential method of learning, an important component of CBT training (Rakovshik & McManus, Reference Rakovshik and McManus2010; Roth et al. Reference Roth, Pilling and Turner2010; Reiser & Milne, Reference Reiser and Milne2012). Because RLS occurs in real time during the therapy session, the supervisee experiences the supervisor's ability to make assessments and interventions in real time. This can give the supervisee the opportunity to ‘walk in the supervisor's shoes’, and experience the psychotherapy model in action. In RLS the supervisor can give guidance immediately when the supervisee needs it. This facilitates state-dependent learning, which can help build procedural psychotherapy skills (Rousmaniere & Frederickson, Reference Rousmaniere and Frederickson2013). Jakob et al. (Reference Jakob, Weck and Bohus2013) examined the use of RLS for supervision of behavioural therapy by giving questionnaires to 10 patients, six trainees, and five supervisors, after a total of 25 RLS sessions. The data indicated very high levels of acceptance of RLS, with 96% of trainees and supervisors reporting that RLS ‘increases the effectiveness of my work’ (Jakob et al. Reference Jakob, Weck and Bohus2013).

Suitability for RLS

For the same reasons cited above, live supervision can be experienced as a very challenging method of training. In RLS, the supervisee must simultaneously track both the supervisor and the client. There is the risk that new clinicians may feel confused or lost during RLS sessions, or may adopt a passive role and simply obey the supervisor's guidance (Bernard & Goodyear, Reference Bernard and Goodyear2014). Furthermore, performing RLS at a geographical distance may heighten these risks. For example, trainees who do not have sufficient skill (technical or interpersonal) to apply their supervisor's interventions may create clinical problems that they are unable to correct later, when the supervisor is not available. Thus, although research has suggested that RLS may be appropriate for in-person supervision of trainees (Bartle-Haring et al. Reference Bartle-Haring, Silverthorn, Meyer and Toviessi2009; Jakob et al. Reference Jakob, Weck and Bohus2013), RLS may be not suitable for trainees located in a distant location or different jurisdiction than the supervisor (Rousmaniere & Frederickson, Reference Rousmaniere and Frederickson2013).

Challenges posed by supervision technology

Researchers have noted a number of potential challenges in the use of supervision technology, including difficulty understanding nonverbal communication due to the limited visual cues permitted by electronic communication (Vaccaro & Lambie, Reference Vaccaro and Lambie2007); the inability for supervisors to provide help from a distance, and possible unfamiliarity with local laws and regulations (Abbass et al. Reference Abbass, Arthey, Elliott, Fedak, Nowoweiski, Markovski and Nowoweiski2011); heightened risks of cultural misunderstandings due to geographical distance between supervisor and supervisee (Powell & Migdole, 2012); heightened anxiety in some supervisees (Sørlie et al. Reference Sørlie, Gammon, Bergvik and Sexton1999) or supervisors (Renfro-Michel et al. Reference Renfro-Michel, Rousmaniere, Spinella, Rousmaniere and Renfro-Michel2015); and the risk that training via videoconference may not be as effective as in-person training or mixed in-person and distance supervision (Sholomskas et al. Reference Sholomskas, Syracuse-Siewert, Rounsaville, Ball, Nuro and Carroll2005). As of yet there is little empirical data to support these concerns, but few studies have been done and more research in these areas is clearly indicated (Rousmaniere, Reference Rousmaniere, Watkins and Milne2014).

Another notable challenge posed by videoconference technology (that may be the cause of considerably anxiety on the part of supervisors) is the protection of client privacy and confidentiality. It is important to use videoconference software that is compliant with national and/or local laws and regulations regarding privacy and confidentiality (McAdams & Wyatt, Reference McAdams and Wyatt2010; Rousmaniere et al. Reference Rousmaniere, Renfro-Michel, Higgins, Rousmaniere and Renfro-Michel2015). Likewise, supervisors should be competent in both videoconference and other technology necessary to ensure internet security. Maintaining competency in internet technology can be no small feat, given that many programs change or ‘update’ on a frequent basis. A list of recommendations for enhancing security in supervision technology is provided in Rousmaniere & Kuhn (Reference Rousmaniere, Kuhn, Rousmaniere and Renfro-Michel2015).

Another challenge of internet-based supervision that invariably causes anxiety is the mixed reliability of internet connections. Most videoconference programs require an internet speed of over 0.5 Mbps. (A free internet speed test is available at Internetwww.speedtest.net.) Even when high-speed internet connections are used, it is common for users to experience occasional technical problems such as dropped calls or poor connectivity. For example, over a 5-year period of using videoconference for supervision at both private practice and university sites, the author encountered connectivity problems in about 20% of RLS sessions, even with high-speed internet. Thus, RLS is only appropriate for supervisors and supervisees who are comfortable with this limitation. To help mitigate this problem, back-up methods of communication should be identified (e.g. phone). Supervisors may take the following steps to help improve videoconference reliability: (a) use a high-speed internet connection, (b) close internet-intensive programs running in the background while using videoconference (e.g. internet-based file-sharing software), (c) avoid ‘screen sharing’ features when connectivity is poor, and (d) use only audio (i.e. turn off the video camera) when internet connectivity is poor (Rousmaniere, Reference Rousmaniere, Watkins and Milne2014).

Technology for evidence-based supervision: an example of use

The following is an example of how technology can be systemically integrated into a clinical training programme at an organizational level in order to encourage and support EBCS. This example is from the practicum for clinical psychology doctoral students at the University of Alaska Fairbanks Student Health and Counseling Center.