Introduction
In an effort to disseminate evidence-based practices (EBP), such as cognitive behavioral therapy for childhood anxiety, computer-assisted cognitive behavioral therapy (CCBT) programs have recently been developed and have demonstrated promising results (e.g. Khanna, and Kendal, Reference Khanna and Kendall2010; Spence et al., Reference Spence, Donovan, March, Gamble, Anderson and Prosser2008; Spence, Holmes, March and Lipp, Reference Spence, Holmes, March and Lipp2006; Wuthrich et al., Reference Wuthrich, Rapee, Cunningham, Lyneham, Hudson and Schniering2012). However, research about the factors that contribute to implementation of CCBT in community mental health centers (CMHC) is limited. As the field moves forward with the development of CCBT for childhood anxiety (Kendall, Khanna, Edson, Cummings and Harris, Reference Kendall, Khanna, Edson, Cummings and Harris2011), and strategies to improve access and implementation of EBP in CMHC are employed (Garland et al., Reference Garland, Haine-Schlagel, Brookman-Frazee, Baker-Ericzen, Trask and Fawley-King2012), it is important to gather feedback from consumers and providers about their perceptions of utilizing CCBT to advance its implementation in CMHC.
There are several factors to consider when developing CCBT programs that determine client acceptability. For example, Cunningham, Rapee and Lyneham (Reference Cunningham, Rapee and Lyneham2006) developed Cool Teens, a 12-week CBT CD-ROM program, and initial feedback from adolescents suggested that they found the program “an interesting and fun way to learn” and they liked the realistic examples and live video clips, but they wanted more scenarios, characters, videos, and better navigation information. Wuthrich et al. (Reference Wuthrich, Rapee, Cunningham, Lyneham, Hudson and Schniering2012) examined barriers commonly encountered over the course of Cool Teens treatment; such barriers included technical difficulties, understanding the content and tasks, therapist support, relevance to the child's problems, amount of personal data collected, and desire to practice the tasks. Additionally, approximately 20% to 30% of the youth reported that very often finding time to use the CD-ROM was difficult; often they lost interest in the program and sometimes they found the CD-ROM boring (Wuthrich et al., Reference Wuthrich, Rapee, Cunningham, Lyneham, Hudson and Schniering2012).
Another CCBT CD-ROM that has undergone initial feedback on the development of the program is Camp Cope-A-Lot (CCAL). Camp Cope-A-Lot, for ages 7 to 13 years, was developed based on the Coping Cat treatment, which is a therapist-led treatment with demonstrated efficacy for childhood anxiety (e.g. Kendall, Reference Kendall1994; Khanna, and Kendall, Reference Khanna and Kendall2010). Results from a treatment development study on CCAL suggested that the program needed to be “faster” paced and more interactive, but that the information was clearly presented and the program was easy to use (Khanna and Kendall, Reference Khanna and Kendall2008).
In the first randomized trial on CCAL, Khanna and Kendall (Reference Khanna and Kendall2010) found no significant difference in anxiety at post and 3-month follow-up between youth who received 12 weeks of CCAL and traditional face-to-face individual CBT (ICBT), and both treatments were superior to the control condition. Children reported significantly higher treatment satisfaction with the CCAL and ICBT than the control condition, and there was no difference in satisfaction between ICBT and CCAL. Similarly, parents reported higher mean satisfaction scores for CCAL and ICBT than the control condition, although there were no significant differences between the three conditions. Khanna and Kendall (Reference Khanna and Kendall2010) suggest that significant differences in satisfaction may be due to the children having direct involvement with treatment content and delivery method, whereas parents’ reports may reflect an overall rating of positive experience with the therapists.
Barriers and solutions to implementing evidence-based CCBT need to be considered at multiple levels (e.g. Collins, Westra, Dozois and Burns, Reference Collins, Westra, Dozois and Burns2004). Implementation science points to the importance of considering multiple perspectives when identifying barriers and designing solutions for implementing EBP (Chorpita and Regan, Reference Chorpita and Regan2009; Southam-Gerow, Rodríguez, Chorpita and Daleiden, Reference Southam-Gerow, Rodríguez, Chorpita and Daleiden2012; Torrey, Bond, McHugo and Swain, Reference Torrey, Bond, McHugo and Swain2012). Accordingly, a mental health systems ecological model that includes the system, organization, therapist, and child and family in the theoretical framework needs to be considered for maximizing adoption of EBP in community-based settings (Southam-Gerow et al., Reference Southam-Gerow, Rodríguez, Chorpita and Daleiden2012). None of the studies to date on CCBT for pediatric anxiety have included feedback from multiple stakeholders, including children, parents, therapists, project coordinators and administrators about the factors that impact implementation of CCBT in CMHC. Therefore, the purpose of this study was to explore consumers’ and providers’ perceptions of utilizing a CCBT program for childhood anxiety in CMHC to identify factors that may impact implementation of CCBT within CMHC.
Method
Design and context
This study employed a mixed-method design using qualitative focus groups and interviews and quantitative surveys. The qualitative methods were based on the phenomenological tradition that aims to provide an in-depth understanding and description of a person's specific experiences (Moustakas, Reference Moustakas1994), and the quantitative surveys were used to triangulate the data exploring consumers’ perceptions of CCBT. This study was part of an open trial where 17 children with a principal diagnosis of an anxiety disorder (i.e. separation anxiety disorder, social phobia, generalized anxiety disorder, specific phobia, or panic disorder) from three CMHC were enrolled to receive CCAL (see Crawford et al., in press). Given that phenomenological studies are designed to examine a person's in-depth experience of a specified occurrence (Moustakas, Reference Moustakas1994), and that the participant needs to have experienced the phenomenon being studied (Creswell, Reference Creswell1998), the two non-completers in the open trial were excluded from this study.
Camp-Cope-A-Lot
Camp-Cope-A-Lot was designed to be “computer-assisted”, meaning that the therapist provides monitoring and “coaching” as the child completes the program. In this study, the “coach” stayed in the room where the child was working on the computerized session in case the child had any questions, and to build therapeutic alliance. There are 12 levels (i.e. sessions) to be completed weekly: 1 to 6 focus on skill-building and 6 to 12 are exposure-based sessions where the therapist provides direct “coaching” (see Khanna and Kendall, Reference Khanna and Kendall2008).
Camp-Cope-A-Lot is interactive as the child is able to select the type of music and pace, and participate in educational games and fun games during reward time. There are several “campers”, including children and a cat named “Charlie” that go to camp to learn coping strategies for anxiety and to overcome fears through “Totem Pole Challenges” (i.e. gradual exposures). Children are given The Go-To-Gadget workbook that includes a review of the content presented in each level and weekly homework assignments. During levels 3 and 7 the therapist meets with the parent to provide psycho-education about childhood anxiety and the rationale for exposures. During these parent sessions, the child completes the CCAL level, while the parent meets in another room with the therapist (see Khanna and Kendall, Reference Khanna and Kendall2008).
Participants
Of the 15 children who completed CCAL, three families from each center were randomly selected, using SPSS 20.0, to participate in focus groups. We chose three families from each CMHC to be consistent with the number of the participants in the other focus groups (therapists, project coordinators, and administrators). Nine families agreed to participate. Two of the parents and their children who agreed to participate were not available at the time of the focus groups, and we were not able to contact them.
The consumers consisted of 7 parents (3 parents from two centers and 1 parent from one center) and 6 children (one child was at summer camp and could not participate). The children were an average of 10.67 years old (SD = 2.42; range 7 to 13), 4 male, 4 White/Non-Hispanic/Latino, 1 Asian/Hispanic/Latino, and 1 identified as White/Hispanic/Latino. All of the children completed all 12 CCAL sessions, and four children were classified as treatment responders based on a rating of “much improved” or “very much improved” on the Clinical Global Impressions Scale (Guy, Reference Guy1976).
Six mothers and one step-mother were involved in the child's treatment (100% White, with two identifying as Hispanic/Latino). Education ranged from high school degree (2), some college or technical school (2), Bachelor's degree (1), and graduate degree (2). The mothers and step-mother who participated were considered the lead parent as they completed the baseline assessment and attended the majority of the sessions. Fathers participated in 26.19% of the treatment sessions involving five children, and during five sessions (5.95%) both parents participated.
There were 3 administrators, 3 project coordinators and 3 study therapists (1 male; mean age = 40.67, SD = 16.53; 8 White and 1 Asian). The administrators had varying degrees (PhD, MD, and MSW) and had 20 to 35 years administrative experience. The project coordinators also had varying education levels (e.g., no degree, BSW, MSW). The 3 study therapists had at least 4 years post masters in counseling education (4 years, 10 months), counseling and psychology (5 year, 10 months) and social work (5 years 5 months).
Procedures
The focus groups occurred within 2 months of completing treatment which was between June 8, 2012 and July 11, 2012. The parents completed written consent and the child provided assent to participate in this study. The providers completed written consent prior to participating in the interviews. This study was approved by All Children's Hospital, St. Petersburg, Florida institutional review board.
The therapist and project coordinator interviews took place at the university, and the parents, children and administrator focus groups and individual interviews occurred via conference calls. There were six focus groups (i.e. therapists, project coordinators, administrators, two parent, one child) and due to scheduling difficulties, one parent and four children were interviewed individually. The focus groups lasted approximately one hour, and the individual interviews lasted approximately 20 minutes. The first and second author conducted all of the focus groups and interviews. There was no compensation provided for volunteering to participate in this study.
Measures
A semi-structured interview guide to elicit perceptions of utilizing CCAL was developed by the first author based on previous research (e.g. Khanna and Kendall, Reference Khanna and Kendall2008; Wuthrich et al., Reference Wuthrich, Rapee, Cunningham, Lyneham, Hudson and Schniering2012). The parents, children, and therapists who all had direct experience with CCAL were asked five questions related to what they liked, found most and least helpful, disliked, and improvement. The therapists were also asked the same questions that were posed to the project coordinators and administrations. These implementation questions about CCAL at the CMHC were about advantages and disadvantages, factors that assisted and challenged the implementation, and improvement.
Demographics of consumers and providers were collected. Consistent with Wuthrich et al. (Reference Wuthrich, Rapee, Cunningham, Lyneham, Hudson and Schniering2012), a 10-item adapted version of the Barriers to Treatment Participation Scale (BTPS) for children about treatment delivery was administered to the children by the second author after the focus groups and interviews. The 8 corresponding parent items from the BTPS (Kazdin, Holland, Crowley and Breton, Reference Kazdin, Holland, Crowley and Breton1997) were administered via an on-line survey to the parents at posttreatment. Responses to the BTPS parent and adapted child version were based on a 5-point Likert scale ranging from 1 (never a problem) to 5 (very often a problem). The Client Satisfaction Questionnaire (CSQ-8; Nguyen, Attkisson and Stegner, Reference Nguyen, Attkisson and Stegner1983) is an 8-item measure rated on a 4-point scale that assesses satisfaction with treatment. It was completed independently by children and parents at the posttreatment assessment. Scores ranged from 8 to 32, with higher scores indicating greater satisfaction with services.
Data analysis
Two undergraduate research assistants transcribed verbatim the audiotapes and two additional undergraduate students reviewed the transcripts to ensure accuracy. The two interviewers (the first and second authors) listened to all audio recordings prior to reading and coding the transcripts. Preliminary themes for the codebook were developed based on the literature (e.g. Cunningham et al., Reference Cunningham, Rapee and Lyneham2006; Khanna and Kendall, Reference Khanna and Kendall2008, Reference Khanna and Kendall2010; Spence et al., Reference Spence, Donovan, March, Gamble, Anderson and Prosser2008; Wuthrich et al., Reference Wuthrich, Rapee, Cunningham, Lyneham, Hudson and Schniering2012). One transcript was randomly chosen to be coded together to better define the codes. The remaining transcripts were coded independently by the two interviewers. Inter-rater agreement of codes was high (92.46%). Consensus of codes was established after coding each transcript, and then all transcripts were re-reviewed independently using the final codebook.
The interviewers analyzed the interconnections among themes, and compared patterns, similarities and differences across groups in order to capture the essence of consumers’ and providers’ experiences of utilizing the CCBT. Member checking and triangulation of qualitative and quantitative data were used to strengthen the rigor of the study (Padgett, Reference Padgett1998). One person from each group (parent, child, therapist, project coordinator, and administrator) was selected to review the results. Only one person was selected from each group in order to minimize participant burden (Padgett, Reference Padgett1998). The participants reported that the results were congruent with their experiences and that the information made sense. However, an administrator noted that the words “potential” or “possible” benefits of the program should be used due to the small sample size.
Results
Qualitative results
There were five child, parent, and therapist implementation themes (receptiveness, therapist factors, treatment components, applicability of treatment, and treatment content) and eight system and organization themes (initial implementation challenges, resources, dedicated staff, support, community outreach, opportunities within the CMHC, payment, and treatment availability). Patterns across stakeholders will be discussed.
Positive receptiveness
All stakeholders reported that they liked that the therapy involved the computer and that children, including older children, were engaged in the program. Noted factors related to receptiveness included the following aspects: the computerized program gave the child control over the treatment, the program was easy to use, and children seemed to benefit from the program, although not all of the children's needs were met. Administrators noted that if treatment outcomes demonstrate effectiveness then it would make it easier for them to decide to continue to implement CCBT within the CMHC.
Therapist: I think it [CCAL] helps kids to warm up to therapy a lot quicker (than talk therapy) because they’re not having to stare at you for the 50 minutes.
Parent: I think it was very helpful for the children to feel like… that they were responsible for their… their own path. They had a lot of ownership in the program. They were sitting at the computer, they were the ones doing this, it wasn't being done to them, so I think that was very powerful.
Therapist factors
The positive therapist factors included: the treatment material was readily available in the program, there was less preparation time needed prior to sessions, and overall the therapy was less time intensive than traditional therapy.
Therapist: I like that it's [CCAL] all in one spot. You can have all your materials, your games, everything like an entire office in a computer program so it makes preparation time for a session a lot less.
Other therapist factors included: standardization of CCBT allows children to receive the same type of treatment with different therapists, even at different centers. Parents reported feeling very supported by the therapists, and parents like that the therapist discussed the child's treatment progress.
Treatment components
Parents, children and therapists liked the Totem Pole Challenges (i.e. the exposures) the most and they indicated that these challenges helped the child face their fears “little-by-little.” Regarding the challenges, the difficulty level, relevance, time at home to complete them, and encouragement to complete them was discussed. Generally, the consumers liked the character Charlie, and the skill-building tools. Sometimes The Go-To Gadgets were confusing, were not challenging enough for children, and it was suggested that a Go-To-Gadget for parents could help them learn more about the program.
Child: The thing I kind of liked about it [CCAL] was that he [referring to Charlie] was kind of going through some anxiety too, so you could see it from his point of view too.
Applicability
The therapists found the program to be a “modernized type of therapy” that was designed for a relatively wide age range of children (7 to 13 years). Other perceived advantages were that the program could be applicable in different settings, such as at home or at school, and also available to a wide range of children, including children who are “gifted”.
Project Coordinator: And I get a lot of feedback from the parents about several of the kids we had were more gifted kids, and the parents were really receptive to the fact that there's no services in our area for typical gifted kids with that label, usually it's the lower functioning or kids with behavioral issues that get services, and they were like “It's great that there is a service here, for our kid”.
Notable limitations to the applicability of CCAL included that the program needed to be more appropriate for older children and that children who could not read or speak English were excluded.
Treatment content
Themes related to content included that the program was easy to use and interactive, but more interaction, better graphics, and parent psychoeducation handouts were suggested. Both therapists and parents wanted to spend more time together to discuss the child's issues and needs, and parents suggested extending the time of the sessions. Children and parents reported that they thought 12 weeks was too long, although the parents recognized that some children may need different amounts of treatment. Therapists liked having the parent as an active participant, such as helping their child carry out the at-home exposures. Therapists and parents suggested that parents needed to see the CCAL so that parents could reinforce what the child was learning.
Initial implementation challenges
Start up costs and clinic resources such as costs associated with purchasing the program, needing laptops or designated computers, designated clinic space that includes two rooms so that the child can work on the computer while the therapist meets with the parents, and technical assistance when problems arise with the computer/internet/computer program were initial challenges.
Resources
Resources mentioned as being helpful were bus passes or gift cards for gas, snacks, materials for project coordinators and therapists to help explain the program, and follow up support after treatment.
Dedicated staff, support, community outreach
Administrators indicated that for successful implementation to occur they needed to ensure that support was provided to the program. Support included weekly support from the research team to help with implementation, having a marketing and outreach plan, designated staff training time for therapists and project coordinators to learn the program and new work flow, and adjustments to productivity requirements until everyone is trained and the systems for patient flow (i.e. screening, assessments, scheduling) are in place. Administrators noted that staff turnover and leave can contribute to implementation challenges. Therapists and administrators indicated the importance of support on multiple levels to assist with implementation. Types of support included clinical supervision to help establish exposures, support of others within the clinic to help carry out exposures with the children, and “hands-on” training about implementing CCAL.
Administrator: The biggest challenge was it wasn't business as usual. We had to do something different and a lot of people had to be involved with that, from the information systems people to the purchasing people, to the time keeping people. So it required everyone in all of the areas to do something a little different in the registering of the client and the recordkeeping for the client. So I think the biggest challenge was trying to identify what needed to be done, who needed to do it, and then getting them on board to do it.
Opportunities within the CMHC, payment, and treatment availability
Administrators recognized that CCBT could be used as a future service line, provide opportunities for individual clinician development and growth, allow for development in system management of patient flow, and grant the option to participate in research,. In terms of payment, parents liked not having to pay for the treatment. Providers discussed the need to make sure CCBT would be reimbursable by third party payers and that this type of service could financially benefit the centers in the long run if it were to be implemented outside of a research study. Providers were concerned with the treatment availability, noting that for parents who were close to the centers it was a convenient option, but for those who lived further away it was more difficult. Parents liked that there was no wait time for treatment, and they wanted flexibility of therapy times.
Quantitative results
Satisfaction score was slightly higher for parents (M = 31.14, SD = 1.46) than for children (M = 27.67, SD = 3.27) on the CSQ-8. Four parents and children reported that almost all of their needs had been met, two parents and children reported most of their needs had been met, and one parent and child reported that only a few needs were met. When asked “Did you get the kind of services you wanted?” 100% of the parents and 67% of the children reported “yes, definitely”.
Results from the parent and child BTPS are consistent with qualitative findings: 100% of children and parents felt supported by the therapists and that the treatment focused on their life and problems; 50% of the children and 86% of the parents reported that they never lost interest in coming to sessions; 66% of the children reported that they never found the program boring; 66% of the children and 86% of the parents understood the tasks and treatment, but some children (33%) and parents (14%) found the information a bit confusing. Consistent with qualitative findings, 66% of the children and 43% of the parents indicated that “once in a while”, to “often” treatment may have been too long. Consistent with qualitative results, some parents (43%) and some children (50%) had a problem “once in a while” to “sometimes” finding time to complete the homework tasks. Most of the children (83%) reported that they never had technical problems with the CCAL.
Discussion
The positive receptiveness from all stakeholders about the implementation of CCBT within CMHC is quite promising. Thirteen themes emerged that addressed child, parent and therapist implementation factors (i.e. receptiveness, therapist factors, treatment components, applicability of treatment, and treatment content) and system and agency/organization implementation factors (i.e. initial implementation challenges, resources, dedicated staff, support, outreach, opportunities with the CMHC, payment, and treatment availability). Consistent with previous studies (e.g. Khanna and Kendall, Reference Khanna and Kendall2010), both parents and children reported high treatment satisfaction levels.
On the child and family level, the issue of applicability is an important implementation factor to consider (Southam-Gerow et al., Reference Southam-Gerow, Rodríguez, Chorpita and Daleiden2012). Implementation issues were related to availability of the program in multiple settings, applicability across different developmental ages, appropriateness for children with different cognitive abilities, and restriction to English only. Since this study did not compare stakeholders’ perceptions of implementation of CCBT to a standard CBT program, it is unclear whether these issues of perceived applicability by the stakeholders would apply to both types of treatment or if they viewed them as specific to the CCBT program.
There were three important findings related to therapists. First, while the time needed to learn an EBP has been suggested as a barrier to implementing EBP (Stewart, Stirman and Chambless, Reference Stewart, Stirman and Chambless2012), the therapists noted that the standardization of the treatment saved time. Time saving is an important factor for both therapists and administrators. Second, while it has been noted that community therapists are often hesitant to use exposures (Kendall, Robin, Hedtke, Suveg and Flannery-Schroeder, Reference Kendall, Robin, Hedtke, Suveg and Flannery-Schroeder2005), the therapists, parents and children all reported that they liked the exposures. However, consistent with research suggesting that there is a need for training and supervision to implement EBP (e.g. Southam-Gerow et al., Reference Southam-Gerow, Rodríguez, Chorpita and Daleiden2012), the therapists reported that the on-going supervision for exposures was one of the main factors that assisted with implementing CCAL. Third, despite concerns about standardized treatments such as CCBT limiting therapeutic alliance, 100% of the parents and children reported feeling supported by the therapist, and the parents specifically noted feeling that the therapists were working with them on the same goals for the child.
A critical factor in the implementation of EBP in CMHC is administrative support and active leadership devoted to the implementation of the new service (Torrey et al., Reference Torrey, Bond, McHugo and Swain2012). The administrators in this study recognized the importance of their support to successful implementation, the need to actively address barriers as they arose, such as making adjustments to workflow, and the importance of providing resources. Administrators are often in the difficult position of balancing the demand to provide effective services and having productive staff and efficient systems that are financially beneficial (Southham-Gerow et al., Reference Southam-Gerow, Rodríguez, Chorpita and Daleiden2012). Importantly, administrators noted many possible benefits of offering CCAL that included the potential effectiveness of treatment, and the ability to provide a new progressive service to the community while simultaneously providing opportunities within the center for professional development and administrative improvements.
Conclusions
The current study had many strengths, including the perspectives of multiple stakeholders in the community centers, child treatment responders and non-responders, independent researchers from the developers of CCAL, and methods to increase rigor (e.g. mixed-methods, triangulation of data, member checking). However, there were also limitations: these included a small sample size and limited generalizability of the findings, exclusion of some consumers, implementation of CCAL as part of a research study, and the providers continuing to work with the authors. Nonetheless, this study on consumers’ and providers’ perceptions of utilizing CCBT for childhood anxiety identifies important factors that may impact the implementation of computer-assisted treatment within CMHC.
Acknowledgements
The contributions of Tyne Pierce, Amanda Krucke, and April Lott at Directions for Living in Largo, FL, Ashley Holden, Elise Ward, Bhagirat Sahas and Pamela Galan at Henderson Behavioral Health in Ft. Lauderdale, FL, Tanya White, Shannon Massingale and John Bilbrey at Lakeview Center Inc. in Pensacola, FL, and Jonathan Lerner, Rachel Baumsteiger, Morgan King, Kara Pineiro and Caitlin Forshier at the University of South Florida, are gratefully acknowledged.
This study was funded by a grant from the Agency for Health Care Research and Quality, (1R18HS018665-01A1) awarded to Eric A. Storch. The content of this article is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.
Comments
No Comments have been published for this article.