More than 40 years of prevention research have demonstrated the efficacy of family-centered approaches for improving a range of child problem behaviors and negative health outcomes, including aggression, delinquency, depression, achievement problems, and substance abuse. Parenting skills training and support for families have consistently been shown to be the most effective interventions for improving long-term child behavioral outcomes across development, with effects lasting from middle school to the young adult years (Dishion, Forgatch, Chamberlain, & Pelham, Reference Dishion, Forgatch, Chamberlain and Pelham2016; Kazdin, Reference Kazdin, Weisz and Kazdin2010, Prinz & Dumas, Reference Prinz, Dumas, Barrett and Ollendick2004; Stormshak, DeGarmo, Chronister, & Caruthers, Reference Stormshak, DeGarmo, Chronister and Caruthers2018). The efficacy of these models has been well established; however, successful dissemination of family-centered prevention into community systems, such as schools, has been more difficult to accomplish. Much of this work was conducted by Dr. Thomas Dishion and his colleagues, and as such, this article is a tribute to his work and legacy. Dr. Dishion's focus on dissemination and implementation toward the end of his career, particularly his work on disseminating the Family Check-Up (FCU) model in schools, has inspired this research and our path forward.

Despite overwhelming evidence suggesting the efficacy of family-centered interventions, attempts to scale up systemwide family-centered, behavioral health treatments in schools have been met with a variety of challenges. One problem is that a high proportion of children and adolescents in schools never receive treatments they need, and a very small percentage of parents participate in parenting or family interventions to address behavior problems (Prinz & Sanders, Reference Prinz and Sanders2007; Zubrick et al., Reference Zubrick, Silburn, Garton, Burton, Dalby, Carlton and Lawrence1995). The result is an overwhelming level of need complicated by limited staff resources or support infrastructures in schools to manage the demand for services. For example, of students in need of behavioral support, two-thirds of their parents never have any contact, help, or guidance from public school professionals (Noel, Stark, Redford, & Zukerberg, Reference Noel, Stark, Redford and Zukerberg2015).

One approach to address the limited access to mental health treatments for children has been an increase in school-based mental health programs. In these programs, schools partner with mental health agencies to provide services to students and families or provide mental health services from within the school. While the number of school-based mental health programs continues to increase in the United States, the many barriers to implementing them effectively have not been overcome, despite more than 20 years of focused attempts (Weist, Reference Weist2005; Weist et al., Reference Weist, Youngstrom, Stephan, Lever, Fowler, Taylor and Hoagwood2014). School-based mental health is most effective when the interventions delivered are targeted and focused on children with externalizing problems (Sanchez et al., Reference Sanchez, Cornacchio, Poznanski, Golik, Chou and Comer2018), because school staff are more likely to observe externalizing behavior and apply interventions to those students. The majority of school-based interventions that target at-risk or high-risk children focus exclusively on the child or on the school context (e.g., Positive Behavioral Interventions & Supports; Horner, Sugai, Todd, & Lewis-Palmer, Reference Horner, Sugai, Todd, Lewis-Palmer, Bambara and Kern2005; Good Behavior Game; Kellam et al., Reference Kellam, Brown, Poduska, Ialongo, Wang, Toyinbo and Wilcox2008). Typically, school-based interventions use a response-to-intervention approach that frames services within a tiered (universal, selected, and indicated interventions) model delivered in the school, but they often lack functional coordination between home and school. Few of these interventions target known risk factors for mental health and substance use problems, such as family management skills, contextual risk factors, and long-term family support. As a result, these interventions are often associated with short-term gains that are context specific and lack generalizability.

Our scientific knowledge base demonstrates that for nearly all effective child and adolescent mental health and substance use interventions, working with families is a primary underpinning. Unfortunately, school-based intervention staff continue to struggle with successfully engaging and collaborating with families (Hoagwood, Reference Hoagwood2005; Jensen & Hoagwood, Reference Jensen and Hoagwood2008; Smolkowski et al., Reference Smolkowski, Seeley, Gau, Dishion, Stormshak, Moore and Garbacz2017; Weist et al., Reference Weist, Youngstrom, Stephan, Lever, Fowler, Taylor and Hoagwood2014). School-based interventionists face an almost insurmountable array of parent engagement challenges, including transportation issues; school operational hours; lack of child care options for younger siblings; school-based staff comfort with, and training for, partnering with families; and the ongoing training issues and expense caused by administrative and staff turnover and other competing educational priorities (Forman, Olin, Hoagwood, Crowe, & Saka, Reference Forman, Olin, Hoagwood, Crowe and Saka2009; Stormshak et al., Reference Stormshak, Brown, Moore, Dishion, Seeley, Smolkowski, Sheridan and Kim2016). As a result, most interventions that target parenting practices are unrealistic for schools (Christensen, Reference Christenson2003). Yet, data strongly suggest that helping parents engage in parent management through motivational strategies creates long-term change (Dishion et al., Reference Dishion, Forgatch, Chamberlain and Pelham2016; Dishion & Kavanagh, Reference Dishion and Kavanagh2003; Kazdin, Reference Kazdin, Nathan and Gorman2002; Stormshak, Fosco, & Dishion, Reference Stormshak, Fosco and Dishion2010) and that interventions for high-risk students in public school environments that target parenting are efficacious (e.g., Atkins, Hoagwood, Kutash, & Seidman, Reference Atkins, Hoagwood, Kutash and Seidman2010). Thus, it seems imperative that we find a cost-effective, efficient, and pragmatic means of improving student education and behavioral health success rates using parenting interventions, in spite of the dire economic situations and parent engagement challenges facing most school behavioral health systems today. It is also critical that any pragmatic intervention solution be designed such that it can be implemented with fidelity without requiring extensive school staff time.

The FCU is a brief, cost-effective preventive model that potentially fits these criteria. The FCU was originally developed to be delivered in community-based settings, including schools. This ecological developmental model of adaptation and risk behavior (see Figure 1) was iteratively developed in a series of randomized trials to prevent the escalation of problem behaviors in young adolescents. The FCU was designed to target early adolescent risk factors known to predict later problem behavior and substance use, such as poor parental monitoring, deviant peer affiliation, and lack of positive parenting (Dishion, McCord, & Poulin, Reference Dishion, McCord and Poulin1999; Patterson, Dishion, & Yoerger, Reference Patterson, Dishion and Yoerger2000). It was developed as a potential alternative to interventions for antisocial behavior of the late 1980s and early 1990s that focused on group-based treatments. Groundbreaking work by Dr. Dishion (Dishion & Andrews, Reference Dishion and Andrews1995) showed these interventions to be iatrogenic and harmful and to lead to increases in behavior problems rather than reductions. This work led to the development of the FCU, an individualized, tailored approach to intervention that did not involve group treatment for problem behavior.

Figure 1.

Ecological model predicting adaptation from middle to late adolescence.

The FCU evolved from a series of efficacy trials (Dishion & Kavanagh, Reference Dishion and Kavanagh2003; Dishion et al., Reference Dishion, Shaw, Connell, Gardner, Weaver and Wilson2008; Dishion & Stormshak, Reference Dishion and Stormshak2007; Stormshak et al., Reference Stormshak, Connell, Véronneau, Myers, Dishion, Kavanagh and Caruthers2011; Stormshak, Dishion, Light, & Yasui, Reference Stormshak, Dishion, Light and Yasui2005), with the goal of implementation within a multitiered school-based family engagement model (Dishion, Reference Dishion2011; Fosco et al., Reference Fosco, Seeley, Dishion, Smolkowski, Stormshak, Downey-McCarthy, Strycker, Weist, Lever, Bradshaw and Owens2014). The original FCU, developed by Dishion and Kavanagh, was applied to middle school youth in a randomized controlled trial (RCT) called Project Alliance, initiated in 1994. Since then, more than 30 peer-reviewed articles, published by multiple authors about these data, have demonstrated the impact of the FCU on both prevention of problem behavior and promotion of healthy child development. Project Alliance 2, beginning in 2005, replicated the original research and expanded the model to include a briefer assessment, culturally relevant observational tasks, and universal delivery in the middle school environment, and families were selected through an adapted, tailored design for additional support and feedback (Dishion & Stormshak, Reference Dishion and Stormshak2007; Stormshak et al., Reference Stormshak, Fosco and Dishion2010). The efficacy of the FCU has been shown in terms of both proximal and distal developmental outcomes, some more than 10 years after intervention exposure. These effects include decreases in substance use through high school and into early adulthood; reduction in antisocial behavior, including fewer arrests; decreases in deviant peer association; lower levels of adolescent depression and suicide ideation; and stabilization of attendance and academic achievement through high school (Fosco, Frank, Stormshak, & Dishion, Reference Fosco, Frank, Stormshak and Dishion2013; Stormshak et al., Reference Stormshak, Fosco and Dishion2010, Reference Stormshak, Connell, Véronneau, Myers, Dishion, Kavanagh and Caruthers2011; Van Ryzin, Stormshak, & Dishion, Reference Van Ryzin, Stormshak and Dishion2012). Similar FCU findings have been demonstrated among samples of early childhood and elementary students, such as increases in school readiness mediated though increases in inhibitory control and language development (Lunkenheimer et al., Reference Lunkenheimer, Dishion, Shaw, Connell, Gardner, Wilson and Skuban2008) and improvements in teacher ratings of problem behavior after delivery of the FCU at kindergarten entry (Garbacz, McIntyre, & Stormshak, Reference Garbacz, McIntyre and Stormshak2018). Analyses have strongly suggested that changes in the parenting constructs of positive parenting, limit setting, and monitoring, and decreased family conflict mediate the positive developmental outcomes and reductions in risk behavior over time (for a more thorough review, see Dishion et al., Reference Dishion, Forgatch, Chamberlain and Pelham2016).

To build on this foundation of efficacy trials, we conducted an effectiveness trial of the FCU in 41 middle schools where the FCU was further adapted to be embedded within a tiered family engagement model (Positive Family Support) synergistic with ongoing schoolwide positive behavior interventions and support programs (Dishion et al., Reference Dishion, Garbacz, Seeley, Stormshak, Smolkowski, Moore, Fosco and Garbacz2019; Fosco et al., Reference Fosco, Seeley, Dishion, Smolkowski, Stormshak, Downey-McCarthy, Strycker, Weist, Lever, Bradshaw and Owens2014; Smolkowski et al., Reference Smolkowski, Seeley, Gau, Dishion, Stormshak, Moore and Garbacz2017). In this effectiveness trial, we encountered many of the same economic and systemic barriers to effective parent engagement. Although we found that at-risk students responded more favorably to the intervention, with parents reporting fewer negative contacts with schools in the intervention group, our findings overall were quite limited. Perhaps even more frustrating was how difficult it was to get school staff to engage caretakers in the FCU intervention component, which likely contributed to our limited findings. In prior RCTs, engagement in the FCU was the most important predictor of improved outcomes (Connell, Dishion, Yasui, & Kavanagh, Reference Connell, Dishion, Yasui and Kavanagh2007; Stormshak, Connell, & Dishion, Reference Stormshak, Connell and Dishion2009; Van Ryzin et al., Reference Van Ryzin, Stormshak and Dishion2012). Multiple factors led to our implementation problems and limited findings, including poor school funding and staffing resources; high teaching staff and administrative staff turnover; changes in staffing for support specialists, including counselors and behavioral support specialists who were delivering the intervention; staff time allocation; and the challenge of motivating school counselors and mental health staff to adapt their work to focus on family and parent systems rather than exclusively on the child (Dishion et al., Reference Dishion, Garbacz, Seeley, Stormshak, Smolkowski, Moore, Fosco and Garbacz2019; Stormshak et al., Reference Stormshak, Brown, Moore, Dishion, Seeley, Smolkowski, Sheridan and Kim2016).

The intensity, frequency, and duration of the challenges we faced engaging parents in the FCU within a school setting suggested that an electronic health (eHealth) intervention developed around the FCU and delivered via the Internet might help solve many of these implementation challenges. Advantages of an eHealth approach include ability to develop an FCU intervention that could be stand-alone (fully automated), thereby reducing delivery and training costs and, concurrently, increasing public health reach. The eHealth version may also include live contact with counseling or coaching staff, either via face-to-face sessions or via phone or video calls. The coach option, when feasible, is hypothesized to help increase adherence through a supportive accountability model (Mohr, Cuijpers, & Lehman, Reference Mohr, Cuijpers and Lehman2011) and has been associated with enhanced outcomes for eHealth interventions when compared with those delivered without a coach or support (Day & Sanders, Reference Day and Sanders2018). A coach model for the FCU would also enable titration of support and intensity of delivery, consistent with the adaptive, tailored model of FCU intervention delivery.

Increases in multimedia technology and Internet usage make it easier for mental health interventions to reach much of the population. In a report by the Pew Research Center in 2018, roughly 90% of adults currently use the Internet and 95% use a mobile phone (77% smartphones). Recent advances in web-based mental health and substance use intervention technology (Danaher, McKay, & Seeley, Reference Danaher, McKay and Seeley2005, Danaher et al., Reference Danaher, Milgrom, Seeley, Stuart, Schembri, Tyler and Lewinsohn2013; Danaher & Seeley, Reference Danaher and Seeley2009) suggest that schools and their families could benefit from a brief, online family management intervention adapted and targeted to their needs.

Improvements and increases in technology and broadband Internet enable interactive Internet access and have been linked with higher levels of engagement in interventions with families (Jones, Reference Jones2014). Several evidence-based parenting curricula have been adapted to web-based programs that are self-administered or assisted by trained human support (Cardamone-Breen et al., Reference Cardamone-Breen, Jorm, Lawrence, Rapee, Mackinnon and Yap2018; Enebrink, Högström, Forster, & Ghaderi, Reference Enebrink, Högström, Forster and Ghaderi2012; Gelatt, Adler-Baeder, & Seeley, Reference Gelatt, Adler-Baeder and Seeley2010; Kling, Forster, Sundell, & Melin, Reference Kling, Forster, Sundell and Melin2010; Sanders, Baker, & Turner, Reference Sanders, Baker and Turner2012; Sanders, Kirby, Tellegen, & Day, Reference Sanders, Kirby, Tellegen and Day2014; Sourander et al., Reference Sourander, McGrath, Ristkari, Cunningham, Huttunen, Lingley-Pottie and Fossum2016; Yap et al., Reference Yap, Mahtani, Rapee, Nicolas, Lawrence, MacKinnon and Jorm2018). A number of web-based parenting interventions have been effective in reducing risk outcomes in children. For instance, the Triple-P Online Brief (TPOL-Brief; Turner & Sanders, Reference Turner and Sanders2013) is a brief web-based self-directed application of the Triple-P (Positive Parenting Program), a multilevel system of both universal and targeted interventions for parents of children with early onset conduct problems (Baker, Sanders, Turner, & Morawksa, Reference Baker, Sanders, Turner and Morawska2017; Sanders et al., Reference Sanders, Baker and Turner2012). Results from an RCT evaluating the efficacy of the TPOL-Brief showed decreases in the use of ineffective parenting strategies and higher levels of parent confidence in self-efficacy, with outcomes sustained at 8-week and 9-month follow-up (Baker et al., Reference Baker, Sanders, Turner and Morawska2017). Results from other RCTs that examined the efficacy of the TPOL-Brief have demonstrated significant improvements in parenting and child behavior outcomes, including lower levels of dysfunctional parenting, lower levels of disruptive child behavioral and emotional difficulties, less reported parental anger and use of negative parenting styles, reductions in partner conflict, and lower levels of maternal adjustment difficulties (Sanders et al., Reference Sanders, Baker and Turner2012; Sanders, Kirby, et al., Reference Sanders, Kirby, Tellegen and Day2014). In a meta-analysis of digital-based parent training for parents of children with disruptive behaviors, researchers found positive improvements in outcomes of child behavior, parent behavior, and parental confidence in self-efficacy (Baumel, Pawar, Kane, & Correll, Reference Baumel, Pawar, Kane and Correll2016). An RCT by Cardamone-Breen et al. (Reference Cardamone-Breen, Jorm, Lawrence, Rapee, Mackinnon and Yap2018) implemented a single-session, individually tailored, web-based parenting intervention that targeted parenting factors known to affect the development of depression and anxiety in adolescents. The intervention resulted in significant improvements in parenting factors but did not support improvement in adolescent depression or anxiety symptoms. In addition, results from a study that examined the effects of Partner in Parenting, a tailored web-based parenting intervention designed to prevent depression and anxiety symptoms in adolescents, indicated positive short-term improvements in parenting behaviors associated with adolescents’ risk for depression and anxiety (Yap et al., Reference Yap, Mahtani, Rapee, Nicolas, Lawrence, MacKinnon and Jorm2018).

Adherence to and effectiveness of online behavioral health treatments can be enhanced by the inclusion of guided human support (e.g., technician-level coach or skilled therapist; Mohr et al., Reference Mohr, Cuijpers and Lehman2011). For instance, an online application of the COMET parent management training program, a web-based program with e-mail support and an online discussion forum, demonstrated lower levels of child behavior problems, with outcomes maintained at 6-month and 18-month follow-up assessments (Enebrink et al., Reference Enebrink, Högström, Forster and Ghaderi2012; Högström, Enebrink, Melin, & Ghaderi, Reference Högström, Enebrink, Melin and Ghaderi2015). In addition, in a study comparing Triple-P Online with and without telephone support, results suggested that although self-directed online approaches to parenting interventions are effective, telephone support enhances effective engagement and outcomes for families (Day & Sanders, Reference Day and Sanders2018). However, in another evaluation study of the Triple-P Online, results demonstrated no significant differences between the types of delivery format (Sanders, Dittman, Farruggia, & Keown, Reference Sanders, Dittman, Farruggia and Keown2014). Clearly, there is support for delivery of parenting skills interventions via the Internet, and a variety of programs have adapted content for online delivery with some success. No programs, however, have been adapted for school-based delivery with the goal of supporting schools in their intervention delivery to families of at-risk youth.

In this study, we tested the efficacy of FCU Online, a web-based, eHealth implementation of the FCU intervention, and compared delivery of the model as a stand-alone, web-based delivery versus a web-based, coach-supported delivery model. Building on the adolescent version of the FCU originally tested in Project Alliance, we randomly assigned 322 families of middle school youth to receive the intervention either delivered with a coach or as a stand-alone web program, or to a waitlist control. We predicted that delivery with the coach would be associated with improved outcomes when compared with web-only or waitlist control, including reductions in child behavior problems, emotional problems, and parent self-efficacy. In prior research using the FCU, we found that high-risk families benefit more than do low-risk families across a number of outcomes, including parenting skills (Dishion et al., Reference Dishion, Shaw, Connell, Gardner, Weaver and Wilson2008; Leijten et al., Reference Leijten, Shaw, Gardner, Wilson, Matthys and Dishion2015; Stormshak et al., Reference Stormshak, Connell and Dishion2009). We have found few moderating effects of gender and race in prior studies, although a preventative effect for depression was found for females who started with a high baseline of antisocial behavior and low levels of depression in middle school, suggesting the FCU prevented the growth of depression over time into the high school years (Connell, Stormshak, Dishion, Fosco, & Van Ryzin, Reference Connell, Stormshak, Dishion, Fosco and Van Ryzin2018). On the basis of this prior research, we predicted that at-risk youth and their parents would show greater improvements than would those who were not at risk (Smolkowski et al., Reference Smolkowski, Seeley, Gau, Dishion, Stormshak, Moore and Garbacz2017). We also examined the potential moderating effects of child gender and race/ethnicity, and family socioeconomic status, but we had no predictions regarding their impact on treatment outcomes.

Method

Participants

Participants were primary caregivers of middle school children across eight schools in Oregon, including two rural, two urban, and four suburban schools ranging in size from 294 to 839 students. Schools were chosen for participation in the project based on the percentage of students who qualified for free or reduced-cost lunch; this population ranged from 52% to 100%, with an average of 68%. During two consecutive school years, all families with a sixth- or seventh-grade student enrolled in one of the participating schools were contacted and invited to participate via an e-mail message sent to families from the principal. From this population, 432 families initially responded. The CONSORT diagram (Moher et al., Reference Moher, Hopewell, Schulz, Montori, Gøtzsche, Devereaux and Altman2010) presented in Figure 2 displays participant flow and allocation to condition. Some of the initial respondents were excluded from the study because they did not return the initial survey (n = 66), declined to participate (n = 8), or did not meet inclusion criteria (n = 11; i.e., not in sixth or seventh grade or not a student at a participating school). Siblings, those who were primarily Spanish speakers, and families who had participated in other FCU studies (n = 25) were assigned to a nonexperimental condition during which they received support but were not included in the analytic sample. Families who consented and returned the baseline assessment (n = 322) were randomly assigned to one of three conditions, including FCU Online with coach (web+coach; n = 108), FCU Online delivered as a web-only intervention (n = 109), and a waitlist control group (n = 105). Our goal was to recruit 300 families for the study, and once we had reached our goal, we discontinued recruitment in the schools.

Figure 2.

Participant flow and allocation to intervention condition.

Table 1 reports demographic characteristics of participants across the three conditions. The parents in the study were predominately female and were average age 40.0 years (SD = 6.1); 35% had a 4-year college degree or graduate training. The children in the study were approximately gender balanced, mostly White, and most were living in suburban environments.

Table 1.

Study demographic characteristics

Procedure

Families were invited to participate in the study via an e-mail sent to them from their school. This e-mail explained the study and included a link to a Qualtrics form where interested parents could provide their contact information. A research staff member then contacted all interested parents to explain the goals of the study and provide details about participation. Parents who indicated interest in participating in the study were then mailed a packet that included a parent consent form, a youth assent form, and the parent and youth surveys. Once families returned their surveys and consent/assent forms, they were randomly assigned to one of three experimental conditions. Parents were then sent an e-mail with information about their next steps, relative to their experimental condition.

Condition

Parents in the two treatment conditions received login information and support to begin the online program. Waitlist control participants were told that they would receive the intervention after the assessment period was completed, and that they would next hear from project staff in 3 months. For families assigned to a treatment condition, one parent per family was chosen as the primary target for the study, although multiple caregivers could participate in the intervention if desired. Participants in the web-only condition were invited to log in to the FCU Online website, where they were asked to complete an intervention assessment, received feedback about their data, and were provided with online tools to support their parenting in areas that were identified as challenges by the assessment. Participants in the web+coach condition logged in to the same FCU Online website, where they had access to the same assessment, materials, and tools. In addition, a family coach contacted parents to help them set goals, talk through their results, offer support via the phone, and motivate them to improve parenting practices. Families in all three conditions were mailed posttest questionnaires 3 months after they completed the pretest, regardless of their condition. Parents were paid $100 for each assessment wave.

Training of therapists

Family coaches in this study were doctoral-level or master's-level psychologists who had been previously trained in the FCU through a variety of means, such as attending a training workshop and working on prior projects that used this model. Specific training content included training on the protocol, developmental norms, motivational interviewing techniques, and specific support for parents of middle school youth (Stormshak & Dishion, Reference Stormshak and Dishion2009). After training, therapists were observed and scored on the COACH rating scale for fidelity over five coached sessions with a parent (Smith, Dishion, Shaw, & Wilson, Reference Smith, Dishion, Shaw and Wilson2013). All therapists’ COACH ratings were within the satisfactory range (ranging from 4.8 to 6.8; minimum score of 4 out of 9). Once therapists were providing the FCU Online independently, weekly group supervision meetings that emphasized case conceptualization and delivery of feedback were held to maximize treatment fidelity. Supervision was provided by licensed, clinical psychologists with experience delivering the FCU.

Family Check-Up (FCU)

The original FCU model involves three steps and a menu of intervention services that are adapted and tailored to families’ needs. It follows a tiered service delivery model (Dishion & Kavanagh, Reference Dishion and Kavanagh2003; Dishion & Stormshak, Reference Dishion and Stormshak2007). Consistent with an response-to-intervention approach to service delivery (Fuchs & Fuchs, Reference Fuchs and Fuchs2006), targeted interventions are modified to fit the families’ needs based on assessment and risk status.

Key features

The FCU's key features make it an ideal intervention for online delivery: (a) interventions are tailored and adapted to family needs on the basis of assessments and collaborative agreements with family caregivers, (b) parents’ motivation to change is explicitly targeted in the feedback session, and (c) a health maintenance model of intervention is followed, with interventions repeated as needed. These features make the FCU cost effective and viable for the prevention of long-term patterns of problem behavior and substance use.

The FCU is a selected intervention in that it engenders training in skilled family management to encourage positive family interactions and improve family management skills. For some families, the FCU will be the primary intervention and will motivate caretakers to engage in systematic changes that will reinforce their youth's well-being. For other families who are at higher risk, follow-up intervention services and support will be indicated. In this study, four types of follow-up intervention services (i.e., positive behavior support, limit setting, monitoring, and effective problem solving and communication) were available to caregivers who were randomly assigned to the FCU Online, with services tailored and adapted to the child's and family's specific needs. The intervention actively promoted self-selection into the most appropriate services.

FCU intervention protocol

The FCU Online version motivates caregivers to promote and use positive parenting skills and effective communication with their adolescent to reduce family conflict and subsequent problem behavior. The original FCU involves at least three sessions, including (a) an initial rapport-building interview, (b) an assessment with the caregiver(s), and (c) a feedback session during which the results of the assessment and initial interview are discussed with the caregivers, attention is focused on the caregivers’ readiness to change, and specific change options are delineated.

The FCU is a motivational intervention designed to elicit improved parenting and enhance contextual supports for children. Motivational interviewing (MI), first developed as a treatment for substance abuse (Miller & Rollnick, Reference Miller and Rollnick2002), elicits change by activating motivation from within a collaborative relationship. MI has been successfully applied to many school mental health programs and is natural to the school setting in that the interventions associated with MI are brief and targeted (Frey et al., Reference Frey, Cloud, Lee, Small, Seeley, Feil and Golly2011). MI has also been successfully adapted to a variety of Internet interventions (Friederichs, Bolm, Oenema, Guyaux, & Lechner, Reference Friederichs, Bolm, Oenema, Guyaux and Lechner2014; Ingersoll et al., Reference Ingersoll, Banton, Gorlin, Vajda, Singh, Peterson and Cox2015; Mohr et al., Reference Mohr, Duffecy, Ho, Kwasny, Cai, Burns and Begale2013).

Basic process of FCU Online

To begin using the FCU Online program, parents in both treatment conditions first logged in to the website on their computer or mobile device using their family identification number. This step allowed families to create a page that was exclusively viewed by the parent or parents who were participating. Parents first identified strengths and challenges to parenting their middle school child as part of the initial assessment. Then they completed the FCU assessment, which included content similar to prior protocols but abbreviated for online delivery. Measures targeted key domains relevant to middle school youth, such as achievement at school, problem behavior, emotional problems, parenting skills, family conflict, and motivation to change. Parents created goals and learned about the FCU Online process. After the assessment was completed, a computer-generated feedback form was available to the family. In the web+coach condition, completion of the assessment prompted the program to notify the coach to call the family to schedule a time to review the feedback and results. In the web-only condition, feedback was delivered in an interactive format, with parents able to explore areas of strengths and challenges and learn more about their own family and assessment results.

The family feedback session occurred either on the phone or via video conferencing (e.g., Skype) for the web+coach version, or automated online for the web-only version. The feedback session was both strengths based and specific. The goals of feedback sessions were to (a) share assessment findings with caregivers regarding areas of strengths and challenges, (b) engage in a motivationally enhanced discussion about promoting positive changes, and (c) provide a menu of resources to facilitate the family change process. Emphasizing areas of strengths and challenges reduces parents’ defensiveness and enables them to feel empowered to pursue strategies for making changes relevant to their family's needs.

After the feedback, a menu of options tailored for the family and based on the assessment was presented. These options included parenting modules guided by our Everyday Parenting curriculum (Dishion, Stormshak, & Kavanagh, Reference Dishion, Stormshak and Kavanagh2011). The choice to continue with the curriculum and pick options based on the assessment or parent interest is part of the model of tailoring and adapting the intervention to the family and integrates the principles of MI by allowing for self-directed change. The parenting modules consist of content and structured activities that are designed to help parents learn each skill. Parents were also given the opportunity to practice parenting skills and track their progress. Parents who chose to engage in this practice could sign up to receive text messages that periodically prompted them to try out new skills learned from the website. A coach checked in with families in the web+coach condition to assess progress and support gains in positive behavior. Parents could log in to the website as often as they would like, to learn more or participate in any of the other modules. Families in the coach condition could choose to participate in at least one therapy/coaching contact; 73% chose to do so, and 72% chose to review the FCU with their coach during the coaching sessions. Of those who received any coaching contact, the average number of contacts was 1.8 (SD = 1.2).

Content of follow-up sessions and web activities

On the basis of assessment results, parents were offered a variety of follow-up parenting skill activities. Interventions delivered online must be brief and effective in order to be viable, and they must capture that attention of participants quickly in order to increase engagement (Kelders, Bohlmeijer, & Van Gemert-Pijnen, Reference Kelders, Bohlmeijer and Van Gemert-Pijnen2013). The tailored and adaptive nature of the FCU intervention gives it an online delivery potential that is brief, meaningful, and interactive. Each of the following online modules derived from our Everyday Parenting curriculum (Dishion et al., Reference Dishion, Stormshak and Kavanagh2011) included content that is animated, video based, and interactive, with activities for tracking behavior and providing ongoing feedback to parents about their success. Each module also highlighted behaviors that target school success at this age, including homework completion, attendance, and home-to-school planning.

Positive behavior support module

Three teaching content areas focus on positive behavior support: making effective requests, praise, and incentives for behavior. The “making requests” area helps parents understand and learn basic skills for promoting behavior change in their child. The “praise” content area shows parents how to develop skills for using positive language with their adolescent. The “incentive” content area helps parents develop skills for using incentives and planning how to manage their child's behavior.

Limit-setting module

This module includes tracking behavior, providing structure and clear instructions, and identifying appropriate consequences. In the “tracking behavior” area, parents are asked to track both desired and undesired behaviors over time, such as homework completion. The “providing structure” area shows parents how to establish clear commands and expectations, and “identifying appropriate consequences” includes working with the adolescent to develop incentives and consequences that are meaningful for reducing negative behavior and increasing positive behavior.

Monitoring

This content area provides guidance in listening skills and clear expectations while tracking the child's activities with friends and at home. Monitoring includes supervising cell phone usage, Internet activities, and peer interactions at home, at school, in the neighborhood, and online. A number of activities to help parents develop these skills are available.

Problem solving and communication

This content area teaches effective problem solving and negotiation skills. The online module presents relevant skills, such as accepting responsibility, recognizing the youth's positive contributions, and building on strengths. Activities and interactive content are delivered to improve problem solving and communication skills, both at home and with school personnel, including activities online and via mobile texting. Additional details regarding the formative development of the FCU Online program are reported in Danaher et al. (Reference Danaher, Seeley, Stormshak, Tyler, Caruthers, Moore and Cardenas2018).

Measures

Demographic characteristics

Family background variables were obtained from primary caregivers through the use of standard demographic questions, including queries about parent gender, age, and education level, as well as child gender and race/ethnicity. A family's geographic location was determined by the school the child attended. Schools were categorized as urban, suburban, or rural, relative to their communities.

Program use

Participants’ usage of the online program was assessed unobtrusively by the website to allow us to observe how often and for how long each participant interacted with the online program overall, as well as with each module of the program (e.g., the “monitoring” module) and with various engagement activities (e.g., videos and animations or practice activities). For the present analysis, we investigated number of unique logins, total time spent in the program, time spent on assessment and in the library of available parenting support materials, and time spent on each of the four modules (positive behavior support, limit setting, monitoring, and problem solving and communication).

Outcomes

Effortful control

Parents reported on their teen's effortful attention control via questionnaire using an eight-item subscale of the Early Adolescent Temperament Questionnaire (Ellis & Rothbart, Reference Ellis and Rothbart2005). In this questionnaire, parents are asked to use a 5-point scale with endpoints of almost always untrue and almost always true to indicate how well each of eight statements describes their child, such as “has a hard time finishing things on time” and “pays close attention when someone tells her/him how to do something” (effortful control M = 3.33, SD = 0.92; α = .91).

Problem behavior

Parents reported on their teen's problem behavior using the Strengths and Difficulties Questionnaire (SDQ; Goodman, Reference Goodman1997), a well-validated measure for assessing problem behavior at this age. The SDQ is a 25-item brief behavioral screening questionnaire that consists of five subscales with five questions each. For the purposes of the present analysis, we focused on three subscales: emotional problems, conduct problems, and hyperactivity. In this questionnaire, parents were asked to indicate the extent to which statements were true of their child's behavior in the past month using a 3-point scale (not true, somewhat true, and certainly true). The emotional problems subscale included items such as “many worries or often seems worried” and “often unhappy, depressed, or tearful” (M = 2.51, SD = 2.24; α = .73). The conduct problems subscale included items such as “often loses temper” and “often fights with other youth or bullies them” (M = 1.29, SD = 1.67; α = .71). The hyperactivity subscale includes items such as “restless, overactive, cannot sit still for long” and “easily distracted, concentration wanders” (M = 3.71, SD = 2.69; α = .80).

Parenting skills and self-efficacy

Using a measure developed for the purposes of this study, parents were provided with a list of 14 parenting skills that closely corresponded to skills that were supported in the FCU Online parenting curriculum, such as “providing praise and encouragement for good behavior” and “setting clear rules and expectations for behavior.” For each parenting skill, parents were asked to indicate the importance of this skill during the middle school years using a 5-point scale with end points of not at all important and very important (sense of parenting importance M = 4.52, SD = 0.29; α = .72). They were also asked to indicate their level of confidence in using each skill with their child via a 5-point scale with endpoints of not at all confident and very confident (sense of parenting confidence M = 4.14, SD = 0.57; α = .87).

Parent self-efficacy was also assessed with eight items adapted from the Parenting Task Checklist (Sanders & Woolley, Reference Sanders and Woolley2005). Using a 5-point scale with endpoints of not confident at all and very confident, parents were asked to indicate their level of confidence in handling eight negative child behaviors, such as “your child arguing with you about rules” or “your child's mood swings” (confidence in teen behaviors M = 2.76, SD = 0.79; α = .92).

Moderators

In addition to child's gender and racial/ethnic minority status, moderators included family socioeconomic status (SES) risk and child's pretest behavior problems. Family SES risk was assessed as a cumulative risk index (range 0−6, M = 1.03, SD = 1.23) measuring multiple risk factors. The index is based on six dichotomous demographic indicators: the parent does not have a high school diploma or GED, the parent is single, the parent's income is below the poverty threshold (relative to family size), the parent is a recipient of financial assistance (e.g., food stamps), the parent is currently unemployed, or the family's home is overcrowded (in the bottom one-third of the sample for room-to-occupant ratio).

Child's pretest behavior problems is a baseline risk behavior measure based on the SDQ total difficulties score, which includes the hyperactivity, emotional problems, and conduct problems subscales, as well as the peer problems subscale. The peer problems subscale is similarly scored to the other subscales and is based on five items, such as “has at least one good friend” and “generally liked by other children.” Scores for each of the four subscales were categorized according to normal, borderline, and abnormal score thresholds, per Goodman (Reference Goodman1997). The pretest behavior risk measure thus identifies the highest threshold (0 to 2) of risk a child scored on any of the four subscales.

Data analysis

Preliminary analyses included a descriptive summary of participant characteristics and use of the website. Descriptive statistics and plots were then used to screen outcomes for normality. Next, chi-square tests and one-way analyses of variance were used to compare conditions on demographic characteristics and pretest values of the outcomes to assess whether randomization produced initially equivalent groups. A multivariate logistic regression model was used to compare study condition, demographic characteristics, and pretest values of the outcomes between participants who dropped out of the study at posttest versus those who completed the posttest assessment.

Intent-to-treat analyses of condition effects were evaluated with fixed-effects growth models using SAS 9.2 PROC MIXED and estimated using full-information maximum likelihood. Variability in outcomes from pretest to posttest was modeled as a function of condition, time (coded in months since pretest), and a Condition × Time interaction term. The Condition × Time interaction term is a test of the efficacy of the intervention. Three a priori dummy-coded comparisons were examined: waitlist control (reference group) versus web-only, waitlist control (reference group) versus web+coach, and web-only (reference group) versus web+coach. Effect sizes for the Condition × Time interaction are based on condition differences in change in the outcomes over time and are equivalent to Cohen's d (Feingold, Reference Feingold2009).

Child's gender, minority status, family SES risk, and pretest behavior problems were examined as potential moderators of intervention effects. Moderation models included main effects of the condition, time, moderator, and all two- and three-way higher order interaction terms. SES risk and behavior problems were mean centered. The critical p value for tests of the Condition × Time effect and tests of moderation was .017, the traditional .05 p value divided by three a priori comparisons, to adjust for the increased likelihood of Type I errors when making multiple comparisons.

Results

Preliminary analyses

A summary of program use metrics is provided in Table 2. Overall, use of the site was robust (an average of 72% of users completed the FCU assessment), but the web+coach group used the program significantly more than did the web-only group. Compared with the web-only group, the web+coach group had approximately twice as many unique logins on average (4.4 vs. 2.1) and spent 47% more minutes on the site (65.0 min vs. 44.3 min). The web+coach group completed significantly more online assessments than did the web-only group (82% vs. 62%).

Table 2.

Program use summary

Note: Time in minutes using the site is reported for only those parents who completed the Family Check-Up online assessment (67 web-only parents and 89 web+coach parents). Group differences between web-only and web+coach for unique logins, completion of online assessments, site navigation, open communication, and total time are statistically significant at p < .05.

The outcomes approximated normal distributions except conduct problems and prosocial behaviors, which were normalized with a logarithmic transformation. Participants in the three conditions did not differ significantly at the p < .05 level on gender, area (rural, suburban, or urban), minority status, SES risk score, or pretest measures of the outcomes, indicating that randomization produced initially equivalent groups. Attrition was 7% at posttest, and results of the multivariate logistic regression model showed that failure to provide posttest data was significantly associated only with pretest measures of hyperactivity (odds ratio = 1.44, p = .005). Parents who reported higher pretest values of hyperactivity for their child were more likely to drop out of the study.

Intent-to-treat analyses

Table 3 provides means and standard deviations for the outcomes at each assessment point for each of the three conditions. Table 4 provides the correlations of the pretest measures (correlations range from r = −.67 to r = .60, mean | r | = .30). Table 5 provides a summary of the Condition × Time effects from the growth models for the three a priori comparisons. A comparison of the waitlist control and web+coach groups showed a significant differential change in emotional problems from pretest to posttest. Relative to the waitlist control group, the web+coach group showed significantly greater decreases in parent report of their child's emotional problems (p = .003, d = −0.32), a small to medium effect (Cohen, Reference Cohen1988).

Table 3.

Descriptive statistics for study outcomes

Note: M, mean. SD, standard deviation.

Table 4.

Pearson correlations among outcomes at pretest

Note: Correlations greater than |.11| are statistically significant at p < .05.

Table 5.

Condition × Time parameter from growth models

Note: Bolded p value and d statistic are significant at p < .017.

No other statistically significant (at p < .017) Condition × Time effects were found, although some trend-level effects (p < .10) are worth noting. Most notable is that emotional problems showed trend-level differences that favored the web-only group relative to the waitlist control group (p = .092, d = −0.17). The impact of the program with regard to decreasing youth emotional problems is thus a consistent finding for both active intervention groups relative to the waitlist-only group. The inclusion of a coach in the online training was associated with greater increases in effortful control (p = .046, d = 0.16) and parents’ confidence in dealing with teen behaviors (p = .057, d = 0.21), relative to waitlist controls. However, these findings were not consistent for the web-only group. The web+coach group further showed greater increases in parents’ confidence in dealing with teen behaviors, relative to the web-only group (p = .018, d = 0.25).

Moderation effects

Parent report of child behavior problems moderated the Condition × Time effect for effortful control (estimate = 0.03, SE = 0.01, t = 2.47, p = .015) and sense of parenting confidence (estimate = 0.03, SE = 0.01, t = 2.42, p = .017) for the web-only versus waitlist control comparison. The Condition × Time effects for effortful control and sense of parenting confidence were examined separately for youth identified as at risk because they had received a borderline or abnormal pretest behavior problem score (n = 60, 28%) and youth not at risk who had scored in the normal range for the four subscales (n = 154, 72%). The study was not powered to significantly detect differences between youth at risk and not at risk, but the effect sizes associated with the Condition × Time effect for each group elucidated the potential moderating effects. The Condition × Time effect size for effortful control for the at-risk group (estimate = 0.22, SE = 0.18, t = 1.27, p = .208, d = 0.26) was approximately 6-fold the effect for the not-at-risk group (estimate = −0.03, SE = 0.09, t = −0.40, p = .693, d = 0.04). Similarly, the Condition × Time effect size for sense of parenting confidence (estimate = 0.32, SE = 0.17, t = 1.87, p = .068, d = 0.57) was approximately 10-fold for the not-at-risk group (estimate = 0.03, SE = 0.08, t = 0.42, p = .677, d = 0.06). These results are consistent with other published effects of the FCU and suggest that at-risk families benefit more from the intervention than do those not at risk.