Schools frequently implement intervention programs that have been developed to support children's learning and development across areas such as literacy, social skills, bullying, and emotional wellbeing. However, not all of the programs used in schools are evidence based. This means that their efficacy has not been established in properly conducted trials. Furthermore, interventions that were originally designed for and tested with typically developing children may need to be adapted for use with students with developmental disabilities. In this article, we begin by discussing the importance of using evidence-based interventions and consider the value of modifying existing programs to meet the needs of students who are developing atypically. We then describe the ways in which two well-established resilience interventions have been adapted for students with developmental disabilities. We conclude by providing some guidelines for program modification and evaluation.
Evidence-Based Interventions in Schools
When selecting interventions to be used in the school setting, or indeed in any setting, it is imperative to consider the evidence for program effectiveness. Numerous interventions are now available for both preventative and therapeutic purposes. At the same time, however, we are increasingly being bombarded with new interventions that are marketed with persuasive, sometimes wildly extravagant, claims. Such claims need to be critically evaluated, yet it seems that many professionals lack the necessary skills to judge the rigour of evidence for program efficacy (Tardif, Doudin, & Meylan, Reference Tardif, Doudin and Meylan2015). As a result, schools may be investing time and money in programs that are unlikely to be successful. Of even greater concern, there may be unintended negative consequences of using interventions that have been inadequately tested.
The Society for Prevention Research has established a set of standards against which programs should be evaluated prior to dissemination (Flay et al., Reference Flay, Biglan, Boruch, Castro, Gottfredson and Ji2005). The minimum requirements are for interventions to be tested in at least two well-designed trials, using psychometrically sound pre- and post-measures, with follow-up to determine whether intervention effects are sustained for at least six months. The inclusion of a control group is essential for demonstrating program efficacy, with participants being assigned randomly to intervention or control conditions. Well-designed studies involve adequate sample sizes, manualised procedures that are monitored for fidelity, and robust statistical analysis. By contrast, interventions that rely purely on testimonials or other forms of anecdotal evidence to support their effectiveness, ones that allude only vaguely to evidence without providing specific citations of peer-reviewed literature, or those that make bold claims about being able to ‘cure’ a range of developmental disabilities should be regarded with suspicion.
Many intervention programs that schools use regularly do have an adequate evidence base. For example, FRIENDS for Life is a structured preventative program for anxiety in children and adolescents that has been extensively evaluated (Barrett & Phal, 2006). In a systematic review of seven randomised controlled trials, Higgins and Sullivan (Reference Higgins and O'Sullivan2015) reported reductions in self-reported anxiety for groups that had received the program. Despite effect sizes being only small to medium, the intervention effects were maintained for periods of 4 to 36 months. Although originally designed to be used one-on-one in clinical settings, FRIENDS for Life has been shown to be effective in schools when delivered to groups by an appropriately trained professional such as a psychologist or counsellor. Another evidence-based program widely used in schools is MultiLit, a reading intervention that has been evaluated in a series of studies by the program's developers (see, e.g., Buckingham, Beaman, & Wheldall, Reference Buckingham, Beaman and Wheldall2012; Wheldall & Wheldall, Reference Wheldall and Wheldall2014). Significant improvements in aspects of literacy such as decoding, fluency, comprehension, and confidence have been demonstrated.
At times, however, schools are investing in interventions that do not have an adequate evidence base. Brain Gym is an example of a program that has been used despite lack of evidence for its efficacy. The developers of Brain Gym attribute learning problems to inefficient integration of visual, auditory, and motor skills, and argue that a range of exercises will boost performance in areas such as reading, memory, organisation, physical coordination, and social relationships, claims that have been strongly criticised by neuroscientists (Teaching and Learning Research Program, 2007). Evidence for the program's efficacy is largely anecdotal. In a review, Hyatt (Reference Hyatt2007) found only five peer-reviewed studies of Brain Gym. One of these was excluded from the review because the author was also a study participant, and the others were found to be seriously flawed. In a careful evaluation of Brain Gym against accepted standards for establishing intervention efficacy, Spaulding, Mostert, and Beam (Reference Spaulding, Mostert and Beam2010) concluded that the program's claims were not supported by the empirical evidence, and a recent trial by Watson and Kelso (Reference Watson and Kelso2014) found no evidence to support the intervention's effectiveness for students with developmental disabilities. Walker (Reference Walker2001) commented that ‘educators are notorious for embracing programs that look good but do no actual good’ (p. 2). Despite widespread condemnation within the scientific community, it seems that many education systems have supported, and may indeed still be supporting, the use of Brain Gym in Australian schools (Stephenson, Reference Stephenson2009).
Interventions for Students With Developmental Disabilities
Interventions often involve a ‘one size fits all’ approach, based on the presumption that mainstream classroom activities can provide for all students without any need to make adjustments to cater for those with special needs (Torr, Reference Torr2013). The majority of programs are developed and trialled with students who are developing typically, and the benefits for children with developmental disabilities who participate in evidence-based interventions in regular classrooms are seldom evaluated separately from the rest of the class (Petrenko, Reference Petrenko2013). While it is possible that children with developmental disabilities benefit in similar ways to their typically developing peers, this cannot be assumed. For example, a trial of the violence prevention program Second Step showed differential effects for students with and without disabilities (Sullivan, Sutherland, Farrell, & Taylor, Reference Sullivan, Sutherland, Farrell and Taylor2015).
There is growing recognition of the need to adapt established approaches such as cognitive behaviour therapy (Rossiter & Holmes, Reference Rossiter and Holmes2013) and psychotherapy (Campbell, Ruble, & Hammond, Reference Campbell, Ruble, Hammond, Prout and Fedewa2015) to make them more suitable for individuals with special needs. Modifications of specific programs for students with disability have also been reported in the literature. For instance, Hubbard, Bandini, Folta, Wansink, and Must (Reference Hubbard, Bandini, Folta, Wansink and Must2014) systematically adapted an established school-based nutrition intervention for students with intellectual and developmental disabilities; Lemons et al. (Reference Lemons, King, Davidson, Puranik, Fulmer and Fidler2015) modified a reading program (Road to the Code) specifically to align with the phenotypic learning profiles of children with Down syndrome; and the school-based behavioural intervention Promoting Alternative Thinking Strategies (PATHS) was modified and evaluated in special education settings (Kam, Greenberg, & Kusche, Reference Kam, Greenberg and Kusche2004).
It seems likely that students with developmental disabilities will benefit more from programs that are specially designed or modified for them. In the next section, we describe two interventions that have been rigorously tested with typically developing students and subsequently adapted and evaluated for their effectiveness in target groups with special needs. The Aussie Optimism program for primary school children and the Resourceful Adolescent Program for secondary school students both meet Flay et al.’s (Reference Flay, Biglan, Boruch, Castro, Gottfredson and Ji2005) intervention standards. The modified versions also meet all of the standards, except that to date only one randomised controlled trial has been reported for each program.
Aussie Optimism
Aussie Optimism is an evidence-based program for promoting children's mental health (Roberts, Ballantyne, & van der Klift, Reference Roberts, Ballantyne and van der Klift2002). It was originally an adaptation of the Penn Prevention Program (PPP; Gillham et al., Reference Gilham, Reivich, Jaycox and Seligman1995). There are three levels of the program: positive thinking skills for 8- to 10-year-old children; social life skills for ages 10–12; and optimistic thinking skills for ages 11–13. Each of the levels has 10 separate modules that are delivered weekly for 1 hour across 10 weeks. All the levels include didactic teaching, discussion groups, activities, role-plays, and homework tasks. There are also parent and family components of the program.
The program has been shown to reduce anxiety and depression in typically developing children (Roberts et al., Reference Roberts, Kane, Bishop, Cross, Fenton and Hart2010; Roberts, Kane, Bishop, Matthews, & Thompson, Reference Roberts, Kane, Bishop, Matthews and Thompson2004), children with conduct disorders (Swannell, Hand, & Martin, Reference Swannell, Hand and Martin2009), and those involved in substance abuse (Roberts et al., Reference Roberts, Williams, Kane, Pintabona, Cross, Zubrick and Silburn2011). Improvements in social skills have also been reported (Mills, Reference Mills2007). However, a recent follow-up study found that reductions in levels of depression and anxiety were not maintained (Rooney, Hassan, Kane, Roberts, & Nesa, Reference Rooney, Hassan, Kane, Roberts and Nesa2013).
Several years ago, the original Aussie Optimism program was adapted for use with children with intellectual disability. In addition to limitations in cognitive and adaptive functioning, these students often have difficulties with communication, attention, self-regulation, and social competence (Harris, Reference Harris2006), as well as elevated rates of mental health problems (Einfeld, Elis, & Emerson, Reference Einfeld, Ellis and Emerson2011). Students with intellectual disability need more time to process information and benefit from a slower pace of delivery, concrete activities, and repetition of new material.
The modified program was named the Aussie Optimism Resilience Skills Program for Children with Intellectual Disability (Roberts et al., Reference Roberts, West, Campbell, Ho, Ward, Gilmore and Shochet2009). In order to increase the accessibility of program content, instructions were simplified and shortened to reduce demands on working memory and attention. Clearer and more concrete explanations were added to reduce the complexity of concepts, and wording was carefully scrutinised for possible ambiguity. Delivery of the program was modified by breaking hour-long modules into shorter sessions of 20 to 30 minutes to accommodate the children's more limited capacity for sustained attention. Their slower pace of learning and need for repetition to consolidate new learning were taken into account, and material was designed to be presented to small groups of 2 to 6 children, rather than whole classes of 25 students. Teaching methods emphasised demonstrations and role-plays, and included both individual and group activities that were guided by specially developed student workbooks and a teacher resource manual. The modified program contains 10 modules that cover areas such as feelings, coping skills, problem solving, communication, social skills, assertiveness, and negotiation. These were selectively drawn from the 30 modules in the three levels of the original program. Particular emphasis was placed on the application of skills such as friendly habits and assertive ways of communicating within peer and family relationships, with the aim of developing networks needed for social support.
Evidence from a randomised controlled trial with 110 children with intellectual disability aged 11–13 years in two Australian cities (Brisbane and Perth) showed that students who were assigned to the intervention group were significantly more likely to report higher perceptions of social support than those who were wait-listed to receive the intervention at a later time (Gilmore et al., Reference Gilmore, Campbell, Shochet, Garland, Smyth, Roberts, West, Prince-Embury and Saklofske2014). This finding was considered to be an important one, given the protective nature of social support (Tennant et al., Reference Tennant, Demaray, Malecki, Terry, Clary and Elzinga2015), including for those with intellectual disability (Lunsky & Benson, Reference Lunsky and Benson2001). Students who received the intervention provided more positive responses to items such as ‘There are people who love and care about me’ and ‘If something bad happens, I can ask my friends for help’, suggesting that the intervention increased their awareness of the availability of such support. This perception does not necessarily mean that children will actually ask for help, but simply knowing that support is available may be as important, or even more important, in terms of its protective value for mental health and wellbeing because it gives children a sense of security and confidence.
There was also a trend towards significant increases in tolerance for the group who received the intervention. Children with intellectual disability often have some limitations in their ability to understand the perspectives of others and consequently may struggle to accept those who are different from themselves. The intervention activities included listening to others, identifying their feelings, negotiating and responding positively which may have led to greater tolerance for the perspectives of others. Ultimately, greater tolerance should reduce conflict and improve the quality of social relationships.
The Resourceful Adolescent Program
The Resourceful Adolescent Program (RAP) is a universal school-based primary prevention program that aims to reduce the risk of adolescent depression by teaching positive coping skills (Shochet et al., Reference Shochet, Dadds, Holland, Whitefield, Harnett and Osgarby2001; Shochet & Wurfl, Reference Shochet and Wurfl2015a, Reference Shochet and Wurfl2015b). The RAP-A intervention is designed as a school-based intervention typically implemented on a universal basis with whole cohorts of students in Years 7, 8 or 9. It is an 11-session program that can be implemented within a typical school term on a weekly basis. RAP is a strength-focused approach that aims at promoting known protective factors. RAP avoids the use of any deficit language such as ‘depression’, focusing instead on identifying strengths and resources. Designed to build resilience and promote mental health in adolescents aged 11 to 15 years, the program integrates cognitive behavioural therapy (CBT) and interpersonal therapy (IPT) approaches, both of which have a solid evidence base for the treatment of adolescent depression (e.g., Clarke et al. Reference Clarke, Hornbrook, Lynch, Polen, Gale, Beardslee and Seeley2001; Merry, McDowell, Wild, Bir, & Cunliffe, Reference Merry, McDowell, Wild, Bir and Cunliffe2004; Rivet-Duval, Heriot, & Hunt, Reference Rivet‐Duval, Heriot and Hunt2011). The program has an adolescent focus (RAP-A) and a separate parent intervention (RAP-P).
RAP-A has developed an extensive evidence base and has been through a number of randomised controlled trials. In the initial controlled trial (N = 260), Shochet et al. (Reference Shochet, Dadds, Holland, Whitefield, Harnett and Osgarby2001) found that the intervention group showed significantly reduced depressive symptoms post-intervention and at 10-month follow-up compared to the control group. In addition, at-risk adolescents in the RAP intervention group were less likely than the control group to go on to develop clinical levels of depressive symptoms. In a subsequent placebo-controlled trial, conducted by independent researchers in New Zealand (Merry et al., Reference Merry, McDowell, Wild, Bir and Cunliffe2004), RAP participants showed significant reductions in depressive symptoms post-intervention that were maintained at follow-up compared to a placebo control group. Qualitative studies have suggested that the major mechanisms underlying the success of the program are the ability to regulate emotions in times of stress and improved interpersonal relationships (Cunningham, Shochet, Smith, & Wurfl, Reference Cunningham, Shochet, Smith and Wurfl2016; Shochet, Montague, Smith, & Dadds, Reference Shochet, Montague, Smith and Dadds2014).
In the last few years, RAP-A has been adapted specifically for adolescents with autism spectrum disorder (ASD) who have very high rates of mental health problems, particularly depression (Mayes, Calhoun, Murray, Ahuja, & Smith, Reference Mayes, Calhoun, Murray, Ahuja and Smith2011). These adolescents experience greater difficulties in regulating emotions and building relationships and belonging. They are often more prone to being bullied by their peers and experience loneliness and self-esteem difficulties. They are also at greater risk of suicide (Ghaziuddin, Ghaziuddin, & Greden, Reference Ghaziuddin, Ghaziuddin and Greden2002; Matson & Nebel-Schwalm, Reference Matson and Nebel-Schwalm2007; Stewart, Barnard, Pearson, Hasan, & O'Brien, Reference Stewart, Barnard, Pearson, Hasan and O'Brien2006; White & Robertson-Nay Reference White and Roberson-Nay2009). At the same time, the very condition of ASD places these students at higher risk of experiencing intense negative emotions with reduced capacity to manage these emotions (American Psychiatric Association, 2013; Kanne, Christ, & Reiersen, Reference Kanne, Christ and Reiersen2009; Konstantareas & Stewart, Reference Konstantareas and Stewart2006; White & Robertson-Nay, Reference White and Roberson-Nay2009). Since living with ASD creates very significant conditions of adversity, adolescents with ASD are in particular need of a resilience-building program such as RAP, which is modified for their particular needs (Shochet et al., Reference Shochet, Saggers, Carrington, Orr, Wurfl, Duncan and Smith2016).
The most significant change from RAP-A to RAP-A-ASD is the move from a group-based delivery to an individual format. It was thought that the social demands in the group-based format might impact negatively on the ability of the adolescents with ASD to engage with the program content. It has been shown in the treatment of anxiety in adolescents with ASD that individual therapy is more effective than group therapy (Kreslins, Robertson, & Melville, Reference Kreslins, Robertson and Melville2015). The length of the program for RAP-A-ASD has also been adjusted to a more flexible 11- to 15-session format to accommodate the time required to engage and build rapport with the young adolescent with ASD, or to deal with disruptions to the program process due to difficulties associated with ASD symptoms such as school absence. Another modification involves the use of computerised sessions for some of the core components (iRAP). Adolescents with ASD engage well with technology (Kientz, Goodwin, Hayes, & Abowd, Reference Kientz, Goodwin, Hayes and Abowd2013), and the computerised sessions, including the use of video and ‘drop and drag’ activities, deliver some more complex components of the overall CBT model. Additional exercises have been included to bolster theory of mind and to cater for variations in sensory processing and special interests.
The accompanying RAP-A-ASD facilitator's manual increases awareness of the specific challenges of living with ASD (e.g., the sensory processing issues associated with ASD) and highlights the importance of structuring the environment when working with young people with ASD. RAP-A-ASD was evaluated with a small pilot randomised controlled trial of young adolescents living with ASD aged from 10 to 13 (N = 30; Mackay, Reference Mackay2013). This mixed quantitative and qualitative evaluation showed initial support in favour of RAP-A-ASD (compared to the comparison group) in terms of significant effects on parent measures of adolescents’ coping efficacy, both at post-test and follow-up. Although there were no significant effects on adolescent self-report measures, qualitative findings provided good triangulation of parent measures of intervention effects. Teachers, parents, and adolescents provided consistent endorsement of themes of perceived improvements in adolescents’ affect regulation, coping self-efficacy, confidence, and social skills.
Conclusions
School psychologists and counsellors play a critical role, not only in the selection of appropriate evidence-based programs, but also in their proper implementation within the school context and in evaluations of their outcomes. The interventions that are most likely to be successful are those that have a strong theoretical base and robust evidence for their efficacy. As discussed above, acceptable interventions should have a minimum of two well-designed studies that demonstrate program effects, with the inclusion of control groups, robust pre- and post-intervention measures, and appropriate statistical analysis. Interventions that are newly developed in schools should be similarly trialled and evaluated, possibly in collaboration with university researchers.
Ideally, programs should be adapted to increase their suitability for children with developmental disabilities, and then tested with those populations. As illustrated in the examples of Aussie Optimism and the Resourceful Adolescent Program, modifications may include using shorter sessions and smaller groups, simpler formats, and repetition of key material. Non-language based techniques, such as drawings, role plays and puppets, are useful to supplement or replace written and oral language activities.
In the school setting, numerous barriers may impede the effective implementation of evidence-based interventions. These include time constraints, limited funds, and lack of support from administrative or senior staff (Forman et al., Reference Forman, Shapiro, Codding, Gonzales, Reddy, Rosenfield, Sanetti and Stoiber2013). Such obstacles are likely to be reduced when intervention programs are consistent with broader school goals and priorities, and when staff training and ongoing support are readily available. In addition, programs that are accompanied by clearly manualised procedures and a range of appropriate resources will generally be most effective in school contexts (Forman & Barakat, Reference Forman and Barakat2011).
Miller and Sawka-Miller (Reference Miller, Sawka-Miller, Lionetti, Snyder and Christner2011) provide a valuable set of guidelines for psychologists to use when evaluating established interventions that are implemented in schools. They argue that skills in program evaluation are essential for school psychologists, and that the question is not whether or not programs should be evaluated, but rather how the evaluations should be undertaken. Merely asking students to rate their enjoyment of the activities, or teachers to rate their perceptions of program effectiveness, are not sufficient for determining the impact of an intervention. Program evaluation involves documenting the process of delivery, monitoring the integrity of implementation, measuring the achievement of targeted outcomes, and assessing the sustainability of intervention effects (Miller & Sawka-Miller, Reference Miller, Sawka-Miller, Lionetti, Snyder and Christner2011).
There is now a wealth of evidence to demonstrate the benefits that can accrue from the use of well-designed and rigorously tested school-based interventions. It is imperative for the wellbeing of students that interventions are carefully selected, competently implemented, and properly evaluated. Of utmost importance is the need to consider the appropriateness of specific programs for children with developmental disabilities and to make whatever modifications are necessary to increase the likelihood that these students too will reap the benefits of intervention.
