Autism spectrum disorder (ASD) is a neurodevelopmental disorder, affecting approximately 51 per 10,000 children in Australia (Parner et al., Reference Parner, Thorsen, Dixon, de Klerk, Leonard, Nassar and Glasson2011). ASD is frequently associated with co-occurring intellectual disability (ID). It has been estimated that 50–70% of individuals with ASD have ID (Charman et al., Reference Charman, Pickles, Simonoff, Chandler, Loucas and Baird2011; Edelson, Reference Edelson2006; Fombonne, Reference Fombonne2005; Salomone et al., Reference Salomone, Kutlu, Derbyshire, McCloy, Hastings, Howlin and Charman2014), although rates vary depending on various factors, including diagnostic criteria used (Matson & Shoemaker, Reference Matson and Shoemaker2009). For children with ASD, a lower childhood IQ is associated with poorer prognosis for early intensive intervention plans (Itzchak, Lahat, Burgin, & Zachor, Reference Itzchak, Lahat, Burgin and Zachor2008) and overall negative adult outcomes (Billstedt, Gillberg, & Gillberg, Reference Billstedt, Gillberg and Gillberg2005). This suggests that children with ASD and ID represent a particularly vulnerable and important population to consider in ASD clinical research and treatment. Nevertheless, this group is seldom the focus of research attention.
Research suggests approximately 70–75% of children with ASD have at least one mental health disorder, including a range of both internalising and externalising disorders in addition to ASD (Brereton, Tonge, & Einfeld, Reference Brereton, Tonge and Einfeld2006; Leyfer et al., Reference Leyfer, Folstein, Bacalman, Davis, Dinh, Morgan and Lainhart2006; Mattila et al., Reference Mattila, Hurtig, Haapsamo, Jussila, Kuusikko-Gauffin, Kielinen and Moilanen2010; Mazefsky, Conner, & Oswald, Reference Mazefsky, Conner and Oswald2010; Simonoff et al., Reference Simonoff, Jones, Baird, Pickles, Happe and Charman2013; Simonoff et al., Reference Simonoff, Pickles, Charman, Chandler, Loucas and Baird2008). While core ASD symptoms impact significantly on the everyday functioning of a child with ASD, mental health disorders are often equally impairing and deserving of intervention (Williams, Matson, Beighley, Rieske, & Adams, Reference Williams, Matson, Beighley, Rieske and Adams2014). Of the extant research examining samples of children with ASD, with and without ID, studies suggest ASD and ID may be independent risk factors for emotional and behavioural disorders (Goldin, Matson, & Cervantes, Reference Goldin, Matson and Cervantes2014; Totsika, Hastings, Emerson, Lancaster, & Berridge, Reference Totsika, Hastings, Emerson, Lancaster and Berridge2011).
Social deficits are a core feature of ASD (Matson, Matson, & Rivet, Reference Matson, Matson and Rivet2007) and have been found to differentiate children with ASD from those with other developmental disorders (Klin et al., Reference Klin, Saulnier, Sparrow, Cicchetti, Volkmar and Lord2007). In typically developing children, poor social and emotional competence has been associated with mental health, personal, social, and academic difficulties (Eisenberg, Reference Eisenberg, Fitzgerald, Lester, Zuckerman, Fitzgerald, Zucker and Freeark2006; Masten & Coatsworth, Reference Masten and Coatsworth1998; Weissberg & Greenberg, Reference Weissberg, Greenberg, Sigel and Renninger1998). Although more research is needed, a recent study found social skills and social reciprocity deficits account for between 44.7–54.7% of the parent- and teacher-reported variance in mental health in school-age children with ASD, both with co-occurring mild ID and without ID (Ratcliffe, Wong, Dossetor, & Hayes, Reference Ratcliffe, Wong, Dossetor and Hayes2015). This implies poor social skills may be associated with the very high levels of mental health disorders in children with ASD, suggesting social-emotional interventions may be a way to target mental health in school-age children with ASD with and without co-occurring ID.
To date, social skills intervention research has focused on verbal children without ID (Ingersoll, Walton, Carlsen, & Hamlin, Reference Ingersoll, Walton, Carlsen and Hamlin2013; Reichow, Steiner, & Volkmar, Reference Reichow, Steiner and Volkmar2013). In general, reviews of such interventions support the utility of social skills interventions for children with ASD without ID (ASD/no-ID; Kaat & Lecavalier, Reference Kaat and Lecavalier2014; Landa, Holman, O'Neill, & Stuart, Reference Landa, Holman, O'Neill and Stuart2011; Matson et al., Reference Matson, Matson and Rivet2007; McConnell, Reference McConnell2002; Reichow & Volkmar, Reference Reichow and Volkmar2010; Scattone, Reference Scattone2007). Existing ASD social skills intervention research tends to be conducted in university clinic settings (Kasari & Smith, Reference Kasari and Smith2013). However, school may be the ideal place to teach social-emotional skills as school is the context in which social-emotional skills are commonly used and teaching children with ASD skills in context helps promote generalisation. There has been scant research evaluating school-based delivery of social-emotional interventions for school-age children with ASD (Kasari & Smith, Reference Kasari and Smith2013). School-based ASD social-emotional interventions have historically employed single-subject designs (Bellini, Peters, Benner, & Hopf, Reference Bellini, Peters, Benner and Hopf2007). More recent group-based intervention studies have lacked a control group (Beaumont, Rotolone, & Sofronoff, Reference Beaumont, Rotolone and Sofronoff2015), utilised unpublished treatment packages (Kasari, Rotheram-Fuller, Locke, & Gulsrud, Reference Kasari, Rotheram-Fuller, Locke and Gulsrud2012; Lopata et al., Reference Lopata, Thomeer, Volker, Lee, Smith, Smith and Toomey2012), or focused on children with ASD/no-ID (Beaumont et al., Reference Beaumont, Rotolone and Sofronoff2015; Johnson, Reference Johnson2014; Laugeson & Park, Reference Laugeson and Park2014; Ratcliffe, Wong, Dossetor, & Hayes, Reference Ratcliffe, Wong, Dossetor and Hayes2014).
Emotion-Based Social Skills Training
Emotion-Based Social Skills Training (EBSST) is a manualised group-based social-emotional intervention specifically designed to improve mental health and wellbeing in school-aged children with ASD. There are two versions of EBSST: EBSST for ASD/no-ID (Wong, Lopes, & Heriot, Reference Wong, Lopes and Heriot2010), recently published as the Westmead Feelings Program 2 (Wong et al., 2018); and EBSST for ASD and Mild ID (Ratcliffe, Grahame, & Wong, Reference Ratcliffe, Grahame and Wong2010), recently published as the Westmead Feelings Program 1 (Ratcliffe et al., Reference Ratcliffe, Wong, Grahame, Brice, Carroll and Dossetor2017). EBSST for ASD/no-ID was previously evaluated in a school-based pretest/posttest control group study involving 217 children (aged 7–13 years) with ASD/no-ID (Ratcliffe et al., Reference Ratcliffe, Wong, Dossetor and Hayes2014). Forty-one school counsellors delivered the 16-session treatment to groups of 3–8 students in their schools. Teachers and parents also received six sessions of EBSST in separate groups. Teacher-reported emotional competence improved after EBSST. The effect size was large and improvements were sustained at 6-month follow-up. EBSST has been modified in a version to cater to the learning needs of children with ASD and co-occurring mild ID (ASD + MID). EBSST for ASD + MID (Ratcliffe et al., Reference Ratcliffe, Grahame and Wong2010) has been described in a case study (Ratcliffe, Reference Ratcliffe, Dossetor, White and Whatson2011). Clinical and pilot studies of EBSST for ASD + MID have shown promise; however, this version has not yet been further evaluated.
Current Study
The primary aim of this preliminary study was to evaluate the effectiveness of delivering EBSST for ASD + MID to children, their parents and teachers by school counsellors in schools in a treatment versus wait list comparison study. It was posited that children with ASD + MID would likely benefit from EBSST in a similar fashion to their peers with ASD without co-occurring ID. Therefore, it was hypothesised that children in the treatment group would improve on teacher- and parent-reported measures of emotional competency. It was further hypothesised that EBSST for ASD + MID would improve results for teacher- and parent-reported social skills and mental health from pretreatment to posttreatment compared to the control group.
Method
Design
A quasi-experimental design was implemented for the purposes of the evaluation. There was one between-participants factor (group: treatment vs. control) and one within-participants factor (time: pretest vs. posttest) for each domain (emotional competence, social skills and mental health).
School counsellors nominated themselves to facilitate EBSST groups. They also nominated 3–8 eligible children in their schools to participate. Groups were non-randomly allocated to treatment or 9-month waitlist control based on availability of counsellors, schools and eligible participants.
Participants
Fifteen school counsellors from different mainstream metropolitan and regional NSW Department of Education and Communities (DEC) schools nominated themselves to participate. DEC school counsellors are trained as classroom teachers, with post-teacher qualification training in psychology. The school counsellors had an average 14.13 years of experience as a school counsellor (range 3–25 years). One school counsellor was transferred and was unable to participate. The 14 remaining school counsellors nominated 3–8 eligible children in their schools to participate in the study.
Of the 75 child participants, eight treatment group school counsellors enrolled 43 children and six waitlist comparison group school counsellors enrolled 32 children. This study predated the release of the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association, 2013). Therefore, for the purposes of this study, children were eligible if they had a diagnosis of any of the three ASD subtypes contained in the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV): autistic disorder, Asperger's disorder, or pervasive developmental disorder not otherwise specified (PDDNOS; American Psychiatric Association, 2000) and a mild ID, defined as an IQ of 50–55 to 70 and associated deficits in adaptive behaviour (American Psychiatric Association, 2000). Assessment and diagnosis of ASD + MID were made and documented prior to the study by a specialist medical practitioner and/or registered psychologist with appropriate clinical experience, and relevant reports documenting children's diagnoses were placed on children's school files. Researchers were blind to the selection and identity of the children, and it was the school counsellors who reviewed children's school files to confirm eligibility. In this study, the parent Social Responsiveness Scale (SRS; Constantino, Reference Constantino2002) scores were utilised to confirm participants' diagnostic features. Parents' pretreatment t scores on the SRS-P were in the severe range for every participant, which was strongly associated with a diagnosis of autism (Murray, Mayes, & Smith, Reference Murray, Mayes and Smith2011; Wilkinson, Reference Wilkinson2010a, Reference Wilkinson2010b). In addition, teacher SRS scores were in the clinical range for each participant. This gives an indication of the validity of school counsellor file reviews, implying that participants were highly likely to have an ASD as documented on the school file. Additional participant demographic information is displayed in Table 1.
Participants' Demographic Information (N = 75)
Note: SRS = Social Responsiveness Scale.
Ethics
Consent was sought from school counsellors, school principals, parents and teachers for all children involved. The study had ethics approval from the Human Research Ethics Committee at the The Children's Hospital at Westmead Human Ethics Committee.
Measures
Demographic questionnaire
School counsellors, teachers and parents all completed brief background information surveys to obtain general demographic information.
Autism severity
The SRS (Constantino, Reference Constantino2002) is a standardised, norm-referenced questionnaire designed to capture the severity of autistic behaviours in 4- to 18-year-olds. The SRS can be used as an effective screener in educational settings and as an aid to clinical diagnosis (Wilkinson, Reference Wilkinson2010a). The SRS items measure the ASD symptoms in the domains of social awareness, social information processing, reciprocal social communication, social anxiety/avoidance, and stereotypic behaviour/restricted interests. The scale demonstrates strong interrater reliability and acceptable internal consistency, and correlates highly with the Autism Diagnostic Interview — Revised (Rutter, Le Couteur, & Lord, Reference Rutter, Le Couteur and Lord2003). The authors report satisfactory psychometric properties for the instrument: Cronbach's alpha = .97; test-retest reliability r = .88; interrater reliability, parents and teachers r = .73 (Constantino, Reference Constantino2002).
Emotional competence
Emotional competence was assessed using the teacher and parent informant report versions of the Emotional Development Questionnaire (EDQ; Wong, Heriot, & Lopes, Reference Wong, Heriot and Lopes2009). The 40-item EDQ is a measure of emotional competence, which was designed to specifically examine competency in the skills taught in EBSST for children with ASD aged 4–18 years. The EDQ assessed the following constructs: emotional understanding, empathy/theory of mind, emotion regulation and parent emotion coaching skills. The EDQ was designed as an overall measure of a child's emotional competence, and each of the domains it assesses maps onto EBSST Modules. The EDQ has been previously found to be a reliable and valid outcome measure for EBSST for school-aged children with ASD/no-ID (Ratcliffe et al., Reference Ratcliffe, Wong, Dossetor and Hayes2014). Using a sample of children with ASD/no-ID, the questionnaire demonstrated excellent internal consistency, with a Cronbach's alpha of .92 for teachers and .91 for parents for the population in this study. Support for the concurrent validity of the EDQ was provided by the strong relationship between teachers' ratings on this measure, and their scores on the Social Skills Improvement System (SSIS), r = .27, p < .001. Moreover, there was a strong relationship between parents' ratings on the EDQ and their scores on the SSIS, r = .54, p < .001, and the SDQ, r = −.36, p < .001 (Ratcliffe et al., Reference Ratcliffe, Wong, Dossetor and Hayes2014). In the current study involving a sample of children with ASD and co-occurring mild ID, the EDQ also demonstrated excellent internal consistency, with a Cronbach's alpha of .95 for teachers and .90 for parents for the population in this study. In addition, in the current study, further support for the concurrent validity of the EDQ was provided by the strong relationship between teachers' ratings on this measure and their scores on the SSIS, r = .55, p < .001. Moreover, there was a strong relationship between parents of children with ASD + MID ratings on the EDQ and their scores on the SSIS, r = .61, p < .001, and the DBC, r = −.20, p < .001.
Social skills
Overall social skills were assessed using the teacher- and parent-report versions of the Social Skills Improvement System (SSIS; Gresham & Elliott, Reference Gresham and Elliott2008). The SSIS is a standardised norm referenced assessment of social skills for children and youth aged 3–18 years. Participants received a total social skills score, which included the subscales of communication, cooperation, assertion, responsibility, empathy, engagement, and self-control. The total score was utilised to provide an overview of participant's overall social skills. The SSIS shows strong psychometric properties in terms of internal consistency and test-retest reliability estimates. Median scale reliabilities of the social skills scale were in the mid-to-upper .90s for every age group on each form. Median subscale reliabilities were in the high .80s for the Teacher Form and the mid .80s for the Parent Form. All alpha coefficients were equal to or exceeded .70. Test-retest indices for Total Social Skills were .82 for the Teacher Form and .84 for the Parent Form (Gresham, Elliott, Vance, & Cook, Reference Gresham, Elliott, Vance and Cook2011).
Mental health difficulties
Mental health difficulties were measured using the teacher and parent informant report versions of the Developmental Behaviour Checklist (DBC; Einfeld & Tonge, Reference Einfeld and Tonge1992). The DBC is a 96-item standardised, norm-referenced questionnaire, which is commonly used in Australian ID clinical practice and research. The DBC is designed to assess a broad range of behavioural and emotional disturbances in young people with ID. Norms are available for different levels of ID, including young people with mild ID. The authors report robust reliability and validity characteristics: Cronbach's alpha = .94; test-retest reliability r = .88. The total score on the DBC provides a measure of overall severity of psychiatric disturbance and correlates with child psychiatrists' ratings of severity of psychopathology (Cox & Rutter, Reference Cox, Rutter, Rutter and Hersov1985). Subjects who have DBC total scores greater than 46 have clinically significant levels of psychiatric disorder, as determined by receiver operating characteristic (ROC) curve analysis (Tonge & Einfeld, Reference Tonge and Einfeld2000).
Emotion-Based Social Skills Training (EBSST)
EBSST (Ratcliffe et al., Reference Ratcliffe, Grahame and Wong2010; Ratcliffe et al., Reference Ratcliffe, Wong, Grahame, Brice, Carroll and Dossetor2017; Wong et al., Reference Wong, Lopes and Heriot2010) is a manualised group-based intervention that aims to teach children with ASD skills in emotional competence. EBSST is divided into three 5-week modules, teaching skills in understanding own and others' emotions (module 1), emotional problem solving and theory of mind (module 2), and emotion regulation skills (module 3), following a developmental theoretical framework. A variety of evidence-based teaching techniques, including social stories, video modelling, and use of concrete visual teaching techniques to cater to children with ASD are employed as part of the multicomponent treatment package (Ratcliffe et al., Reference Ratcliffe, Wong, Dossetor and Hayes2014; Ratcliffe et al., Reference Ratcliffe, Wong, Grahame, Brice, Carroll and Dossetor2017).
EBSST was originally developed for children with ASD/no-ID (Wong et al., Reference Wong, Lopes and Heriot2010; Wong et al., Reference Wong, Lopes, Heriot, Brice, Carroll, Ratcliffe and Dossetor2018a, Reference Wong, Lopes, Heriot, Brice, Carroll, Ratcliffe and Dossetor2018b), and this version has been described elsewhere (Ratcliffe et al., Reference Ratcliffe, Wong, Dossetor and Hayes2014). EBSST has previously been adapted to meet the learning needs of children with ASD + MID (Ratcliffe et al., Reference Ratcliffe, Grahame and Wong2010). In the adaptation, EBSST for ASD + MID retained the underlying theoretical grounding and program structure of EBSST with ASD/no-ID. However, in terms of teaching content, developmentally appropriate social-emotional skills were targeted. An overview of EBSST is provided in Table 2. In module 1 of EBSST for ASD/no-ID, children learn the four primary emotions — happy, sad, worried, and angry. In each session, children also learn emotions ‘like’ the four primary emotions. For example, relaxed and calm are emotions like happy. This allows children to recognise and communicate about a broader range of emotions. For EBSST for ASD + MID, module 1 retained a focus on ‘understanding emotions'. However, owing to the cognitive and language difficulties in this population, module 1 focuses on just the four primary emotions — happy, sad, worried, and angry. Module 2 retained a focus on ‘understanding others' emotions'. However, children were taught a developmentally appropriate visual perspective-taking skill. That is, they were encouraged to learn that two people can have different feelings in the same situation depending on what they see — a skill gained around the age of 4 or 5 years in typically developing children (Gzesh & Surber, Reference Gzesh and Surber1985), but impaired in children with ASD (Hamilton, Brindley, & Frith, Reference Hamilton, Brindley and Frith2009). The higher order skill of identifying that people have different feelings and thoughts in the same situation depending on what they see, hear or experience was excluded owing to the difficulties that children with ASD + MID have with language and understanding of abstract concepts. A simplified version of emotional problem solving was retained in module 2. Module 3 retained a focus on ‘emotional regulation’, teaching children to select from several strategies to help change their not so good feelings to good feelings by selecting from one of six (reduced from eight in the original program) strategies in the ‘feelings control kit’. EBSST for ASD + MID utilises a range of evidence-based techniques in order to teach skills within the modules.
Summary of EBSST for ASD and Mild ID Program Content for Children, Parents and Teachers
A parent and teacher version of EBSST for ASD + MID (Ratcliffe et al., Reference Ratcliffe, Grahame and Wong2010) was adapted from EBSST for ASD/no-ID (Wong et al., Reference Wong, Lopes and Heriot2010). EBSST for parents and teachers promotes generalisation and maintenance of child skills via psycho-education, parent cognitive behaviour therapy, training parents as emotion coaches and training in self-monitoring and evaluation. Given the additional cognitive and language difficulties in children with ASD + MID, it was identified that parents and teachers would play an important role in facilitating children to practice the skills learnt in group via use of the visual worksheets at home.
Procedure
EBSST ASD + MID Training
School counsellors received a complete EBSST for ASD + MID kit and completed two days of skill-based training by the authors of the EBSST at the Children's Hospital at Westmead prior to running EBSST groups. Training included didactic instruction, videos, live demonstrations and role play. A two-part, competency-based assessment was employed as part of the two-day training. Assessment 1 was a written quiz consisting of 30 multiple choice questions based on information provided in the training course and the EBBST manuals. Assessment 2 was an in vivo role-play of one of 12 key sections of the manualised program. The role-play was assessed by the authors of the program against a checklist for content (delivery as described in the manual) and process (appropriate use of materials, visual supports and positive behaviour strategies), and all counsellors were accredited as competent to delivery EBSST.
Three proxy fidelity measure checks were used for delivery of the intervention: (1) school counsellors were trained and their competency was based on 80% or higher quiz and role play; (2) they were given access to online support forum and telephone and email contacts with the trainers to support program implementation, and (3) they completed a self-rating of their treatment implementation fidelity via rating adherence to the treatment manual on a 7-point Likert-type scale, ranging from not at all closely to very closely. All school counsellors rated their adherence to the treatment manual as either closely or very closely.
Data collection
All participating school counsellors were sent data collection packs for each participant. Packs contained copies of the demographic and outcomes measures for teachers and parents to complete prior to commencing EBSST and immediately after EBSST. School counsellors administered and collected all assessment information. School counsellors then forwarded deidentified, coded assessment protocols to the Children's Hospital at Westmead where clinical psychologists scored the assessment measures and entered the data into a database.
EBSST ASD + MID treatment group
EBSST for ASD + MID was run in small groups of 3–8 participants. Children participating in EBSST were withdrawn from class at an agreed day or time each week. The treatment sessions were typically delivered in a quiet room, away from the regular classroom, that had access to a video and whiteboard. The 16-session treatment was divided into three modules over three school terms. Within each of the three modules, children received weekly 90-minute sessions for five consecutive weeks in groups of between 3 to 8 children. Teachers and parents received a session prior to and following each module. All child, teacher and parent sessions were facilitated by the school counsellor, with assistance from a member of school staff, most typically a teacher or teacher's aide. School counsellors self-selected to report on their fidelity to the treatment manuals. School counsellors reported following the manual and using the materials as per their training in EBSST ranging from ‘closely’ to ‘very closely’.
Waitlist treatment group
Children in the waitlist control group did not receive any intervention during the waitlist period. After the wait period ended, children in the waitlist group and their parents and teachers were invited to participate in EBSST groups with their school counsellor at school.
Statistical Method
All statistical analyses were conducted using the SPSS software program, version 22, except the mixed linear regression model, which was conducted using R, version 3.5.1. The Bonferroni adjustment was made where appropriate to adjust for multiplicity. Missing responses on outcome measures were managed using pairwise deletion in SPSS, meaning analyses were computed where all data were required for the specific analysis.
A series of baseline analyses (independent t tests) were conducted to detect any difference between groups in terms of age, or any of the teacher or parent versions of any dependent measure (EDQ, SSIS or DBC). A chi-squared analysis was used to detect differences between groups in gender. Cronbach's alphas were computed for the various measures used to assess the internal consistency of responses given in the current dataset.
The main research questions, does EBSST improve emotional competence, social skills and mental health in the treatment group compared to the control group, were addressed using mixed linear regression. Using a mixed-effects model, pairwise deletion of individuals with missing values was not required and all preintervention test results for EDQ, SSIS and DBC were included in the model, even when the posttest results were missing. There were 5–7 individuals removed from the mixed-effects models, as they had missing baseline values. Given this was a small proportion of the total sample, imputation was deemed necessary in this case, as it would have a negligible effect on the results. Separate analyses were conducted for teacher and parent report for each of the outcome measures. The total emotional competence, social skills and mental health scores at pre- and posttreatment were entered as dependent variables. The main independent variables were time (pre- vs. posttreatment) and group (treatment vs. control). A significant interaction between group and time was used to test whether there was a greater change in the scores in the treatment group, relative to the control group. The other independent variables that were adjusted for in each analysis were gender and pretreatment SRS scores, and ID was treated as a random effect.
Results were further examined in terms of Cohen's d analyses. The coefficients for the interaction between group and time were converted to a Cohen's d. Cohen's d effect sizes reflect the clinical significance of results, as they provide an indication of the standardised differences in scores. They may be interpreted as small (.2), medium (.5) and large (.8) (Cohen, Reference Cohen1988). However, these values are arbitrary and should not be interpreted rigidly (Thompson, Reference Thompson2007). Small effect sizes can have large consequences, particularly in mental health interventions where, for example, an intervention that leads to a reliable reduction in suicide rates with an effect size of d = 0.1 (Lakens, Reference Lakens2013). Preliminary analyses were conducted to ensure no violation of the assumptions of normality, linearity, homogeneity of variances, homogeneity of regression slopes and reliable measurement of the covariate.
Results
The non-random design did have the potential to influence treatment and control group equivalence. However, a series of baseline analyses (independent t tests) confirmed that the treatment and control groups did not differ significantly on any of the teacher or parent versions of any dependent measure (EDQ, SSIS or DBC) reported below at pretreatment (T1; all p > .05). Independent samples t tests indicated that there was no significant difference between the two groups in terms of age (p > .05). However, there were differences between both parent- and teacher-reported SRS scores. For parents' reports, the treatment group had significantly higher mean SRS scores compared with the control group (p = .03). For teachers' reports, the treatment group had significantly lower SRS mean scores than the control group (p = .001). Preliminary chi-square analyses indicated that there were also significant gender differences between the treatment and comparison groups, chi (1, n = 75) = .03. Specifically, there were a greater proportion of females in the treatment group compared to the control group. Pretreatment differences between groups found on gender and parent and teacher rated SRS scores were adjusted for in all relevant analyses.
Performance of Measures
Outcome measures were assessed using the pretreatment data for the treatment and control groups combined.
Cronbach's alpha scores were computed for the various measures used to assess the internal consistency of responses given in the current dataset (see Table 3). For the EDQ-T, EDQ-P and the DBC-T internal consistency was excellent. Internal consistency for the DBC-P, and the SSIS and was also good.
Teacher and Parent Outcome Measures — Intercorrelations and Internal Reliability
Note: EDQ-T = Emotions Development Questionnaire — Teacher form; EDQ-P = Emotions Development Questionnaire — Parent form; SSIS-T = Social Skills Improvement System — Rating Scales — Teacher form; SSIS-P = Social Skills Improvement System — Rating Scales — Parent form; DBC-T = Developmental Behaviour Checklist — Teacher form; DBC-P = Developmental Behaviour Checklist — Parent form.
*p < .05, two-tailed, **p < .01, two-tailed.
Teaching Ratings
Teacher-rated results on the various outcome measures at pretreatment and posttreatment for the treatment and control groups are presented in Tables 4 and 5. The return rate was 94.4% for treatment group teachers and 77% for control group teachers.
Means and Standard Deviations for Teacher EDQ, SSIS, DBC for Treatment Group at Pretreatment, Posttreatment, and Control Group Pretreatment and Posttreatment
Note: Standard deviations are presented in parentheses.
Mean Difference Between Posttreatment and Pretreatment Scores, Relative to the Waitlist Control Group on the Teacher and Parent EDQ, SSIS and DBC, p Values and Cohen's d Following Multiple Mixed Linear Regression Analyses for Each Outcome Measure
Note: Estimated mean differences are the mean difference in scores (posttreatment–pretreatment), relative to the control group after adjusting for gender and pretreatment SRS scores.
On the EDQ-T, the mean difference between the post- and pretreatment score was significantly higher for the treatment group compared to the control group (mean difference = 16.98; 95% CI: [1.57, 32].4; p = .032). However, this result was no longer statistically significant after the Bonferroni adjustments was applied (Bonferroni-adjusted α for comparison = .017). The magnitude of the difference between groups was small to medium (Cohen's d = .32)
On the SSIS-T, the mean difference between the post- and pretreatment SSIS scaled score was not significantly higher for the treatment group compared to the control group (mean difference = 1.18; 95% CI [−4.96, 7.32]; p = .7; Bonferroni-adjusted α for comparison = .017).
There was no statistically significant difference between the treatment and the control group on the DBC-T (p = .14). A small Cohen's d effect size was observed (Cohen's d = .22).
Parent Ratings
Parent-rated results on the various outcome measures at pretreatment and posttreatment for the treatment and control groups are presented in Tables 5 and 6. The return rate was 70% for treatment group parents and 60% for control group parents. There was a statistically significant difference found between the treatment and control group on the EDQ-P (mean difference = 10.13; 95% CI [0.17, 20.9]; p = .047). However, this result was no longer statistically significant after the Bonferroni adjustments was applied (Bonferroni-adjusted α for comparison = .017). The magnitude of the difference between groups was small to medium (Cohen's d = .34). There was no statistically significant difference between the treatment and the control group on the SSIS-P (p = .51) or the DBC-P (p = .92).
Mean and Standard Deviations for Parent EDQ, SSIS, DBC for Treatment Group at Pretreatment, Posttreatment, and Control Group Pretreatment and Posttreatment
Note: Standard deviations are presented in parentheses.
Discussion
This pilot study aimed to evaluate the effectiveness of EBSST for children with ASD + MID in a ‘real world’ school setting. Overall, results found partial support for the hypothesis that school counsellor delivery of EBSST to children with ASD + MID, their teachers and parents in schools would improve child emotional competence. Parents and teachers both reported significant improvements in emotional competence in the treatment group compared to a waitlist control, as measured by the Emotions Development Questionnaire (EDQ). However, results were not significant after the Bonferroni adjustment was applied. Effect sizes were consistently in the small to medium range. Parent- and teacher-reported results on the measures of mental health and social skills were not statistically significant; however, they trended in the expected directed of improvements in the treatment group compared to the waitlist control.
The current findings indicate that EBSST for ASD + MID results in improved emotional competence at school and home. Although the results were not significant after the Bonferroni adjustment, the small- to medium-effect sizes are noteworthy and clinically significant in a sample of children with ASD + MID, who by diagnostic criteria present with impaired social-communication, cognitive and language difficulties. That is, any incremental gains in skills in this population are clinically noteworthy. The current evaluation of EBSST in a sample of children with ASD + MID is consistent with a previous treatment versus control trial of EBSST for children without ID in schools, which also found that EBSST improved teacher reported emotional competence at school (Ratcliffe et al., Reference Ratcliffe, Wong, Dossetor and Hayes2014). The current study replicates and extends these previous results, indicating that EBSST for ASD + MID also improves parent-reported emotional competence. It may be surprising that children with ASD with more complex learning needs associated with co-occurring mild ID improved their emotional competence skills, at school and home, following delivery of a school-based intervention, whereas children with ASD/non-ID improved only their skills in emotional competence at school (Ratcliffe et al., Reference Ratcliffe, Wong, Dossetor and Hayes2014). Effective social skills programs for ASD utilise a variety of modalities when teaching strategies and practising skills (Reichow & Volkmar, Reference Reichow and Volkmar2010). EBSST is a multicomponent treatment package. In comparing the teaching techniques in EBSST for ASD + MID to EBSST for ASD/no-ID, the former has a greater reliance on visual supports, particularly video modelling. In an evaluation of several commonly used social skills interventions for children with ASD, video modelling was found to be evidence-based and highly effective (Bellini & Akullian, Reference Bellini and Akullian2007; Reichow & Volkmar, Reference Reichow and Volkmar2010). It is possible that this version was able to improve both emotional competence at home and because of a greater reliance on video modelling. It is also possible that teachers and parents may have been aware that due to the children's cognitive impairments, they would need support to practise and generalise skills, and may have provided more support and scaffolding to nurture skill development as a result.
In the current study, EBSST did not result in statistically significant improvements in teacher- and parent-reported social skills. The null findings for social skills may suggest that children with ASD + MID were only able to obtain measurable improvements in emotional competence, as emotional competence skills are taught explicitly in EBSST. Social skills were not taught explicitly in the program. Nevertheless, results trended in the expected direction, and a small effect size was observed at home. This finding is noteworthy and suggests gains in emotional competence may have led to some mild improvements in social skills in a population of children with complex neurodevelopmental disorders of ASD + MID, in which gains are made in small incremental steps. These current findings are consistent with results of a previous study of children with ASD/non-ID (Ratcliffe et al., Reference Ratcliffe, Wong, Dossetor and Hayes2014). Previous research does suggest that children with ASD have difficulty generalising new skills (Bellini et al., Reference Bellini, Peters, Benner and Hopf2007; Lord et al., Reference Lord, Bristol-Power, Cafiero, Filipek, Gallagher, Harris, Wetherby, Lord and McGee2001; Radley, Jenson, Clark, & O'Neill, Reference Radley, Jenson, Clark and O'Neill2014) to different skills and contexts. The current findings suggest the children were able to better generalise their emotion competence skills from their sessions at school with the school counsellor to a different school context (namely the classroom) and home, because there were more opportunities to explicitly practise the emotional competence skills in untrained settings with their parents and teachers. It is very possible that children with ASD + MID may require an explicit program in pragmatic social skills to obtain measurable social skills gains following an intervention for teachers compared with parents, to facilitate practice and reinforcement of skills.
In terms of mental health, no statistically significant treatment effects for parents or teachers were found. This indicates EBSST may not improve mental health of children with ASD + MID. This may be because EBSST targets emotional competence and is not an explicit mental health treatment. It is possible that the DBC measure may not have been sensitive enough to detect change in this school-based community sample of children with ASD + MID, as it was designed to measure emotional and behavioural competence in children with ID. Indeed, the lack of standardised measures available to assess mental health and wellbeing in non-clinical, school-based samples of children with ASD (Yager & Iarocci, Reference Yager and Iarocci2013) may have impeded research and development in this area. These issues are exacerbated for children with ASD + MID, who have been all but ignored in the development of appropriate outcome measures and further research in this area is urgently needed.
Limitations
The current study was a preliminary evaluation of the effectiveness of EBSST for ASD + MID, which is the first group social-emotional intervention package specifically developed for school children with ASD and co-occurring mild ID. Autism-related interventions with proven efficacy are rarely adopted or successfully implemented in public schools due, at least in part, to the ‘lack of fit’ between the intervention and the needs, structures, and capacities of the school setting (Locke, Kasari, & Wood, Reference Locke, Kasari and Wood2014). Thus, the school setting was chosen for this preliminary study, in order to engage end-users, including school children with ASD + MID, parents, teachers and school counsellors in the research to determine whether EBSST for ASD + MID is effective under real-life conditions. It is important to note that this study was considered a critical first step before conducting efficacy studies of EBSST for ASD + MID in order to increase research integrity, uptake and applicability, thereby maximising the likelihood of eventual knowledge translation from research to practice (Dew & Boydell, Reference Dew and Boydell2017).
Owing to the nature of the ‘real world’ design, it is possible that child, parent, teacher and school counsellor characteristics could have influenced the study. For example, it is possible that student characteristics such as behavioural and/or attentional difficulties, access to support or intervention external to the school setting (e.g., psychological intervention), and environmental factors, such as staff-to-student ratios in groups, may have influenced the effectiveness of the content delivered during each treatment session. In addition, parents and teachers participated both as intervention agents and raters of children's social-emotional functioning, indicating their reports might be subject to bias. Nevertheless, the correlations between parent- and teacher-rated variables provide some indication of the reliability of their responses. School counsellors who delivered EBSST were self-selected, suggesting they may have been motivated to participate in delivering the program. Thus, it is possible that the current findings may not generalise to schools where staff are less motivated to be trained in, and to deliver the intervention. However, as it is typical practice for school counsellors in NSW schools to self-select to attend training courses and to self-select to provide group interventions to children in the schools in which they work, the findings may well reflect ‘real world’ conditions. The return rate for parents (60–77%) and teachers (70–94%) in this study was higher than previous studies using parent or teacher report in ASD previous research in (Chandler et al., Reference Chandler, Charman, Baird, Simonoff, Loucas, Meldrum and Pickles2007; Laugeson, Frankel, Mogil, & Dillon, Reference Laugeson, Frankel, Mogil and Dillon2009). Children participated in EBSST during their regular school hours. The return rate therefore reflects teachers and parents returning questionnaires, rather than a loss of data due to children not completing the program, as children participated in EBSST during their regular school hours. School counsellors were required to collect data at pretreatment and posttreatment, as well as running student, teacher and parent groups in addition to their typical duties. Given the nature of the field trial and demands on the school counsellors, teachers and parents, the response rate was higher than might have been expected and may provide an indication of the motivation of the school counsellors, parents and teachers to participate in the research. It is unclear whether the teachers' and school counsellors' prior knowledge and experience in working with children with ASD and ID varied. Although school counsellors all attended a two-day accreditation and training course, and teachers were trained by accredited school counsellors, it is possible varying levels of experience may have influenced the delivery and uptake of the program and this should be measured in future studies.
Future efficacy studies employing gold standard methodology are now recommended in order to evaluate whether the EBSST for ASD + MID intervention package is effective for children, parents and their teachers under ideal clinic conditions with accredited EBSST practitioners. Future studies should include confirming participants' diagnoses through a more standardised and consistent process, random assignment and direct, blinded observational outcome measures. Monitoring participant attendance, and follow-up of participants over time would also be important to assess generalisability of effects over time.
Clinical Implications and Future Research
Research suggests children with ASD have benefited from social-emotional interventions (Kaat & Lecavalier, Reference Kaat and Lecavalier2014; Landa et al., Reference Landa, Holman, O'Neill and Stuart2011; Matson et al., Reference Matson, Matson and Rivet2007; McConnell, Reference McConnell2002; Reichow & Volkmar, Reference Reichow and Volkmar2010; Scattone, Reference Scattone2007), particularly group-based social interventions (Reichow & Volkmar, Reference Reichow and Volkmar2010). Although most research is conducted in university clinic settings (Kasari & Smith, Reference Kasari and Smith2013), some recent studies have evaluated school-based delivery of social-emotional interventions for children with ASD (Beaumont et al., Reference Beaumont, Rotolone and Sofronoff2015; Johnson, Reference Johnson2014; Laugeson & Park, Reference Laugeson and Park2014; Ratcliffe et al., Reference Ratcliffe, Wong, Dossetor and Hayes2014). However, there have been no previous controlled trials that have been specifically designed to assess the effectiveness of a group-based social-emotional intervention for children with ASD who also have co-occurring mild ID. Previous research has neglected this vulnerable population, despite the high co-occurrence of ASD and ID (Brereton et al., Reference Brereton, Tonge and Einfeld2006; Leyfer et al., Reference Leyfer, Folstein, Bacalman, Davis, Dinh, Morgan and Lainhart2006; Mattila et al., Reference Mattila, Hurtig, Haapsamo, Jussila, Kuusikko-Gauffin, Kielinen and Moilanen2010; Mazefsky et al., Reference Mazefsky, Conner and Oswald2010; Simonoff et al., Reference Simonoff, Jones, Baird, Pickles, Happe and Charman2013; Simonoff et al., Reference Simonoff, Pickles, Charman, Chandler, Loucas and Baird2008).
It is also feasible that EBSST for ASD + MID could be clinically effective for other populations. EBSST for ASD + MID is designed for verbal children with basic preliteracy skills at the level of about a typically developing 4- to 5-year-old child, who is not yet about to read and write. This implies children with ASD/no-ID aged 4–5 years, children aged 8–12 years with ID alone, adolescents with mild-moderate ID, and young children with a range of social communication difficulties could potentially benefit from the intervention delivered by schools counsellors at school.
In order to further increase generalisation to the home as well as the school setting, future research could explore options to increase parent engagement in EBSST; for example, providing parent sessions via webinar. In order to further increase generalisation, plans are currently underway to develop apps of visual worksheets to be held on students' personal technology devices (e.g., iPads) to enhance consistency and continuity of implementation of the EBSST skills across school, home and community environments. In addition, future studies could explore family factors, including parental mental health, as a potential mediating variable in parent informant outcomes. Future studies comparing EBSST to other social-emotion intervention programs used for school-age children with ASD and/or mild ID would also be of interest.
Conclusion
The current preliminary study evaluated a novel intervention package for children with ASD + MID. In this study, EBSST for ASD + MID was delivered in ‘real life’ school settings by trained school counsellors. Results indicated EBSST resulted in moderate improvements in parent and teacher ratings of children's emotional competence, compared to a waitlist control. This initial study provides valuable pilot data for future research exploring the utility of delivering EBSST as a group-based, social-emotional intervention for children with ASD and co-occurring mild ID in schools.
Author ORCIDs
Belinda Ratcliffe, 0000-0003-1586-8155
Acknowledgements
This research project was supported by an Australian Postgraduate Award (APA) PhD Scholarship and University of Sydney Merit Award to Dr Belinda Ratcliffe. The researchers would like to gratefully acknowledge and thank the Children's Hospital at Westmead and the University of Sydney for their support of this project. The researchers would also like to acknowledge and thank the NSW Department of Education and Communities for their partnership in this project and their support in organising the study and the time given by the school counsellors, teachers, the students and their parents. Special thanks to our Research Assistant Amy Talbot and to Dr Russell Thomson for assistance in conducting the statistical analysis.
Declaration of interest
The Children's Hospital at Westmead receives royalties from the sale of EBSST program materials (renamed the Westmead Feelings Program). A/Prof David Dossetor, Dr Michelle Wong and Dr Belinda Ratcliffe were employed by the Children's Hospital at Westmead at the time of the study and as authors of the Westmead Feelings Program, receive a share of royalties.
