Social communication skills are paramount to forming meaningful social relationships (Owens, Granader, Humphrey, & Baron-Cohen, Reference Owens, Granader, Humphrey and Baron-Cohen2008). Successful social interaction requires an array of skills, including identifying and appropriately using non-verbal responses such as eye contact, gesture, facial expression, voice tone, interpersonal space, starting and maintaining conversations (Spence, Reference Spence2003), as well as recognising emotions in others and adjusting responses accordingly (Barnhill & Myles, Reference Barnhill and Myles2001; McPartland & Klin, Reference McPartland and Klin2006). While typically developing children acquire these skills usually without the need for direct teaching, individuals with autism spectrum disorder (ASD) often find this a challenge. It is not uncommon for them to display inappropriate facial expressions, poor eye contact, and/or respond with limited gestures (Woodbury-Smith & Volkmar, Reference Woodbury-Smith and Volkmar2009). In addition, children with ASD may experience difficulties engaging in reciprocal interactions, accepting suggestions from their peers, and monitoring the effect their conversations and behaviours have on others (Attwood, Reference Attwood2000; Barry et al., Reference Barry, Klinger, Grofer, Joycelyn, Palardy, Gilmore and Bodin2003). Albeit unintentionally, they can also easily come across as insensitive due to their difficulties in offering comfort to others at times of stress or unhappiness (Klin & Volkmar, Reference Klin and Volkmar2003).
The impact of social skills impairment on children with ASD is extensive, especially among those with higher intellectual functioning (Klin, McPartland, & Volkmar, Reference Klin, McPartland, Volkmar, Volkmar, Paul, Klin and Cohen2005). These children are reported to have fewer friends than their typically developing peers (Koning & Magill-Evans, Reference Koning and Magill-Evans2001). They are stuck in a vicious cycle of social isolation — that is, they often desire close friendships with their peers, but fall short of the skills and experiences to form them (Bauminger, Reference Bauminger2002; Bauminger & Kasari, Reference Bauminger and Kasari2000). High-functioning children on the spectrum also demonstrate increased self-awareness of their social skills deficits (Knott, Dunlop, & Mackay, Reference Knott, Dunlop and Mackay2006). This self-awareness, coupled with repeated social failures, are reported to result in significantly higher levels of anxiety and depression among these children than their typically developing peers (Groden, Baron, & Groden, Reference Groden, Baron, Groden, Baron, Groden, Groden and Lipsitt2006; Lopata et al., Reference Lopata, Toomey, Fox, Volker, Chow, Thomeer and Shmerbeck2010). Their idiosyncratic behaviours can also easily make them targets of bullying and teasing (Little, Reference Little2001; Van-Roekel, Scholte, & Didden, Reference Van-Roekel, Scholte and Didden2009).
Intervention programs aimed at teaching social skills have been introduced to attenuate the enduring impact of these social skills deficits (Cappadocia & Weiss, Reference Cappadocia and Weiss2011). Frankel and colleagues (2010) demonstrated the efficacy of a 12-week Children's Friendship Training (CFT) social skills program for 76 high-functioning children aged 7–10 years with ASD. Target skills included conversational, play and negotiation skills. Post-intervention data revealed significant improvements on child measures of popularity, loneliness and parent measures of social skills. However, treatment gains did not generalise to the school setting, as indicated by teacher-report measures, and only parent-report improvements were maintained at 3-month follow-up. Cotugno (Reference Cotugno2009) evaluated the effectiveness of a 30-week social skills program for 18 children aged 7–11 years. Skill demonstrations, role-play exercises and ongoing feedback were provided to teach children the target skills of joint attention, anxiety and stress management, and coping with change. Although results at post-intervention demonstrated significant improvement across the targeted skills, the study was limited by the lack of a no-treatment ASD control group and the absence of follow-up data to assess the maintenance of treatment gains.
Despite the prevalence of social skills intervention for high-functioning children with ASD that demonstrate positive outcomes, investigations of the successful generalisation of skills outside treatment settings, the use of a non-treatment control group and/or follow-up data remains scarce. Rao, Beidel, and Murray (Reference Rao, Beidel and Murray2008) reviewed 10 social skills evaluation studies for high-functioning individuals with ASD. While seven out of the ten reported positive treatment outcomes, only three studies had included components to promote skills generalisation. Nonetheless, only one study (Barry et al., Reference Barry, Klinger, Grofer, Joycelyn, Palardy, Gilmore and Bodin2003) demonstrated partial generalisation, while the other two (Barnhill, Cook, Tebbenkamp, & Myles, Reference Barnhill, Cook, Tebbenkamp and Myles2002; Marriage, Gordon, & Brand, Reference Marriage, Gordon and Brand1995) reported disappointing results. Additionally, only one (Barry et al., Reference Barry, Klinger, Grofer, Joycelyn, Palardy, Gilmore and Bodin2003) out of the ten studies conducted follow-up assessment, albeit relatively short-term. Similar research gaps were also highlighted in a recent review by Flynn and Healy (Reference Flynn and Healy2012) where three studies (DeRosier, Swick, Davis, McMillen, & Matthew, Reference DeRosier, Swick, Davis, McMillen and Matthew2010; Dotson, Leaf, Sheldon, & Sherman, Reference Dotson, Leaf, Sheldon and Sherman2010; Kroeger, Schultz, & Newsom, Reference Kroeger, Schultz and Newsom2007) reviewed demonstrated improvements in participants’ social skills, but only one (Dotson et al., 2010) had collected generalisation and maintenance data. For this study, participants achieved only partial generalisation.
In view of this research gap, Beaumont and Sofronoff (Reference Beaumont and Sofronoff2008) evaluated a nine-session multi-component social skills program called the Secret Agent Society (SAS; formerly known as the Junior Detective Training Program) for high-functioning children with ASD aged 8–12. In this program, social skills learning is facilitated through a computer game, small group sessions, parent training sessions and teacher handouts. Results from the randomised controlled trial showed that participants who received the program had significant improvements in their social skills at home and in school, as indicated by parent- and teacher-report measures, with improvements maintained at 5-month follow-up. The program has since been made commercially available (see www.sst-institute.net for further details).
The Current Study
Originally designed as a small group-based intervention, SAS has also been tailored for individual delivery (Beaumont, Reference Beaumont2010). This tailoring was undertaken because of the challenges some practitioners face in recruiting groups of children for the program, as well as Australia's publicly funded health insurance scheme only funding interventions delivered either individually or in groups of six or more people, thus limiting the public availability of SAS. This pilot study therefore aimed to extend Beaumont and Sofronoff's (2008) research and evaluate whether individually delivered SAS would result in similar improvements in children's social-emotional skills. With the program adopting a family-based approach in its delivery, the study included children's siblings and one or more caregivers, to examine the impact the intervention had on their social-emotional skills.
Method
Participants
Three boys were selected from 21 families who expressed interest in the project. Selection was based on meeting the inclusion criteria; that is, they were required to: (a) have a paediatrician diagnosis of Asperger's disorder, high functioning autism or pervasive developmental disorder — not otherwise specified according to the Diagnostic and Statistical Manual — Fourth Edition, Text Revision (DSM-IV-TR; American Psychiatric Association, 2000); (b) have a Wechsler Abbreviated Scale of Intelligence — Second Edition (WASI-II; Wechsler, Reference Wechsler2011) pro-rated IQ score of 85 or above; (c) be aged 8–12 years at the time of intake assessment; (d) not be receiving any psychotherapy or social skills intervention during the course of study; and (e) have a typically developing sibling aged 8–12 years. Participants’ parents were also required to have no more than mild symptoms of depression, anxiety and stress on the Depression, Anxiety, Stress Scales 21 (DASS 21; Lovibond & Lovibond, Reference Lovibond and Lovibond1995) and a Parenting Scale total score of less than 3.2 (Arnold, O'Leary, Wolff, & Acker, Reference Arnold, O'Leary, Wolff and Acker1993). This was to ensure that parents had the emotional and parenting capacity to support their children throughout the program.
Participant A was an 11-year-old boy with superior IQ who tended to engage in conversations with people on topics of his own interest. His parents noted that he was often not aware of his listeners’ interest and would continue talking despite his audience disengaging. Appropriate eye contact was described as being limited except when he spoke about topics that he was interested in. At school, he had difficulty participating in group activities and maintaining conversations with his peers. Participant A also had a history of becoming easily upset over minor issues, but would calm down when hugged and listened to. His twin sister was not noted to have any significant developmental difficulties.
Participant B was an 11-year-old boy who enjoyed spending time with his schoolmates. Due to his ‘stickler for rules’ character, however, he would easily become agitated when they misbehaved. His parents also noted that he had difficulty maintaining conversations, often leaving out important contextual information but expecting others to understand and then getting angry when they did not. Participant B was more often angry at home than when out of the house. He would become aggressive when angry and required a few hours to calm down. Diagnosed with attention deficit hyperactive disorder (ADHD) at the age of 3, he was prescribed methylphenidate to help him stay more focused at school. Participant B shared a close relationship with his 12-year-old brother.
Participant C was an 8-year-old boy who enjoyed playing with others and engaging in conversations of his own interest. He received speech pathology intervention when he was younger and his speech improved significantly. However, his parents reported that he was unable to initiate or join in conversations and activities with his peers. Turn-taking and extending conversations to topics outside of his areas of interest were described as difficult. He also had difficulty recognising emotions in himself and others. Participant C had a 21-year-old brother and 10-year-old sister whom he enjoyed playing with.
Design
The study staggered the introduction of the intervention by 2 weeks across participants and included a follow-up assessment 6 weeks after each participant completed the intervention.
Measures
Developmental History Questionnaire
Parents were asked to complete the questionnaire, adapted from Beaumont (Reference Beaumont2010), at the intake assessment to ensure that their child met the inclusion criteria for the study.
Childhood Asperger Syndrome Test (CAST; Scott, Baron-Cohen, Bolton, & Brayne, Reference Scott, Baron-Cohen, Bolton and Brayne2002)
The CAST was used to confirm that participants met DSM-IV-TR criteria for Asperger's syndrome. This test has high sensitivity and specificity (Williams et al., Reference Williams, Scott, Stott, Allison, Bolton, Baron-Cohen and Brayne2005), and has been shown to be effective in screening children with Asperger's syndrome in the general population (Scott et al., 2002). The clinical cut-off score was 15 for possible Asperger's syndrome or related social communication difficulties.
Wechsler Abbreviated Scale of Intelligence—Second Edition (WASI-II; Wechsler, Reference Wechsler2011)
The WASI-II, which has high internal consistency and acceptable to excellent concurrent validity with the Wechsler Intelligence Scale for Children — Fourth Edition (WISC-IV), was administered to the children at the intake assessment to assess their intellectual functioning and ensure they met the inclusion criteria. The four-subtest form, which comprises versions of the Similarities, Vocabulary, Block Design and Matrix reasoning subtests of the WISC-IV, was used to generate an estimate of each child's full-scale IQ (FSIQ).
Depression, Anxiety, Stress Scales 21 (DASS 21; Lovibond & Lovibond, Reference Lovibond and Lovibond1995)
The DASS 21 is a 21-item self-report measure that assesses depression, anxiety and stress in adults. The DASS 21 is widely used with high internal consistencies across the depression, anxiety and stress scales (Henry & Crawford, Reference Henry and Crawford2005). The current study required parents to complete the DASS-21 to assess their emotional capacity to support their child through the program. To be eligible for the study, parents were required to score within the non-clinical range for the DASS-21, that is, no more than mild symptoms of depression, anxiety and stress (Lovibond & Lovibond, Reference Lovibond and Lovibond1995).
The Parenting Scale (PS; Arnold, O'Leary, Wolff, & Acker, Reference Arnold, O'Leary, Wolff and Acker1993)
The 30-item PS was employed to assess the extent to which parents display dysfunctional parenting styles across three categories: overreactivity, laxness and hostility. Each item is presented on a 7-point Likert scale anchored with one effective and one ineffective strategy (e.g., When my child misbehaves: ‘I raise my voice or yell’ vs. ‘I speak to my child calmly’). Parents rate their tendency to use these strategies in response to their child's misbehaviour. High scores on the PS are reflective of dysfunctional parenting styles. This measure is reported to have adequate internal consistency across its scales (Rhoades & O'Leary, Reference Rhoades and O'Leary2007), and was used to confirm the capacity of parents to support their children through the program.
Social Skills Questionnaires (SSQ) — Parent (SSQ-P) and Teacher (SSQ-T; (Spence, Reference Spence1995)
The SSQ-P and SSQ-T are rating scales for parents and teachers to evaluate children's social competence. The scales require respondents to rate the applicability of each of 30 statements in relation to a child's behaviour over the past 4 weeks. Both SSQ-P and SSQ-T have good internal consistency (Butterworth et al., Reference Butterworth, Hodge, Sofronoff, Beaumont, Gray, Roberts and Einfield2014) and have been used in previous studies (Beaumont & Sofronoff, Reference Beaumont and Sofronoff2008; Mackay, Knott, & Dunlop, Reference Mackay, Knott and Dunlop2007) to evaluate the efficacy of social skills interventions with children with AS. The current study used both SSQs to assess the social skills of the children, but only the SSQ-P was used for the siblings.
Emotion Regulation and Social Skills Questionnaire (ERSSQ) — Parent (ERSSQ-P) and Teacher (ERSSQ-T; Beaumont & Sofronoff, Reference Beaumont and Sofronoff2008; Butterworth et al., Reference Butterworth, Hodge, Sofronoff, Beaumont, Gray, Roberts and Einfield2014)
The ERSSQ-P and ERSSQ-T, designed to evaluate the emotion regulation and social skills of high-functioning children on the spectrum, require respondents to indicate on a 5-point scale the frequency with which children exhibit various social-emotional behaviours. The ERSSQ-P consists of 27 items and the ERSSQ-T, 25 items. Both scales have shown high internal consistency (Butterworth et al., Reference Butterworth, Hodge, Sofronoff, Beaumont, Gray, Roberts and Einfield2014) and were used in this study to evaluate participants’ improvements in the specific social-emotional skills taught in the SAS intervention. The ERSSQ-P was also administered to assess siblings’ social skills.
The Child and Adolescent Social Perception measure (CASP; Magill-Evans, Koning, Cameron-Sadava, & Manyk, Reference Magill-Evans, Koning, Cameron-Sadava and Manyk1995)
The CASP was used in this study to assess children's abilities to infer the emotional state of others using contextual and non-verbal cues such as facial expressions and body postures. The measure contains 10 unrelated videotaped scenes portraying common day-to-day social interactions. The audio tracks of the videotaped scenes are filtered to make the sound incomprehensible, but leave the tone and rate of speech as cues for the participants to describe the characters’ emotional states as well as provide reasons for their answers. Recordings are standardised across children and individually administered. Two sets of scores, the Emotion Score (ES), which is based on correct identification of the characters’ feelings, and the Non-Verbal Cues Score (NCS), which assesses the ability to accurately use the cues to infer emotions, were generated. Both ES and NCS have been shown to display good internal consistency (Koning, Magill-Evans, Volden, & Dick, 2011). The measure has been previously used by researchers with AS populations (Castorina & Negri, Reference Castorina and Negri2011; Koning et al., 2011).
James and the Maths Test (Attwood, Reference Attwood2004a)
This task was used to assess participants’ knowledge of anxiety management strategies. It involved the co-investigator reading aloud to each participant a scenario about James, a boy who felt anxious about completing a math test in class. Participants were instructed to suggest strategies to help James cope with his anxiety. Administration was standardised across the participants, with the co-investigator transcribing responses and later awarding a point for each appropriate strategy. Scoring was checked for interrater reliability using an independent rater blind to the study. Interrater agreement was within acceptable range (K = .81).
Dylan is Being Teased (Attwood, Reference Attwood2004b)
This task, which assesses participants’ knowledge of anger management strategies, involved a similar protocol to ‘James and the Maths Test’ but using a different scenario. Participants were asked to provide suggestions for how the story character, Dylan, could better manage bullying at school. Scoring procedures mirrored that for ‘James and the Maths Test’. Interrater agreement for the present study was within acceptable limits (K = .91)
Piers-Harris Self-Concept Scale (PHS; Piers, Reference Piers1984)
The PHS is an 80-item yes/no self-report measure that assesses how children and adolescents perceive themselves across six domains: physical appearance and attributes, intellectual and school status, happiness and satisfaction, behavioural adjustment, freedom from anxiety, and popularity. This scale provides individual scores for each domain and a composite score that reflects a child's overall self-concept. In this study, the 12-item popularity subscale was used to evaluate participants’ perceived popularity, including their ability to make friends and involvement in social activities. The popularity subscale is internally consistent (Piers, Reference Piers1984) and has been used previously to evaluate a social skills intervention for children with ASD (Frankel et al., Reference Frankel, Myatt, Sugar, Whitham, Gorospe and Laugeson2010).
Program Satisfaction Questionnaire — Parent and teacher forms (Beaumont, Reference Beaumont2010)
These questionnaires, specially designed to assess respondents’ views on the appropriateness and effectiveness of the SAS program, were completed by parents and teachers following completion of the intervention.
Procedure
Recruitment
Participants were recruited via a number of disability support groups within the Perth metropolitan area. Interested parents were provided the study information sheet and consent form. Upon receiving the parental consent, intake assessments consisting of the WASI-II, CAST, DASS-21 and PS were conducted to assess children's eligibility for the study. Three participants and their siblings who met the eligibility criteria were recruited for the study.
Intervention
Preintervention assessments were administered to each participant prior to the commencement of the study (pretest 1) and the introduction of the intervention delivered at staggered intervals (pretest 2). To ensure that the parents were aware of the requirements of the SAS program, an initial parent information session was conducted. Intervention commenced the week following the briefing by the author, who was trained to deliver the SAS program. The program of nine weekly 75-minute sessions started with the author spending the first 60 minutes of each session engaging the child and his sibling in various SAS activities.Footnote 1 Each session ended with a 15 minutes discussion with the parents to review the skills taught. Weekly tip sheets were also provided to the teachers via the parents to ensure the skills learnt in the program were reinforced in school.Footnote 2 Post-intervention assessments were administered after the conclusion of the intervention.
Follow up
Follow-up assessments were administered at 6 weeks’ post-intervention.
Treatment fidelity
To ensure treatment fidelity, the therapist completed session checklists indicating activities completed for each session. The overall mean percentage completion rate of session activities was 89%, with a range between 36% and 100%. All sessions were video-recorded and 20% of them were randomly checked by an independent observer to ensure accuracy of the checklist data. The reliability check found 96% agreement between raters.
Data analysis
Clinical significance analysis based on Jacobson and Traux (1991) was conducted to investigate participants’ pretest 2-post intervention and pretest 2-follow-up scores on each outcome variable. The analysis involved first assessing the statistical reliability of the change—that is, whether improvement in scores was greater than would be expected from measurement error. Change was defined as reliable when the reliable change index calculated as per formulae outlined by Jacobson and Traux (1991) was greater than 1.96. A test of clinical significance based on published normative data for the functional and clinical population was subsequently conducted to determine the clinical cut-off scores for each outcome measure. As per Jacobson and Truax's (1991) recommendations, where non-clinical norms were unavailable, a cut-off of two standard deviations above the clinical mean was used. Where clinical norms were unavailable, a cut-off of two standard deviations below the non-clinical mean was used. When both clinical and non-clinical norms were available, the cut-off would be the midpoint of the two aforementioned cut-offs. Clinically significant change was defined as: (a) a pretest 2 score in clinical range, (b) a post-intervention and/or a follow-up score in the non-clinical range, and (c) a reliable change score greater than 1.96.
Further analyses were conducted to evaluate participants’ overall performance across outcome measures. Two percentages were computed for each participant: (a) the percentage of measures with pretest 2 scores capable of showing reliable change that did show a reliable change, and (b) the percentage of measures with pretest 2 scores in the clinical range that evidenced a clinically significant change.
Results
Participants’ scores on the outcome measures at pretest 1, pretest 2, post-intervention and follow-up are presented in Table 1, and their overall performances across the outcome measures are presented in Table 2.
Psychometric Measures: Overall Results and Outcomes
Note: T1 = Pretest 1; T2 = Pretest 2; T3 = Post-intervention; T4 = 6-week follow-up; SSQ-P = Social Skills Questionnaire – Parent form; SSQ-T = Social Skills Questionnaire-Teacher form; ERSSQ-P = Emotion Regulation and Social Skills Questionnaire-Parent form; ERSSQ-T = Emotion Regulation and Social Skills Questionnaire-Teacher form; CASP-ES = Child and Adolescent Social Perception Measure- Emotion Score; CASP-NCS = Child and Adolescent Social Perception Measure-Non Verbal Cue Score; PHS = Piers-Harris Self-Concept Scale; ∆Statistically reliable change; ▲Statistically reliable and clinically significant improvement.
Percentage of Measures That Evidenced a Reliable Change and Clinically Significant Change
Note: Number of measures that showed reliable change and clinically significant change out of those possible in parentheses.
Social skills and emotion regulation measures: Parent report
Participants’ social and emotion regulation skills were assessed using the parent version of the SSQ and ERSSQ. Results at post-intervention indicated that all participants’ SSQ-P and ERSSQ-P scores were within the non-clinical range. Specifically, on both measures, Participant B showed clinically significant improvement, Participant C demonstrated reliable improvement while Participant A remained unchanged. Siblings’ scores on the SSQ-P and ERSSQ-P scores were generally unchanged and within non-clinical range, with the exception of Participant C's sister who showed reliable improvement on the SSQ-P. At follow-up, participants’ scores were maintained. Participant C's sister's SSQ-P, while still within the non-clinical range, no longer met the reliable change criteria.
Social skills and emotion regulation measures: Teacher report
Analysis of teacher data was performed for two of the three participants whose teachers returned completed forms at pre- and post-intervention. Results on the SSQ-T indicated that Participant A showed reliable change and Participant B remained unchanged. At post-intervention, both participants achieved SSQ-T scores within the non-clinical range. For the ERSSQ-T, both participants did not demonstrate a reliable change and scores remained within the clinical range. Follow-up data were not available as the assessment coincided with the school holidays.
Emotion recognition
Changes in participants’ emotion recognition skills were assessed using the CASP. Analysis of the CASP-ES's pre- and post-intervention scores revealed that only Participant A and B showed reliable improvement in successfully identifying the emotions of presented characters in the videos. However, these improvements were not clinically significant. At post-intervention, only Participant A reported CASP-ES scores within the non-clinical range. Follow-up data revealed that while Participant A's and B's CASP-ES scores remained within the non-clinical and clinical range respectively, their scores had dropped such that they no longer met the criteria for reliable change. For the CASP-NCS, Participant A demonstrated clinically significant improvement at post-intervention, whereas Participant B's and C's responses suggested no change in these skills. At follow-up, Participant A maintained his performance on CASP-NCS.
Knowledge of emotion management strategies
Pre- and post-intervention data from the Dylan is Being Teased measure revealed that Participant B showed clinically significant improvement in his knowledge of appropriate anger-management strategies. Participant C showed a reliable improvement but did not meet the criteria for clinical significance. Participant A's scores were unchanged and remained within the clinical range. At follow-up, Participant B maintained improvement while Participant C no longer met the reliable change criteria. For James and the Maths Test, none of the three participants recorded a reliable change in their pre–post knowledge of appropriate anxiety-management strategies. With the exception of Participant A, participants’ scores on the James measure remained within the clinical range at post-intervention. At follow-up, Participant B met the criteria for clinical significance while Participants A and C remained unchanged.
Self-report popularity measure
Results from the PHS indicated that none of the three participants demonstrated reliable change in their self-reported popularity at post-intervention and/or at follow-up. Nonetheless, post-intervention and follow-up scores across participants were all within the normal range.
Program Satisfaction Questionnaire
Overall feedback from the individually delivered SAS program was positive. Exemplars of narrative feedback by parents and teachers are provided below:
The program equipped the children with the ability to calm themselves and regulate their emotions. He is much better at recognising emotions in himself and others now. We also have definite tools to point them to now, like the 02 regulator. (Participant C's mum)
He tends not to get as upset or angry. He listens to others’ conversations and joins in. We are a little calmer. We have told him what behaviours we expect of him and reinforce them positively. (Participant B's mum)
His emotional development has been so important in the way he reacts to his peers and has been delightful to watch. It is great to see his social and emotional growth. I think the program has been very worthwhile for him. (Participant A's teacher)
Discussion
Two participants, Participants B and C, showed reliable change on the parent-report social skills and emotion regulation measures. Participant B, in particular, achieved clinical significance on both measures. Participant A's performance on these measures did not meet the criteria for either reliable or clinically significant change; this was likely due to a ceiling effect on these measures that rendered it difficult for improvement to be demonstrated. Participant A's post-intervention scores on both the parent-report measures were higher than Participant B, who met the criteria for clinical significance on the same measures. Furthermore, all participants were scoring within the non-clinical range at post-intervention on the parent-report measures. As predicted, participants’ performance on the parent-report measures of social skills and emotion regulation were maintained at 6-weeks’ follow up. This suggests that treatment gains were generally enduring, although a longer-term follow-up study would be required to verify this.
Teacher-reports of participants’ social skills and emotion regulation were mixed. Participant A demonstrated reliable change on the SSQ-T while Participant B's performance remained unchanged within the non-clinical range. The finding that Participant A had improved on the SSQ-T is particularly encouraging given that the SSQ was designed to assess important positive social interactions among typically developing children. Although it may be expected that this improvement would also be reflected on the ERSSQ-T, reliable change was not demonstrated on Participant A's ERSSQ-T. This may be partially due to the limited sensitivity of the SSQ to capture the subtle social difficulties often experienced by children with ASD relative to the ERSSQ-T, which was specifically developed to assess the social-emotion challenges that characterise children on the spectrum (Beaumont & Sofronoff, Reference Beaumont and Sofronoff2008; Butterworth et al., Reference Butterworth, Hodge, Sofronoff, Beaumont, Gray, Roberts and Einfield2014). Nonetheless, Participant A's pre–post ERSSQ-T scores were trending towards a reliable change, and teacher feedback on the Program Satisfaction Questionnaire highlighted his social and emotional growth (i.e., ‘Participant A has become more interactive with his peers and has been able to deal with mistakes he makes better than previously’). At post-intervention, Participant B's scores on both SSQ-T and ERSSQ-T had decreased, albeit the change was not considered reliable. It should be noted that parents and teacher of Participant B reported an extremely stressful period for him nearing the end of the intervention phase, having classmates who were exceptionally disruptive and misbehaving at school. Caution must therefore be exercised when interpreting these scores, as external factors are likely to influence one's ability to interact and regulate their emotions in social situations. As such, no conclusive interpretations can be drawn about the generalisation of participants’ social-emotional skills to the school environment. Follow-up data from teachers was also unavailable, as the assessment coincided with the school holidays.
Despite having had high scores at pretest, one sibling continued to demonstrate reliable change on the SSQ. At follow-up, the sibling's score was no longer reliably greater than at pretreatment (although it remained near perfect). As for the other two siblings, near perfect scores reported at preintervention may have significantly reduced their potential to show a reliable improvement.
Participant A demonstrated reliable change on the CASP-ES and clinically significant improvement on the CASP-NCS. The latter was maintained at 6 weeks’ follow-up, but his CASP-ES scores had slipped, albeit remaining within the non-clinical range. This suggests that Participant A's ability to attend to non-verbal cues in emotion recognition was within the performance of his typically developing peers by the conclusion of the intervention and at follow-up. While Participant B's scores on the CASP-NCS were unchanged, reliable pre- to post-intervention improvement was noted on the CASP-ES, although this was not maintained at follow-up. Caution is warranted when interpreting these findings, as Participant B was avoidant of the CASP task after having completed it several times over the course of the study. Arguably, this lack of motivation may have impaired his CASP performance. There was no reliable change in Participant C's CASP post-intervention and follow-up scores relative to preintervention. This is likely due to the discrepancy between the emotions focused on in the intervention with participant C — that is, more emphasis on simple as opposed to complex emotions and those assessed in the CASP (i.e., more complex emotions such as those with incongruent situational and emotional cues). Alternate emotion recognition measures such as The Cambridge Mindreading Face–Voice Battery for Children (Golan & Baron-Cohen, Reference Golan and Baron-Cohen2006), which assess both basic and complex emotions, are recommended for future studies. Nevertheless, feedback provided by his parents on the Program Satisfaction Questionnaire indicated that Participant C had developed better emotion-recognition skills, although the change was not captured on the CASP.
On measures of knowledge of emotion management strategies, post-intervention results indicated that Participant B and C demonstrated reliable improvement on the Dylan but not on the James measure. Specifically, Participant B's performance on the Dylan measure improved from the clinical to the normal range. The finding that both participants demonstrated improvements in their knowledge of appropriate anger but not anxiety management strategies was particularly interesting. This may be attributed to the fact that Participants B and C each identified themselves as angry rather than anxious children. As such, they may have had more opportunities to review and practise the anger-management (as opposed to anxiety-management) strategies taught in the program in everyday contexts, resulting in higher scores on the anger-management as opposed to the anxiety-management measure. At follow-up, Participant B maintained improvement on the Dylan measure while Participant C no longer met the criteria for reliable change. However, it should be noted that Participant C received most of the intervention during the school holidays. In the absence of opportunities at school for skills rehearsal and application, it is not surprising that Participant C did not maintain treatment gains at follow-up. Importantly, on the James measure, Participant B showed clinically significant improvement at follow-up. This is consistent with results on the Dylan measure and parent feedback on the Program Satisfaction Questionnaire, which noted improvements in his knowledge of emotions regulation strategies and ability to stay calm. For Participant A, performance on the James and Dylan measures was unchanged throughout the study. It is possible that this was due to him having high scores on both measures at pretest relative to Participants B and C, and hence having less room to show reliable improvement.
Post-test and follow-up scores of self-reported popularity for all participants remained unchanged. While previous studies have reported that children with ASD have a lower self-concept (Capps, Sigman, & Yirmiya, Reference Capps, Sigman and Yirmiya1995), the observed high scores at preintervention, post-intervention and at follow-up instead suggest that participants in the current study either had high quality friendships and positive self-identity (Bauminger, Shulman, & Agam, Reference Bauminger, Shulman and Agam2004), or a lack of awareness of their social and/or friendship challenges.
Limitation and Future Directions
Several methodological limitations of the current study must be acknowledged. First, as with studies with small N designs and only male participants, replication with a larger sample and female participants is warranted to ensure that current results can be generalised to a larger population. Second, verbal reports and participants’ behaviour suggested that repeat administration of measures at pretest 1, pretest 2, post-intervention and at follow-up affected participants’ motivation to complete the tasks. Since motivation is essential to ensure optimal performance, it is recommended that future studies limit test administration to a pre- and post-assessment. Third, medication use, as a confounding factor, was not monitored throughout the course of this study. This limitation should be addressed in future evaluation trials to ensure participants maintain a stable dose of medication use during the study. In addition, it must be acknowledged that changes were made to the protocol for one participant (i.e., sessions being collapsed across time and conducted over the holidays) and data was also not collected on whether teachers received the weekly tip sheets from parents. Although these factors reflect ‘real-world’ practice, they may have limited the potential impact of the intervention.
Future research should also address other limitations of this study; specifically, with the delivery and choice of measures used. This includes the reliance on parent and teachers’ reports, whose responses may have been positively biased by expectations of improvements, given that they were not blind to the study. The inclusion of behavioural observations as an unbiased measure would allow for more rigorous evaluation of treatment outcome. Although the current study had screened participants’ diagnoses using the CAST, the use of gold standard screening tools such as the ADOS and ADI-R would have been much preferred. This was not possible due to the cost and limited availability of training in such measures. Additionally, it is recommended for the entirety of the intervention to be conducted within the school term, not only to allow for more accurate assessment of participants’ own self-report popularity among peers, but to also enable follow-up assessments with the teachers. The absence of self-report emotion regulation in the current study should also be acknowledged as a limitation. This did not occur as research has shown that limited self-awareness among higher-functioning individuals with ASD (Mazefsky, Kao, & Oswald, Reference Mazefsky, Kao and Oswald2011) could compromise the test validity of self-report measures. Despite these limitations, the use of multi-method approach to assessing participants’ social skills and emotional regulation highlights the rigorous evaluation adopted by the current study.
Conclusion and Clinical Implications
Notwithstanding the limitations highlighted above, this exploratory study provided preliminary support for the effectiveness of the SAS program delivered individually. All three participants who completed the individually delivered SAS program showed significant improvements on half of the measures assessing social-emotional competence. Two of the participants demonstrated improvements to within the range of their typically developing peers on a third of these measures. Follow-up results suggested that improvements were maintained at the 6-week post-intervention. This is an important finding for both professionals and families, given there have previously been no formal evaluations of the program delivered in this format. Finally, with the practical ease of delivering services individually versus in a group format, the current findings are an important first step in extending the reach of the program to families with an ASD child who requires social skills training.
Acknowledgments
This research was supported in part by a grant awarded to Trevor Mazzucchelli and Lynette Tan from the School of Psychology and Speech Pathology, Research Allocation Fund RAF-2013-03. Practitioner and parent resources, as well as practitioner training, were provided by the Social Skills Training Institute, a division of Triple P International Pty Ltd. We thank Amy Hamilton, Yeow May Tan and Nick Buckley for assisting in scoring the test protocols, as well as Theresa Kidd and Josette Hamilton for helping with the recruitment of participants.
Conflict of Interest Statement
The third author of this article, Dr Renae Beaumont, is the developer of the Secret Agent Society social-emotional skills program, and receives royalties on all program materials sold.
