Introduction
Externalizing behaviours, such as disobedience, aggression and lying, are the most common form of maladjustment in school aged children (Dishion and Patterson, Reference Dishion, Patterson, Cohen and Cicchetti2006). Whereas the normative level of aggressive behaviour declines when children are between 4 and 9 years old (Campbell, Spieker, Burchinal and Poe, Reference Campbell, Spieker, Burchinal and Poe2006), for a small group of children (5% to 11%) the aggressive behaviour remains stable and becomes problematic (Prinzie, Onghena and Hellinckx, Reference Prinzie, Onghena and Hellinckx2005). This is especially true for children with elevated aggression at the start of elementary school (Broidy et al., Reference Broidy, Nagin, Tremblay, Bates, Brame and Dodge2003). Several developmental trajectory studies indicate that stable aggressive behaviour in elementary school places children at risk for future difficulties such as rejection by peers, school failure, and more serious externalizing problems (Moffit, Reference Moffit1993; Patterson, Reid and Dishion, Reference Patterson, Reid and Dishion1992). Over time, these children also have increased risk of developing diagnosable psychopathology (Oppositional Defiant Disorder and Conduct Disorder), substance abuse, and delinquency in adolescence (Broidy et al., Reference Broidy, Nagin, Tremblay, Bates, Brame and Dodge2003). Moreover, in an 18-year longitudinal study of 10-year-old children, it has been demonstrated that costs to society are 10 times higher for children with elevated levels of externalizing behaviour compared to children without elevated levels of externalizing behaviour (Scott, Knapp, Henderson and Maughan, Reference Scott, Knapp, Henderson and Maughan2001). Given these findings, there is a strong need for effective preventive interventions designed to interrupt the developmental trajectory towards more serious behaviour problems for elementary school-children with aggressive behaviour problems (Buckley, Reference Buckley2009).
The aim of the present study was to test the effectiveness of an individual school-based preventive cognitive behavioural training (Stay Cool Kids) to reduce aggressive behaviour among indicated elementary school children. The intervention was originally developed in real world practice and is routinely used in clinical practice. However, its effectiveness has not yet been examined. Therefore, we aimed to study the effectiveness of this program as actually delivered in daily practice.
Much research has been dedicated to understanding the etiology of aggression (e.g. Bandura, Reference Bandura1973; Patterson, Reference Patterson, Reid, Patterson and Snyder2002). One of the well grounded models explaining the development and persistence of aggressive behaviour is the social information processing model, based on social cognitive theory (Dodge, Reference Dodge and Perlmutter1986). According to this model aggressive children have problems in processing information, which lead to inappropriate responses in social settings.
Stay Cool Kids aims to prevent externalizing behaviour problems by targeting problems in social information processing. In this model behaviour is seen as a result of six mental steps: (1) encoding of cues; (2) interpretation of these cues; (3) clarification of goals; (4) response access or construction; (5) response decision; and (6) enacting selected responses. A set of internal social schemas of others and self, derived from individual experiences, is the knowledge base for the social information processing steps (Lemerise and Arsenio, Reference Lemerise and Arsenio2000). It has been suggested that children with aggressive behaviour show problems in their social information processing patterns (Crick and Dodge, Reference Crick and Dodge1994; Dodge, Reference Dodge and Perlmutter1986). For example, at the encoding stage, aggressive children attend to other cues from social situations (Horsley, de Castro and van der Schoot, Reference Horsley, de Castro and van der Schoot2010). When making interpretations, they over attribute hostile intentions to peers and adults when provoked (de Castro, Veerman, Koops, Bosch and Monshouwer, Reference De Castro, Veerman, Koops, Bosch and Monshouwer2002). They consequently hold more instrumental and less affiliative goals (Salmivalli, Kaukianinen, Kaistanieme and Lagerspetz, Reference Salmivalli, Kaukiainen, Kaistaniemi and Lagerspetz1999), generate less prosocial solutions and are more likely to opt for an aggressive solution (Matthys and Lochman, Reference Matthys, Lochman, McMurran and McGuire2005). More specifically, children showing more reactive aggression (which can be seen as an angry, emotional reaction to a presumed threat or provocation) specifically make more hostile attributions, whereas proactive aggressive children (planned “cold blooded” aggression to dominate or intimidate) evaluate aggressive responses more positively (Crick and Dodge, Reference Crick and Dodge1994; de Castro, Merk, Koops, Veerman and Bosch, Reference De Castro, Merk, Koops, Veerman and Bosch2005; Dodge, Reference Dodge, Pepler and Rubin1991). In addition to these aspects of social information processing, aggressive children seem to differ from their peers in social cognitive schemas, notably in their self-perception. From a social learning theory perspective (Bandura, Reference Bandura1973), lasting experiences of coercive parenting, conflicts, and rejection by peers can make children uncertain of their worth and competence. This is reflected in highly variable self-esteem and self-perceived social competence in aggressive children. Apparently, many aggressive children with such uncertain self-views feel they need to maintain a facade of high status to others (and perhaps even themselves). When their competence or worth is challenged these children try to defend their uncertain self-regard from external threats (e.g. Bushman et al., Reference Bushman, Baumeister, Thomaes, Ryu, Begeer and West2009). Thus, uncertainty about one's worth or competencies can lead to perceiving others as threatening, hostile and rejecting, which in turn can cause hostile, defensive and aggressive behaviour (de Castro, Brendgen, van Boxtel, Vitaro and Schaepers, Reference De Castro, Brendgen, van Boxtel, Vitaro and Schaepers2007). These behaviours supposedly lead to rejection by others, which confirm the child's uncertainty about himself (Donnellan, Trzesniewski, Robins, Moffit and Caspi, Reference Donnellan, Trzesniewski, Robins, Moffitt and Caspi2005). Especially proactive aggression in children has been suggested to be related to distorted self-perceptions (Salmivalli, Reference Salmivalli2001).
Interventions based on the social information processing model (e.g. Brainpower program; Hudley and Graham, Reference Hudley and Graham1993; Coping Power Program, Lochman and Wells, Reference Lochman and Wells2002a) have several general characteristics. They explicitly provide training on one or more of the social information processing steps (Crick and Dodge, Reference Crick and Dodge1994). Trainers put emphasis on cognitive skills or thinking processes, and they make use of structured tasks to teach children cognitive skills that can be applied in social situations (Wilson and Lipsey, Reference Wilson and Lipsey2006). These social information processing interventions are effective (d = .26) in reducing aggressive behaviour of school-age children who show increased levels of externalizing behaviour (k = 47 studies, Wilson and Lipsey, Reference Wilson and Lipsey2006). However, large differences in effect sizes across studies were found, which can be explained by the considerable variability among intervention programs. Interventions for children with externalizing behaviour were found to be most effective when they focused on higher risk children in regular education (Wilson and Lipsey, Reference Wilson and Lipsey2006), individually delivered and better implemented (Wilson and Lipsey, Reference Wilson and Lipsey2007).
The Stay Cool Kids program includes a number of the characteristics that have been suggested to be most promising. First, it is a school-based intervention, aimed at children displaying a (sub) clinical level of aggression according to their teachers, within regular education. Schools are excellent locations for prevention activities as children can be reached in a systematic and efficient way. Moreover, stable aggressive behaviours start to impair social and academic functioning at school age, and since school is the most common setting for social interaction, it is also the place where much interpersonal aggression among children can occur. Offering prevention activities in a school setting may make treatment more acceptable to parents (Catron and Weiss, Reference Catron and Weiss1994) and there is no dependence on parents remembering their child's appointments. Besides, locating the intervention at school increases the likelihood of generalization and maintenance of treatment effects to the natural environment (Evans, Langberg and Williams, Reference Evans, Langberg, Williams, Weist, Evans and Lever2003). School-based interventions for children indicated with externalizing behaviour problems are in general effective (k = 108 studies, d = .29; Wilson and Lipsey, Reference Wilson and Lipsey2007).
Second, Stay Cool Kids is an indicated-type intervention that targets children at higher risk for developing more serious problem behaviours as a consequence of early onset aggressive behaviour. Interventions can be distinguished into three categories: universal, selective, and indicated preventive interventions (Saxena, Jané-Llopis and Hosman, Reference Saxena, Jané-Llopis and Hosman2006). Universal preventive interventions are offered to a whole population, not identified because of elevated levels of problem behaviour. For example, the KiVa anti-bullying intervention, in which lessons to prevent bullying in schools are offered to all students in a school (Salmivalli, Kärnä and Poskiparta, Reference Salmivalli, Kärnä and Poskiparta2011). Selective prevention is for those individuals or subgroups at higher risk for developing psychosocial problems, because of biological, psychological or social risk factors. For instance, the Strengthening Families Program, to prevent behaviour problems in children living in high-risk families (Kumpfer and Alvarado, Reference Kumpfer, Alvarado, Botvin, Schinke and Orlandi1995). Finally, indicated prevention is for at-risk individuals who are displaying symptoms of behavioural disorders, but who do not yet meet diagnostic criteria. For example, the Coping Power Program, for children with elevated levels of externalizing behaviour (Lochman and Wells, Reference Lochman and Wells2002b). School-based interventions exist at these three different levels (Reinke, Splett, Robeson and Offutt, Reference Reinke, Splett, Robeson and Offutt2009; Walker et al., Reference Walker, Horner, Sugai, Bullis, Sprague and Bricker1996). School-based universal interventions are often “curriculum” interventions, in which lessons that focus on reducing aggression or violence in schools can be given to a whole class or school population (e.g. Promoting Alternative Thinking Strategies; Greenberg, Kusche, Cook and Quamma, Reference Greenberg, Kusche, Cook and Quamma1995). Children at higher risk might not respond to universal interventions, so for these children selective preventive interventions can be offered (e.g. Primary Mental Health Project; Cowen et al., Reference Cowen, Hightower, Pedro-Carroll, Work, Wyman and Haffey1996). For a small group of children, displaying more severe problems, indicated or targeted interventions, as Stay Cool Kids, might be more beneficial (Gottfredson and Wilson, Reference Gottfredson and Wilson2003). An important advantage of targeted interventions, in contrast to universal and selective interventions, is that they can be adapted for individual children in need of more than a standard curriculum (Gottfredson and Wilson, Reference Gottfredson and Wilson2003).
Third, Stay Cool Kids is an individual intervention and therefore may prevent children from exposure to deviancy training. Most of the targeted interventions for aggressive behaviour take place in groups of selected children, although several studies (e.g., Dishion, McCord and Poulin, Reference Dishion, McCord and Poulin1999) indicated that peer-group interventions may increase problem behaviour, especially for high-risk youth. In addition, it is more difficult to focus on specific needs and difficulties of an individual child in group interventions. However, in a meta-analysis on the effectiveness of individually delivered indicated school-based interventions for externalizing behaviour problems at elementary schools we found that only 24 studies evaluated effects in the last 35 years (k = 24 studies, d = .30; Stoltz, van Londen, Deković, de Castro and Prinzie, in press). Stay Cool Kids is not just an individually delivered intervention, it is also an individualized (adaptive) intervention, which means that it can be adapted to particular individual differences, for example to cultural values or gender of the child. It has been recommended that an emphasis should be placed on tailoring interventions to specific youth (Frick, Reference Frick2000; La Greca, Silverman and Lochman, Reference La Greca, Silverman and Lochman2009).
Recently, it has been found that implementation of an intervention in schools is highly dependent on the skills of school staff, funding, and the capacity to provide coaches for the program (Loman, Rodriguez and Horner, Reference Loman, Rodriguez and Horner2010). A unique aspect of the Stay Cool Kids program is that it is provided to children in the school setting by professional mental health care workers, who work intensively together with school staff. It has been suggested that prevention programs offered by trained interventionists are more effective than programs delivered by classroom teachers (Baranowski, Cullen, Nicklas, Thompson and Baranowski, Reference Baranowski, Cullen, Nicklas, Thompson and Baranowski2002), because of more and higher quality training (Gottfredson and Gottfredson, Reference Gottfredson and Gottfredson2002). Moreover, indicated individual interventions, as Stay Cool Kids, are time-consuming and teachers may not be able to devote as much time to providing one-to-one interventions. Additionally, teachers often have their own “history” with a child displaying elevated levels of externalizing behaviour, since these children disrupt classroom routines, detract from learning opportunities, and have negative influences on classmates (McConaughy and Skiba, Reference McConaughy and Skiba1993). Therefore, it might be beneficial to let an independent trainer conduct the intervention.
Finally, to make sure that a program is effective in changing child behaviour, it is important to involve the context as well (e.g. Incredible Years; Reid and Webster-Stratton, Reference Reid and Webster-Stratton2001). In the Stay Cool Kids training parents and teachers are involved with the goals of helping the child with generalization of new learned skills and behaviour to the school and home setting.
The present study extends previous intervention studies in several ways. First, it focuses on an intervention that includes several specific promising characteristics: it is an indicated-type school-based intervention, delivered by trained professionals on an individual basis that still involves the context. Second, in contrast to studies concerning optimally implemented “demonstration programs” for research purposes, the present study focuses on a routinely delivered program in daily practice, and therefore can give information about effectiveness under real-world conditions (e.g. Hautmann et al., Reference Hautmann, Hoijtink, Eichelberger, Hanisch, Plück and Walter2009; Kratochwill and Shernoff, Reference Kratochwill and Shernoff2004). Third, in addition to general effectiveness, we also examined for whom the intervention works best by including gender and ethnic background as moderators. Little is known about possible ethnic differences in response to the intervention, despite the inclusion of ethnic minority families in intervention study samples (Yasui and Dishion, Reference Yasui and Dishion2007). As a result of different parental socialization practices, cultural values and traditions, possible negative experiences with discrimination, and experienced challenges of the acculturative process, it is important to include ethnicity as moderator (Kellam and van Horn, Reference Kellam and van Horn1997; Yasui and Dishion, Reference Yasui and Dishion2007). Moreover, despite numerous studies showing differences in manifestations of aggression for boys and girls (e.g. Crick and Grotpeter, Reference Crick and Grotpeter1995), gender differences in intervention effects are less clear (Brestan and Eyberg, Reference Brestan and Eyberg1998). Fourth, multiple informants (children, mothers, fathers and teachers) and multiple types of outcome behaviours (reactive and proactive aggression, social cognitions and self-perception) are included to provide a comprehensive view of changes in child behaviour after the intervention (Achenbach, McConaughy and Howell, Reference Achenbach, McConaughy and Howell1987; Grietens et al., Reference Grietens, Onghena, Prinzie, Gadeyne, Van Assche and Ghesquière2004). Primary outcome variables are reactive and proactive aggression. It has been suggested that these two types of aggression require different interventions (Vitaro, Brendgen and Tremblay, Reference Vitaro, Brendgen and Tremblay2002). We expect Stay Cool Kids to affect reactive aggression, since the training provides exercises on modifying encoding and interpretation of social cues and generating responses to social provocations. We also expect changes in proactive aggression, since the training focuses on modifying choosing responses to react and enacting behaviour. Because of the social cognitive basis of this intervention, we examine in addition to aggressive behaviour changes in child self-perception, hostile intent attribution, response generation and response evaluation and we expect that children will have more optimal levels of self-perception, will attribute less hostile intent, and will be more likely to select prosocial solutions.
Finally, clinically relevant changes are examined in addition to intervention effects on mean levels of outcome (e.g. Scott, Reference Scott2005). Clinically relevant changes in every day life, in addition to the classical approach of evaluating intervention effects, are rarely tested in intervention studies (Kendall, Reference Kendall1999). Therefore, in the present study we investigated for how many children externalizing behaviour actually changed from a (sub) clinical level to a normative level (i.e. recovery; Ogles, Lunnen and Bonesteel, Reference Ogles, Lunnen and Bonesteel2001).
Method
Design
Figure 1 shows the randomization process and participant flow. Forty-eight elementary schools (i.e. 4-th grade classrooms), in two urban regions in the Netherlands, were randomly assigned to one of three groups. Each group of schools participated in the intervention as well as in the control condition, but in a different order. In group 1, schools were assigned to the intervention condition in the first and second year and to the control condition in the third year. In group 2, schools received the intervention in the first year, were in the control condition in the second year, and again in the intervention condition in the third year. In group 3, schools were assigned to the control condition in the first year, and were in the intervention condition in the second and third year. Thus, treatment condition was randomized, and each school provided both intervention and control condition children. In this way we ensured that intervention effects could not be due to school factors, because the same schools were in both conditions. Moreover, schools were more willing to participate in the control condition when they were assured of receiving 2 years of training. In different years, different teachers selected children. However, in some schools the same teachers selected children for the 3 years of the study. There were two assessment periods: prior to the beginning of the intervention (T1), and at intervention termination (after 11 weeks, T2). The study was approved by the Dutch Central Committee on Research Involving Human Subjects.
Flow chart of randomization design
To control for the design effect (randomization at school level could lead to biased standard errors), we computed the design effect as recommended by Muthén (Reference Muthén, Rose, Chassin, Presson and Sherman2000), which is expresses as d = 1 + ρ (c – 1), where ρ is the average intraclass correlation (.18) and c is the common cluster size (i.e. the average number of children per school, 4.2). The design effect was 1.58, which is smaller than 2.0 and can therefore be ignored (Muthén, Reference Muthén, Rose, Chassin, Presson and Sherman2000).
Procedure
First, parents of all children in fourth grade received a general information letter about the study and a consent form to give permission for teachers to fill out the Teacher Report Form (TRF; age 6–18; Achenbach, Reference Achenbach1991, see Measures) about their child. Teachers then nominated children from their classes with the highest levels of externalizing behaviour (the top 30%) and filled out the 32-item externalizing scale of the TRF. Next, researchers selected children based on their T-scores. The inclusion criterion was a T-score > 60, indicating a (sub) clinical level of externalizing behaviour, which places children at risk for various negative outcomes. Other studies used similar criterion as high-risk indicators in prevention studies with elementary school-aged children (Lochman and CPPRG, Reference Lochman1995). If a child was selected, primary caretakers were contacted by phone and in meetings at school to gather their informed consent to participate in this study. After consent was obtained, the baseline pre-assessment was conducted. Children's measures were collected in their school settings and were administered to children by trained research assistants. Parents received questionnaires in meetings at school or by mail. They were asked to return the questionnaires within a week and were reminded by teachers and research staff if needed. Children received a small gift for their participation. Parents and teachers received a small monetary reimbursement for their time.
Participants
Teachers initially selected 437 children. Of these selected children, some children did not meet the inclusion criteria (n = 96), some parents refused to participate (n = 52), some children already participated in other forms of youth care (n = 15) or were diagnosed with Autism Spectrum Disorder (n = 3) (see Figure 1). Three children discontinued the intervention, four children discontinued in the control condition. Children in the control condition did not receive any intervention or received care as usual (e.g. remedial teaching, universal remediation programs). Moreover, the Stay Cool Kids intervention is offered to children in the control condition after the final assessment when still necessary (a year after treatment termination).
There were no significant differences in child gender or TRF score between parents who consented to participate and those who did not. Children in the intervention and control group did not differ significantly on demographic characteristics at Time 1.
The final sample consisted of 264 children (n = 191 boys, n = 73 girls), 197 mothers, 130 fathers, and 155 teachers at Time 1. For 114 children both parents filled in the questionnaire at Time 1. For 83 children only mothers, and for 14 children only fathers filled in the questionnaire at Time 1. As can be seen in Table 1, most children (76%) are from double-parent families. During the 3-year period most teachers (n = 79) had only one child receiving the intervention in their class, whereas some teachers had two (n = 54), three (n = 19), or four (n = 5) children from their classes being selected by researchers for the intervention. As a result, for 185 children another classmate also participated in the same condition, but children always participated individually. Table 1 lists demographic characteristics for the intervention (n = 191) and control (n = 73) group.
Sample characteristics by condition
Consistent with other studies on immigrant groups in the Netherlands (Eichelsheim et al., Reference Eichelsheim, Buist, Deković, Wissink, Frijns and van Lier2009), a child was considered “immigrant” if either the child or one of the parents was born in another country than the Netherlands. If both child and parents were born in the Netherlands, the child was considered “native Dutch”. In total, 27% of the participating children were immigrants.
At Time 2, 99% of the children in the intervention actually had completed the intervention. All children who completed the intervention filled out the questionnaires at Time 2 (n = 264). For 24 children (9%), teachers did not complete the questionnaire at Time 2, because of lack of time or personal circumstances. These missing values at Time 2 were completely random (Little's MCAR test: χ2 /2 = .05, p = .95). For both mothers (T1 n = 67, T2 n = 93) and fathers (T1 n = 134, T2 n = 164) missings were also completely random (Mothers: Little's MCAR test: χ2 / 7 = .99, p = .44; Fathers: Little's MCAR test χ2 / 4 = .85, p = .49). We did not use listwise deletion as it can result in discarding a large proportion of the data, which can lead to biased results, because the power of the study decreases. Multiple imputation is currently recommended as a modern missing data handling technique (Baraldi and Enders, Reference Baraldi and Enders2010). We used Multiple Imputation techniques module of LISREL8.7 with the Expected Maximization (EM) algorithm. Imputed data were used in further analyses.
Intervention
The Stay Cool Kids training is a social cognitive intervention that focuses on reciprocal relations between cognitions, emotions, and behaviour of the child. The training is designed to reduce aggressive behaviour in highly aggressive children at elementary schools. The long term goal is to prevent the development of disruptive behaviour disorders. Trainers worked individually, in 8 weekly sessions of 45 minutes, with the target child. Children were seen during the school day from January until March. The trainer met with parents and teachers before the start of the training, during a mid term evaluation, and at the end of the training.
The training consisted of two phases. In the first phase, trainers investigated child's specific needs and competences. The first session starts with a general introduction, which is the same for all children. Next, trainers were able to choose two from six exercises that were best suited for the individual child, for the second and third session. After the third session, an individual analysis of child's competences was made and discussed with parents and teachers during a midterm evaluation, resulting in an individual intervention plan. For the intervention plan trainers chose five from nine program components that were most appropriate for the individual child's needs, as described in the trainer manual. Before phase 2 (session 4–8) started, a contract between the trainer and child was signed, in which the training program is described. Exercises focused on: 1) self-perception (less negative, realistic self-perception); 2) social cognitions (attribution of benign intent in ambiguous situations, accurate representation of other children emotions); 3) anger management (emotion-regulations strategies, e.g. “stop-think-act”); and 4) aggressive behaviour (generation of less aggressive responses to social provocations). Parents and teachers received information after each training session about what was done during the training. Also, they were asked to practise together with the child his or her newly learned skills.Footnote 1
Treatment fidelity
In this study, an existing frequently used implemented intervention is evaluated and participating Stay Cool Kids trainers from youth mental health care centres (n = 33) are typical providers of the intervention within these schools. Trainers have a background in clinical child psychology, and have to be certified as a Stay Cool Kids trainer. To become certified, trainers have to run three pilot training sessions, under supervision from accredited Stay Cool Kids trainers. These initial pilot training was prior to implementing the program, and these trained children were not included in the study. During the intervention period for this study trainers had two-weekly meetings to discuss training and get feedback from other trainers and supervisors. Trainers filled in logs after every intervention session. Logs completed by the trainers indicated that 99% of the training sessions were completed as planned. The average training session lasted 45 minutes. Trainers changed the content of their training session in 6.5% of the cases, when the planned training session did not work for the specific child. Overall, it was found that Stay Cool Kids was conducted as intended.
Parents and teachers satisfaction with program
Parents and teachers in the intervention condition were asked to fill in a short questionnaire at Time 2 to assess the level of satisfaction with the program. Overall, parents and teachers were satisfied with the program at post test (scale 0–6; Mteachers = 4.4 (1.24), Mmothers = 4.6 (1.35), Mfathers = 4.6 (1.18)).
Measures
Child aggressive behaviour
Reactive and proactive aggression according to teachers were measured with the Teacher Rating of Aggression (TRA; Dodge and Coie, Reference Dodge and Coie1987; Dutch version; Hendrickx, Crombez, Roeyers and de Castro, Reference Hendrickx, Crombez, Roeyers and de Castro2003). Items for both reactive aggression (e.g. “When this child has been teased or threatened, he or she gets angry easily and strikes back”; 3 items) and proactive aggression (e.g. “This child uses force to dominate peers”; 3 items) were rated on a 5-point scale (1 = never to 5 = always). Parents and children also reported on the child's reactive and proactive aggression, with adapted parent and child versions of the TRA (Dodge and Coie, Reference Dodge and Coie1987; Hendrickx et al., Reference Hendrickx, Crombez, Roeyers and de Castro2003). Cronbach's alphas were as follows. For teachers: reactive T1 = .84, T2 = .86; proactive T1 = .79, T2 = .88; children: reactive T1 = .53, T2 = .58; proactive T1 = .70, T2 = .75; mothers: reactive T1 = .72, T2 = .80; proactive T1 = .77, T2 = .76; and fathers: reactive T1 = .74, T2 = .77; proactive T1 = 80, T2 = .78.
As a screening measure the Externalizing subscale of the Teacher Report Form (age 6–18, Achenbach, Reference Achenbach1991; Verhulst, van der Ende and Koot, Reference Verhulst, van der Ende and Koot1997) was used. To determine if the mean level of externalizing behaviour for children in the intervention condition was still in the clinical range after the intervention, we also included the broad band externalizing scale of the TRF as outcome measure (32 items; T1 α = .85, T2 α = .84).
Self perception
To assess self perception, children were asked to fill in the subscale “Behaviour attitude” of the Dutch version of the Self Perceived Competence Scale for Children (Harter, Reference Harter1982; Veerman, Straathof, Treffers, van den Bergh and Ten Brink, Reference Veerman, Straathof, Treffers, van den Bergh and Ten Brink1997) (e.g. “Some kids usually get in trouble because of the things they do – other kids don't do things that get them into trouble”). Children first had to decide which of the items in the pair better described them, and then they had to choose between “sort of true” or “really true”. With this format the effects of a pull for social desirability are reduced (T1 α = .68, T2 α = .73).
Hostile intent attribution
Four hypothetical stories were presented to children (Social Information Processing test, de Castro et al., Reference De Castro, Merk, Koops, Veerman and Bosch2005). The stories all concerned being hindered by a peer whose intentions are ambiguous (e.g. “Imagine: You and a friend are taking turns at a game at the pin-ball machine. When one has lost the game, it is the other's turn. Now it's your turn, and you are doing great. You will soon earn an extra ball, so you are trying very hard! The boy you are playing with watches the game over your shoulder. He looks into the pin-ball machine to see where the ball is. Then he shouts ‘Watch out! The ball will drop from the right side!’ But all of the sudden the ball appears from the left side and now you have lost the game!”). Immediately after hearing a story the child was asked to indicate why the peer in the story might have acted the way he or she did. Responses were written down by the interviewer and scored as 0 (benign intent) or 1 (hostile intent). Mean kappa was calculated to measure intercoder agreement, which was .95 to 1.00. Disagreements were resolved through discussions until consensus was reached. An open-answer hostile attribution variable was created by counting the number of stories with hostile answers (0 = never a hostile attribution to 4 = always a hostile attribution). Furthermore, the child was asked to indicate the peer's intent on a 10-point rating-scale ranging from 1 = to be “nice” to 10 = to be “mean”. Scores were averaged over the four stories. Because the open-answer and rating-scale variables were strongly correlated (r = .74), they were combined by standardizing each variable and then computing their average (Cronbach's alpha T1 = .64, T2 = .62).
Response generation
Next, children were asked what they would do if the events in the vignette would actually happen to them. Responses were written down by the interviewer and scored 0 (not aggressive), 1 (verbally aggressive or coercive response), 2 (physically aggressive response), and scores were averaged over the vignettes. In a previous study (de Castro et al., Reference De Castro, Merk, Koops, Veerman and Bosch2005) findings showed that an accurate interval scale of response aggressiveness could be created by weighting physical aggression with two points, verbal aggression with 1 point, and non-aggression with zero points. Inter-rater's agreement was found to be high: 95%. Cronbach's alphas were .65 at T1 and .62 at T2.
Response evaluation
Next, a possible behavioural response to the problem was presented (e.g. “if this happens to me, I will hit the child who pushed me”), to measure Approval of Aggression. This was measured with a 10-point scale where the child had to indicate whether he or she thought this was a good response (0 = not a good response to 10 = a good response). Ratings were averaged over the stories (Cronbach's alpha T1 = .66, T2 = .66). Enactment of Aggression was also measured. Children similarly had to indicate on a 10-point scale to what extent they would enact a presented aggressive response themselves. Cronbach's alphas were .76 at T1, and .66 at T2.
Data-analyses
First, we tested for possible differences at baseline scores between the intervention and control group. Next, intervention effects were examined using a series of analyses of covariance (ANCOVA) on the post test scores on reactive aggression, proactive aggression, social cognitions, and self-perception, using the pre test scores as covariate and condition status (0 = control group; 1 = intervention group) as a fixed factor. Adjusting for pretest scores in ANCOVAs is recommended for testing intervention effects (Rausch, Maxwell and Kelly, Reference Rausch, Maxwell and Kelly2003). To test for moderating effects of gender and ethnicity, we conducted additional ANCOVAs in which gender (0 = boys; 1 = girls) or ethnicity (0 = native Dutch; 1 = Immigrant) were added as fixed factors, and interactions with condition status were tested.
Effect sizes were calculated as the standardized mean difference, with mean gain scores. The adjusted posttest mean (posttest mean minus baseline mean) of the control group was subtracted from the adjusted posttest mean of the intervention group, and the results were divided by the pooled standard deviation (Lipsey and Wilson, Reference Lipsey and Wilson2001). Effect sizes were coded such that positive values mean a positive outcome for the experimental group, compared to the control group. An effect size of .20 was considered small, .50 was considered medium, and an effect size of .80 was considered large (Cohen, Reference Cohen1988).
Next, we examined whether the level of externalizing behaviour of children declined to a subclinical or normal range for more intervention than control children, using the norms of the Teacher Report Form (TRF). We calculated the percentage of children in the normal, subclinical, and clinical range at pre- and post test and compared this between intervention and control group with chi-square tests.
Results
Preliminary analyses
Before running the analyses, the normality in the distribution of variables was assessed. Absolute values of skewness (skew) > 2, and absolute values of kurtosis (kurt) > 1 represent deviations from normal distributions (Muthén and Kaplan, Reference Muthén and Kaplan1992). Mother reported proactive aggression at post test (skew = 1.84, kurt = 5.10), and child reported proactive aggression (T1 skew = 1.95, kurt = 4.18; T2 skew = 2.07; kurt = 5.37), response generation (T1 skew = 2.06, kurt = 4.31; T2 skew = 2.25; kurt = 5.19), approval of aggression (T1 skew = 1.86, kurt = 2.99; T2 skew = 1.98; kurt = 3.78), and enactment of aggression (T2 skew = 1.73, kurt = 2.57) appeared to be somewhat skewed. Log transformations were performed for these scales, and after transformation scales were normally distributed. Correlations among variables measured at pretest are presented in Table 2.
Intercorrelations among assessed variables at pretest
Note. *p < .05, **p < .01
Although transformed scores were used in the analyses, Table 3 shows, for ease of interpretation, non transformed means and standard deviations for the intervention and control group on outcome measures at pre and post treatment, as well as effect sizes. Children in the intervention and control group differed only on teacher reported proactive aggression at pre test, with higher levels of proactive aggression for children in the intervention condition (t = 2.26, p = .03). There were no differences on outcome behaviour for boys and girls. Native Dutch and immigrant children differed only on teacher reported reactive aggression at pretest, with higher levels of reactive aggression for immigrant children (t = 3.35, p = .00)
Means, standard deviations and effect sizes for outcome variables
Note: 1Standardized scores
Intervention effects
Reactive and proactive aggression
For child reported reactive aggression, the ANCOVA revealed a significant condition effect (F(2,263) = 4.06, p = .02), indicating that Stay Cool Kids decreased reactive aggression at post test. Moreover, the intervention had a significant effect on child reported proactive aggression (F(2,263) = 8.02, p = .02). For teacher reported reactive aggression, effects were significant (F(2,263) = 3.81, p = .05). No significant intervention effect was found for teacher reported proactive aggression. Significant intervention effects were found for mother reported reactive (F(2,263) = 10.71, p < .001) and proactive aggression (F(2,263) = .3.81, p = .05), indicating that children showed less reactive and proactive aggression after the intervention compared to children in the control condition. For father reported reactive aggression, condition did not predict scores at post test. However, the intervention had a significant effect for father reported proactive aggression (F(2,263) = 6.02, p = .02). Intervention children showed lower levels of proactive aggression at post test in intervention group compared to control group.
Social cognitions and self-perception
Children who received the Stay Cool Kids intervention reported higher levels of self-perception at posttest compared to children in the control condition (F(2,263) = 17.37; p < .001). The intervention also had a significant effect on level of approval of aggression at post test, indicating that the intervention reduced approval of aggression (F(2,263) = 4.69, p = .04). For hostile intent attribution, response generation and enactment of aggression, there were no significant intervention effects.
Moderator analyses
To test whether gender and ethnicity moderated intervention effects, we added gender and ethnicity (separately) in ANCOVA models, and tested the gender × condition and ethnicity × condition interactions. For child reported proactive aggression a significant gender × condition interaction effect was found (F(3,263) = 8.02, p = .02). Post hoc analyses revealed that the intervention effects were stronger for boys (d = .55) than for girls (d = .23).
For teacher reported reactive aggression a significant ethnicity x condition interaction was found (F(3,263) = 7.89, p = .01). Post hoc analyses showed stronger intervention effects for immigrant children (d = .87) than for native Dutch children (d = .06).
Also for child response generation, a marginally significant ethnicity x condition interaction was found (F(3,263) = 4.14; p = .05). Post hoc analyses showed that intervention effects on aggressive response generation were stronger for children with an immigrant background (d = .58) than for native Dutch children (d = .15).
Recovery
(Sub) clinical externalizing behaviour problems
Children were selected by their teachers, based on their subclinical and clinical levels of externalizing behaviour. At pretest, teachers reported child externalizing behaviour of the children in the intervention group in the clinical range for 61% of the children and in the subclinical range for 39%. For the control group, 67% was rated as clinical, and 33% as subclinical. These small differences between groups were not significant (χ2 (1, n = 264) = .98, p = .61). However, at post test, for the intervention group, teachers reported child externalizing behaviour for 26% in the normal range, for 22% in the subclinical range, and for 52% in the clinical range. For the control group, teachers reported child externalizing behaviour for 19% in the normal range, for 11% in the subclinical range, and for 70% in the clinical range. These differences in percentages between conditions were significant (χ2 (2, n = 264) = 5.89, p = .01), indicating that the intervention significantly increased recovery according to teacher ratings.
Discussion
In the present study, we examined whether the routine practice of the Stay Cool Kids intervention was effective in reducing reactive and proactive aggression and enhancing social cognitions and self-perception. In addition, through moderator analyses, we examined ethnic and gender differences in response to the intervention. Overall, Stay Cool Kids significantly reduced aggressive behaviour according to children, mothers, fathers and teachers, and clinical externalizing behaviour problems according to teachers. Children showed more positive levels of self-perception and reported lower approval of aggression.
These results demonstrate that the intervention can reduce the level of aggression, across informants, in children at risk of developing more serious behaviour problems, and therefore may help prevent children from having various negative outcomes later in life. Effect sizes on aggressive behaviour for teachers and children ranged from .21 to .30. For father and mother reported aggression, there was a larger variability in effect sizes ranging from .11 to .32. The effect sizes can be considered as small to modest effects. However, even small effect sizes can be of practical significance, because a small reduction in aggressive behaviour may break a vicious cycle. Therefore, it is important to neither exaggerate nor minimize both the theoretical and practical significance of small to modest effect sizes (McCartney and Rosenthal, Reference McCartney and Rosenthal2000). A notable finding was that fathers reported a significant decrease in proactive aggression, but not in reactive aggression. Reactive aggression might be observable less early in the home setting, since there are less conflict situations with peers. In general, fathers spend less time with their children than mothers (Jones, Forehand and Beach, Reference Jones, Forehand and Beach2000) and therefore may have less opportunity to see their child in situations with peers. For teachers, we found the opposite. There was a significant intervention effect on reactive aggression, but not on proactive aggression. Reactive aggression may be more prominent in the school situation, as reaction on provocation by peers.
Effect sizes found in the current study are comparable to mean effect sizes found in meta-analyses on cognitive behavioural interventions for aggressive children (ES = .26; Wilson and Lipsey, Reference Wilson and Lipsey2006) and on selected school-based interventions (ES = .29; Wilson and Lipsey, Reference Wilson and Lipsey2007). It is noteworthy that most intervention studies are demonstration or research programs that usually result in larger effects sizes (Kratochwill and Shernoff, Reference Kratochwill and Shernoff2004). Findings of this study show comparable effect sizes, but these effects were found under routine-practice conditions.
Regarding child self-perception and social cognitions, we found that children were more positive about their own behaviour after the intervention, and that they were less likely to evaluate an aggressive response as a good way to obtain desired outcomes. This indicates that the Stay Cool Kids intervention can have some impact on problems in social cognitive functioning. However, although we found significant effects on two social cognitive outcomes, several other aspects of the social cognitive processes did not change as a result of the intervention. It is possible that these aspects (e.g. hostile intent and enactment of aggression) are less sensitive to change and that more time is needed to internalize the newly learned cognitions. Changes in approval of aggression and perception of selves might well result in benign schemata that, in turn, can change social information processes (Lochman and Wells, Reference Lochman and Wells2002b). The present study was unable to test whether changes in self-perception and approval of aggression caused the reduction in child aggressive behaviour. Longitudinal data with more time points are needed to test for mediational processes.
How do present results compare to results obtained with similar interventions? The Coping Power Program is a social-cognitive school-based group and parent intervention designed to reduce conduct problems. The intervention is effective in reducing aggression and disruptive classroom behaviour and improving self-esteem in boys. However, teachers did not report effects (Lochman, Burch, Curry and Lampron, Reference Lochman, Burch, Curry and Lampron1984). Lochman and colleagues state that behavioural changes were modest and apparently situation-specific. Results of the current study show that the Stay Cool Kids intervention has the potential to reduce aggression (although effects are modest in size) across situations, which might be a result of the individual, tailor-made format. Another comparable, although multisite, intervention focusing on a high-risk sample is the Fast Track intervention (McMahon et al., Reference McMahon, Bierman, Coie, Dodge, Greenberg and Lochman1999; Slough, McMahon and CPPRG, Reference Slough and McMahon2008). Effects were found on emotional and social coping skills and on observed rates of problem behaviour at school. Teachers and parents did not report significant improvements in child behaviour. The authors state that major reductions in aggressive-disruptive behaviour are needed in high-risk children before losing their “status” as disruptive by teachers (McMahon et al., Reference McMahon, Bierman, Coie, Dodge, Greenberg and Lochman1999). As in the current study, parents did not select children for the Fast Track intervention, which might explain why parents did not report reductions in aggressive behaviour in the Fast Track trial. In the current study, parents and teachers do report (modest) effects of the training, which strengthens our conclusion that the training may have some impact on child behaviour in different situations.
To examine whether subgroups of children benefited more from the intervention, moderator analyses were performed. We found that gender and ethnicity moderated some of the intervention effects. Effects on teacher reported reactive aggression were stronger for children with an immigrant background than for native Dutch children. Initial levels of reactive aggression according to teachers were higher for immigrant children, which is in line with findings in other studies on mental health in immigrant (Moroccan) children in the Netherlands (Stevens and Vollebergh, Reference Stevens and Vollebergh2008). Therefore, perhaps there was more room for improvement (Wilson and Lipsey, Reference Wilson and Lipsey2007). Immigrant children showed also a stronger decrease in aggressive response generation compared to native Dutch children. Because Stay Cool Kids is an individualized intervention, trainers perhaps focused more on this specific social cognitive aspect for immigrant children, and therefore they learned to generate less aggressive responses in social situations. The increasing diversity of the Dutch population requires an intervention that is sensitive and effective for ethnically diverse children. Most evidence-based interventions are designed for majority children, and adapted afterwards to specific needs of minority children (Yasui and Dishion, Reference Yasui and Dishion2007). In the Stay Cool Kids intervention, the content of the training is based on an individual competence analysis, in which the child's individual needs are investigated. Because of this, there is flexibility to address specific preferences and needs of specific ethnic groups. Gender emerged as moderator only for child reported proactive aggression, with stronger effects for boys. It has been speculated that gender differences in response to intervention can occur as a consequence of differing therapy preferences of boys and girls (Garber, Reference Garber2006). There were no other moderating effects of gender. Therefore, we can conclude that the Stay Cool Kids intervention can be effective in reducing aggressive behaviour as well in at-risk boys as in girls. This enriches our knowledge about effectiveness of interventions for girls, since gender differences were less clear as a result of overrepresentation of boys in study samples (Brestan and Eyberg, Reference Brestan and Eyberg1998).
Another way to evaluate the effects of a program is to examine to what extent changes are clinically relevant (Kendall, Reference Kendall1999). Therefore, we examined in this study whether child's elevated level of externalizing behaviour, which was the selection criterion of this study, changed significantly as a result of the intervention. We found clinically relevant changes, reported by teachers. Externalizing behaviour of children who received the Stay Cool Kids intervention was rated more frequently in the normal or subclinical range, compared to children in the control condition. For children in the control condition, the externalizing behaviour was most frequently rated in the clinical range. This indicates that the intervention can, at least for part of the participating children, reduce clinical levels of externalizing behaviour to subclinical or normal levels.
Although many school-based interventions have shown significant initial effects on child's aggressive behaviour, findings on long-term effects are mixed. For some studies positive short-term effects maintained or emerged at follow-up (e.g. Conduct Problems Prevention Research Group, 2002; Scott, Reference Scott2005), whereas for other studies effects faded with time (August et al., Reference August, Lee, Bloomquist, Realmuto and Hektner2004; Tolan and Gorman-Smith, Reference Tolan and Gorman-Smith2002). In the current study we did find posttest intervention effects; however, it is important to test whether the Stay Cool Kids training may also set the stage for improvement of behaviour in middle and high school years. Longitudinal data are needed to detect impact of the intervention in the long term, so called “sleeper-effects”, and the developmental processes over time.
There are several limitations to this study that may have affected effect sizes and should be addressed in future research. First, it should be noted that the unit of analyses (individual level) did not match the unit of randomization (school level), which might have increased the risk of false-positive finding (Stice, Shaw, Bohon, Marti and Rohde, Reference Stice, Shaw, Bohon, Marti and Rohde2009). The present design permitted us to study actual effects of an intervention implemented in routine school practice. Allowing each school to participate both in control and intervention conditions and blockwise randomization made it practically feasible for schools to participate. This would not have been possible with a traditional individual randomization to either control or intervention condition, because ethical issues concerning assignment of individual children in the same classroom to different conditions and lack of motivation to participate in the control condition would have prevented schools, teachers and parents from participating in the study. Moreover, the present design prevented biases due to school factors by delivering intervention and control conditions in the same schools.
Second, intervention effects are based on child, parent and teacher reported data, and informants were not blind to the condition. Although there are problems inherent in ratings of parents and teachers, especially when they are involved in some way in the intervention, the fact that intervention effects were found across all informants and that the pattern of findings was similar across informants lends credibility to the findings. It is important to note that teachers and parents did not conduct the intervention themselves, as is frequently the case in other intervention studies. Therefore, they still may be critical to changes in child's behaviour. However, for future studies we suggest a multi-method strategy; including observational measures would increase reliability and generalizibility of results (e.g., Shores et al., Reference Shores, Jack, Gunter, Ellis, DeBriere and Wehby1993). Observational data have been shown to be resistant to biases originating in observed persons (e.g. when the child attempts to fake good behaviour) or in trained coders (i.e. no self-fulfilling expectations of behaviour). However, observational data have some limitations too. Observers tend to gather data during limited time intervals and have less access to the diversity of behaviour settings found in schools (Ladd and Profilet, Reference Ladd and Profilet1996). Therefore, data tend to be based on restricted samples of behaviour. Furthermore, aggressive behaviour often happens with peers, not in the presence of adults (e.g. Dodge and Coie, Reference Dodge and Coie1987). Peer ratings, in addition to self-, teacher- and parent-reports, might provide useful information. Using a multi-method strategy in intervention studies, which includes observational measures and peer ratings in addition to questionnaires (e.g. Lochman and Wells, Reference Lochman and Wells2002b), would result in the most comprehensive picture of changes in child behaviour. Many intervention studies, however, share this limitation of relying on questionnaires (e.g. Domitrovich, Cortes and Greenberg, Reference Domitrovich, Cortes and Greenberg2007). These studies still made great efforts to include empirical estimates that take into consideration recent recommendations (combine reports from different informants, use broad-assessment with information from adults having different relationships with the child) to use comprehensive, reliable and valid measurements (Grietens et al., Reference Grietens, Onghena, Prinzie, Gadeyne, Van Assche and Ghesquière2004).
It is also worth noting that children were selected for this intervention by their teachers because of elevated levels of externalizing behaviour at schools. Selecting children for the intervention based on a combination of informants (teachers, parents and possibly peers) on the child's externalizing behaviour might lead to larger effect sizes, as a result of more involvement in the intervention. Moreover, the control group in this study did not receive any treatment. It is possible that the beneficial effects of Stay Cool Kids are simply the consequence of receiving attention and that some other treatment would have the same effect.
Fourth, the intervention is conducted by trained professionals rather than by school personnel. Although there are several advantages from using trained professionals, it has been found that interventions provided by staff that is working full-time in school (for example teachers) are delivered with greater intensity, are operated more frequently, used more regularly, and are offered to more students. But to realize this a large portion of the regular job of teachers should be dedicated to the intervention activity (Gottfredson and Gottfredson, Reference Gottfredson and Gottfredson2002). Implementation may be more guaranteed when working with trained professionals (Silvia and Thorne, Reference Silvia and Thorne1997).
Finally, with the research design used in the current study, twice as many intervention students as control students participated. The reason for this is that we plan to examine the effects of a booster intervention for part of the intervention group. However, this is beyond the scope of the current study. Including more intervention students than control students does not inflate the effect sizes in this study. In a meta-analysis (Stice et al., Reference Stice, Shaw, Bohon, Marti and Rohde2009) it has been found that design factors, such as randomization at school level, are not related to the magnitude of effect sizes.
Despite these limitations, the results of this study are encouraging. The individualized preventive intervention, Stay Cool Kids, delivered to children at-risk for externalizing behaviour problems can have some impact on reactive and proactive aggression, some aspects of social cognitive functioning, and self-perception. Based on the high treatment completion rate (99%), treatment integrity and high level of satisfaction with the training, it can be concluded that this program is an example of how prevention activities can be successful using the school as location. Moreover, this study showed that the program is particularly effective for children with an immigrant background and it proved to be effective for girls as well. The current study fills a gap in the literature because it evaluated the effects of an already existing real-world program, which is delivered in a one-to-one setting in schools and is tailor-made. We were able to show that Stay Cool Kids children showed clinically relevant changes in teacher-reported aggressive behaviour. The intervention can move externalizing problem behaviour into a normative range, which strengthens implications for clinical practice. This study meets several of the standards for evidence developed by the Society for Prevention Research regarding efficacy, effectiveness and dissemination (Flay et al., Reference Flay, Biglan, Boruch, Gonzales Castro, Gottfredson and Kellam2005).
These results provide information to clinicians working in schools with children showing elevated levels of externalizing behaviours and to policymakers concerning options for how to prevent the development of later, more serious, externalizing behaviour problems in at-risk youth at elementary schools. Effectiveness under real-world conditions, across informants, and on diverse outcomes, is demonstrated.
Comments
No Comments have been published for this article.