Introduction
Externalizing problem behaviour comprises symptoms of Attention-Deficit/Hyperactivity Disorder (ADHD), Oppositional Defiant Disorder (ODD) and Conduct Disorder (CD) as defined by the Diagnostic and Statistical Manual of Mental Disorders (American Psychiatric Association, 2000). These symptoms often cluster together and there are high prevalence rates of ADHD and ODD in both preschool children and school-age children (Cuffe, Moore and McKeown, Reference Cuffe, Moore and McKeown2005; Gadow, Sprafkin and Nolan, Reference Gadow, Sprafkin and Nolan2001; Maughan, Rowe, Messer, Goodman and Meltzer, Reference Maughan, Rowe, Messer, Goodman and Meltzer2004). Children with early externalizing problem behaviour have a higher risk for adverse developmental outcomes in youth and adulthood, such as ongoing psychiatric problems, academic underachievement and substance use (Biederman et al., Reference Biederman, Monuteaux, Mick, Spencer, Wilens, Silva, Snyder and Faraone2006; Fergusson, Horwood and Ridder, Reference Fergusson, Horwood and Ridder2005; Mason et al., Reference Mason, Kosterman, Hawkins, Herrenkohl, Lengua and McCauley2004; Spira and Fischel, Reference Spira and Fischel2005). Therefore, effective interventions are needed. From the perspective of developmental psychopathology, these interventions should start early before problems become consolidated and disturb subsequent developmental tasks (Ialongo et al., Reference Ialongo, Rogosch, Cicchetti, Toth, Buckley, Petras, Neiderhiser, Cicchetti and Cohen2006).
Parents and children mutually influence each other (Maccoby, Reference Maccoby2000). Numerous studies document that dysfunctional parenting and externalizing problem behaviour are associated (Bender et al., Reference Bender, Allen, McElhaney, Antonishak, Moore, Kelly and Davis2007; Chamberlain, Reid, Ray, Capaldi and Fisher, Reference Chamberlain, Reid, Ray, Capaldi, Fisher, Widiger, Frances, Pincus, Ross, First and Davis1997; Cunningham and Boyle, Reference Cunningham and Boyle2002). For children with antisocial behaviour, the parent-child interaction is often characterized by low levels of parental involvement in children's activities, poor supervision of offspring, and harsh and inconsistent discipline practices (Hinshaw and Lee, Reference Hinshaw, Lee, Barkley and Mash2003). Parent management training (PMT) tries to utilize and to modify the influence that parents exert on their children (Kazdin, Reference Kazdin2005).
The present study investigates the 1-year follow-up data of a PMT developed in Germany within the Prevention Program for Externalizing Problem Behaviour (PEP, Plück, Wieczorrek, Wolff Metternich and Döpfner, Reference Plück, Wieczorrek, Metternich and Döpfner2006). Like many other PMTs, PEP has a cognitive behavioural foundation and is based on published prevention and treatment manuals for children with externalizing behaviour problems (Barkley, Reference Barkley1997; Döpfner, Schürmann and Frölich, Reference Döpfner, Schürmann and Frölich2007; McMahon and Forehand, Reference McMahon and Forehand2003). Parents are trained to notice the antecedents and consequences of the problem behaviour of the children. One key feature of PEP is contingency management. Consequences that follow compliant and noncompliant child behaviour should be immediate, specific, and consistent. To attain this goal, individually tailored rules for specific problem behaviour are developed with the parents during the training sessions. Methods for rewarding the child (e.g. token systems) are presented as well as methods for adequately punishing the child (e.g. time out). The sessions are followed by parental homework. See the Methods section of this paper for more details about the program.
In general, PMTs are evaluated less often than child-focused treatments (Weisz, Doss and Hawley, Reference Weisz, Doss and Hawley2005). Nevertheless, several reviews judge PMT to be evidence-based for children with externalizing behaviour problems (Connor et al., Reference Connor, Carlson, Chang, Daniolos, Ferziger, Findling, Hutchinson, Malone, Halperin, Plattner, Post, Reynolds, Rogers, Saxena and Steiner2006; Eyberg, Nelson and Boggs, Reference Eyberg, Nelson and Boggs2008; Farmer, Compton, Burns and Robertson, Reference Farmer, Compton, Burns and Robertson2002; McCart, Priester, Davies and Azen, Reference McCart, Priester, Davies and Azen2006; Nixon, Reference Nixon2002). For children with ADHD, PMT is seen as an environmental way to help the child cope with self-regulation deficits (Weisz, Reference Weisz2004) and it is regarded as empirically supported, either alone or in combination with stimulant medication (Chronis, Jones and Raggi, Reference Chronis, Jones and Raggi2006). Positive short-term effects of PEP, including parent training and pre-school-teacher training, have been demonstrated in a randomized controlled efficacy study (Hanisch et al., Reference Hanisch, Freund-Braier, Hautmann, Meyer, Plück, Brix and Döpfner2009; Hanisch et al., Reference Hanisch, Plück, Meyer, Brix, Freund-Braier, Hautmann and Döpfner2006).
In general, however, there is a paucity of data on the long-term outcomes of the treatment of externalizing problem behaviour in children (Farmer et al., Reference Farmer, Compton, Burns and Robertson2002; Kazdin, Reference Kazdin1997). Only a few studies report follow-up data at more than 6 months after treatment. Positive treatment effects were found at 1-year follow-up and longer for Webster-Stratton's Incredible Years program (Webster-Stratton, Reference Webster-Stratton, Jensen and Hibbs2005) and Eyberg's Parent-Child Interaction Therapy (Nixon, Sweeney, Erickson and Touyz, Reference Nixon, Sweeney, Erickson and Touyz2004; Querido and Eyberg, Reference Querido, Eyberg, Hibbs and Jensen2005). In general, these programs are evaluated as efficacy trials conducted in well-controlled settings.
In the present study, PEP was delivered in routine care settings. PEP was offered by employees of different counseling and mental health services. This study can, therefore, be characterized as an effectiveness trial. In contrast to efficacy trials, effectiveness studies assess treatment effects under real-world conditions (Lutz, Reference Lutz2003; Nathan, Stuart and Dolan, Reference Nathan, Stuart and Dolan2000; Weisz, Donenberg, Han and Weiss, Reference Weisz, Donenberg, Han and Weiss1995). That is, effectiveness studies have high external validity, but this is often obtained at the expense of low internal validity. To date, only a few outcome studies have been carried out as effectiveness trials (Glasgow, Lichtenstein and Marcus, Reference Glasgow, Lichtenstein and Marcus2003). In general, the results of these effectiveness trials are less promising than those of efficacy trials. In effectiveness trials of traditional child psychotherapy, average effect sizes range from –.08 (Weiss, Catron, Harris and Phung, Reference Weiss, Catron, Harris and Phung1999) to .01 (Weisz and Jensen, Reference Weisz and Jensen1999). However, none of these studies assessed the effectiveness of parent training in isolation.
One of the few trials that has tested a PMT for externalizing behaviour problems under real-world conditions is that of Ogden and Hagen (Reference Ogden and Hagen2008). Using the Parent Management Training Oregon model (PMTO, Forgatch, Reference Forgatch1994) they obtained positive results when applied in the health system in Norway. In comparison with treatment as usual, PMTO was effective in reducing parent-reported child externalizing problems, in improving teacher-reported social competence, and in enhancing parental discipline immediately after treatment. For Webster-Stratton's Incredible Years program (Webster-Stratton, Reference Webster-Stratton, Jensen and Hibbs2005), there is also evidence that treatment effects can persist for a long time when applied in routine care conditions (Gardner, Burton and Klimes, Reference Gardner, Burton and Klimes2006; Hutchings et al., Reference Hutchings, Gardner, Bywater, Daley, Whitaker, Jones, Eames and Edwards2007; Scott, Reference Scott2005). In the study of Hutchings et al. (Reference Hutchings, Gardner, Bywater, Daley, Whitaker, Jones, Eames and Edwards2007), conduct problems of the child and parenting were both significantly improved at 6-month follow-up in the intervention group compared with the control group. Scott (Reference Scott2005) demonstrated stable effects for externalizing problem behaviour when the post-treatment scores of the intervention group were compared with the follow-up scores 1 year later. Gardner et al. (Reference Gardner, Burton and Klimes2006) found that conduct problems and parenting behaviour remained stable when the intervention group post-treatment scores were compared with follow-up scores 18 months later. All three of these trials were conducted in the health care system of the United Kingdom.
The aim of the present study was to determine the effectiveness of PEP at 1-year follow-up under routine care conditions in Germany. To our knowledge, this is the first study that considers the long-term effects of a PMT in routine care in Germany. Despite differences in the health care systems of the United Kingdom and Germany, it was expected that treatment effects for externalizing problem behaviour and parenting would remain stable at follow-up. That is, comparable results with the studies of Gardner et al. (Reference Gardner, Burton and Klimes2006) and Scott (Reference Scott2005) were expected.
Method
Design
In this study, PEP was evaluated using a within-subject control group design. There were two assessment points before treatment: the first assessment (pre1) occurred 3 months before treatment; the second assessment (pre2) was after a 3-month waiting period and immediately before treatment. Changes in outcome variables during this waiting period served as the control condition and were compared with changes during treatment, i.e. between pre2 and post (the assessment conducted immediately after treatment). Follow-up assessments were conducted at 3 months and at one year after treatment (1-year fu). The present analysis considers pre1, pre2, post, and 1-year fu data. The ethics committee of the University Hospital, Cologne, approved the study.
Participants
To be considered an effectiveness trial, the PEP courses had to be conducted by members of local counselling services, pediatric primary care centres, and psychotherapy practices. Altogether, 37 different institutions located in North Rhine-Westphalia (Germany) were recruited for the study. The investigators of these participating institutions decided which families fulfilled the study inclusion criterion. The only inclusion criterion was a 3- to 10-year-old child with externalizing problem behaviour. No limits were defined for symptom severity. The families were informed that prior to the intervention there was a 3-month waiting period.
A total of 324 families were included at the pre1 assessment. Of these families, 265 supplied questionnaire data at the pre2 assessment, 210 at the post assessment, and 101 at the 1-year fu assessment. Families who never attended the training were excluded from the analysis. Thus, conclusions about the effectiveness of treatment are only valid for families who attended at least 1 unit of PEP training. After this correction, the number of families at each assessment point was: pre1 (n = 270), pre2 (n = 248), post (n = 210) and 1-year fu (n = 101). The 270 families at pre1 constituted the sample considered in this analysis. Families with missing values at pre2, post and 1-year fu were included in the statistical analysis. The strategy used for handling missing data is given in the missing data section below.
Of the 270 families at pre1, 79.3% children were male and mean age was 6.5 years (SD = 2.0). Mothers mean age was 36.4 years (SD = 5.2) and 15.9% of the families had an immigration background. In 63.3% of the families, both biological parents lived together, 24.8% of the children stayed with their biological mothers only, and 11.9% of children had other family backgrounds.
PEP trainers
The PEP trainers were experienced child therapists and employees of the 37 different institutions taking part in the study. Overall, 59 trainers were involved in conducting PEP parent trainings including didactic presentation, modelling group discussion and practising. The PEP trainers attended a 2-day course held by the project members to learn how to conduct the PEP training. The PEP trainers were: psychologists (37.3%), social or educational workers (23.5%), educationalists (15.7%), remedial teachers (15.7%), or belonged to other professions (7.8%).
PEP training
PEP is designed for children with externalizing behaviour problems aged 3 to 10 years and has a parent training component and a (pre-school) teacher training component. The lessons for parents and teachers are given separately. Both training components comprise 12 units: 6 basic units and 6 additional units. Each unit takes 90–120 minutes to deliver and is ideally for between 4 and 8 participants in each group. Each unit is individually tailored to the needs of each participating family. Therefore, at the beginning of the training, the specific problem of each child is defined. The parents are taught how to solve these specific problems by using the different interventions discussed in the units. In the present study, only parents were trained.
In the first basic unit for parents, target problems of the child as well as competencies are identified. In the second unit, the coercive interaction process (cf. Patterson, Reference Patterson1982) is identified with the parents. This serves as an explanatory model for the target problems of the child. In addition, positive play time is introduced as a means to strengthen positive parent-child interactions. In the third basic unit, methods for parents to cope with daily hassles are put together. In the fourth basic unit, firm and secure rules are developed with the parents for the target problems of the child. The parents are taught how to communicate effective commands. In the fifth basic unit, methods for rewarding the child (e.g. token systems) are presented for when the child complies with the rules. In the last basic unit, parents are informed how to punish the child adequately (e.g. time out) when the child has broken the rules.
The first additional unit precedes the basic units and is an initial get-together with a brief introduction to the contents of the program. The remaining additional units are delivered after the basic units. In the second additional unit, methods for managing problem behaviour in public are discussed. In the third additional unit, ways to cope with enduring quarrels between children (e.g. siblings) are presented. The fourth additional unit focuses on how to strengthen persevering play of children. The fifth additional unit is aimed at parents of school-age children and teaches methods to strengthen attention and finish homework. The last additional unit is a summary of the content of the units of PEP.
In the present study, the trainers were obliged to give the 6 basic units, but the remaining additional units were delivered at the discretion of the trainer and according to the needs of the parents. On average, the 59 trainers offered 7.9 units (SD = 1.4). Parents attended on average 4.6 (SD = 1.6) of the 6 basic units.
Outcome measures
Data were collected from mothers via questionnaire booklets.
Child behaviour problems
The Child Behavior Checklist for ages 4–18 (CBCL/4–18, Achenbach, Reference Achenbach1991) is designed to assess a variety of child-specific behaviour problems. Items are scored from 0 to 2, with higher scores indicating more severe problems. Various studies proved the German version to be a factorially valid, robust and highly reliable rating scale (Döpfner, Berner, Schmeck, Lehmkuhl and Poustka, Reference Döpfner, Berner, Schmeck, Lehmkuhl, Poustka and Sergeant1995). For this report, the externalizing syndrome scale (CBCL-EXT) with 33 items and an internal consistency of Cronbach's α = .89 was used.
The Symptom Checklist Attention-Deficit/Hyperactivity Disorder (SCL-ADHD, Döpfner, Görtz-Dorten and Lehmkuhl, Reference Döpfner, Görtz-Dorten and Lehmkuhl2008) assesses the diagnostic criteria of DSM-IV (American Psychiatric Association, 1994) and ICD-10 (World Health Organization, 1993) for ADHD. The instrument consists of 20 items, each scored on a 0 to 3 severity scale; scores of 2 and above are considered clinically relevant. Reliability of this instrument has been shown (Döpfner et al., Reference Döpfner, Görtz-Dorten and Lehmkuhl2008). In our sample, internal consistency for the total score was Cronbach's α = .92.
The Symptom Checklist Disruptive Behaviour Disorder (SCL-DBD, Döpfner et al., Reference Döpfner, Görtz-Dorten and Lehmkuhl2008) contains the diagnostic criteria of DSM-IV and ICD-10 for ODD and CD. For this study, only the 9 items of the ODD subscale were considered (SCL-ODD). Reliability of this instrument has been shown (Döpfner et al., Reference Döpfner, Görtz-Dorten and Lehmkuhl2008). Items are scored on a 0 to 3 severity scale. Internal consistency for SCL-ODD was Cronbach's α = .91.
Parenting
The Self-Efficacy Scale (SEFS) is the German adaptation of the Parenting Sense of Competence Scale developed by Johnston and Mash (Reference Johnston and Mash1989) and the Self Efficacy for Parenting Task Index by Coleman and Karraker (Reference Coleman and Karraker2000). The SEFS comprises 15 items measuring parents' perception of self-efficacy on a 0 to 3 scale with higher values indicating more competencies. In this sample, one item was deleted due to low item total correlation. Internal consistency of the remaining items was Cronbach's α = .85.
The German adaptation of the Problem Setting and Behaviour Checklist (PSBC) developed by Sanders, Markie-Dadds, Tully and Bor (Reference Sanders, Markie-Dadds, Tully and Bor2000) measures the perceived ability to solve difficult parenting situations. Items are scored on a 0 to 3 scale with higher scores reflecting a stronger ability to deal with difficult parenting situations. Internal consistency for the overall score in our sample was Cronbach's α = .91.
Statistical analysis
Data were analysed by multilevel modelling (Goldstein, Reference Goldstein2003; Hox, Reference Hox2002; Raudenbush and Bryk, Reference Raudenbush and Bryk2002; Snijders and Bosker, Reference Snijders and Bosker1999) using the sixth version of HLM software (Raudenbush, Bryk, Cheong and Congdon, Reference Raudenbush, Bryk, Cheong and Congdon2004; Raudenbush, Bryk and Congdon, Reference Raudenbush, Bryk and Congdon2008). For the present analysis, piecewise linear growth models were computed (Raudenbush and Bryk, Reference Raudenbush and Bryk2002; Singer and Willett, Reference Singer and Willett2003; Snijders and Bosker, Reference Snijders and Bosker1999). That is, different growth rates were taken into account for different time periods. Altogether, three time periods and, therefore, three different growth rates (β10, β20, β30) were considered. The first time period was the waiting period from pre1 to pre2. Changes during this period were covered by growth rate β10. The treatment period from pre2 to post was the second time period and was covered by growth rate β20. The third time period was the follow-up period from post to 1-year fu. Changes during this period were covered by growth rate β30. The intercept of model was considered to be random and the growth rates were fixed for reason of model identification.
The analysis had two main objectives. First, to show that growth rate β20 (change during treatment) was significantly higher than growth rate β10 (change during waiting period) as a test for treatment effects. To check this, contrasts were defined. The results of this analysis should replicate previous findings of the study where a less advanced statistical procedure was used (Hautmann, Hanisch, Mayer, Plück and Döpfner, Reference Hautmann, Hanisch, Mayer, Plück and Döpfner2008).
The second main objective was to test whether initial treatment effects were maintained or improved over time, i.e. that the growth rate β30 was either not significant or indicated a significant improvement.
For the regression coefficients, the significance tests were based on robust standard errors. We further calculated effect sizes using Cohen's d (Cohen, Reference Cohen1988).
Missing data
In multilevel modelling, incomplete cases remain in the analysis (Maas and Snijders, Reference Maas and Snijders2003). Little (Reference Little1995) has shown that this strategy is appropriate if missing data are missing at random (MAR, Rubin, Reference Rubin1976). For all other analyses except the multilevel model, missing data were imputed by the expectation maximization procedure (EM, McLachlan and Krishan, Reference McLachlan and Krishan1996) of SPSS (SPSS, 2007). EM also assumes MAR. In our study, several comparisons were made to get an indication of whether or not data were MAR. This was especially important because there was considerable drop-out over the course of the study (the number of participating families decreased from n = 270 at pre1, to n = 248 at pre2, to n = 210 at post, to n = 101 at 1-year fu).
We tested whether families who participated in the training but who dropped out of the study either at post (n = 60) or at 1-year fu (n = 169) differed from those who attended the training and provided data. Participants with missing data at post did not differ in any of the outcome measures at pre1 from those whose data were available at post (CBCL-EXT: t(268) = 0.61, p = .543; SCL-ADHD: t(268) = 1.67, p = .097; SCL-ODD: t(268) = −0.14, p = .893; SEFS: t(268) = −1.88, p = .061; PSBC: t(268) = −1.25, p = .211). For SEFS, significance was only just missed. In this case, those who dropped out were less impaired. Participants with missing data at 1-year fu did not differ in any of the outcome measures at pre1 from those whose data were available at 1-year fu (CBCL-EXT: t(268) = −0.33, p = .740; SCL-ADHD: t(268) = 0.96, p = .341; SCL-ODD: t(268) = −0.13, p = .898; SEFS: t(268) = −1.37, p = .173; PSBC: t(268) = −0.68, p = .496). Patients with missing data at 1-year fu did not differ from patients with full data sets at 1-year fu regarding their changes during treatment (difference pre2 to post) on any of the outcome variables (CBCL-EXT: t(196) = 1.51, p = .132; SCL-ADHD: t(196) = 0.77, p = .442; SCL-ODD: t(196) = 37, p = .709; SEFS: t(196) = −1.68, p = .094; PSBC: t(196) = −1.39, p = .167). In sum, we found no evidence that missing data were not MAR. We therefore hypothesized that the prerequisites for the chosen missing data handling strategies were appropriate.
We also tested whether families who did not participate in the training (n = 54) and were not considered for this analysis differed from those who attended the training and were analysed (n = 270). Families who did not participate in the training did not differ in any of the outcome measures at pre1 from those who took part in the training (CBCL-EXT: t(321) = −0.26, p = .793; SCL-ADHD: t(322) = −0.84, p = .401; SCL-ODD: t(322) = 0.50, p = .618; SEFS: t(322) = 1.14, p = .257; PSBC: t(322) = −0.30, p = .761).
Results
Table 1 summarizes the means and standard deviations for the outcome measures at the four assessment points.
Means and standard deviations for outcome variables at four different measurement points for the sample of families who attended at least 1 unit of the PMT (missing values not imputed)
Notes: CBCL-EXT = Child Behavior Checklist/4–18 externalizing syndrome scale; SCL-ADHD = Symptom Checklist Attention-Deficit/Hyperactivity Disorder total score; SCL-ODD = Symptom Checklist Disruptive Behaviour Disorder subscale Oppositional Defiant Disorder; SEFS = Self-Efficacy Scale total score; PSBC = Problem Setting and Behaviour Checklist total score
Based on intercept-only models, the intra-class correlation ρ for the various outcome measures were computed. For CBCL-EXT ρ = .30, for SCL-ADHD ρ = .34, for SCL-ODD ρ = .31, for SEFS ρ = .37, and for PSBC ρ = .43. That is, about one-third of the total variance of the child behaviour problem variables (CBCL-EXT, SCL-ADHD, SCL-ODD) was variance between individuals, and about two-thirds was variance within individuals across time. For the parenting variables (SEFS, PSBC) more than one-third of the total variance could be attributed to variance between individuals.
The results of the multilevel models are presented in Figures 1 to 5. For the waiting period (pre1 to pre2), the growth rate β10 was negative and significant for all child behaviour problem variables indicating a significant decrease during the waiting period (CBCL-EXT: β10 = −0.488, t(825) = −4.30, p < .001; SCL-ADHD: β10 = −0.033, t(825) = −4.03, p < .001; SCL-ODD: β10 = −0.035, t(825) = −3.49, p = .001). For the parenting variables SEFS and PSBC, the growth rate β10 was not significant, although PSBC only just missed statistical significance (SEFS: β10 = 0.005, t(825) = 0.77, p = .442; PSBC: β10 = 0.014, t(825) = 1.96, p = .05).
Average course of Child Behavior Checklist for ages 4–18 externalizing syndrome scale (CBCL-EXT) based on parameters of the multilevel model (N = 270); β10 = slope waiting period (pre1/pre2); β20 = slope treatment period (pre2/post); β30 = slope 1-year follow-up period (post/1-year fu); by contrast significant difference between β10 and β20 was tested; *p < .05
Average course of Symptom Checklist Attention-Deficit/Hyperactivity Disorder total score (SCL-ADHD) based on parameters of the multilevel model (N = 270); β10 = slope waiting period (pre1/pre2); β20 = slope treatment period (pre2/post); β30 = slope 1-year follow-up period (post/1-year fu); by contrast significant difference between β10 and β20 was tested; *p < .05
Average course of Symptom Checklist Disruptive Behaviour Disorder subscale Oppositional Defiant Disorder (SCL-ODD) based on parameters of the multilevel model (N = 270); β10 = slope waiting period (pre1/pre2); β20 = slope treatment period (pre2/post); β30 = slope 1-year follow-up period (post/1-year fu); by contrast significant difference between β10 and β20 was tested; *p < .05
Average course of Self-Efficacy Scale (SEFS) based on parameters of the multilevel model (N = 270); β10 = slope waiting period (pre1/pre2); β20 = slope treatment period (pre2/post); β30 = slope 1-year follow-up period (post/1-year fu); by contrast significant difference between β10 and β20 was tested; *p < .05
Average course of Problem Setting and Behaviour Checklist (PSBC) based on parameters of the multilevel model (N = 270); β10 = slope waiting period (pre1/pre2); β20 = slope treatment period (pre2/post); β30 = slope 1-year follow-up period (post/1-year fu); by contrast significant difference between β10 and β20 was tested; *p < .05
For the treatment period (pre2 to post), the growth rate β20 was significant for all variables, indicating a decrease in child behaviour problems and an increase in self-reported parenting competencies during treatment (CBCL-EXT: β20 = −1.213, t(825) = −8.66, p < .001; SCL-ADHD: β20 = −0.077, t(825) = −7.77, p < .001; SCL-ODD: β20 = −0.085, t(825) = −7.50, p < .001; SEFS: β20 = 0.046, t(825) = 5.24, p < .001; PSBC: β20 = 0.061, t(825) = 6.34, p < .001).
In the next step, the first main analysis of the study was conducted. By defining contrasts (see Figures 1 to 5), we calculated whether the growth rate during the treatment period (β20) was significantly larger than the growth rate during the waiting period (β10). This was the case for all outcome variables – for externalizing behaviour problems (CBCL-EXT: χ2(1) = 12.77, p = .001), ADHD symptoms (SCL-ADHD: χ2(1) = 9.38, p = .003), ODD symptoms (SCL-ODD: χ2(1) = 8.54, p = .004), self efficacy of parenting (SEFS: χ2(1) = 10.30, p = .002), and the perceived ability to solve difficult parenting situations (PSBC: χ2(1) = 10.93, p = .001) as rated by parents.
The second main objective of this study pertained to the growth rate β30 during the 1-year follow-up (see Figures 1 to 5). For child behaviour problems, there was no significant change over time during follow-up, indicating stability (CBCL-EXT: β30 = −0.034, t(825) = −0.59, p = .553; SCL-ADHD: β30 = −0.004, t(825) = −0.89, p = .373; SCL-ODD: β30 = −0.002, t(825) = −0.44, p = .660). The growth rates for the parenting variables were positive and significant, indicating a further increase in parenting competence (SEFS: β30 = 0.010, t(825) = 3.61, p = .001; PSBC: β30 = 0.011, t(825) = 4.23, p < .001).
To assess the magnitude of these results, Cohen's d effect size (Cohen, Reference Cohen1988) was computed for three time intervals: (a) the waiting period (pre1 to pre2), (b) the intervention period (pre2 to post) and (c) the follow-up period (post to 1-year fu). The Cohen's d effect sizes for the different outcome measures are given in Table 2. According to Cohen (Reference Cohen1988), effect size values ranging from 0.2 to 0.5 are considered as small, from 0.5 to 0.8 as medium, and greater than 0.8 as large.
Cohen's d effect sizes for waiting period (pre1/pre2), treatment period (pre2/post) and follow-up period (pre2/1-year fu) for sample of families who attended at least 1 unit of the PMT (N = 270)
Notes: Missing values were imputed by expectation maximization; CBCL-EXT = Child Behavior Checklist/4–18 externalizing syndrome scale; SCL-ADHD = Symptom Checklist Attention-Deficit/Hyperactivity Disorder total score; SCL-ODD = Symptom Checklist Disruptive Behaviour Disorder subscale Oppositional Defiant Disorder; SEFS = Self-Efficacy Scale total score; PSBC = Problem Setting and Behaviour Checklist total score
For child behaviour problems, the effect sizes were small for the waiting period, medium for the intervention period, and negligible for the follow-up period. For parenting, effects sizes were negligible for the waiting period, small for the intervention period, and small to medium for the follow-up period.
Clinical significance of the findings was further investigated by normative comparisons. For CBCL-EXT (Arbeitsgruppe Deutsche Child Behaviour Checklist, 1998), SCL-ADHD (Döpfner et al., Reference Döpfner, Görtz-Dorten and Lehmkuhl2008) and SCL-ODD (Döpfner et al., Reference Döpfner, Görtz-Dorten and Lehmkuhl2008) normative data was available. We investigated how many children were above the 90th percentile for the respective measurement points. For CBCL-EXT (pre1: 68.1%; pre2: 61.1%; post: 46.7%; 1-year fu: 44.8%), SCL-ADHD (pre1: 47.0%; pre2: 41.1%; post: 23.3%; 1-year fu: 20.7%), and SCL-ODD (pre1: 50.7%; pre2: 44.1%; post: 30.4%; 1-year fu: 24.1%), there was a progressive decrease in the percentage of children scoring above the 90th percentile over the course of the study.
Discussion
In the present study, a group of children with externalizing behaviour problems was observed for three different time periods. The first 3-month waiting period served as a control period to detect naturally occurring changes in parenting or child behaviour problems. In the second period, the parent management training of PEP was provided. Changes in outcome measures during this time interval indicated changes during treatment. The third time period covered the time from the end of treatment up to 1 year post treatment.
The study had two main objectives. First, to show that changes during the treatment period were significantly greater than those during the waiting period thereby testing the effectiveness of PEP in routine care. This short-term effectiveness was shown for all outcome variables. Compared with doing nothing in the waiting period, participation in PEP resulted in significantly improved parenting and externalizing child behaviour problems in children referred for these kinds of behaviour problems under routine care conditions. The effects were small for parenting and medium for externalizing behaviour. Thus, previous findings based on a less rigorous statistical approach than used in the present analysis were replicated (Hautmann et al., Reference Hautmann, Hanisch, Mayer, Plück and Döpfner2008).
The second main objective of the study was to show that the treatment gains were maintained over time or even increased. Our results show that externalizing problem behaviour was maintained (i.e. stable) over the 1-year follow-up period and that parenting self-efficacy and perceived parenting ability showed further improvement in the small to medium range. With respect to externalizing problem behaviour, this study replicates the findings of the effectiveness trials of Scott (Reference Scott2005) at 1-year follow-up and Gardner et al. (Reference Gardner, Burton and Klimes2006) at 18-months follow-up. However, we obtained somewhat different results for parenting behaviour than these previous studies. Gardner et al. (Reference Gardner, Burton and Klimes2006) found stability when post-treatment scores were compared with 18-month follow-up scores, whereas we found an increase in parenting competencies during the 1-year follow-up period. This difference in parenting behaviour might be due to the different lengths of follow-up in the studies, but this remains speculative and is unlikely as the change in the present study was in the medium range, indicating a substantial improvement.
Scott (Reference Scott2005) as well as Gardner et al. (Reference Gardner, Burton and Klimes2006) conducted their effectiveness trials under routine conditions in the United Kingdom. The present study is the first study conducted in Germany to demonstrate beneficial long-term effects under real-world conditions. These findings are especially promising as the results of effectiveness trials often are less positive than those of efficacy trials (Weiss et al., Reference Weiss, Catron, Harris and Phung1999; Weisz and Jensen, Reference Weisz and Jensen1999). Relevance of the findings was further investigated in terms of their clinical significance. For child behaviour problems, we examined how many children scored above the 90th percentile when compared with normative data. At the beginning of the study, between 47.0% and 68.1% of the children were classified as severely impaired. At 1-year follow-up, the corresponding range was 20.7% to 44.8%. That is, over the course of the study, there was a substantial decrease in the proportion of children within the clinical range. Nevertheless, the results show there remained a large proportion of children who were in need of further support.
An improvement of perceived parenting during follow-up did not translate into further improvement of child behaviour during this period. This may be because parenting is only one risk or protective factor that contributes to externalizing problem behaviour of children. Child variables (e.g. genetic make-up) as well as other environmental variables of the children (e.g. deviant peer group) also exert an influence on the course and have to be taken into consideration (Lahey, Waldmann and McBurnett, Reference Lahey, Waldmann and McBurnett1999; Nigg, Reference Nigg2006).
Child behaviour improved substantially during the waiting period. There may be several reasons for this. Possible explanatory models include repeated measurements effects, regression toward the mean (Nesselroade, Stigler and Baltes, Reference Nesselroade, Stigler and Baltes1980), “real changes” during the waiting period, or improvement due to expected help. On the other hand, we found significant differences in the magnitude of change for all outcome measures, demonstrating relevance of the treatment success.
Study limitations
This study has several limitations, some of which can be attributed to the nature of effectiveness studies in general. A within-subject control group design is less rigorous than a randomized control trial. Data were gathered exclusively by mother questionnaire and a third-person rating would have been useful. The results therefore primarily reflect the views of the mothers who, in general, also participated in the treatment. Therefore, the observed symptom reduction may primarily reflect effort gratification of the mothers. Further analyses that also consider the views of the fathers will clarify this question (Hautmann et al., Reference Hautmann, Hoijtink, Eichelberger, Hanisch, Plück, Walter and Döpfner2009). The drop-out rate from the post measurement to the 1-year follow-up measurement was quite high; a higher persistence rate over the long-term course of the study would have been desirable. On the other hand, preliminarily analyses showed that families who dropped out of the study did not differentially profit from treatment and that drop-out could be considered to be at random. Furthermore, because of the general paucity of long-term data, these results are valuable.
Conclusions
The results of this study are promising. In general, only a few studies investigating PMT are conceived as effectiveness trials (e.g. Ogden and Hagen, Reference Ogden and Hagen2008), and even fewer studies report results for longer follow-up periods. Long-term effects under routine care settings in the United Kingdom have already been demonstrated by Gardner et al. (Reference Gardner, Burton and Klimes2006) and Scott (Reference Scott2005). This study confirms their findings under conditions of routine care in Germany. This indicates that PMT can have long-lasting effects even when applied under real-world conditions in different European health care systems.
Acknowledgements
The study was funded by the Deutsche Forschungsgemeinschaft (DFG; DFG grant DO 620/2). We thank all the families who participated in PEP and all the institutions and trainers who offered the training. We kindly thank Deirdre Elmhirst for carefully reading the manuscript.
Comments
No Comments have been published for this article.