Participants

The study ran from 2002 to 2006 with follow-up three completed by April 2006. Eligible families were recruited over a period of 21 months between January 2003 and September 2004. This study reports on the 104 intervention families from the previously reported randomised controlled trial, which compared intervention v. waiting list control at baseline and follow-up one. Intervention took place between these two time points. The trial took place in 11 Sure Start areas across North and Mid-Wales. ^{Reference Hutchings, Bywater, Daley, Gardner, Whitaker and Jones10} Ethical approval was granted by North West Wales research ethics committee (reference number 02/12). The trial registration number is: ISRCTN46984318. Families were eligible if they were disadvantaged, the primary carer was able to attend group times, the index child resided with the primary carer within a Sure Start area, was aged 3–4 years and scored above the clinical cut-off as rated by the parent on either the Eyberg Child Behavior Inventory ^{Reference Eyberg and Ross17} (ECBI) problem (11) or intensity (127) scale (norms 6.9 (s.d. = 7.8) for problem and 103.8 (s.d. = 34.6) for intensity). For non-clinical children aged 2–7 (based on a cohort of 1526), 10.4% of children scored in the clinical range on the ECBI. ^{Reference Burns, Patterson, Nussbaum and Parker18}

Online Table DS1 presents demographic data highlighting the ‘at risk’ nature of the children for developing conduct disorders, for example, young/single parents, low income.

There were no significant differences between intervention and control groups at baseline. The waiting list control group were not followed past follow-up one as they were then offered the intervention. This paper reports on the long-term findings for the intervention condition only. The 18, 22 and 25 participants lost to follow-up one, two and three were included in an intention-to-treat analysis. There were no differences between the families retained in the study and those lost to follow-up on any measure (online Tables DS1 and DS2).

Intervention programme

The Incredible Years BASIC parent programme was used. This was run by two trained leaders for one 2 to 2.5-hour session per week for 12 weeks, introducing a structured sequence of topics using a collaborative approach. ^{Reference Hutchings, Bywater, Daley, Gardner, Whitaker and Jones10} All leaders had run at least one group previous to the study, received 3 h of weekly supervision from a certified trainer, and attained group leader accreditation during, or shortly after, completion of the study. ^{Reference Hutchings, Bywater and Daley11}

Measures and procedures

The evaluation examined parent-completed questionnaires and ‘masked’ observation of parent–child interactions during two separate home visits at each time point, baseline and three follow-ups (6, 12 and 18 months after baseline). The parent-reported measures of child problem behaviour include the primary outcome measure, the ECBI and secondary outcome measures: Strengths and Difficulties Questionnaire (SDQ), ^{Reference Goodman, Renfrew and Mullick19} Conners Abbreviated Parent Rating Scale (CAPRS) ^{Reference Conners and Maruish20} and Kendall Self-Control Rating Scale (SCRS). ^{Reference Kendall and Wilcox21}

Parent-reported measures of parenting competencies, mood and demographics include the Parenting Stress Index – Short Form (PSI–SF), ^{Reference Abidin22} Arnold et al's Parenting Scale ^{Reference Arnold, O'Leary, Wolff and Acker23} and Beck Depression Inventory (BDI). ^{Reference Beck, Ward, Mendelson, Mock and Erbaugh24}

In addition to the above parent-completed questionnaires, an objective observational measure was used, Dyadic Parent–Child Interaction Coding System (DPICS). ^{Reference Eyberg and Robinson25} The summary variables reported here are positive parenting (the sum of positive affect, physical positive, labelled and unlabelled praise and problem-solving behaviours) and critical parenting (the sum of negative commands and critical behaviours) and deviant child behaviours (the sum of cry, whine, yell, physical negative, smart talk and destructive behaviours). Observer interrater reliability levels were maintained above 70% through weekly training. Twenty per cent of all home observation visits were interrated with agreement levels exceeding 70%. Parents also completed a Client Service Inventory (CSI) ^{Reference Knapp26} for their child's previous 6 months' service use.

Randomisation

The study utilised a pragmatic randomised controlled trial design with a sample size calculation. ^{Reference Hutchings, Bywater, Daley, Gardner, Whitaker and Jones10} After baseline assessment the first author masked and randomly block-randomised participants by area at a 2:1 ratio, intervention to control (waiting list) after stratification by gender and age of the index child, using a random number generator. ^{Reference Hutchings, Bywater, Daley, Gardner, Whitaker and Jones10} A 2:1 design allows evaluation of a larger intervention sample than a 1:1 ratio, with only a small loss of statistical power and is the favoured design in this type of research. ^{Reference Pocock27} This paper examines intervention families only across all data points.

Masking

The researchers were initially masked to allocation. ^{Reference Hutchings, Bywater, Daley, Gardner, Whitaker and Jones10} Only intervention families were followed after follow-up one. A fifth of all observation visits were carried out by two researchers to ensure interrater reliability. Researchers had to maintain 70% interrater reliability, calculated at weekly coder reliability meetings.

Analysis strategy

All families allocated to intervention (n = 104) were included in the analysis, irrespective of uptake of intervention; a further analyses was performed to only include outputs from parents and children for whom complete data-sets across all time points were available (a per-protocol analyses). The intention-to-treat analysis, in which those lost to follow-up were included (n = 25 from the intervention condition by follow-up three) assumed no change since the last available measurement for those lost to follow-up. ^{Reference Pocock27} Use of the last-observation-carried-forward rather than expectation-maximisation method of estimating missing values resulted in mean values at the three follow-up time points that were conservative estimates of the effect of the intervention. The fact that there was no difference in child behaviour at baseline between completers and those lost to the study by follow-up three also suggests this is the appropriate analysis. A linear mixed model command in SPSS version 13 for Mac was used to perform repeated measures analysis of variance (ANOVA) over time with no assumption of equal variances, using an unstructured covariance matrix, within-participants for time. All time points were compared with scores at follow-up one. A significant difference existed between baseline and follow-up one. ^{Reference Hutchings, Bywater, Daley, Gardner, Whitaker and Jones10} If positive outcomes have been maintained there should not be a significant difference between follow-ups one and three. Effect sizes were calculated using Cohen's guidelines. ^{Reference Cohen28}

As previously analysed, ^{Reference Edwards, Ceilleachair, Bywater, Hughes and Hutchings29} service use was costed using national costs measured from a multi-agency public sector perspective ^{Reference Netten and Curtis30} (n = 72). Costs from 2003/04 were inflated to 2006/07 using hospital and community health service inflation indices. ^{Reference Curtis31} Costs at follow-up three were discounted at 3.5%. ³² No cost-effectiveness analysis was carried out as a result of a lack of a control condition after follow-up one.

Programme attendance

Of the 104 parents allocated to the intervention condition, 79 completed all three post-intervention assessments (Fig. 1). Mean session attendance did not differ for the intervention sample with complete data at follow-up one, two, and three (9.2 (s.d. = 3.2), 9.32 (s.d. = 3.04) and 9.4 (s.d. = 2.97) respectively). Eighty-five per cent of those who completed all follow-ups attended 7 or more of the 12 sessions, which is comparable to the 88% attending 6 or more sessions quoted by Webster-Stratton. ^{Reference Webster-Stratton3}

Short-term findings

The programme had a positive effect on observed and reported parent and child behaviour for the intervention condition at follow-up one. ^{Reference Hutchings, Bywater, Daley, Gardner, Whitaker and Jones10,Reference Edwards, Ceilleachair, Bywater, Hughes and Hutchings29}

Long-term effectiveness findings

Tables 1 and 2 and online Tables DS3 and DS4 show no statistical differences between follow-up one, two and three on the vast majority of measures in both the intention-to-treat and complete data per-protocol analyses. The improvements were maintained for the primary child outcome measure (ECBI) and secondary child outcome measures: CAPRS, SCRS, SDQ and observed child deviance. Similar findings were found for the parent outcome measures: observed positive and critical parenting, Parenting Scale, BDI and PSI. Very small effect sizes reflect these findings. The long-term findings were identical for both the intention-to-treat group and the per-protocol group, therefore demonstrating that the results are robust. However, there was a small, yet significant, negative change in scores for one measure, the Parenting Scale, between follow-up one and follow-up three whereby the mean scores worsened from 2.81 to 2.9 in the intention-to-treat analyses (Table 1).

Table 1

Intention-to-treat (n = 104) parent outcome measures: summary of long-term results for all families allocated to intervention condition

					Estimated mean differences using mixed model ANOVA
	Intervention n = 104, raw scores: mean (s.d.)							Baseline to follow-up 1			Follow-up 1 to follow-up 2			Follow-up 1 to follow-up 3
Parent measures (cut-off)	Baseline	Follow-up 1	Follow-up 2	Follow-up 3	Mean difference (95% CI) P	Effect size (95% CI)	Mean difference (95% CI) P	Effect size (95% CI)	Mean difference (95% CI) P	Effect size (95% CI)
PSI (90)	100.67 (23.85)	84.03 (22.62)	85.18 (22.25)	85.56 (22.95)	16.64 (11.79 to 21.5) <0.001	0.94 (0.67 to 1.22)	1.15 (-1.63 to 3.94) 0.414	0.11 (-0.16 to 0.39)	1.53 (-1.59 to 4.65) 0.333	0.14 (-0.14 to 0.41)
BDI (19)	17.09 (10.67)	10.95 (10.08)	11.71 (9.8)	11.12 (9.66)	6.14 (4.0 to 8.27) <0.001	0.79 (0.51 to 1.06)	0.76 (-0.69 to 2.21) 0.302	0.15 (-0.15 to 0.42)	0.15 (-1.47 to 1.78) 0.188	0.03 (-0.25 to 0.3)
Parenting Scale a	3.44 (0.8)	2.77 (0.7)	2.81 (0.8)	2.9 (0.78)	0.68 (0.51 to 0.85) <0.001	1.17 (0.88 to 1.47)	0.04 (-0.06 to 0.134) 0.446	0.11 (-0.17 to 0.36)	0.13 (0.31 to 0.24) 0.011	0.36 (0.08 to 0.67)
Positive parenting — observation b	22.6 (18.74)	30.37 (19.08)	28.52 (21.25)	28.85 (19.79)	-7.77 (-12.24 to -3.3) 0.001	-0.48 (-0.76 to -0.2)	-1.85 (-6.23 to 2.54) 0.405	-0.12 (-0.39 to 0.16)	-1.52 (-5.9 to 2.86) 0.493	-0.1 (-0.37 to 0.18)
Critical parenting — observation b	19.12 (14.57)	11.52 (11.31)	11.82 (12.33)	12.3 (12.13)	7.6 (4.96 to 10.23) <0.001	0.79 (0.52 to 1.07)	0.3 (-1.52 to 2.12) 0.746	0.05 (-0.23 to 0.32)	0.78 (-1.14 to 2.69) 0.422	0.11 (-0.16 to 0.39)

PSI, Parenting Stress Index; BDI, Beck Depression Inventory.

a. No cut-offs for this measure. Higher score, poorer parenting.

b. No cut-offs for this measure. Frequency count in 30 min.

Table 2

Intention-to-treat (n = 104) child outcome measures: summary of long-term results for all families allocated to intervention condition

					Estimated mean differences using mixed model ANOVA
	Intervention n = 104, raw scores: mean (s.d.)							Baseline to follow-up 1			Follow-up 1 to follow-up 2			Follow-up 1 to follow-up 3
Child measures (cut-off)	Baseline	Follow-up 1	Follow-up 2	Follow-up 3	Mean difference (95% CI) P	Effect size (95% CI)	Mean difference (95% CI) P	Effect size (95% CI)	Mean difference (95% CI) P	Effect size (95% CI)
ECBI—I (127)	146.84 (27.02)	122.25 (35.06)	123.84 (36.26)	123.66 (37.9)	24.59 (18.43 to 30.75) <0.001	1.1 (0.82 to 1.37)	1.59 (-3.02 to 6.2) 0.496	0.1 (-0.18 to 0.37)	1.41 (-03.87 to 6.7) 0.597	0.19 (-0.52 to 0.91)
ECBI—P (11)	16.41 (6.63)	10.62 (7.86)	9.87 (8.09)	9.7 (8.74)	5.8 (4.19 to 7.41) <0.001	0.99 (0.72 to 1.27)	-0.75 (-1.86 to 0.36) 0.183	-0.19 (-0.46 to 0.09)	-0.91 (-2.15 to 0.32) 0.145	-0.2 (-0.48 to 0.07)
CAPRS (15)	16.53 (6.95)	12.34 (7.48)	12.6 (7.39)	12.94 (7.71)	4.19 (2.98 to 5.4) <0.001	0.95 (0.68 to 1.23)	0.26 (-0.62 to 1.14) 0.56	0.08 (-0.2 to 0.36)	0.61 (4.47 to 1.66) 0.256	0.16 (-0.12 to -0.44)
SCRS (160)	125.49 (29.73)	113.98 (29.92)	112.24 (32.65)	112.13 (31.82)	11.5 (6.14 to 16.86) <0.001	0.59 (0.32 to .87)	-1.74 (-6.23 to 2.75) 0.444	-0.11 (-0.38 to 0.17)	-1.9 (-6.5 to 2.79) 0.43	-0.11 (-0.39 to 0.17)
Child deviance — observation a	25.84 (34.37)	15.58 (23.6)	15.08 (21.03)	14.02 (20.36)	10.27 (3.84 to 16.7) 0.002	0.44 (0.16 to 0.72)	-0.5 (-5.17 to 4.17) 0.832	-0.03 (-0.3 to 0.26)	-1.56 (-0.6.25 to 3.14) 0.512	-0.9 (-0.37 to 0.44)

ECBI—I, Eyberg Child Behaviour Inventory — Intensity scale; ECBI—P, Eyberg Child Behaviour Inventory — Problem scale; CAPRS, Conners Abbreviated Parent Rating Scale, for hyperactivity; SCRS, Kendall Self-Control Rating Scale.

a. Frequency count in 30 min.

Indices of change

Indices of significant change are given in Table 3 and online Table DS5. Using the value of 0.3 of one standard deviation as a minimum index of significant change, from baseline to follow-up three, two-thirds of the intention-to-treat sample (63%) made at least this modest change in ECBI problem behaviour. Using a stricter criterion of at least 0.8 standard deviation indicating a large effect size, ^{Reference Cohen28} just over half of the sample (54%) made at least this large change and over a third (39%) made a very large change (1.5 standard deviations). For those families who had complete data at every time point the findings are even more marked, with 77% demonstrating modest change, 68% large and 41% very large change. The effects were even greater for those over the clinical cut-off at baseline (n = 65/79) for the ECBI problem scale where 82%, 75% and 56% made moderate, large and very large improvements (eligible children were over the cut-off on either the problem or intensity scale). These findings are also reflected in all other child and parent outcomes (Table 3 and online Table DS5).

Table 3

Proportion of parents who underwent moderate, large and very large changes from baseline to follow-up three

	Proportion (%) of parents from intention-to-treat							Proportion (%) of parents from ‘completers’
	In whole sample			Sample over clinical cut-off at baseline			In whole sample			Sample over clinical cut-off at baseline
Measures (cut-off), size of change a	n	%	n	%	n	%	n	%
Parental depression: Beck Depression Inventory (19)	104		41		79		34
≥0.3 s.d.		51.9		80.5		63.3		88.2
≥0.8 s.d.		31.2		63.4		38.0		67.6
≥1.5 s.d.		19.2		48.8		21.5		50.0
Parental Stress: Parent Stress index (90)	104		70		79		55
≥0.3 s.d.		55.8		68.6		68.4		80.0
≥0.8 s.d.		38.5		51.4		51.9		61.8
≥1.5 s.d.		16.3		22.9		17.7		23.6
Parenting competencies: Arnold et al
Parenting Scale b	104		n/a		79		n/a
≥0.3 s.d.		56.7		n/a		69.6		n/a
≥0.8 s.d.		38.5		n/a		48.1		n/a
≥1.5 s.d.		19.2		n/a		24.1		n/a
Positive parenting, observed b	104		n/a		79		n/a
≥0.3 s.d.		45.2		n/a		54.4
≥0.8 s.d.		29.8		n/a		36.7
≥1.5 s.d.		14.4		n/a		17.7
Critical parenting, observed b	104		n/a		79		n/a
≥0.3 s.d.		49.0		n/a		58.2	n/a
≥0.8 s.d.		27.9		n/a		34.2		n/a
≥1.5 s.d.		14.4		n/a		19.0		n/a

n/a, not applicable.

a. 0.3 denotes a modest improvement; 0.8 denotes a large improvement; 1.5 denotes a very large improvement.

b. No cut-off.

Fig. 1

Consort diagram: flow of participants through the trial. ECBI, Eyberg Child Behavior Inventory.

Long-term child health, social and special educational service use

Tables 4 and 5 show the frequency and associated cost of service use over time. This data is categorised as primary and secondary healthcare, social and special education services. Table 4 shows that by follow-up three total mean costs per child for all services, with the exception of special education, had fallen.

Table 4

18-month follow-up of health, social care and special education service costs of children whose parents received the Incredible Years parent programme ^a

Type of service	Baseline (n = 73)	At 6-month follow-up (n = 73)	At 12-month follow-up (n = 72)	At 18-month follow-up (n = 72)
Primary care	96.20	70.70	32.87	27.63
Hospital services	213.72	203.72	184.60	60.41
Special education	621.45	710.96	774.18	708.60
Social services	64.44	76.25	41.41	29.74
Total	995.81	1061.63	1033.06	826.38

a. Figures are mean total cost per child, (£) pounds sterling 2005/06, for preceding 6-month period. See short-term cost effectiveness paper ^{Reference Edwards, Ceilleachair, Bywater, Hughes and Hutchings29} for explanation of different n to main outcome study.

Table 5

Mean frequency (visits) of service contacts for preceding 6-month time period at baseline, 6, 12 and 18 months ^a

Type of service	Baseline (n = 73)	At 6-month follow-up (n = 73)	At 12-month follow-up (n = 72)	At 18-month follow-up (n = 72)
Primary care	3.92	2.74	1.51	1.07
Hospital services	0.658	0.699	0.694	0.431
Special education	42.99	52.70	52.60	53.35
Social services	2.52	2.33	2.01	1.50

a. See short-term cost effectiveness paper ^{Reference Edwards, Ceilleachair, Bywater, Hughes and Hutchings29} for explanation of different n to main outcome study.

Strengths and weaknesses

This long-term study has demonstrated the stability of positive intervention outcome for families within this disadvantaged, at-risk, population, using both self-report and objective observational data. The relatively low drop-out rate and high mean attendance reflects the effectiveness of the intervention in engaging typically hard-to-reach families. Similar findings in both the intention-to-treat and per-protocol analysis offer reassurance about the real world significance of the results. The fact that only 18% of the children in the intervention group above the ECBI problem scale cut-off at baseline did not demonstrate a small improvement at follow-up three highlights that the positive effects of the Incredible Years programme are maintained over time for many children and families who are in greatest need. Up to 40% of untreated children diagnosed with conduct disorders develop later problem behaviours including drug misuse, criminal and violent behaviour. ^{Reference Coid, Farrington and Coid4} The lack of long-term control data is a weakness of this, and most longitudinal intervention studies, but given the wealth of evidence for the effectiveness of such programmes, and the long-term risks for untreated children, it is unethical to withhold intervention from families in need. An additional weakness of this study is that observers were not masked to condition after follow-up one, although the results of this trial show remarkable stability between findings at follow-up one through to three. Despite the long follow-up period in this trial, further follow-up is needed to examine the important influence of transition into school on children's behaviour.

Comparison with other studies

This early preventive study demonstrates comparable results to samples of referred children in USA and UK long-term parenting outcome trials, ^{Reference Gardner, Burton and Klimes16,Reference Scott, Spender, Doolan, Jacobs and Aspland34} highlighting the stability of this evidence-based parenting programme in the longer term with disadvantaged, at-risk, multicentre community samples.

Meaning and implications of the study

The stability of the intervention in the longer term, with reduced reliance on health and social services, holds important lessons for UK and other governments. The use of an evidence-based programme delivered with serious attention to fidelity ^{Reference Hutchings, Bywater and Daley11} achieves remarkable long-term outcomes with a high-risk population. These results are in stark contrast to the Sure Start initiative in England, whereby services initially used a variety of programmes (some without an evidence-base), delivered without fidelity checks, without rigorous evaluation and that were generally ineffective and detrimental for those particularly disadvantaged children. ^{Reference Belsky, Melhuish, Barnes, Leyland and Romaniuk35}

The early results of the Welsh Sure Start study ^{Reference Hutchings, Bywater, Daley, Gardner, Whitaker and Jones10} prompted the Welsh Assembly Government to fund training in the Incredible Years programme as part of its Parenting Action Plan ³⁶ and it is hoped that other national governments will follow suit. It is important for governments to commission effective programmes for high-risk children with conduct problems that have been demonstrated to work longer term and to ensure that the resources are provided for them to be delivered with fidelity making the programmes likely to be good value for money.

Questions and future research

Future research should focus on examining the impact of early intervention using the Incredible Years programme on subsequent longer-term child development, including tracking progress through early school years, middle childhood and adolescence. The combination of the Incredible Years parent intervention with the Incredible Years child and classroom intervention programmes delivered to nursery nurses, teachers or directly to the child could potentially further enhance the long-term benefits. ^{Reference Webster-Stratton, Reid and Hammond37}