Introduction
Anxiety disorders in youth have received considerable research attention over the past two decades. This research has confirmed the high prevalence rates (Kashani & Orvaschel, Reference Kashani and Orvaschel1990), the often chronic course of the disorders (Last et al. Reference Last, Hersen, Kazdin, Finkelstein and Strauss1987), that high comorbidity is the rule rather than the exception (Wittchen et al. Reference Wittchen, Nelson and Lachner1998) and also that anxiety disorders may often be a precursor to other difficulties such as depression and substance abuse (Kendall et al. Reference Kendall, Safford, Flannery-Schroeder and Webb2004).
The literature indicates that anxiety may increase the risk of the child not participating in age- related activities, may result in reduced academic functioning, low self-esteem, poor peer relations and family problems (Ollendick & King, Reference Ollendick, King and Kendall1994; Costello & Angold, Reference Costello, Angold and March1995). Increased focus on this group of children and early intervention is thus essential given the high prevalence and bleak prognoses if left untreated (Last et al. Reference Last, Perrin, Hersen and Kazdin1996).
Over the last 20 years treatment programmes based on cognitive behaviour therapy for children and adolescents with anxiety disorders have been developed (Barrett, Reference Barrett2004; Kendall & Hedtke, Reference Kendall and Hedtke2006) and extensively evaluated (Barrett et al. Reference Barrett, Dadds and Rapee1996, Reference Barrett, Dadds and Rapee2001; Kendall & Southham-Gerow, Reference Kendall and Southam-Gerow1996; Kendall et al. Reference Kendall, Hudson, Gosch and Flannery-Schroeder2008). These treatments are today characterized as probably effective (Hibbs, Reference Hibbs2001). While such programmes are extensively used in many countries, structured treatment programmes for emotional disorders have not been available in Norway until recently. However, the research related to the programmes is often based on efficacy studies where the conditions are close to ideal, using highly motivated therapists and children that might not be recruited from the ordinary referral system. A general concern is therefore that ordinary clinicians in outpatient clinics meeting children with multiple problems will not obtain the same positive results (Weisz et al. Reference Weisz, Jensen-Doss and Hawley2006). To bridge the gap between efficacy research and ordinary clinical care it is thus important to study the usefulness of such interventions in local settings with attention given to the effects across diverse patients, therapists and settings as suggested by the APA Task Force (2006). To increase the use of evidence-based treatments in ordinary clinical care, research also has to be informed by and sensitive to issues raised by practitioners (Kendall & Beidas, Reference Kendall and Beidas2007).
The Friends programme has been translated and adapted to the Norwegian culture. In this pilot study the programme has been introduced in three outpatient clinics in Norway and this paper will focus on the effect of treatment and alternative ways of measuring this effect in addition to traditional statistical analysis. As statistically significant change may not give information about how the treatment has impacted on the client's life in terms of better quality of life, this paper will also focus on clinical meaningful change (Jacobsen & Truax, Reference Jacobsen and Truax1991). Two approaches will be used for this purpose; changes in severity rating (CRS) at diagnostic level and the Reliable Change Index (RCI; Jacobsen & Truax, Reference Jacobsen and Truax1991) on changes in self-report measures.
Last, we discuss the acceptability of the structured treatment among the families and therapists in this pilot study.
Method
Participants and procedure
Children (n = 18) aged 7–12 years (mean = 9.81 years, 48.4% girls) participated in the study. All were ethnic Norwegian. The children were recruited through the ordinary referral system of three different outpatient clinics in Southern and Eastern Norway. All children (n = 111) in the targeted age range (7–12 years) referred to the clinics in the spring of 2006 were screened using the Children's Behavior Checklist (CBCL; Achenbach, Reference Achenbach1991). Thirty-two children showing elevated levels of internalizing problems (T score >65) were assessed with child and parent measures.
Inclusion criteria were principal diagnosis of separation anxiety disorder (SAD), generalized anxiety disorder (GAD) or social phobia (SP) according to DSM-IV criteria (APA, 1994). The children and the parents were assessed separately to assess for anxiety disorders. Diagnosis was based on the results from the Anxiety Disorders Interview Schedule conducted with the parents (ADIS-P; Silverman & Albano, Reference Silverman and Albano1996). Diagnosis at post-treatment was established by a blind rater.
Exclusion criteria were IQ <70. Twenty-one of the 32 children meeting the criteria of elevated scores of internalizing problems also met the inclusion criteria of GAD, SP or SAD. Eleven children were thus excluded. Families who declined to participate in the study were offered treatment as usual. As these 21 children were treated at three clinics, 1–2 intervention group(s) were formed at each clinic and no control condition could be established. The groups varied in size with a range from 4 to 7 children. Three children were lost to post-measurement due to high anxiety levels in the family, family problems and having started medication for attention deficit hyperactivity disorder during treatment. Final analysis was thus performed on 18 children.
The children completed a mean number of 9.2 sessions, with two booster session provided 3 and 4 months after the programme ended. Two children had only one booster session and two children did not attend booster sessions. There were three dropouts; after sessions 1, 2 and 5, respectively.
Measures
A number of measures were used in the study. The measures included in this paper focus on the symptoms and diagnoses of the children pre- and post-intervention.
Diagnosis
The ADIS-C/P is a structured interview to diagnose anxiety disorders in children and also screen for other common DSM-IV diagnoses. Severity (CSR) is rated on a Likert scale from 0 to 8 where a rating of 8 is considered most serious and 4 is the cut-off for diagnosing the disorder. The reliability of the symptom scales of SAD, SP, specific phobia and GAD has been characterized as ‘excellent’ (Silverman et al. Reference Silverman, Saavedra and Pina2001).
Child self-reports
The Multidimensional Anxiety Scale for Children (MASC; March, Reference March1997) is a 39-item measure of anxiety in children aged 8–19 years. The questions cover four areas; physical symptoms, avoidance, social anxiety and separation anxiety/panic. Test–retest reliability is satisfactory and found to be between 0.7 and 0.9 on the different subscales (March et al. Reference March, Parker, Sullivan, Stallings and Connors1997). The MASC has been shown to differentiate children with anxiety from children with other disorders (March, Reference March1997). The Recent Mood and Feelings Questionnaire (MFQ; Angold, Reference Angold and Thompson1989) measures symptoms of depression in children and adolescents. Thirty-four items are measured on a 3-point scale. The MFQ has acceptable reliability and is a satisfactory screen for depressive disorders in children (Wood et al. Reference Wood, Kroll, Moore and Harrington1995).
Parent reports
Parents reported their child's internalizing and externalizing problems on the CBCL (Achenbach, Reference Achenbach1991). The CBCL has been extensively evaluated and has excellent psychometric properties.
The parents completed a Norwegian version of a user-satisfaction questionnaire, Fragebögen zur Beurteilung der Behandlung (FBB; Mattejat & Remschmidt, Reference Mattejat and Remschmidt1998) after the treatment. The scale goes from 0 to 1.5 (not satisfied), 1.5–2.5 (partly satisfied), 2.5–3.5 (largely satisfied) to 3.5–4.0 (completely satisfied). FBB has good psychometric properties.
Treatment materials
A Norwegian version of the Friends for Life programme (Barrett, Reference Barrett2004) was used as a treatment protocol in the pilot study. The treatment is conducted in a group format and the parents are included at the end of every session. The programme is based on three major components; Learning/behaviour, Cognitive, and Physiological, and skills are taught in all areas. These skills are summarized in the name of the programme, FRIENDS, which serves as an acronym, helping the children remember the skills they learn. The programme is structured with 10 group sessions and two booster sessions. Homework activities are given after each session, and the children use workbooks throughout the programme while the group leaders are provided with a treatment manual.
Two psychologists in the project were trained and licensed as Friends facilitators at Paula Barrett's clinic: Pathways Research Centre. Psychologists from the project team ran the group sessions together with clinicians from the outpatient clinic acting as co-therapists. These clinicians participated in a 2-day training in the programme followed by monthly supervision meetings during the study.
Results
Diagnostic status, severity, comorbidity and change in diagnostic status
Table 1 indicates that many children (83%) met the criteria for more than one anxiety disorder at T1, with a mean severity rating varying from 5.6 to 6.5. SAD was the most frequent diagnosis found in the sample.
Anxiety and severity present at T1 (n = 18)
Taking all disorders into consideration, the most common comorbid disorder other than anxiety was depression/dysthemia with a mean severity rating of 5.4. Of the 18 children included, three scored over the clinical range (T > 64) on the externalizing scale of the CBCL.
Investigations of changes in diagnostic status from pre- to post-treatment yielded a mixed picture. We found ~33% reduction in number of diagnoses at T2. SAD and depression showed the greatest absolute change with fewer children meeting the diagnostic criteria at T2. Four children no longer met the criteria for SAD and depression after treatment (see Table 2).
Change in diagnostic status from pre- to post-treatment (n = 18)
Clinical meaningful change at diagnosis level-severity rating (CRS)
The mean severity rating for the group (n = 18) at T1 was 6.15. This was only slightly reduced at T2 with a mean of 5.84.
At T2 the number of children meeting the criteria for any disorder was reduced from 18 to 15 as three children no longer fulfilled the criteria for any diagnosis. However, while 15 children still met the criteria for the diagnosis, the severity rating may have been reduced at T2, which may constitute clinically meaningful change. However, for most of the disorders there was no or only a modest reduction in severity rating at T2 (see Table 3). The exception was for SAD where pairwise t tests indicated a significant reduction in mean severity rating [t(6) = 2.714, p < 0.05] from pre- (mean = 6.86, s.d. = 0.90) to post-treatment (mean = 5.57, s.d. = 1.51). SP showed a rather strong reduction in severity rating from 6.57 to 5.29, but with no significance [t(6) = 1.996, p = 0.93].
Severity rating pre- and post treatment for children still meeting diagnostic criteria at T2
For depression as a comorbid condition the severity rating at T1 ranged from 5 to 8 with a mean of 5.5. As four of the five children not longer met the diagnostic criteria for depression at T2, severity rating for only one child remained at T2, being unchanged at 8.
Clinical meaningful change using self-report – RCI
Earlier, clinical significant change has been studied with regard to the change in severity status of the children at diagnosis level. The change may also be reflected in self-report measures. t tests showed no significant changes in the group from pre- to post-treatment concerning anxiety symptoms [MASC: t(17) = 0.400, p = 0.694] or depression symptoms [MFQ: t(16) = −0.731, p = 4.75]. The RCI is a statistical approach for measuring individual change in psychological disorders assessed by self-report. If the change found by RCI exceeds what can be expected based on measurement error alone, we have a statistical approach to measure clinically significant change.
All the children (n = 18) reported their symptoms of anxiety and depression pre- and post-treatment. RCI was calculated for 17 children, as one child was excluded due to incomplete data, regardless of diagnostic status.
The results in Table 4 indicate that six of the 18 children reported a significant reduction in their anxiety symptoms and one child (number 6) showed a negative change. None of the changes in self-reported depression symptoms were significant.
RCI for self-report measures MASC and MFQ
RCI, Reliable Change Index; MASC, Multidimensional Anxiety Scale for Children; MFQ, Mood and Feelings Questionnaire.
RCI ≥1.96 indicates significant change; negative sign = symptom reduction = positive effect. Reliability values used for the calculations of MASC total score: test–retest reliability, r = 0.933 (March, Reference March1997), MFQ total score: test–retest reliability, r = 0.80 (Sund et al. Reference Sund, Larsson and Wichstrøm2001).
User satisfaction – FBB
Another important feature of the study was to evaluate user satisfaction with treatment. The FBB measurement was used for this purpose and was scored by the parents. Most of the parents rated the programme positively with 12 of the parents rating the treatment as ‘largely satisfied’ or ‘completely satisfied’ with a mean score of 3.
However, user satisfaction was rated on two dimensions; Successfulness of treatment and Treatment process. Treatment process indicates if the parents considered that their children were appropriately treated. All parents rated Treatment process as ‘largely satisfied’ or ‘completely satisfied’ with a mean score of 3.4. In general, Successfulness of treatment was rated lower with only four parents reporting ‘largely satisfied’ or ‘completely satisfied’ and 10 rating ‘not satisfied’ or ‘partly satisfied’ on this measure. The mean score here was 2.2.
Evaluation of a structured intervention by the clinicians
All clinicians at the outpatient clinics participating in the study were asked to evaluate the usefulness of the intervention in ordinary clinical care and were invited to discuss their experiences at a 1-day conference. All the therapists wanted to continue using the structured treatment. They reported that using a high-quality structured manual was useful when treating children with anxiety disorders. Conducting treatment in a group format was also evaluated positively by the clinicians and as a possible economical way of treating these children. Moreover, it was put forward that the clinics had gained more knowledge regarding both methodology and screening measures.
Discussion
The aims of the pilot study were to evaluate a structured treatment manual in ordinary clinical care and to test the acceptability of the programme.
The children included in the study had high levels of comorbidity and most met the diagnostic criteria for more than one anxiety disorder, and also had other disorders such as depression and externalizing problems. This range of difficulties was included as comorbid conditions are typical in ordinary clinical settings. At a diagnostic level the largest change observed was for SAD where 7/11 children no longer qualified for the diagnosis after treatment. For depression, 4/5 children no longer met the diagnostic criteria at T2.
Clinical meaningful change was considered by looking at changes in severity rating for the children still having the disorder at T2 and by using a statistical approach, the RCI, for evaluating clinically meaningful change at a self-reported symptom level. For the disorders SAD and SP in particular we found a substantial reduction in severity rating at T2 and for SAD the change was also statistically significant. Statistical analysis of self-report measures further indicated that 1/3 of the children showed clinically meaningful change at T2. None of the changes in self-reported depression were significant which is surprising as change in depression was noted at the diagnostic level. However, diagnosis is based on information from the parents, and discrepancy between parent information and child self-report is often reported in the literature (Kazdin, Reference Kazdin, Hibbs and Jensen1986).
The effects found in this pilot study may indicate that the results found in previous studies in university clinics are difficult to replicate in ordinary clinics. However, this study does have several limitations which may explain the low effect observed.
The acceptability of the programme indicates that structured approaches are welcome in ordinary clinical practice. Most of the parents were satisfied with the treatment in general. They considered that their children were treated in an appropriate way (treatment process) but were less satisfied with the successfulness of the treatment. This finding is understandable given the relatively high number of children still having symptoms of anxiety after treatment.
Limitations
Undertaking treatment research in Norway involves certain methodological challenges, and the present study has several limitations. The size of the outpatient clinics and the catchment area for recruiting children in Norway are relatively small and it was challenging to recruit enough children to establish a control condition. The children in the study had a wide range of comorbid problems, including externalizing problems, a condition not targeted by the programme. This may have contributed to the lack of response to the programme.
Implications for future studies
Despite the limitations and challenges in conducting outcome research in ordinary clinical care in Norway, this pilot study is the first to evaluate the Friends for Life programme in Norway and has provided important knowledge for future research. Given the small treatment effects, different actions could be taken to improve the effectiveness of the programme in a future study. One possibility might be to increase the specificity in the groups. This could be done by limiting the range of comorbidity and possibly having fewer anxiety disorders represented. The intensity in the programme might be increased by including more exposure in the sessions as is done in other programmes. The dosage could also be increased by having longer or more sessions and thus more time to cover the content of the programme. For some of the anxiety disorders, e.g. SP, an individual format could also be considered as being more beneficial than a group situation.
Furthermore, in order to assess ‘meaningful change’ future studies could benefit from including measures that better capture this idea. One possibility could be to operationalize each child's treatment goal in behavioural terms and have the parents assess these pre- and post-treatment.
Given the difficulty of recruiting the necessary number of children into the study, future research on structured interventions in a group format will probably have to include multiple sites. These sites would ideally be located in close proximity so that treatment groups could be conducted at one site with participants recruited from a larger catchment area.
As most therapists in the present study were positive towards the treatment approach, it is likely that ordinary clinics could find it beneficial to participate in a larger study. Training of the therapists both with regard to the programme, and assessment as well as providing continuous supervision is essential to ensure the quality of the diagnostic process and fidelity to the programme used. However, this might be a challenge with regard to the ecological validity of the results.
Addressing these limitations and dilemmas in future studies may enable us to evaluate structured interventions in ordinary clinical care more fully and thus contribute to the effectiveness research which eventually may be crucial in bridging the gap between research and practice.
Summary of main points
• Structured treatments are welcome in ordinary clinical care.
• Effect of treatment may be measured using alternative approaches.
• Clinical meaningful change may be a useful measure of bridging the gap between research and clinic.
• Conducting treatment research in ordinary clinics presents challenges with regard to recruiting a sufficient number of children and thus establishing control conditions.
Declaration of Interest
None.
Learning objectives
• Considering alternative ways of measuring effect of treatment – clinical meaningful change.
• Evaluating usefulness of a structured treatment programme for clients, parents and practitioners.
Comments
No Comments have been published for this article.