Introduction
Cognitive behaviour therapy (CBT) is the most studied psychological treatment for anxiety disorders in children and adolescents (both age groups are hereafter referred to as youths, unless age differences are specifically considered). A recent meta-analysis of psychotherapeutic treatment of youth anxiety disorders identified 55 RCT studies, of which 48 were of CBT (Reynolds, Wilson, Austin and Hooper, Reference Reynolds, Wilson, Austin and Hooper2012). Although the efficacy of CBT for youth anxiety disorders is well established, studies of the specific components of CBT have been somewhat neglected. Homework assignments are a standard component in CBT for anxiety disorders in youth, and are often referred to as an essential component in successful CBT treatment (Hudson and Kendall, Reference Hudson and Kendall2002). Research on homework in CBT has mainly focused on two aspects: 1) the effect of including homework in therapy (i.e. experimental studies comparing treatment conditions with and without homework assignments); and 2) the association between homework adherence (i.e. the quantity or quality of homework carried out [also referred to as homework compliance]) and treatment outcome.
The latest meta-analysis of the effect of including homework assignments in CBT with no age restrictions found that treatments that included homework assignments had significantly better pre-post outcome compared to similar treatments that did not include homework assignments, with a controlled mean effect size of d = 0.48 (based on nine controlled studies) (Kazantzis, Whittington and Dattilio, Reference Kazantzis, Whittington and Dattilio2010). Of the total 46 studies identified in Kazantzis et al. (Reference Kazantzis, Whittington and Dattilio2010), 26 were of anxiety disorders and none of these included youths. The first meta-analysis to examine the association between homework adherence (HA) and treatment outcome identified 20 studies, of which four were of anxiety disorders, and found an overall mean effect size of r = .22;r = .27 for anxiety disorders (Kazantzis, Deane and Ronan, Reference Kazantzis, Whittington and Dattilio2000). A more recent meta-analysis, including only studies from 2000 to 2008, identified 23 studies, of which eight were of anxiety disorders, and found small to medium mean effect sizes (r = .26; r = .22 for anxiety disorders) (Mausbach, Moore, Roesch, Cardenas and Patterson, Reference Mausbach, Moore, Roesch, Cardenas and Patterson2010). Results were significantly heterogeneous. Moderator analyses in Mausbach et al. (Reference Mausbach, Moore, Roesch, Cardenas and Patterson2010) indicated that results were robust across target symptoms (e.g. anxiety and depression), but differed by source, timing, and type of homework rating. Larger effect sizes were found with: 1) combined client and therapist ratings, compared to therapist only or objective assessments; 2) retrospective ratings after therapy, compared to contemporaneous assessments during the course of therapy; and 3) Likert type assessments, compared to percentage of homework completed (Mausbach et al., Reference Mausbach, Moore, Roesch, Cardenas and Patterson2010).
In prior meta-analyses of HA and treatment outcome, only a single study involving youths with anxiety disorders was included (Hughes and Kendall, Reference Hughes and Kendall2007). An updated systematic search in PsycINFO and PubMed, using the same criteria as Mausbach et al. (Reference Mausbach, Moore, Roesch, Cardenas and Patterson2010) from September 2008 till September 2014, revealed one additional study of HA and treatment outcome for youths with obsessive compulsive disorder (OCD) (Park et al., Reference Park, Small, Geller, Murphy, Lewin and Storch2014).
The study by Hughes and Kendall (Reference Hughes and Kendall2007) included 138 children, ages 9 to 13 years, diagnosed according to DSM-III with primary diagnoses of either overanxious disorder, avoidant disorder or separation anxiety disorder. The sample was pooled from two prior efficacy studies in which children received 16 individual CBT sessions (Kendall, Reference Kendall1994; Kendall et al., Reference Kendall, Flannery-Schroeder, Panichelli-Mindel, Southam-Gerow, Henin and Warman1997). The therapists rated the child's HA at each session using a 7-point Likert scale ranging from 1 (no homework adherence) to 7 (total adherence). The therapeutic relationship between the child and the therapist was also rated at each session. The results of separate multiple regression analyses showed that therapeutic relationship predicted outcome at posttreatment and 1-year follow-up, but that HA was not a significant predictor of outcome at either time-point (Hughes and Kendall, Reference Hughes and Kendall2007). The study by Park et al. (Reference Park, Small, Geller, Murphy, Lewin and Storch2014) consisted of 30 youths, aged 8 to 17 years, with a primary diagnosis of OCD who received 10 individual CBT sessions. The therapist rated the youth's HA at each session using a 7-point Likert scale ranging from 0 (did not complete any assigned homework) to 6 (completed all homework and made efforts above and beyond assignments). The aim of their randomized controlled trial was to examine if D-cycloserine could improve HA in CBT, but no association was found. However, using the pooled sample, results of a random effects model indicated that HA significantly predicted decline in OCD severity during treatment (Park et al., Reference Park, Small, Geller, Murphy, Lewin and Storch2014).
Studies of the association between HA and CBT outcome for youths with depression are also few and with inconsistent findings. One study has found an association between HA and change in adolescents’ depressive symptoms (Simons et al., Reference Simons, Marti, Rohde, Lewis, Curry and March2012), while two other studies have found no such association (Clarke et al., Reference Clarke, Hops, Lewinsohn, Andrews, Seeley and Williams1992; Shirk, Crisostomo, Jungbluth and Gudmundsen, Reference Shirk, Crisostomo, Jungbluth and Gudmundsen2013). While a number of studies of CBT for adults seem to indicate a small to medium effect size for an association between HA and treatment outcome, it is clear that more studies are needed within the area of youth anxiety disorders. The purpose of the present study was to examine the association between HA and the outcome of a generic group CBT treatment program for youths diagnosed with DSM-IV anxiety disorders.
Method
Participants
Participants were 98 youths (55 girls [56.1%]), ages 7 to 16 years (M = 11.80, SD = 2.47), with a primary anxiety disorder and who had received treatment as part of a randomized wait-list controlled trial of a group CBT program (Cool Kids; Rapee et al., Reference Rapee, Lyneham, Schniering, Wuthrich, Abbott and Hudson2006) for youth anxiety disorders that showed high effect sizes (Arendt, Thastum and Hougaard, in press). Analyses in the present study were conducted on the pooled data from participants in the CBT condition and the wait-list condition after they had received treatment (there were no significant differences in mean outcome measures at pretreatment between the two groups). Only participants who had completed treatment and filled out the relevant measures were included in the analyses, which excluded 11 of the original 109 participants. Of the 11 excluded, 7 did not wish to receive treatment after the wait-list period; 3 had participated in less than 7 of the 10 treatment sessions and were thus considered dropouts; one had registered less than 6 of the 9 ratings for homework adherence (see below). All participants were diagnosed with the Anxiety Disorders Interview Schedule for DSM-IV, Parent and Child Versions (Silverman and Albano, Reference Silverman and Albano1996). Primary diagnoses at pretreatment were: 32 (32.7%) separation anxiety disorder, 22 (22.4%) generalized anxiety disorder, 16 (16.3%) social phobia, 13 (13.3%) specific phobia, 8 (8.2%) OCD, 5 (5.1%) agoraphobia without panic disorder, and 2 (2.0%) panic disorder with agoraphobia.
Measures
The Anxiety Disorders Interview Schedule for DSM-IV, Parent and Child Versions (ADIS-IV C/P) (Silverman and Albano, Reference Silverman and Albano1996) is a semi-structured diagnostic interview based on DSM-IV. Separate interviews are conducted with youths and their parents, and all diagnoses are rated by youths, parents, and clinicians with a clinical severity rating (CSR). CSRs range from 0 (no interference) to 8 (extreme interference) and severity ratings of 4 or above indicate the presence of a disorder. Only CSRs rated by the clinicians are used in the present study. The primary diagnosis was defined as the most impairing diagnosis, as assessed by the clinician. All diagnostic interviews were conducted by clinical psychologists or graduate students trained in the use of ADIS-IV C/P. Student assessors received supervision on all their interviews.
ADIS-IV C/P has demonstrated good to excellent test-retest reliability (Silverman, Saavedra and Pina, Reference Silverman, Saavedra and Pina2001) and strong concurrent validity (Wood, Piacentini, Bergman, McCracken and Barrios, Reference Wood, Piacentini, Bergman, McCracken and Barrios2002). In the original RCT study, the interrater reliability (κ) for the primary anxiety diagnosis was .77 and the intraclass correlation coefficient for the CSR of the primary anxiety diagnosis was .69 (see Arendt et al., in press, for further details of the study).
The Spence Children's Anxiety Scale (SCAS) is a self-report rating scale for assessing youth anxiety symptoms (Spence, Reference Spence1997). The scale contains 44-items (including six positive filler items), rated from 0 (never) to 3 (always). It consists of six subscales reflecting symptoms of specific anxiety diagnoses: social phobia (SoP), panic disorder and agoraphobia, generalized anxiety disorder (GAD), obsessive compulsive disorder (OCD), separation anxiety disorder (SAD), and specific phobias (SP [fear of physical injury]). Each subscale may be scored separately as well as added together for a score of overall anxiety symptoms. The Danish translation of SCAS has demonstrated excellent internal consistency for the total scale (α = .89) in a sample of youths with anxiety disorders, and good test-retest reliability after 2 weeks (r = .84) and 3 months (r = .83) in a community sample (Arendt, Hougaard and Thastum, Reference Arendt, Hougaard and Thastum2014). Internal consistency for the total scale in the current sample was excellent (α = .91).
Homework adherence was measured at the beginning of each session. Youths and parents were asked to estimate “How much time have you spent on average per day doing homework since the last session?” Answer categories were: 5, 10, 20, 30, 40, 50, 60 or more than 60 minutes. The last category (more than 60 minutes) was used eight times across five different raters and was, for the purpose of analyses, changed to a score of 70 minutes. An average of minutes spent on doing homework per day was calculated for each participant by summing the scores for all sessions and dividing it by the number of ratings. Youths and parents were seated together during the HA assessment and could see each other's ratings. Youths rated how much time they had spent on homework during the past week before the mothers and fathers filled out a consensual rating of how much time they (i.e. the parents) on average had spent on doing homework the past week. Assessment of parental HA included the amount of time parents spent on assisting youths with their homework assignments as well as the time parents spent on doing homework assignments specific for parents (see below).
Clinician rated ADIS CSR on the primary diagnosis and child reported SCAS served as outcome criterion measures in the study (for further outcome measures in the study, see Arendt et al., in press).
Treatment
The treatment consisted of 10 weekly 2-hour group sessions with a manual-based CBT program for mixed anxiety disorders in youths (Cool Kids; Rapee et al., Reference Rapee, Lyneham, Schniering, Wuthrich, Abbott and Hudson2006). The treatment focuses on teaching youths to recognize their emotions, restructure negative automatic thinking, and gradually confront feared situations. Each group was led by a clinical psychologist and consisted of six to seven youths (in three age groups: 7–9, 10–12, or 13–16 years) and their parents. Furthermore, three graduate students participated in each group and assisted families during in-session assignments. Three months after the end of treatment participants were offered a one-hour booster group session (for further details about the treatment, see Arendt et al., in press). The Cool Kids Program has been shown to be efficacious in studies conducted in Australia by its developers (Hudson et al., Reference Hudson, Rapee, Deveney, Schniering, Lyneham and Bovopoulos2009) and in an independent Danish efficacy study (Arendt et al., in press).
The homework assignments in the Cool Kids program consisted mainly of cognitive restructuring (from session 1), gradual exposure (from session 4), and training social skills and assertion (from session 7). In addition to assisting the youths in implementing homework assignments in everyday life, parents had additional homework that consisted of learning to reward youths for non-anxious behaviour (i.e. approach behaviour instead of avoidance and utilizing exercises for cognitive restructuring) and to react more appropriately to the youth's anxious behaviour (e.g. less reassurance and overprotective behaviour).
Procedure
All participating families had referred themselves to the Anxiety Disorder Clinic, Aarhus University, Denmark, in response to the clinic's webpage, newspaper advertisements, or recommendations from local community health services. Families were required to send in a letter or an e-mail to the clinic, describing the youth's anxiety symptoms, and relevant families were invited to the clinic for an initial diagnostic assessment (ADIS-IV-C/P). All families completed diagnostic ADIS-IV interview and electronically administrated rating scales, including SCAS, before starting the treatment. Assessment at posttreatment and 3 months after treatment consisted of ADIS-IV C/P interviews and electronically administered rating scales. At post and follow-up assessments, ADIS assessors were blinded to the youth's prior diagnoses.
Statistical analyses
Preliminary multiple regression analyses were conducted separately for gender and age to examine if these variables predicted outcome at posttreatment or 3-month follow-up. When pretreatment scores were also entered as predictors, neither gender nor age were significant predictors of outcome on SCAS or ADIS CSR at posttreatment or 3-month follow-up. Thus, age and gender were not included as predictors in the main analyses.
Main analyses were conducted as stepwise multiple regression analyses with two separate criteria variables (SCAS and ADIS CSR) at posttreatment and at 3-month follow-up. The pretreatment criterion variable was inserted in the first step, while HA was inserted in the second step. Separate analyses were conducted for both of the criteria variables at posttreatment and at 3-month follow-up with youth, as well as parent-reported HA, which resulted in a total of eight analyses.
Analyses were based on data from the 98 treatment completers (at least seven sessions) with at least six homework assessments. Some participants had missing assessment points: one SCAS assessment was missing at pretreatment, five were missing at posttreatment, and eight were missing at 3-month follow-up; for ADIS CSR, four assessments were missing at 3-month follow-up. Listwise exclusion was used for missing assessment points in the analyses. All statistical analyses were carried out using Statistical Product and Service Solutions (SPSS) version 21.0.0.2.
Results
Descriptive results
Mean scores and standard deviations for SCAS and ADIS CSR across the three time points can be seen in Table 1. On average, youths reported having spent 16.91 (SD = 10.01) minutes on homework per day, while parents reported having spent 16.57 (SD = 9.61) minutes per day. Correlations between youth and parent reports of HA were very high (r = .88, p < .001). Correlations between youth reported HA and pretreatment severity was r = .38, p < .001 for SCAS and r = .10, p = .333 for ADIS CSR. Correlations between parent reported HA and pretreatment severity was r = .30, p = .003 for SCAS and r = .09, p = .376 for ADIS CSR.
Means and standard deviations (in parentheses) for SCAS and ADIS CSR across time
Notes: ADIS = Anxiety Disorder Interview Schedule for DSM-IV; CSR = Clinician Severity Rating (ADIS); SCAS = Spence Children's Anxiety Scale
Homework adherence and treatment outcome
Tables 2 and 3 show the results of the multiple regression analyses at posttreatment and 3-month follow-up respectively. At posttreatment, HA was not a significant predictor of clinician rated or child self-reported outcome. At 3-month follow-up, only parent rating of HA was significant for clinician rated outcome, with more time spent on homework predicting lower anxiety severity.
Summary of multiple regression analyses for ADIS CSR and SCAS at posttreatment
Notes: ADIS = Anxiety Disorder Interview Schedule for DSM-IV; CSR = Clinician Severity Rating (ADIS); HC = homework compliance; SCAS = Spence Children's Anxiety Scale
Summary of multiple regression analyses for ADIS CSR and SCAS at 3-month follow-up
Notes: ADIS = Anxiety Disorder Interview Schedule for DSM-IV; CSR = Clinician Severity Rating (ADIS); HC = homework compliance; SCAS = Spence Children's Anxiety Scale
Discussion
The study examined the association between HA and outcome of a generic CBT program (Cool Kids) for youths with anxiety disorders and their parents. Only two prior studies have examined this association and they found mixed results. Hughes and Kendall (Reference Hughes and Kendall2007) found no association between HA and outcome of an individual CBT program for children diagnosed with overanxious, avoidant, or separation anxiety disorder according to DSM-III. In contrast, Park et al. (Reference Park, Small, Geller, Murphy, Lewin and Storch2014) found a significant association between HA and outcome of a CBT treatment program for youths with a primary diagnosis of OCD. Homework in their study almost exclusively consisted of exposure and response prevention, components that have been found to be essential in CBT for OCD (Rosa-Alcázar, Sánchez-Meca, Gómez-Conesa and Marín-Martínez, Reference Rosa-Alcázar, Sánchez-Meca, Gómez-Conesa and Marín-Martínez2008).
The results of the present study are in line with a similar study by Hughes and Kendall (Reference Hughes and Kendall2007) in which there was no indication of the importance of homework adherence in CBT for youth anxiety disorders. In the present study, little variance in outcomes were accounted for by HA (R 2 range: .000–.046) as pretreatment scores of criteria variable explained the majority of variance in the models (R 2 range: .062–.470). The one significant association found between the HA of parents and outcome on CSR at 3-month follow-up (p = .034) in the present study could possibly be due to chance, since it was only one of eight investigated associations. There is some debate as to whether Bonferroni correction should be applied to multiple regression analyses (Mundfrom, Perrett, Shaffer, Piccone and Roozeboom, Reference Mundfrom, Perrett, Shaffer, Piccone and Roozeboom2006) and, in case of doing so, to which analyses it should be corrected. Even with correction for only the two outcome criterion measures (p < .025), the found association would be insignificant. Furthermore, it could be argued that prediction of outcome immediately after treatment should be considered the primary outcome criterion, as HA during treatment was measured, which is probably not a plausible explanation for the maintenance of treatment gains during the follow-up period.
As mentioned in the introduction, there are no experimental studies on the importance of homework in CBT for youth anxiety disorders and, including the present study, only two studies on HA for what may now be considered genuine anxiety disorders in youth; both studies show non-significant findings (OCD is no longer considered an anxiety disorder in DSM-5; American Psychiatric Association, 2013). Only one of three studies found an association between HA and outcome in CBT for depression in youths.
Although results from the literature on HA in CBT for adults are also inconsistent, significant mean effects (with small to medium effect sizes, r = .22–.27) of HA on outcome were found for adults with anxiety disorders (Kazantzis et al., Reference Kazantzis, Deane and Ronan2000; Mausbach et al., Reference Mausbach, Moore, Roesch, Cardenas and Patterson2010). It is unclear why similar results have not been found in the two studies of youths with mixed anxiety disorders. Perhaps it is more difficult to measure reliably the amount of client homework in youths (perhaps especially in children), even if it is assessed by therapists. Also, different methodical approaches may play a role, as indicated by the moderator analyses in the meta-analyses by Mausbach et al. (Reference Mausbach, Moore, Roesch, Cardenas and Patterson2010) where larger effect sizes were found for studies that, in contrast to all studies of HA in youths with anxiety disorders, used retrospective assessment after therapy instead of continuous assessment after each session. Retrospective assessment should, however, be assumed to be less reliable and more susceptible to contamination between predictor and criterion variables (i.e. clients as well as therapists may be biased to recall higher HA if the client had a good outcome).
The families in the present study were highly dedicated to treatment with a high attendance and low dropout rate (only three [2.9%] attended less than 7 of the 10 sessions in the original study). Thus it seems likely that the majority of the participating youths and their families were dedicated to carrying out enough homework to achieve an acceptable treatment result. However, based on the standard deviations of the mean HA of youths (M = 16.91, SD = 10.01) and parents (M = 16.57, SD = 9.61), there seems to be sufficient variation in the amount of homework conducted to allow for a significant association to be found, if there had been one.
Correlational process-outcome research can be susceptible to bias due to an association between a process component and the patient's need for the component (cf. Stiles and Shapiro, Reference Stiles and Shapiro1989). Thus, the finding of no correlation between length of treatment and outcome in some naturalistic psychotherapy studies has been explained by therapy length being tailored to clients’ needs (Stiles, Barkham, Connell and Mellor-Clark, Reference Stiles, Barkham, Connell and Mellor-Clark2008). In a similar way, it could be proposed that amount of homework would be adjusted to youths’ needs; for instance, some of those doing well in therapy might tend to do less homework, as they no longer feel the need for it, while some of those with less improvement might try to strengthen their efforts resulting in more homework. The moderate high correlations found between pretreatment SCAS severity and child and parent HA (respectively .38 and .30) could indicate such an adjustment of HA to youths’ needs, since higher severity could call for a larger amount of homework.
Such an adjustment could hold for CBT for adults as well as for youths, but for youths, homework should be expected to be primarily initiated by, and carried out together with, their parents. Parents could thus invest a great deal of effort to motivate their children to do the relevant homework, even in case of youth's demoralization or giving up. The high correlation between parent and child HA (.88) in the study could indicate such a high degree of parental influence on homework. Differences in the amount of homework adjustment to needs, due to parental influence on homework for youths, might thus be an explanation for the divergent results on HA in CBT for youths and for adults.
The association between need for and amount of homework could limit conclusions from correlational HA research. Other problems with such research consist of the many different ways HA has been measured. There is clearly a need for a more convergent use of measures of homework, as well as for the development of objective and valid measures (Kazantzis, Deane and Ronan, Reference Kazantzis, Deane and Ronan2004; Mausbach et al., Reference Mausbach, Moore, Roesch, Cardenas and Patterson2010). It is, however, difficult to envisage an easy to apply, more adequate measure of amount of homework than contemporaneous assessments by participants. Modern IT technology, for instance cell phone applications, might however provide a promising possibility in future studies.
It is also possible that different sorts of homework are of different value. As assessed in the Park et al. (Reference Park, Small, Geller, Murphy, Lewin and Storch2014) study on youths with OCD, it could be that amount of exposure homework is a more relevant variable than overall amount of HA in CBT for anxiety disorders. It was, however, not possible to specify the precise amount of homework spent on exposure in the present study.
It has also been suggested that quality of homework could be a more relevant indication of HA than amount of homework (Kazantzis et al., Reference Kazantzis, Deane and Ronan2004). A study of CBT for panic disorders in adults found that the quality of the homework completed was a better predictor of outcome than the quantity of homework carried out (Schmidt and Woolaway-Bickel, Reference Schmidt and Woolaway-Bickel2000). However, no studies have examined the association between treatment outcome and the quality of homework carried out by youths with anxiety disorders. Future studies would benefit from a clearer distinction between the quantity and quality of homework assignments carried out, and their distinctive predictive value of treatment outcome. Measuring the quality of specific homework assignments in a valid and objective way may, however, be difficult and time consuming (e.g. did the exposure task generate an appropriate amount of fear? Were safety behaviours used during exposure tasks? Was sufficient evidence evaluated for and against anxious thoughts in cognitive restructuring?). However, global measures of the quality of homework have been developed (Kazantzis et al., Reference Kazantzis, Deane and Ronan2004).
The present study has some limitations. First, HA was only assessed by quantity (minutes spent on doing homework) and no assessment of which assignments or the quality of the homework was done. Second, the assessment of parental HA may be biased as they saw the youths fill out their rating first. Furthermore, the assessment of parental HA was a consensual rating between mothers and fathers, although the approach and time spent on homework may differ between mothers and fathers. Finally, although the last category of the HA rating (more than 60 minutes) was only used eight times, the change here to a score of 70 minutes might eliminate relevant differences.
In conclusion, the present study is one of only two studies to examine the association between HA and outcome of CBT for youths with mixed anxiety diagnoses. The study found no convincing evidence that HA, in the form of parent- and youth-reported amount of time spent on homework, was related to outcome in a generic group CBT treatment program for youths with mixed anxiety disorders. The results are in line with the only other similar study, where HA in individual CBT for mixed anxiety disorders also did not predict outcome (Hughes and Kendall, Reference Hughes and Kendall2007). However, it is in contrast with mean results from HA studies on CBT for adults (Kazantzis et al., Reference Kazantzis, Deane and Ronan2000; Mausbach et al., Reference Mausbach, Moore, Roesch, Cardenas and Patterson2010). The different findings of homework adherence in CBT for youths and for adults might indicate methodological problems within correlational studies of homework that are especially prominent in CBT for youths.
Acknowledgements
The authors should like to thank Professor Jennifer Hudson for her insightful comments on the final draft of this paper.
Financial support
This study was funded by TrygFonden (grant ID n10691).
Conflict of interest
The authors have no conflicts of interest with respect to this publication.
Comments
No Comments have been published for this article.