Anxiety disorders and depression are highly prevalent among children and adolescents (Beesdo, Knappe, & Pine, Reference Beesdo, Knappe and Pine2009; Birmaher et al., Reference Birmaher, Ryan, Williamson, Brent, Kaufman, Dahl, Perel and Nelson1996; Cartwright-Hatton, McNicol, & Doubleday, Reference Cartwright-Hatton, McNicol and Doubleday2006; Mesman & Koot, Reference Mesman and Koot2001). These disorders negatively affect children's current (Donovan & Spence, Reference Donovan and Spence2000) and future functioning, and are likely to deteriorate when left untreated (Harland et al., Reference Harland, Reijneveld, Brugman, Verloove-Vanhorick and Verhulst2002). Only a minority of children with anxiety and depression receive professional care for their problems (Tick, Van der Ende, & Verhulst, Reference Tick, Van der Ende and Verhulst2008; Verhulst & Van der Ende, Reference Verhulst and Van der Ende1997; Zwaanswijk et al., Reference Zwaanswijk, Van der Ende, Verhaak, Bensing and Verhulst2005). Early prevention of these disorders is therefore important to prevent a negative spiral of mental health problems, which may be more difficult to tackle in adulthood.
FRIENDS for Life is a school-based prevention program targeting anxiety disorders and depression in children aged from 8 to 12 years (Barrett, Reference Barrett2004a, Reference Barrett2004b). It teaches children skills to cope more effectively with their feelings of anxiety and depression, and it aims to build emotional resilience, problem-solving abilities, and self-confidence. Children are taught how to recognise feelings of anxiety and depression, how to relax, what to do when feeling anxious or sad, how to identify negative thoughts, and how to change these into positive ones. They are also taught to approach difficult situations by making a step-plan, and to reward themselves when having tried and done the best, instead of focusing on failures. The program can be implemented in various forms: (1) as universal prevention for all children regardless of their risk for a disorder; (2) as selective prevention for specific groups at risk for developing a disorder; (3) as indicated prevention for children who show symptoms of anxiety, but who do not meet criteria for a clinical disorder; and (4) as treatment for children with a diagnosed anxiety disorder. Because previous studies showed the program's efficacy in reducing symptoms of anxiety and depression in children (e.g., Dadds, Spence, Holland, Barrett, & Laurens, Reference Dadds, Spence, Holland, Barrett and Laurens1997; Barrett, Sonderegger, & Sonderegger, Reference Barrett, Sonderegger and Sonderegger2001; Essau, Conradt, Sasagawa, & Ollendick, Reference Essau, Conradt, Sasagawa and Ollendick2012), it has been recommended by the World Health Organization (2004).
However, results regarding the efficacy of FRIENDS for Life as an indicated prevention program are inconsistent (Bernstein, Layne, Egan, & Tennison, Reference Bernstein, Layne, Egan and Tennison2005; Hunt, Andrews, Crino, Erskine, & Sakashita, Reference Hunt, Andrews, Crino, Erskine and Sakashita2009; Cooley-Strickland, Griffin, Darney, Otte, & Ko, Reference Cooley-Strickland, Griffin, Darney, Otte and Ko2011; Dadds et al., Reference Dadds, Spence, Holland, Barrett and Laurens1997; Miller et al., Reference Miller, Laye-Gindhu, Liu, March, Thordarson and Garland2011), and previous studies were conducted under carefully controlled conditions rather than in daily practice.
A recent Dutch study was the first to evaluate the effectiveness of FRIENDS for Life as an indicated prevention program when implemented in daily school practice. This study showed the program was able to reduce self-reported symptoms of childhood anxiety and depression up to 12 months postintervention (Kösters et al., Reference Kösters, Chinapaw, Zwaanswijk, Van der Wal and Kootin press).
Effectiveness of a prevention program is, however, not the only prerequisite for a successful implementation. Participants’ evaluation of a program is important as well, since this can provide insight into program aspects best and least liked by the participants. This may provide suggestions about how to fine-tune the program's implementation to best fit the needs of the target population.
Previous studies have shown that children generally evaluate FRIENDS for Life positively (e.g., Barrett, Moore, & Sonderegger, Reference Barrett, Moore and Sonderegger2000; Barrett, Sonderegger et al., Reference Barrett, Sonderegger and Sonderegger2001; Cooley, Boyd, & Grados, Reference Cooley, Boyd and Grados2004; Gallegos, Rodríguez, Gómez, Rabelo, & Gutiérrez, Reference Gallegos, Rodríguez, Gómez, Rabelo and Gutiérrez2012; Gallegos-Guajardo, Ruvalcaba-Romero, Garza-Tamez, & Villegas-Guinea, Reference Gallegos-Guajardo, Ruvalcaba-Romero, Garza-Tamez and Villegas-Guinea2013; Iizuka, Barrett, Gillies, Cook, & Miller, Reference Iizuka, Barrett, Gillies, Cook and Miller2014). The majority of these studies used standardised questionnaires to assess children's opinions of FRIENDS for Life. We believe that investigating children's opinions using qualitative methods will add to the current knowledge because it offers the opportunity to obtain more detailed information about participants’ views.
Most studies investigating participants’ opinions, or the ‘social validity’, of FRIENDS for Life included only children as informants. However, Barrett, Shortt, Fox, and Wescombe (Reference Barrett, Shortt, Fox and Wescombe2001) showed that parents, children, and adolescents each had different views on the FRIENDS program. This underlines the need to include multiple informants when studying the social validity of the program.
This study aims to investigate children's and parents’ evaluations of the Dutch version of FRIENDS for Life using qualitative methods: online focus groups (OFGs) and interviews. Previous research has shown that OFGs are a feasible tool for collecting data from children (Tates et al., Reference Tates, Zwaanswijk, Otten, Van Dulmen, Hoogerbrugge, Kamps and Bensing2009; Zwaanswijk et al., Reference Zwaanswijk, Tates, Van Dulmen, Hoogerbrugge, Kamps and Bensing2007) as they allow children to use their own words in formulating responses and provide them the opportunity to resist the researcher's control of the research process (Mayall, Reference Mayall1994). Moreover, the anonymity in OFGs has been shown to allow participants to speak more freely, particularly regarding sensitive topics (Montoya-Weiss, Massey, & Clapper, Reference Montoya-Weiss, Massey and Clapper1998; Joinson, Reference Joinson2001; Reid & Reid, Reference Reid and Reid2005; Walston & Lissitz, Reference Walston and Lissitz2000). This may be particularly useful for anxious children.
We investigated FRIENDS for Life as an indicated prevention program for children who show symptoms of anxiety or depression. Children's and parents’ general opinions about the program, their perceptions of the effectiveness of the program, and children's opinion about the group atmosphere were studied. In addition, parents’ suggestions for improvement of the program were investigated.
Methods
This study is part of a larger research project designed to evaluate the effectiveness of FRIENDS for Life as a school-based indicated prevention program for children with symptoms of anxiety or depression, in Grades 6, 7 and 8 of primary schools in Amsterdam, the Netherlands (Kösters et al., Reference Kösters, Chinapaw, Zwaanswijk, Van der Wal, Utens and Koot2012). The Medical Ethics Committee of VU University Medical Centre approved the study protocol.
The Dutch translation of FRIENDS for Life (Utens & Ferdinand, Reference Utens and Ferdinand2006a, Reference Utens and Ferdinand2006b) has been implemented at schools in Amsterdam since 2007. The program consists of 10 weekly sessions of 90 minutes each. Each group is led by two prevention workers from a local mental health organisation. Because the prevention workers noticed time constraints of schools and low attendance of parents, the implementation of only one booster session (1 month after the program) and one parent session has become common practice in Amsterdam over the years, which is in contrast to the original protocol (Barrett, 2004a, Reference Barrett2004b). Instead of a second parent session, parents and children are invited for a meeting with the prevention workers in which the child's progress is evaluated. This meeting is organised after the last regular program session. All program sessions are organised during school time.
A total number of 328 children attending 23 elementary schools in Amsterdam participated in FRIENDS for Life during the school years 2010–2011 and 2011–2012. The intention to treat sample consisted of 339 children, of whom 6 did not start and 5 withdrew during the program. Most children who did not finish the program were removed from the program because of externalising behaviour, and nine of them were boys. There were no significant differences between children who finished and those who did not finish the program with respect to child age or child-reported anxiety or depression at baseline. Children participating in the program were 8–13 years old (mean = 10.6, SD = 0.9), and 61.7% were girls (Table 1).
Characteristics of Participants of FRIENDS for Life and Online Focus Groups
Note: RCADS = Revised Child Anxiety and Depression Scale; M = mean; SD = standard deviation. aIntention to treat sample. bNumber of children for whom data were available. Data on children's appraisal were available for only 229 children, since these indicators were added during the study. cAge before participating in FRIENDS for Life (T1). *Difference between FRIENDS for Life group and OFG group is statistically significant (chi-square test, p < .05). Ranges: Socioeconomic status: -7.2–3.2; Usefulness and Enjoyableness: 0–4; RCADS Anxiety: 0–111; RCADS Depression: 0–30.
In total, 35 FRIENDS for Life groups were organised. The program started two times per school year during two succeeding school years, leading to four periods of implementation. Children attended on average 9.1 (SD = 1.3) out of 10 sessions. Before (T1) and after their participation in FRIENDS for Life (T2), children were asked to rate the presence of symptoms of anxiety and depression by using the Revised Child Anxiety and Depression Scale (RCADS; Chorpita, Yim, Moffitt, Umemoto, & Francis, Reference Chorpita, Yim, Moffitt, Umemoto and Francis2000). At T2, children's appraisal of the program was measured by asking them to rate on a 5-point Likert scale how useful (very little = 0, little, not little/much, much, very much = 4) and how enjoyable (very boring = 0, boring, not boring/fun, fun, a lot of fun = 4) they considered the program. Socioeconomic status (SES) scores were retrieved via children's postal codes. The Netherlands Institute for Social Research (2012) computed an SES score based on education level, income and labour market position of the inhabitants of every postal code area in the Netherlands. For more information, see Kösters et al. (Reference Kösters, Chinapaw, Zwaanswijk, Van der Wal, Utens and Koot2012).
Participants and Procedure of OFGs
We had permission to contact 227 of the children who participated in FRIENDS for Life. After their participation in the program, these children were asked by letter to participate in one of four OFGs, which were conducted in 2011 and 2012. A reminder was sent 2–3 weeks after the initial letter. Written consent was obtained from 85 children, of whom 44 actually participated. Mean age of the OFG participants was 10.7 years (SD = 0.9). Most OFG participants (70.5%) were female (Table 2).
Online Focus Groups and Parent Interviews: Response Rates and Characteristics of Participants
Note: M = mean, SD = standard deviation. Age of the participating parents was not known.
The OFGs were conducted using a web-based application that was developed as part of a previous study (Tates et al., Reference Tates, Zwaanswijk, Otten, Van Dulmen, Hoogerbrugge, Kamps and Bensing2009; Zwaanswijk et al., Reference Zwaanswijk, Tates, Van Dulmen, Hoogerbrugge, Kamps and Bensing2007). On average, they were organised 37 days after the last (10th) session of FRIENDS for Life. The OFGs were conducted in an asynchronous form (Rezabek, Reference Rezabek2000; Montoya-Weiss et al., Reference Montoya-Weiss, Massey and Clapper1998), that is, participants could read others’ comments and could respond at any time, not necessarily when anyone else was participating. This allowed children to respond from their home at any time convenient to them. The participants received individual log-in names and passwords with which they could anonymously access the OFG during one week. On the third day, children who had not yet reacted received a reminder by email.
A new question was introduced by the researchers on each of the first 5 days. Questions concerned children's general opinion about the program, their opinion about the group atmosphere (e.g., What did you think about the other children in the group? What did you think about the trainers? Did you dare to say everything you wanted?), and their perception of the effectiveness of the program (e.g., Do you use the skills you have learned? What have you learned from FRIENDS for Life?). On the fifth day, children were also invited to introduce new issues they considered relevant. Questions of the previous days remained open for responses during the whole week. The researchers monitored the discussion and asked additional questions if necessary.
In total, 242 reactions were posted, with a mean of 5.5 postings per child (range: 1–11). Most children reacted to each question at least once.
Participants and Procedure of Parent Interviews
Of each of the 35 FRIENDS for Life groups, one or two parents were randomly selected to participate in a brief structured telephone interview after their child had finished the program. Forty-five parents were approached, and 38 participated (84.4%). If parents did not sufficiently master the Dutch language, an interpreter conducted the interview in Turkish, Berber or Arabic (n = 6). Most participants (78.9%) were female (Table 2). Parent interviews were conducted in 2011 and 2012. Questions concerned parents’ general opinion about the program, their perception of the effectiveness of the program (e.g., Has your child learned something from the program? If so, what?), and suggestions for improvement of the program (Have you missed certain aspects in FRIENDS for Life?).
Data Analysis
Chi-square tests and t tests with p < .05 were used to test differences between children who participated in FRIENDS for Life and OFG participants.
Key aspects of children's and parents’ opinions about FRIENDS for Life were selected. The two authors each read the transcripts independently and constructed a preliminary thematic coding scheme. Disagreements were discussed until consensus was achieved.
Results
Representativeness of the OFG Sample
OFG participants were representative of the sample participating in FRIENDS for Life regarding gender, SES, children's appraisal of the program, and child-reported anxiety and depression symptoms assessed before and after participation in the program (Table 1). A small but significant difference between the FRIENDS for Life group and the OFG groups was found for child age, with a mean age of 10.6 (SD = 0.9) for the FRIENDS for Life group and a mean age of 10.3 (SD = 0.9) for the OFG group (t test; p = .02).
Opinions About FRIENDS for Life in General
When asked how they evaluated the participation in FRIENDS for Life overall, almost all children (n = 42, 95.5%) and nearly two thirds of the parents (n = 25, 65.8%) expressed positive opinions about the program:
I liked the fact that the program was organised during school time. That's simple and it feels safer for the child. She learns that she's not the only one [having problems] and pupils can learn from each other. (Parent)
In addition to their positive opinions, 17 children (38.6%) also expressed negative opinions about FRIENDS for Life. They most frequently reported that they disliked having to miss other school activities because of their participation in the program (n = 7, 15.9%). Some found the program boring because topics were repeated too often (n = 6, 13.6%). Two children reported that the program had not sufficiently helped them.
Five parents (13.2%) expressed a negative opinion about FRIENDS for Life. Two parents indicated that the program should include more sessions. The other three parents criticised the content of the program (e.g., the step-plan not being clear enough).
Although we did not register parents’ attendance during the parent session and the evaluative meeting (see Methods), parents’ responses to the interview gave an indication of their involvement. Only 31.6% (n = 12) of the parents who were interviewed attended the parent session, and 36.8% (n = 14) attended the evaluative meeting.
Almost all of the parents who attended the meetings (n = 11 for the parent session; n = 12 for the evaluative meeting) evaluated them positively. The main reasons for not attending these meetings were: too busy/having other obligations (n = 10, 37.0% of the non-attending parents of the parent session; n = 8, 33.3% of the non-attending parents of the evaluative meeting) and not having been informed about the date of the meeting (n = 5, 19.2% for the parent session; n = 8, 33.3% for the evaluative meeting).
Children's Opinions About the Group Atmosphere
The majority of the children (n = 35, 79.5%) expressed a positive opinion about their fellow participants: ‘I made (. . .) friends because of my participation in the program’ (Child).
Ten children (22.7%) were negative about the other participants. Their main criticism was that the other children were noisy or disobedient.
Most children (84.1%) expressed a positive opinion about the FRIENDS for Life trainers: ‘The trainers were sweet and they were able to help us and they could talk with us like we were a friend and they were not strict like a teacher’ (Child).
Eight children (18.2%) expressed a negative opinion about the trainers. Their comments mainly concerned the strictness or authority of the trainers:
If you didn't want to say something in the group, you were forced to do it. (Child)
If you hadn't done your homework or had forgotten your book, you quite often got a reprimand. (Child)
Three children (6.8%) found that the trainers acted too childishly.
Most children (n = 27, 61.4%) indicated that the FRIENDS for Life group made them feel safe enough to express their opinions. An important part of this safety had to do with not sharing information from within the group with outsiders: ‘Everything stayed within the FRIENDS group, so nothing was told to anyone else’ (Child).
A minority (n = 7, 15.9%) did not experience this kind of safety: ‘I didn't dare to say everything I wanted. I was afraid that someone would laugh at me’ (Child).
Opinions About the Effectiveness of FRIENDS for Life
We asked children and their parents whether the children used the skills (see Introduction) they had learned. Twenty-six children (59.1%) and 18 parents (47.4%) confirmed this. Two children (4.5%) reported that they did not use these skills. This percentage was higher when parents were consulted: 23.7% of the parents reported that their child did not use any of the skills.
Twenty-seven children (61.4%) indicated that the program had been effective, that is, that they had learned something from FRIENDS for Life, whereas the majority of the parents reported so (n = 29, 76.3%). Being less nervous for performances (e.g., taking a test or giving a lecture) was mentioned most by children (n = 14, 51.9% of the children who reported to have learned something). The effect mentioned most by parents was the child being more assertive or less shy in contacts with others (n = 19, 65.5% of the parents who reported that their child had learned something). Other responses are reported in Table 3. Six parents (15.8%) explicitly mentioned that their child's problems had not improved and/or that their child had not changed.
Effectiveness of FRIENDS for Life as Perceived by Participating Children and Their Parents
Note: *Percentages of the total number of children or parents who indicated that the child had learned something from FRIENDS for Life.
The majority of children (79.5%) and parents (71.1%) indicated that they would recommend FRIENDS for Life to others:
My problem has decreased somewhat because of the project. And it's just a lot of fun, so I'd surely recommend it to other children. (Child)
If parents think that their child has an anxiety problem, they could always try [to participate]. Even if the child doesn't have a problem, it would be good to participate, because the program teaches children how to deal with children who do have a problem. (Parent)
Parents’ Suggestions for Improvement of FRIENDS for Life
Nine parents (23.7%) mentioned suggestions for improvement of FRIENDS for Life. They most frequently mentioned that they would have liked to be more involved in the program and to have received suggestions about how to support their child (n = 3; 7.9%). Other suggestions for improvement mentioned by parents (n = 3; 7.9%) had to do with organisational aspects of the program; for example, indicating more clearly when the parent session was planned.
Discussion
We investigated children's and parents’ opinions of FRIENDS for Life as an indicated prevention program for children with symptoms of anxiety or depression. Overall, the majority of children and parents evaluated FRIENDS for Life positively and indicated that they would recommend the program to others. This is in line with previous studies (Barrett et al., Reference Barrett, Moore and Sonderegger2000; Barrett, Sonderegger, et al., Reference Barrett, Sonderegger and Sonderegger2001b; Cooley et al., Reference Cooley, Boyd and Grados2004; Gallegos et al., Reference Gallegos, Rodríguez, Gómez, Rabelo and Gutiérrez2012; Iizuka et al., Reference Iizuka, Barrett, Gillies, Cook and Miller2014; Stallard et al., Reference Stallard, Simpson, Anderson, Carter, Osborn and Bush2005). Some children (n = 17) also expressed negative opinions about the program, in addition to their positive responses. They mainly disliked having to miss other school activities and they criticised the repetition of topics in the program. Comparable multifaceted views, with children expressing positive as well as negative opinions, were found when we investigated children's opinions about their fellow participants and the FRIENDS for Life trainers. We believe that by using a qualitative method to investigate children's opinions, our results add to the current knowledge about FRIENDS for Life, since our method produced more detailed information about participants’ views than previously reported.
In line with a previous study (Stallard et al., Reference Stallard, Simpson, Anderson, Carter, Osborn and Bush2005), nearly two thirds (61.4%) of the children indicated that they had learned something from FRIENDS for Life. When asked about the specific behaviours the child had learned, children and parents expressed complementary views, thereby showing the added value of including multiple informants in our study.
An important positive finding was the fact that most children felt safe enough to express their opinions in the FRIENDS for Life group. This feeling of safety could mainly be ascribed to not sharing information from within the group with outsiders, which is one of the rules established during the first session of FRIENDS for Life (Utens & Ferdinand, Reference Utens and Ferdinand2006a), and which was repeatedly emphasised by the FRIENDS for Life trainers in our study.
Although most children and parents were positive about FRIENDS for Life, suggestions for improvement can also be noted. Efforts should be made to increase the involvement of parents in the program. Although the number of parent sessions was reduced compared to the original protocol, still only a minority of the parents attended these meetings. Almost all parents who attended one or both meetings evaluated them positively. Apparently, parents’ low attendance could mainly be ascribed to organisational issues (i.e., parents not being adequately notified about the date of the parent session or time constraints; cf. Barrett, Shortt et al., Reference Barrett, Shortt, Fox and Wescombe2001) rather than to parents’ disinterest or unwillingness to participate. Indeed, some parents indicated that they would have liked to be more involved in the program and to receive suggestions about how to support their child. Parents’ attendance may be increased by organising meetings in the evening or on weekends, instead of during school time, and by adequately informing parents about the meetings.
On the other hand, one may question the added value of increasing parents’ involvement in FRIENDS for Life. Although theoretical models stress the role of parenting in the development and maintenance of childhood anxiety, a meta-analysis (McLeod, Wood, & Weisz, Reference McLeod, Wood and Weisz2007) showed that parenting accounted for only 4% of the variance in childhood anxiety. A substantial proportion of the variance in childhood anxiety can apparently be attributed to genetic effects and non-shared environmental influences such as peer relationships or traumatic events (McLeod et al., Reference McLeod, Wood and Weisz2007). Moreover, a recent meta-analysis (Teubert & Pinquart, Reference Teubert and Pinquart2011) showed additional parent sessions not to influence the effectiveness of prevention programs targeting childhood and adolescent anxiety. Because data about the attendance during parent sessions is often lacking, it is, however, unclear whether parent sessions indeed do not have an added value, or whether low parental attendance made it hard to detect an effect (Teubert& Pinquart, Reference Teubert and Pinquart2011). Additional research is therefore needed.
Limitations
Ideally, when evaluating a prevention program, data about participants who failed to complete the program should be included to obtain a complete picture of participants’ opinions. Participants who are unhappy about a program are the ones most likely to withdraw from it. In our study, only 3.2% of children who were selected to participate in FRIENDS for Life did not start the program (n = 6) or withdrew during the program (n = 5). Children who withdrew from the program were also asked to participate in the OFGs. Despite the low attrition rate, the results of the OFGs may have been positively biased, because only children who completed the program participated.
Children participating in FRIENDS for Life were slightly older than OFG participants (mean age: 10.6 and 10.3, respectively), but the groups were comparable with respect to child gender, SES, children's appraisal of the program, and child-reported anxiety and depression symptoms assessed before and after participation in the program. Considering the small size of the age difference, we regard our OFG sample as representative for our whole FRIENDS for Life group.
Whereas our response rate for the parent interviews was good (84.4%), the overall response rate for the OFGs was 19.4%, with fairly large differences between the four OFGs (range: 12.5% to 24.2%). Low response rates have been reported before in qualitative studies involving children (cf. Goodenough, Williamson, Kent, & Ashcroft, Reference Goodenough, Williamson, Kent and Ashcroft2003; Kendall et al., Reference Kendall, Sloper, Lewin and Parsons2003; Krol, Sixma, Meerdink, Wiersma, & Rademakers, Reference Krol, Sixma, Meerdink, Wiersma and Rademakers2013; Siebes et al., Reference Siebes, Wijnroks, Ketelaar, Schie, Vermeer and Gorter2007). A previous study using OFGs in a sample of children with paediatric cancer resulted in a response rate of 23% (Zwaanswijk et al., Reference Zwaanswijk, Tates, Van Dulmen, Hoogerbrugge, Kamps and Bensing2007). Because these children were in active treatment for cancer, we expected a higher response rate in our sample. However, we would like to stress that OFGs are primarily meant to explore the views and opinions of small groups of people, instead of aiming to generalise findings (Stewart & Williams, Reference Stewart and Williams2005).
Conclusions
Overall, children and parents evaluated FRIENDS for Life positively, and the majority thought that the program had been useful in changing the child's behaviour. Considered together with the growing evidence of the effectiveness of FRIENDS for Life in preventing anxiety and depression in children, these results indicate that FRIENDS for Life can be successfully implemented as an indicated prevention program by prevention workers in primary schools.
Declaration of Interest
None.
Acknowledgments
This study was funded by ZonMw, the Netherlands Organisation for Health Research and Development (no. 50-50405-98-142). The authors wish to thank M. Winckers, P. van der Heijden and R. Otten for their technical support in the development and maintenance of the OFG application.
