Introduction

One in five young people experience psychological problems associated with some distress, and estimates are that by 2020 mental illness will be one of five most common causes of morbidity, mortality and disability in children (U.S. Department of Health and Human Services, 1999). Many adult mental health (MH) disorders start in childhood. In a very large cross sectional study carried out in the United States, half of all MH disorders had emerged by age 14 with disorders such as anxiety starting even younger (50% by age 11) (Kessler et al. Reference Kessler, Berglund, Demler, Jin, Merikangas and Walters2005). Longitudinal studies also suggest stability over time and that children do not get better spontaneously (Meltzer et al. Reference Meltzer, Gatward, Goodman and Ford2000), and childhood onset cases are more costly than when mental illness onsets in adulthood (Suhrcke et al. Reference Suhrcke, Fahey and McKee2008). The importance therefore of timely, effective and appropriate intervention to young people cannot be underestimated.

Specialist child and adolescent mental health services (CAMHS) have been developed for the treatment of children with mental illness, yet despite high prevalence rates, known and prolonged personal and financial adverse effects, many children in need of service do not access them. In a large robustly conducted study in the United Kingdom (n=10 438) of the 10% found to have MH problems, 30% were not accessing any clinical service (Meltzer et al. Reference Meltzer, Gatward, Goodman and Ford2000). In Ireland, despite high prevalence rates, (Martin et al. Reference Martin, Carr, Burke, Carroll and Byrne2006), only 2% of all children aged 5–17 (n=16 664) were attending any of the 58 community-based CAMHS (Health Service Executive, 2014). Of those enrolled, the non-attendance rate was also high (18.3%).

Dissatisfaction with services may be one reason why, despite the existence of specialist services, few attend or when they do, disengage or drop out prematurely. In fact service users’ satisfaction has become increasingly recognized as important in the evaluation of clinical services. It has been linked to health care seeking, adherence and health outcomes (Strasser et al. Reference Strasser, Aharony and Greenberger1993; Powell et al. Reference Powell, Holloway, Lee and Sitzia2004; Jha et al. Reference Jha, Orav, Zheng and Epstein2008). In recent years, greater emphasis has been placed on obtaining children’s views, either proxy measures via parental perceptions or directly (Hennessy, Reference Heflinger, Simpkins, Scholle and Kelleher1999).

Given the paucity of data available to help guide Irish service development, the authors undertook this systematic qualitative and quantitative study of both parents and adolescent’s satisfaction levels with CAMHS.

Methodology

Three CAMHS clinics from geographically distinct urban regions in Ireland were involved in the study. All clinic attendees in a given month were mailed out a leaflet regarding the study and parents and children over 12 were invited to participate. The Client Satisfaction Questionnaire-8 (CSQ) was the main outcome measure, having been previously used to assess parental and adolescent satisfaction with mental health services with good reliability (αs=0.78–0.94) and construct validity (Brannan et al. Reference Brannan, Sonnichsen and Heflinger1996). Respondents were asked about their level of satisfaction to eight different areas, using a 4 point Likert scale, from 1 (quite dissatisfied) to 4 (very satisfied), giving an overall range of 8–32.

The study specific postal questionnaire collected demographic information: age/gender of child, preferred opening hours, and distance lived from the clinic. Clinical details included waiting time, duration and frequency of attendance, diagnosis, treatment, and perceived impairment at referral and at time of completing the questionnaire to allow a calculation of symptom improvement to be calculated. Ethical approval was the received from relevant ethics committees at all clinical sites.

Results

Participation rate and sample demographics

Of the 871 families sent study information, returned questionnaires were received from 426, giving a response rate of 49%. A total of 270 responses came from clinic 1 (158 parents, 112 adolescents), 129 came from a second urban city (106 parents, 23 adolescents) and 27 from clinic 3 (16 parents, 11 adolescents). Parents (280) answered demographic questions and questions regarding their children, of whom 70% were male with a mean age of 11 (range 3–18 years) (see Table 1 for details). The majority (90%) of the parents described themselves as ‘White Irish’.

Table 1

Demographic characteristics of CAMHS service users (parents)

The majority (204, 72.7%) of children attending had been given a MH diagnosis by the service. Of those who named the diagnosis (85, 42%), the most common was ADHD (55, 65%), followed by an anxiety disorder (12, 14%), PDD (8, 9%), with fewer numbers having an eating disorder (4), depression (3), or a behavioral disorder (3). Just over one third of respondents stated their child was on medication (93, 34%). Of the 77 who gave the medication type, the majority of prescribed medication was for ADHD (54, 70%) followed by antidepressants (17, 22%) and neuroleptics (6, 8%). 48 (18%) of parents reported that their child had a co-morbid medical illness, with a few mentioning asthma. Of those parents who responded to duration of attendance (n=263), more than half (138, 53%) were attending for over 12 months, with 59 (28%) attending greater than 2 years. For the majority, attendance was at least monthly (135, 66.5%). There was a range in time spent on a waiting list, from having been seen within 3 months of referral (116, 44.6%) to waiting over a year (60, 23.1%). The majority of families (n=149, 55%) lived within a 5 miles radius of the clinic. The majority of parents felt that the clinic opening hours were satisfactory (147, 34.5%) or excellent (114, 26.8%), however if given a choice, more than half (145, 53%) would opt for clinic opening hours outside of 09:00 a.m. to 17:00 p.m.

Parental and adolescent satisfaction with services

The CSQ showed good internal consistency, with a Cronbach α coefficient for parents of 0.96 and adolescents of 0.89. The general satisfaction level with CAMHS was high with more than 3/4 of parents or adolescents rating each area as ‘mostly’ or ‘very’ satisfied (Table 2). For both groups, the mean total CSQ was 26.7 (s.d.=4.7) with the proportion of ‘mostly’ or ‘very’ satisfied ratings ranged between 77.1% (extent that program met user’s needs) and 93.1% (recommendation of program to a friend if he/she were in need of similar help) regarding specific areas of service satisfaction.

Table 2

Service satisfaction levels: differences between adolescents and parents

No significant differences were observed between parents and adolescents regarding their general satisfaction with CAMHS services (Mann–Whitney U-test; Z=−0.255; p=0.799). However, despite adolescents being significantly more likely than their parents to state that they got the kind of help they wanted (χ ²=13.3; p=0.004), and that the program met their needs (χ ²=9.1; p=0.028), when asked if they would come back to the program if they were to seek help again, they were less likely than their parents to endorse this ‘χ ²=8.9; p=0.030) (Table 2).

Factors associated with parental satisfaction

Age, gender and ethnicity

The age of the child was significantly correlated with a higher CSQ score (Spearman’s ρ=0.159; p=0.017), indicating that parents with older children had a higher level of service satisfaction compared to parents with younger children. Particularly, parents of older children stated significantly more that the program had met their needs often (Kruskal–Wallis test; χ ²=11.3; df=3; p=0.010). No significant differences regarding parental level of satisfaction were observed with respect to gender or ethnicity of children.

Diagnosis and medication

Satisfaction with services was significantly higher when the child had been given a MH diagnosis by professionals compared to those without (Mann–Whitney U-test; Z=−2.14; p=0.032), although the specific diagnostic type did not predict satisfaction. Similarly psychotropic medication, but not specific type, was also associated with increased satisfaction (Mann–Whitney U-test: Z=−2.31; p=0.021). Parents were asked to rate their child’s impairment level at time of referral and at time of completing the questionnaire, on a scale from 0 (no problems) to 10 (significant impairment). There was a reduction in mean scores of impairment from the time of referral (8.3, s.d. 1.9) to current ratings (5.6, s.d. 2.5) and there was a strong negative correlation between reduction in impairment and CSQ levels of satisfaction (ρ=−0.44, n=275, p<0.000), suggesting that parents were more satisfied if there was a greater reduction in impairment.

Distance, duration and frequency of attendance, waiting time before initial assessment

The distance to a service was significantly linked with parental satisfaction; the shorter the distance the more satisfied service users were (χ ²=9.24; df=3; p=0.026). Frequency of attendance predicted satisfaction; the higher the frequency of being seen the more satisfied service users were (χ ²=8.74; df=3; p=0.033). Interestingly waiting time for initial assessment was not significantly associated with parental satisfaction neither was duration in service.

Generalized linear model

As the level of service satisfaction was generally high among service users, a more conservative model for service dissatisfaction was used. Service dissatisfaction was defined as being dissatisfied in at least one of the eight CSQ questions, scoring either a 1 or 2. Using this model, with any service dissatisfaction, age (older) of the child and frequency of attendance remained significant factors (Table 3).

Table 3

Generalized linear model (GLM) for prediction of being dissatisfied with service (at least one poor or fair rating in any of the eight CSQ areas)

Discussion

Both parents and adolescents responding to this study expressed a general high level of satisfaction with services offered. They perceived CAMHS to be meeting their needs, they would be likely to return and would recommend the service to a friend, all of which are strong endorsements for the service. This is very similar to a recent study using very similar methodology evaluating satisfaction levels among 162 adult MH service users in the west of Ireland (mean CSQ score 26.7, s.d. 4.6) (Lally et al. Reference Lally, Byrne, McGuire and McDonald2013). There is the suggestion of an improvement in satisfaction levels, at least with adult services, in that the study by Lally et al. report higher satisfaction levels than 2 earlier Irish studies (8% in Hill et al. Reference Hill, Turner, Barry and O’Callaghan2009, and 86% in Jabbar et al. Reference Jabbar, Casey, Schelten and Kelly2011). There has been one published study reporting on satisfaction levels in a subsample of CAMHS attendees in Ireland also using the CSQ. In all, 65 parents and 35 adolescents responded, although details of how these were recruitment are not outlined. The average overall item rating was high, 3.6/4 (1.9–4) for parents and 3.2/4 (1.1–4) for adolescents. No tests of association or comparison between groups were reported. There are similar reports of high satisfaction levels with CAMHS in the United Kingdom although children were less satisfied than parents, especially if they had conduct disorder. The link with disruptive disorder and lower satisfaction levels was also found in a US study (Turchik et al. Reference Turchik, Karpenko, Ogles, Demireva and Probst2010).

In this study, there were no statistically significant differences between parents and adolescents on the CSQ total scores; however subtle differences existed on some of the subscales. Adolescents more often endorsed that the program met their needs and they had received the kind of help they needed. However they were less likely than their parents to endorse returning to the program following relapse, perhaps in part explaining the 40–60% premature dropout rates amongst adolescents (Wierzbicki & Pekarik, Reference Wierzbicki and Pekarik1993). It is understandable and documented that there would be differences of opinion between adolescents and parents in term of service evaluation (Turchik et al. Reference Turchik, Karpenko, Ogles, Demireva and Probst2010). For example a child/adolescent may want appointments in school times, they may want to be seen alone, with little reference to their parents, they may want to be offered autonomy and be central to decision making process. Research suggests that adolescent satisfaction is affected by their attitude before the actual visit, their perceptions of the provider’s performance, the acceptance of the service to their peers, and may be as much a reflection of their own psychosocial characteristics as it is of the actual service received (Litt, Reference Litt1998). Parents’ satisfaction on the other hand may be linked to certain structural or economic aspects of care, such as cost, access and convenience, they may prefer appointments after school or work to minimize disruption. Their satisfaction may depend on their child’s treatment progress and their own relationship with the therapist. Parents attending due to behavioral difficulties with their child may be nervous of any change in parental hierarchy, at least at the start, and believe that parental authority should be re-enforced, that meetings should be either with parents alone or family and they may not welcome any efforts by the therapist to increase autonomy in their child. Fathers, although infrequently surveyed, have been shown to have less positive opinions than mothers (Brannan et al. Reference Brannan, Sonnichsen and Heflinger1996), perhaps associated with less frequent attendance and therefore less likely to be involved in treatment (Holmboe et al. Reference Holmboe, Iversen and Hanssen-Bauer2011). All of these idiosyncratic reasons may cloud some of the broader aspects of service user’s evaluation of services being offered. As the responsibility of many areas of health care shifts from parents to adolescents, it is important to ensure that perspectives of both parents and adolescents are included in service evaluation. Supplementing quantitative measures with more qualitative methods may throw light on the differing perspectives held by various family members, and explain the low correlations in the literature between parents and adolescents, despite both groups indicating general high satisfaction levels.

This study examined the associations between various clinical and demographic factors and levels of satisfaction. We found that living in close proximity to CAMHS, and attending frequently were linked with higher satisfaction levels. This is consistent with findings from a very large Norwegian study (n=7906) which found that accessibility to the clinic, along with parental involvement, explained most of the variance in parent satisfaction (Holmboe et al. Reference Holmboe, Iversen and Hanssen-Bauer2011). However, the Irish study examining adult MH services users perspectives’ did not find this, unless mode of transport was considered, whereby patients who travelled by car preferred services to be based in a community setting (Lally et al. Reference Lally, Byrne, McGuire and McDonald2013).

Parents in this study also rated higher satisfaction levels if the child had received a diagnosis from the clinician, irrespective of type, and if their child had been prescribed medication. Given that CAMHS are considered specialist services for those with severe and enduring mental illness, in which medication may often play a valuable and evidenced based role, it is perhaps not surprising that parents of children meeting these criteria were satisfied. Children, who had not received a diagnosis, may have problems that did not fall within this area and might be in need of alternative services and interventions. A delay in giving a diagnosis might also reflect atypicality or complexity of presentation, with a reduced treatment effect, and for which there is some supporting evidence in adults (Hasler et al. Reference Garland, Haine and Lewczyk Boxmeyer2004; Garland et al. Reference Godley, Fiedler and Funk2007), and may explain the lower levels reported if presenting with disruptive disorders (Turchik et al. Reference Turchik, Karpenko, Ogles, Demireva and Probst2010). In this study, perceived severity of problems at referral, and reduction in problem severity over time, were highly correlated with satisfaction levels. This is consistent with a few studies (Steinhausen, Reference Steinhausen1983; Godley et al. Reference Hasler, Moergeli, Bachmann, Lambreva, Buddeberg and Schnyder1998) but not others which report a consistent lack of association between satisfaction levels and other clinical characteristics, including diagnosis type, severity, or symptomatic change (Heflinger et al. Reference Hennessy2004; Garland et al. Reference Godley, Fiedler and Funk2007; Turchik et al. Reference Turchik, Karpenko, Ogles, Demireva and Probst2010). Researchers have suggested that satisfaction taps into unique and differing domains of patient’s experience of services offered and taken up, and should not be considered equivalent to symptomatic change (Turchik et al. 2008).

This loose and inconsistent association between satisfaction levels and clinical outcome, or symptomatic change, remind us of the need to ensure that measures of clinical outcome are essential when evaluating services. Robust psychometrically validated measures of satisfaction for adolescents and parents, combined with some qualitative enquiry, will help tease out why someone may be satisfied with service, even in the absence of any symptomatic improvement. Qualitative work carried out by the authors and submitted for publication highlight many interpersonal experiences valued by both parents and adolescents, often with little reference to symptom reduction (Coyne et al. Reference Coyne, Sakar, McNamara and McNicholas2015). In focus groups conducted with parents, they placed a high value on parental support and the provision of treatment and disorder specific information, highlighting the distinction between symptom improvement and knowledge (Coyne et al. Reference Coyne, Sakar, McNamara and McNicholas2015). Staff turnover was also perceived very negatively, again independent on clinical outcome. Adolescents welcomed the respect and autonomy offered by the therapists, even in the absence of any notable symptom reduction. They repeatedly emphasized the importance of, and need for, information, warmth, understanding, supportive relationships, and continuity of care, all factors which might not link to clinical outcome or symptomatic improvement (Coyne et al. Reference Coyne, Sakar, McNamara and McNicholas2015).

Despite the service users high level of satisfaction, there are still areas in which services could be improved. More than half of respondents would opt for clinic opening hours outside of 09:00 a.m. to 17:00 p.m. However there were a small group of parents whose responses indicated either ‘quite’ or ‘mild’ levels of dissatisfaction on all items (n=10). There were eight boys, mean age 11 (s.d. 3.9), 6 who had received a diagnosis, and one on medication. On inspection, there was no significant difference on any of the main variables.