Introduction
A recent report into the intentional injury of children in the EU found that Ireland has the highest rate of female suicide (2.09/100 000) and the second-highest rate of male suicide (5.12/100 000) in persons aged 0–19 years (MacKay & Vincenten, Reference MacKay and Vincenten2014). These figures, in turn, reflect a much wider incidence of self-harm in this population (Hawton et al. Reference Hawton, Saunders and O’Connor2012; McMahon et al. Reference McMahon, Keeley, Cannon, Arensman, Perry, Clarke, Chambers and Corcoran2014). Depression, anxiety, and emotional stress are commonly experienced by Irish adolescents (Connolly et al. Reference Connolly, Leahy, Bury, Gavin, McNicholas, Meagher, O’Kelly, Wiehe and Cullen2012), but remain undetected and untreated in many instances (Coughlan et al. Reference Coughlan, Tiedt, Clarke, Kelleher, Tabish, Molloy, Harley and Cannon2014). Research in Ireland and internationally has also shown that adolescent mental health issues often co-exist with, and are exacerbated by, the use of alcohol and illicit drugs (Connolly et al. Reference Connolly, Leahy, Bury, Gavin, McNicholas, Meagher, O’Kelly, Wiehe and Cullen2012; Degenhardt et al. Reference Degenhardt, Coffey, Romaniuk, Swift, Carlin, Hall and Patton2013; James et al. Reference James, Smyth and Apantaku-Olajide2013; Coughlan et al. Reference Coughlan, Tiedt, Clarke, Kelleher, Tabish, Molloy, Harley and Cannon2014).
Epidemiological data indicates that the onset of most psychiatric disease occurs within a discrete timeframe between 12 and 24 years (Kessler et al. Reference Kessler, Berglund, Demler, Jin, Merikangas and Walters2005; Patel et al. Reference Patel, Flisher, Hetrick and McGorry2007). Onset of mental illness in this phase of life, when many young people are completing their education, making early career choices and establishing relationships, may have economic and social consequences that extend into adulthood (Patel et al. Reference Patel, Flisher, Hetrick and McGorry2007). Though not unequivocal, there is evidence to indicate that early intervention could reduce ‘direct’ healthcare costs and may lead to better clinical outcomes in patients with a wide range of mental health disorders including, but not limited to, ADHD (Trillingsgaard et al. Reference Trillingsgaard, Trillingsgaard and Webster-Stratton2014), psychosis (Mihalopoulos et al. Reference Mihalopoulos, McGorry and Carter1999; McGorry et al. Reference McGorry, Killackey and Yung2007a, Reference McGorry, Killackey and Yung2008; Mihalopoulos et al. Reference Mihalopoulos, Harris, Henry, Harrigan and McGorry2009), bipolar disorder (Berk et al. Reference Berk, Hallam, Lucas and Hasty2007), depression (Lynch & Hornbook, Reference Lynch and Hornbrook2005; Allen et al. Reference Allen, Hetrick, Simmons and Hickie2007), and substance abuse (Lubman & Hides, Reference Lubman and Hides2007). This early promise has led early intervention in youth mental health to be described as a ‘best buy’ for both patient and healthcare provider (McGorry et al. Reference McGorry, Purcell, Hickie and Jorm2007b).
Increased utilisation of primary care is central to strategies that have been devised to tackle the crisis in youth mental health both in Ireland (Department of Health & Children, 2006) and internationally (NICE, 2005; United Kingdom Department of Children, Schools and Families, 2010; NSW Centre for the Advancement of Adolescent Mental Health, 2013). As the first point of contact for individuals seeking healthcare and because of the ongoing support that it offers, primary care is ideally placed to facilitate early detection and intervention in youth mental health problems (Cullen et al. Reference Cullen, Broderick, Connolly and Meagher2012). However, barriers exist in getting young people to present in primary care with mental health issues. These include young people’s own beliefs about mental illness (Haller et al. Reference Haller, Sanci, Patton and Sawyer2007), lack of awareness that general practitioners (GPs) can assist with psychological and social problems (Biddle et al. Reference Biddle, Donovan, Gunnell and Sharp2006) and resistance to accepting a mental health diagnosis (van Voorhees et al. Reference van Voorhees, Fogel, Houston, Cooper, Wang and Ford2005). When young people do attend a practice, physicians’ own attitudes and beliefs can also impede prompt detection and treatment of mental illness (Iliffe et al. Reference Iliffe, Williams, Fernandez, Vila, Kramer, Gledhill and Miller2009). Correct diagnosis is further complicated by the fact that presentation of mental illness in young people tends to be atypical (Cullen et al. Reference Cullen, Broderick, Connolly and Meagher2012).
Continuing medical education (CME) programs in the areas of youth mental health and substance abuse may help GPs overcome some of these barriers. Educational interventions aimed at increasing the detection of adolescent depression and suicidality by GPs have reported considerable success (Gledhill et al. Reference Gledhill, Kramer, Iliffe and Garralda2003; Fallucco et al. Reference Fallucco, Conlon, Gale, Constantino and Glowinski2012; Kramer et al. Reference Kramer, Iliffe, Bye, Miller, Gledhill and Garralda2013). A randomised controlled trial of an educational intervention to address deficiencies in Australian GPs’ care of adolescents was found to increase GPs’ skills, knowledge, and confidence in this area (Sanci et al. Reference Sanci, Coffey, Veit, Carr-Gregg, Patton, Day and Bowes2000) and these gains were still evident in a 5-year follow-up study (Sanci et al. Reference Sanci, Coffey, Patton and Bowes2005).
This paper describes the development and evaluation of a pilot educational intervention in youth mental health and substance misuse for primary care practitioners.
Methods
Participants, recruitment, and ethical approval
GPs and other professionals working in the Mid-West region of Ireland were invited to take part in an educational session on youth mental health and substance abuse at the host institution. Email invites were circulated to GPs affiliated with the host institution, local specialist GP training programmes, staff within the Faculty of Education & Health Sciences at the host institution, staff within HSE Mental Health Services and HSE Primary Care in the region. Applicants could register online or by post. Upon registration, participants were asked to indicate questions and issues they would like the session to address. Thirty-nine participants registered for the session, 30 of whom attended. Eight participants were GPs, but in keeping with the multidisciplinary ethos of the session, participants with backgrounds in psychology, psychotherapy, nursing, and the social sciences also participated.
Twenty-four (80%) of those who attended completed evaluation questionnaires. Participants were informed that the session was a pilot study, and that by agreeing to participate, their responses would be used for the purposes of research. The educational session was approved for CME purposes by the Irish Council of General Practitioners (ICGP). The ICGP Research Ethics Committee approved the study.
Overview of the educational session
The learning objectives and content (see Box 1) were informed by recently developed evidence-based consensus guidelines outlining the five domains in which primary care can play a role in the treatment of youth mental health (see Box 2; Schaffalitzky et al. Reference Schaffalitzky, Leahy, Cullen, Gavin, Latham, O’Connor, Smyth, O’Dea and Ryan2014) and feedback from registration forms.
The session was designed to be 2 hours in duration and interactive in format. Interactive methods, such as role-play and discussion with clinical experts, are techniques that have been shown to be superior to didactic, theory-based methods in changing physician behaviour and patient health outcomes (Bloom, Reference Bloom2005). This approach has previously been received favourably by Irish GPs, participating in an educational intervention to help GPs manage alcohol abuse among problem drug users (Klimas et al. Reference Klimas, Lally, Murphy, Crowley, Anderson, Meagher, McCombe, Smyth, Bury and Cullen2014).
The session also aimed to encourage the use of standardised screening tools in the assessment of young people to increase the detection of mental health disorders in this population (Katon et al. Reference Katon, Russo and Richardson2008). One established framework for assessing adolescent mental health is the ‘HEEADSSS interview’ (Goldenring & Rosen, Reference Goldenring and Rosen2004; Klein et al. Reference Klein, Goldenring and Adelman2014). This interview technique is recommended for use by the Royal Australasian College of Physicians (2008), has been recommended as a screening tool for risk-taking behaviour in young women (Carr-Gregg et al. Reference Carr-Gregg, Enderby and Grover2003), and is currently being adapted for use in Australian ‘headspace’ centres, which are multidisciplinary centres for the treatment of youth mental health (Parker et al. Reference Parker, Hetrick and Purcell2010). It has been shown to be useful in a diverse range of healthcare settings in which adolescents present, including surgery (Wilson et al. Reference Wilson, Bostock, Phillip, Shannon, Payne and Kennedy2012) and emergency medicine (van Amstel et al. Reference van Amstel, Lafleur and Blake2004; Cappelli et al. Reference Cappelli, Gray, Zemek, Cloutier, Kennedy, Glennie, Doucet and Lyons2012).
An expert panel helped deliver the session, and included the Director of a Student Counselling Service, a Consultant Child and Adolescent Psychiatrist, the Regional Drugs Co-ordinator, the Deputy Manager of the Community Substance Misuse Team, and a Clinical Psychologist.
The session began with a 10-minute powerpoint presentation outlining the learning objectives and how the different components of the session would be organised. Following this, a series of case studies were presented to participants, which were then discussed in small groups and with the expert panel. Participants were then shown a video in which a GP carried out the ‘HEEADSSS’ interview with an adolescent patient (see Box 3). A question-and-answer session with the expert panel followed the demonstration. Discussion, feedback, and questions were sought from participants throughout the session.
At the conclusion of the session, participants were presented with an information pack, which included information on youth mental health and substance abuse services in the region, a copy of the ICGP ‘Guidelines on the Treatment of Adolescent Mental Health in Primary Care’, and reference sheets on consent and confidentiality, identifying common mental health disorders, and improving communication styles with young people.
Measures
Post-session questionnaires were given to all participants. Using a five-point Likert scale (where 1=strongly disagree and 5=strongly agree), participants were asked to rate their level of agreement with statements on knowledge acquisition and the relative usefulness of each part of the session.
Information on participants’ professional backgrounds (including professional experience and training in youth mental health), was sought. Open-ended questions asked participants to indicate what had been good about each component of the session, and what could have been improved.
Analysis
Participant responses were coded numerically from 1 to 5, with 1 representing ‘strongly disagree, and 5 representing ‘strongly agree’. Data were collated and analysed using SPSS Version 22.0. The first two sections of the questionnaire, measuring knowledge acquisition and perceived usefulness, were summed to give total scores for these variables. Descriptive statistics were computed for all of the quantitative measures described above (Tables A1–A4). Potential differences in knowledge acquisition and perceived usefulness of session components between participants that had previous training in youth mental health and substance use, and those that did not, were investigated by examining the mean scores for each group.
Content analysis of the open question responses was conducted. Similar responses were categorised under the same code, and the number of participants were counted for each code.
Results
Participant characteristics
Twenty-four of the 30 participants that took part in the session returned questionnaires. Participants’ professional backgrounds and previous training in youth mental health are presented in Table A1. The majority of the sample (71%) was directly involved in the provision of care to young people. Just under half of participants had previous training relating to youth mental health (45%) and youth substance abuse (45%).
Evaluation scores for the educational intervention
Participants tended to strongly agree or agree that they were able to state the importance of early intervention in youth mental health (M=4.52, s.d.=0.51) and their understanding of the importance of primary care in youth mental health (M=4.52, s.d.=0.51) following the session. Participants tended to be neutral about their knowledge gains in relation to the conduct of a ‘HEEADSSS’ evaluation on a young person (M=3.65, s.d.=1.19) and in understanding what ‘HEEADSSS’ stands for (M=3.74, s.d.=1.18). The mean of the summed scale items for knowledge acquisition (Table A2) was 46.45 (s.d.=4.96, maximum=55).
Participants found the case studies and small group discussions the most useful part of the session (M=4.46, s.d.=0.51), followed closely by the question-and-answer session with clinical experts (M=4.42, s.d.=0.58) and peer interaction (M=4.42, s.d.=0.58). Participants rated the powerpoint presentation as being the least useful part of the intervention, although this component was still rated very highly at 4.36 (s.d.=0.58). The mean summed score for perceived usefulness (Table A3) was 26.48 (s.d.=2.60, maximum=30) on a scale with a maximum of 30, indicating positive perceptions on average. There was no significant difference in mean knowledge acquisition scores or mean ratings of usefulness for the different components of the intervention between participants that had previous training in youth mental health, and those that did not.
Qualitative evaluation (Table A4)
Participants gave more responses for ‘what was good about [each component of the session]?’ than ‘how can [this component] be improved?’ Many responses were general comments indicating that a component was useful and beneficial (e.g. ‘good’, ‘very good’, ‘excellent’).
The characteristic of the presentation that participants most valued was that it delivered ‘clear and concise messages’. One participant, however, felt the presentation could have been clearer and one wanted more time given to this component.
The characteristics of the case studies/small group discussion that participants valued were the interactive and dynamic quality of this component, particularly the sharing of knowledge and ideas and having access to ‘expert opinions’. Five participants felt that more time should have been given to this component and three thought a larger, quieter venue would have been more suitable.
Participants found the video presentation to be a useful medium for outlining the ‘HEADDSSS interview’ technique and provided a basis for ‘thinking through issues’. One participant suggested that there should be more discussion during the video, by stopping at key points to discuss the issues, and one thought better time management was needed during this component.
The characteristic of the question-and-answer session with clinical experts, which participants most liked was the ‘helpful guidance’ and ‘useful insights’ they received from the experts. Participants had no suggestions for improvement of this component.
The characteristics of the interaction with peers, which participants most liked, were the opportunities to ‘meet other specialities’ and to ‘share knowledge and experience.’ Two participants suggested that more time be given to this component, one participant suggested provision of a contact list and one advised that similar sessions be held in other locations.
Participants thought the supplementary information pack was a ‘very useful resource’, although a few participants had not had a chance to read it at the time of completing the evaluation form. Just one participant had a suggestion for improvement of the pack and advised including information relevant to a particular region in Ireland.
When asked how the session could be made more useful overall, participants advised that a larger venue be used and suggested having more of these types of workshops.
When asked what would help them better address youth mental health and substance abuse in practice, participants felt they could better address youth mental and substance use issues if they had more information on available resources and services to refer patients to, including psychology resources. Participants also felt that greater access to relevant services and resources was needed, for example, in rural areas, including access to counselling in primary care.
Discussion
Key findings
A brief educational intervention developed to help primary care practitioners detect and treat youth mental health and substance use issues and to increase youth-friendliness in practice, was positively received by participants. Knowledge gains were rated highest with respect to understanding the role of primary care, and the importance of early intervention, in treating youth mental health. Participants reported clinical case studies and interaction with peers and experts to be the most useful components of the session.
How these findings relate to other literature
The acceptability and positive reception of the present intervention is in keeping with previous literature that has shown a preference among CME participants for interactive, problem-based teaching methods (Smits et al. Reference Smits, Verbeek and de Buisonjé2002). A number of participants took the opportunity in the general feedback section to request more interventions of this type, indicating a general preference for an interactive format. A total of 76% of participants indicated an interest in pursuing a higher qualification in this area, a figure that highlights a desire among professionals for more CME in this area. The 2-hour duration of the intervention was positively received by participants. This timespan appears to be optimal for an intervention of this type, having been utilised successfully in a previous study by this research group (Klimas et al. Reference Klimas, Lally, Murphy, Crowley, Anderson, Meagher, McCombe, Smyth, Bury and Cullen2014) and in previous CME interventions in youth mental health (Gledhill et al. Reference Gledhill, Kramer, Iliffe and Garralda2003; Falluco et al. Reference Fallucco, Conlon, Gale, Constantino and Glowinski2012; Kramer et al. Reference Kramer, Iliffe, Bye, Miller, Gledhill and Garralda2013).
Similar to previous work by this research group, peer interaction was rated by participants as being one of the top three most useful components of the educational session (Klimas et al. Reference Klimas, Lally, Murphy, Crowley, Anderson, Meagher, McCombe, Smyth, Bury and Cullen2014). Additional detail emerged from the open-ended question feedback. Participants derived great benefit from interacting with colleagues from different backgrounds, such as social work or psychology; one participant suggested providing a contact list to aid further consultation beyond the session. This is encouraging when considered in the context of previous research that identified collaboration between primary care professionals, other health professionals, and community agencies as enhancing the provision of mental health treatment to adolescents (Roberts & Bernard, Reference Roberts and Bernard2012; Leahy et al. Reference Leahy, Schaffalitzky, Armstrong, Bury, Cussen-Murphy, Davis, Dooley, Gavin, Keane, Keenan, Latham, Meagher, McGorry, McNicholas, O’Connor, O’Dea, O’Keane, O’Toole, Reilly, Ryan, Sanci, Smyth and Cullen2013). The benefits of a multidisciplinary presence in educational interventions have also been documented in a systematic review that isolated features of successful educational interventions aimed at improving the management of depression in primary care (Gilbody et al. Reference Gilbody, Whitty, Grimshaw and Thomas2003). It is notable that the greatest amount of feedback with respect to improvement was elicited for the ‘case studies/small group discussion’ component of the session, and that this mostly comprised requests for additional time.
Participants rated their knowledge gains as lowest with respect to the HEEADSSS interview. This may be because it was the most specific and complex of the knowledge areas evaluated. It may also reflect a ceiling effect for an intervention of this duration. Though participant feedback was positive with respect to the length of the intervention, there may be limits to what can be learned in 2 hours. It should be noted that a full outline of the HEEADSSS interview, along with suggested questions, was provided in the information pack, ensuring further information was available for those who felt this component was not long enough in the educational session.
Methodological considerations
We acknowledge a number of factors that are likely to limit the generalisability of our findings. The sample size was small and comprised volunteers and thus participants were likely to share characteristics that made them particularly amenable to an intervention of this type. Though the invitation was circulated widely, we have no data on those who did not respond. In particular, although participants represented diverse professional backgrounds, this information was not always provided on feedback forms; the small number of participants and thus concerns regarding anonymity of participants meant this was not an essential item of information for participants to provide on the study instrument. Finally, a precise picture of knowledge gain was difficult to ascertain in the absence of a baseline assessment of knowledge, and an objective test of knowledge post-intervention and we recommend that such an approach be adopted in future research with such aims.
Recommendations for future research
The present research was carried out as a pilot study in line with the MRC Guidelines for the development and evaluation of complex health interventions (Craig et al. Reference Craig, Dieppe, Macintyre, Michie, Nazareth and Petticrew2008). Further evaluation of the present intervention would require, at the least, a replication of the present study in a wider range of locations and with a greater number of participants. It may be preferable, however, to employ a more rigorous design, such as including the intervention in a randomized trial. In addition, direct measurements of outcomes in GP practice, such as differences between detection rates pre- and post-intervention, would offer a more objective measure of the benefit of the intervention. Such data would allow comparison between the present intervention and other interventions that have been successful in increasing GP detection rates of mental ill-health in adolescents (Sanci et al. Reference Sanci, Coffey, Veit, Carr-Gregg, Patton, Day and Bowes2000; Gledhill et al. Reference Gledhill, Kramer, Iliffe and Garralda2003; Sanci et al. Reference Sanci, Coffey, Patton and Bowes2005; Falluco et al. Reference Fallucco, Conlon, Gale, Constantino and Glowinski2012; Kramer et al. Reference Kramer, Iliffe, Bye, Miller, Gledhill and Garralda2013).
Conclusions
The present intervention shows considerable promise as a format for the improvement of practitioners’ skills in dealing with the issues of adolescent mental health and substance abuse. The results of this pilot show that the intervention is both feasible and acceptable to participants. Future research can expand upon these initial findings with a more rigorous experimental design, by including objective measures of participant knowledge gain, and by direct measurement of clinical outcomes post-intervention.
Acknowledgements
The authors gratefully acknowledge the support of the Health Research Board (HRB) of Ireland who funded the development and implementation of the intervention through the ‘Health Research Awards Programme’ (HRB_HRA 2010/4) and the UL-GEMS’ Summer Studentship whose support enabled J.B. to work on this project. The authors thank all healthcare professionals who participated in this study and especially colleagues who contributed to the development and delivery of the training and especially Mr Rory Keane (HSE Midwest Addiction Services).
Appendix
Learning objectives for the session
1. Understanding the role of primary care in mental health and substance use among adolescents and young adults.
2. Appreciation of the value of early intervention in the area of youth mental health.
3. Gaining understanding of the barriers to accessing health services in the community and how healthcare professionals can become more ‘youth friendly’.
4. Developing competency of a brief intervention for engaging with young people and focusing on the issues that are important in relation to their mental health.
PATIO domains
P
Prevention/promotion: it is important that youths and adolescents be aware that emotional and social problems are something that general practitioners can help them with. In making adolescents aware of this capacity, PCPs may be able to increase adolescent presentation at clinics, and hence, detection and diagnosis
A
Assessment: improvements in this domain can be achieved with the use of proper screening tools (e.g. HEEADSSS), knowledge of diagnostic criteria, and knowledge of what represents normal and abnormal behaviour in this population
T
Treatment: knowledge of treatment options including psychological, pharmacological, and referral to specialist treatment, can improve practitioner confidence in treating youth mental health
I
Interfacing and interaction: collaboration and communication with other healthcare professionals in primary and secondary care is important for effective treatment of youth mental health
O
Ongoing care: this domain refers to the importance of monitoring treatment outcomes and establishing plans for when treatment fails
HEEADSSS headings
Home: assesses an adolescent’s basic living situation and relationships with family members.
Education/employment: assesses an adolescent’s educational goals and performance. Employment status is assessed as a potential stressor (e.g. working too many hours).
Eating: assesses adolescent eating behaviour and body image, which are important with respect to self-esteem issues and possible eating disorders.
Activities: assesses social activities with friends and family. Adolescents may be asked about participation in sports or leisure clubs, possible sources of stress and/or social support.
Drugs: screens for tobacco, alcohol, and illicit drug use. The former two have been identified as predictors of presentation with psychosocial issues in Irish primary care settings (Connolly et al. 2012)
Sexuality: assesses sexual activity, romantic relationships, and patient practice and understanding of safe sexual activity.
Suicide: assesses adolescents for depressive symptoms, self-harm, and suicidality.
Safety-exposure: identifies dangerous behaviour or exposure of the adolescent to dangerous situations. The former includes impulsive or reckless behaviour that may result in injury. The latter deals with exposure to violence including assault, physical and sexual abuse, and rape.
Self-reported prior training and background in the areas of youth mental health
Self-reported ratings of knowledge gained in the area of youth mental health
Self-reported ratings of the usefulness of the different components of the workshop
Open question responses
