Introduction
Suicide has been a national concern for Japan ever since numbers first exceeded 30 000 per year in 1998 (Shiho et al. Reference Shiho, Tohru, Shinji, Manabu, Yuka, Eriko, Ikuko and Naoki2005). However, psychiatric services have not been readily accessible to those with depression in Japan, and more than half of individuals with new onset depression consult their general medical practitioners first, often delaying their receipt of psychiatric care (Fujisawa et al. Reference Fujisawa, Hashimoto, Masamune-Koizumi, Otsuka, Tateno, Okugawa, Nakagawa, Sato, Kikuchi, Tonai, Yoshida, Mori, Takahashi, Sato, Igimi, Waseda, Ueno, Morokuma, Takahashi and Sartorius2008). Health professionals across a range of fields are therefore in a position to have important first contacts to detect and intervene with patients with mental health problems, but they require appropriate knowledge and skills to do this (Sato et al. Reference Sato, Kawanishi, Yamada, Hasegawa, Ikeda, Kato, Furuno, Kishida and Hirayasu2006).
In Japan, sufficiently detailed training for health professionals such as physicians on suicide management of individuals in mental crisis has been lacking. Yet such training would seem vital for them in their role as gatekeepers for persons with health problems, particularly as they are expected to assume important roles in suicide prevention (Mann et al. Reference Mann, Apter, Bertolote, Beautrais, Currier, Haas, Hegerl, Lonnqvist, Malone, Marusic, Mehlum, Patton, Phillips, Rutz, Rihmer, Schmidtke, Shaffer, Silverman, Takahashi, Varnik, Wasserman, Yip and Hendin2005). We hypothesized, therefore, that suicide management training for medical residents could help them increase their competency in managing persons with mental health problems, and ultimately contribute to suicide prevention.
This intervention study aimed to improve the competency of medical residents, at individual level, in managing suicidal persons they encounter in medical settings through their attendance at a brief structured training programme for suicide management. Its effectiveness was compared with the conventional training that medical residents receive on suicidality.
Method
Study design
This study employed a cluster randomized controlled trial design to avoid contamination of the programme among individual medical residents and to facilitate group-level participatory intervention. The trial involved five institutions in Japan (Hokkaido University, Iwate Medical University, Keio University, Yokohama City University and Kyushu University). Each institution offers a training programme for medical residents on suicide prevention and the programmes were provided during their psychiatric rotation (July 2009–March 2010). This study was registered in the national clinical trial registry, the UMIN-CTR (registration number: UMIN000002245), on 24 July, 2009.
Participants
Participants were second-year medical residents on their psychiatry rotation. In the Japanese medical education system, newly certified physicians must go through 2 years of postgraduate training as general medical residents before they chose a clinical specialty (Morriss et al. Reference Morriss, Gask, Webb, Dixon and Appleby2005). During these 2 years, psychiatry was one of the required areas for training for at least 1 month in the second year, the time when this research was conducted (Kozu, Reference Kozu2006; Teo, Reference Teo2007).
Randomization
Lists of medical residents in each institution in 2009 were obtained. The unit of randomization was the group of medical residents on their psychiatric rotation during the study period. A cluster constituted a mix of groups completing a psychiatry rotation in the participating institutions. The rotation groups comprised 1–9 medical residents and the groups were equivalent in terms of content of training and individual factors such as preference for future speciality. Since there was variation in the size of the groups, we operationally created a cluster containing several rotation groups in order to make a cluster with 9–18 members. After randomization to clusters, a training session was assigned to each cluster and all medical residents in the rotation group were then invited to participate in the study.
In each institution, two clusters (first half and second half groups) were created and a random number of medical students were assigned to the first group of each institution, yielding two intervention groups and control groups. Consequently, alternate intervention/control conditions were assigned to the second group in each institution to avoid an institutional effect as a whole. Registration of the clusters was conducted by YS, and the randomization procedure was carried out by MF, an independent researcher who was not involved in the intervention. Random numbers were produced by a computer program and four clusters were randomly assigned to the intervention or control group (two clusters each group) (Fig. 1). The nature of the intervention meant that allocation was not concealed.
Flow of the participants in the trial. T0: baseline response before intervention; T1: response immediately after the intervention; T2: response 1 month after the intervention; T3: response 3 month after the intervention.
Interventions
The training programme for suicide prevention implemented in this study for Japanese medical residents (hereafter, Gatekeeper Training Program) was developed by the research team based on the Mental Health First Aid (MHFA) programme implemented in Australia, which was originally developed for the general public. The MHFA programme is a training programme designed to help non-health professionals handle contacts with persons in mental health crisis and this programme has been well researched (Kitchener & Jorm, Reference Kitchener and Jorm2006) and its effectiveness demonstrated in Australia (Jorm et al. Reference Jorm, Kitchener, O'Kearney and Dear2004, Reference Jorm, Kitchener and Mugford2005).
This programme was a 2-hour structured educational programme on managing people with depression and suicidal thoughts. Since the participants in this study were medical residents, the programme was condensed by minimizing the knowledge-based content of the lecture and including more experiential learning, an essential feature of the MHFA programme. To fit the Gatekeeper Training Program into the tightly arranged residency curriculum for medical residents, the only topic taught in the programme was depression and suicide, including current needs to respond to suicide prevention. With reference to the existing mental health first aid guidelines for Japan (Colucci et al. Reference Colucci, Kelly, Minas, Jorm and Suzuki2011), the training covered the approaches which are universally endorsed, rather than taking a specific approach relevant only to Japanese culture. The first part of the programme presented, in lecture format, factual information on depression and suicide in Japan. Action plans for management of people with depression and suicidal thoughts were provided. In the second part, a clinical scenario was shown in a demonstration DVD in which a hospitalized patient with a physical problem and suicidal thoughts was referred to a physician at the bedside. The facilitator then prompted the participants to role-play the scene by taking on different roles of the patient, family member and medical staff. An interactive discussion that followed was facilitated by certified facilitators to address the questions and comments the participants raised. The details of the programme and results of a pilot study are described elsewhere (Kato et al. Reference Kato, Suzuki, Sato, Fujisawa, Uehara, Hashimoto, Sawayama, Hayashi, Kanba and Otsuka2010).
For comparative purposes, one of the researchers presented the usual training session, a didactic lecture on depression and suicidality, the contents of which were based on a standard psychiatric textbook (Nishijima, Reference Nishijima2008).
Measures
A self-administered questionnaire comprised of the following inventories and items were distributed to the study participants.
The Suicide Intervention Response Inventory (SIRI)
To assess the participants' competency in managing suicide intervention, we used the SIRI, a 25-item self-rated questionnaire comprised of forced-choice responses to brief clinical interview excerpts with suicidal persons. The SIRI is a well-validated and reliable instrument that is sensitive to change in suicide interview skills as a result of training (Neimeyer & Bonnelle, Reference Neimeyer and Bonnelle1997). On the SIRI, respondents are asked to rate the appropriateness of each response alternative on a 7-point Likert scale ranging from −3 (very inappropriate) to 3 (very appropriate).
The SIRI was chosen to evaluate competency in suicide management skills quantitatively because, to the best of the authors' knowledge, this is the only assessment tool validated to measure the competency level in suicide intervention in Japan (Kawashima et al. Reference Kawashima, Kawano and Ito2010). Also, when physicians encounter a suicidal patient in a general medical setting, they are expected to show an empathetic, non-judgemental attitude, and this is one of the issues that the SIRI aims to measure.
Originally, the SIRI score was calculated by summing the number of appropriate responses made, with a score ranging from 0 to 25 (SIRI-1 score). However, since this method of scoring demonstrated a ceiling effect, the use of a revised scoring system (hereafter, SIRI-2 score) was recommended (Neimeyer & Pfeiffer, Reference Neimeyer and Pfeiffer1994). The SIRI-2 score is calculated by summing the discrepancy between participants' ratings of the appropriateness of the responses made and a set of criterion ratings by a panel of American suicidology experts (hereafter, SIRI-2 Original score). The SIRI-2 Original score ranges from 12.90 to 247.28, with lower scores indicating better management of suicidal persons. The SIRI has been translated into Japanese and validated among Japanese suicidologists (Kawashima et al. Reference Kawashima, Kawano and Ito2010), and thus the discrepancy between the participants' ratings and Japanese experts' ratings can also be calculated (hereafter, the SIRI-2 Japanese score). The primary outcome was chosen as the SIRI-1 score in this study, and the SIRI-2 Original and SIRI-2 Japanese scores were calculated as secondary measures.
The SIRI was administered before the intervention (T0), immediately after the intervention (T1), 1 month after the intervention (T2) and 3 months after the intervention (T3).
Item for assessing confidence in managing suicidal persons
The participants were asked to indicate their confidence in managing suicidal persons on a 5-point Likert scale, which was originally developed for this study, ranging from 1 (not confident at all) to 5 (very confident).
Medical Outcome Study (MOS) Quality of Life (QOL) Questionnaire-Short Form
The MOS QOL Questionnaire-Short Form (SF-8) was used to assess the participants' health status (Fukuhara & Suzukamo, Reference Fukuhara and Suzukamo2004). The SF-8 is an 8-item self-report questionnaire to assess health-related QOL. It yields two summary measures of physical and mental health: the Physical Component Summary and Mental Component Summary. QOL was measured in this trial because QOL among medical residents generally tends to be poor and is one of the targets for improvement (Drolet et al. Reference Drolet, Christopher and Fischer2012), and they commonly encounter cases of suicide, which brings significant psychological stress that would affect their QOL (Fang et al. Reference Fang, Kemp, Jawandha, Juros, Long, Nanayakkara, Stepansky, Thompson and Anzia2007). Moreover, a previous study on the MHFA programme demonstrated positive effects on non-professional participants' QOL (Kitchener & Jorm, Reference Kitchener and Jorm2006).
Background characteristics of participants
Information was collected on the demographic and educational characteristics of the participants, as well as their own and close relatives' experience of mental health problems.
Sample size calculation
Regarding the primary outcome of SIRI, the Gatekeeper Training Program group was expected to show a greater increase in SIRI-1 score between T0 and T2 than the control group. Based on our pilot study with a pre-post design, the mean (standard deviation, s.d.) of the SIRI-1 score was 18.31 (2.0) at T0 (unpublished data). The mean difference between SIRI-1 score at T2 and T0 was 1.05 (1.87). From the results of this pilot study, we expected the Gatekeeper Training Program group to increase their SIRI-1 score by 1.5 (2.0) at a power of 0.8 and alpha level of 0.05, with the 4 clusters. With intraclass correlates of 0.15 (Jorm et al. Reference Jorm, Kitchener, O'Kearney and Dear2004), the number of participants needed for one cluster was 22, giving 88 participants for 4 clusters. The number of participants needed for recruitment was estimated as 104, based on 10% attrition at T2.
Analysis
Analysis was by the intention-to-treat principle. In the adjusted analysis, we applied a random (mixed) effects model under the missing-at-random assumption to the intention-to-treat population. We included variation of the allocated clusters in the model in order to allow for the possibility that outcome measures from individuals within the allocated cluster might not be independent in the analysis of the cluster randomized trial (Campbell & Grimshaw, Reference Campbell and Grimshaw1998). Mean differences in the two groups from T0 to T1, T2 and T3 and 95% confidence intervals (CI) were calculated. SAS version 9.1 and JMP version 6.0J were used for all analyses.
Results
From among a total of 120 medical residents on their psychiatric rotation during the study period, six declined to participate and data from the remaining 114 residents (95%) were analysed. The characteristics of the Gatekeeper Training Program and the control programme participants are presented in Table 1. No notable difference was observed in the baseline characteristics of the two groups.
Participant characteristics (N = 114)
SIRI-1: sum of the number of appropriate responses made on SIRI. Higher score indicates appropriate competency in managing suicidal persons.
SIRI-2 Original version: sum of the discrepancy between participants' ratings of appropriateness of the responses made and a set of criterion ratings by a panel of American suicidology experts in SIRI. Higher score indicates fewer competencies in managing suicidal persons compared with the competencies of American suicidology experts.
SIRI-2 Japanese version: sum of the discrepancy between participants' ratings of appropriateness of the responses made and a set of criterion ratings by a panel of Japanese health workers on SIRI. Higher score indicates fewer competencies in managing suicidal persons compared with the competencies of Japanese health workers.
SF8: MOS QOL questionnaire-short form.
First, with regard to the primary outcome, no group difference was observed in increase in SIRI-1 score at any time point (Table 2). Next, a comparison of the SIRI-2 Original score between the Gatekeeper Training Program and the control groups revealed that the scores decreased in both groups, indicating that the rating gap between Western suicidologists and study participants had narrowed after the training. However, there was no difference between the Gatekeeper Training Program group and the control group. At T3, the difference in the score between T3 and T0 was −8.32 (s.d. = 6.14) in the Gatekeeper Training Program group and 1.00 (12.88) in the control group, showing a group difference, although the proportion of responses was very low at this final time point. Last, a comparison of the SIRI-2 Japanese version score between the two groups revealed a difference between them: score at T1 was 61.28 (18.06) for the Gatekeeper Training Program group and 53.05 (15.79) for the control group (p = 0.017), and the difference in the score between T1 and T0 was 9.98 (95% CI; 4.39–15.56) between the two groups in the adjusted model, with a significant group difference (p = 0.001).
SIRI score for Gatekeeper Training Program group and usual training programme group
Adjusted by score at T0 and cluster effects.
SIRI-1: sum of the number of appropriate responses on SIRI.
SIRI-2 Original version: sum of the discrepancy between participants' ratings of appropriateness of intervention and a set of criterion ratings by a panel of American suicidology experts on SIRI.
SIRI-2 Japanese version: sum of the discrepancy between participants' ratings of appropriateness of intervention and a set of criterion ratings by a panel of Japanese health workers on SIRI.
With respect to the residents' interest in mental health problems and confidence in their care of suicidal persons, the interest score lowered immediately after the training, although it did so in both groups, and the confidence score increased by about 1 unit in both groups (Table 3). There was no difference in change in QOL between the two groups at any time point in either the physical or mental domain on the SF-8 (Table 4).
Scores for interest in mental health problems and confidence in care of suicidal persons for Gatekeeper Training Program group and usual training programme group and mean differences
Adjusted by score at T0 and cluster effects.
QOL score determined by the MOS QOL questionnaire-short form (SF-8) for Gatekeeper Training Program group and usual training programme group and mean differences
Discussion
This study evaluated the effectiveness of the Gatekeeper Training Program, a brief suicide management training programme, in comparison with the usual programme involving a didactic lecture on depression and suicidality. When measuring competency in managing suicidal persons using the SIRI-1 score, there was no difference between the two groups at individual level. As a cluster was a combination of rotation groups, we cannot generalize the results to the medical residents at group level; however, it may be applicable at individual level. Below we take a closer look at different scoring methods used in this study, discuss further the programme contents and explore issues of cultural sensitivity.
Evaluation of the Gatekeeper Training Programme according to different SIRI scoring methods
First, with respect to the primary outcome measure, SIRI-1, which involves simple arithmetic addition of 25 items, we failed to find improved competency in suicide management. One possible reason for this is that the medical residents already had basic medical knowledge on depression and suicide-related behaviour, therefore, the degree of improvement was limited (a ceiling effect). In addition, the problem associated with the SIRI-1 score is that zero is assumed for any participant that did not respond. As there was a high proportion of such cases in our dataset, the distribution of data became skewed to zero, indicating wrongly that the score decreased regardless of the programme's effect.
Second, the Gatekeeper Training Program was found to be marginally effective when measured using the SIRI-2 Original score at T3. Given the medical residents' already higher level of knowledge than non-health professionals', our programme intended to further improve the residents' competency through participatory learning. It may take longer and more training to effect a change in competency than a change in knowledge because the participants must take back what they have learned and examine whether it really works or not within their own clinical practice, although evidence suggests that those who work in suicidality on a regular basis early in their career benefit most from training (Saunders et al. Reference Saunders, Hawton, Fortune and Farrell2012). In the present study, the actual number of valid returns was low at T3, hence there is the possibility that only self-selected medical residents who were clearly interested and motivated in suicide management answered the questionnaire at T3. Bearing this in mind, a modest interpretation must be made of the programme's effectiveness.
Third, the SIRI-2 Japanese score increased at T1 (immediately after the intervention), indicating a discrepancy in judgement between the participants and Japanese suicidologists that we had not expected, although a previous study had similarly reported a remarkable difference in SIRI-2 Japanese score between intervention programme participants and Japanese suicidologists (Kawashima et al. Reference Kawashima, Kawano and Ito2010). In examining the correlation of the three scores, Pearson's correlation coefficient was 0.5907 (p < 0.01) between the SIRI-2 Original and SIRI-2 Japanese, suggesting a moderate correlation. In addition, the correlation coefficient was −0.8347 (p < 0.01) for baseline scores on the SIRI-1 and SIRI-2 Original, and −0.4251 (p < 0.01) for baseline scores on the SIRI-1 and SIRI-2 Japanese, suggesting a strong correlation between SIRI-1 and SIRI-2 Original and a moderate correlation between SIRI-1 and SIRI-2 Japanese. It should be noted that the Japanese experts who provided the ratings from which the SIRI-2 Japanese score was derived tended to judge the appropriateness of suicide management response modestly compared with Western suicidologists (Kawashima et al. Reference Kawashima, Kawano and Ito2010), and this might explain why, after receiving the Gatekeeper Training Program which was originally developed based on Western views, participants' judgement was more congruent with those of Western suicidologists.
Ambiguity over the principles of suicide management among first aiders was pointed out as a characteristic of Japanese mental health professionals, as many selected the ‘do not know, or it depends’ option in the consensus building process (Colucci et al. Reference Colucci, Kelly, Minas, Jorm and Suzuki2011). Furthermore, the role that first aiders, especially non-specialist health professionals, should play in the suicide prevention framework and the way they should approach persons with suicidal ideation was not well recognized by Japanese mental health professionals, as suggested by the consensus building process results not endorsing that persons with suicidal ideation themselves or others close to them can manage their psychological crisis (Colucci et al. Reference Colucci, Kelly, Minas, Jorm and Suzuki2011). Thus, while the standards of care for persons with suicidal behaviours have been established (Kuwabara et al. Reference Kuwabara, Kawanishi, Kawano and Ito2009), consensus is yet to be reached on the approach people should take in suicide management in Japan. This evolution of suicide prevention strategy in Japan might have had an effect on the results. In spite of there still being the need to firm up experts' views, from a cultural perspective it is clearly important for participants to bring their suicide management skills more in line with those of Japanese experts.
Evaluation of the Gatekeeper Training Programme in relation to medical residents' confidence and QOL
In this study, we adapted the MHFA programme that was originally developed for lay persons and which was reported by previous evaluation studies to have improved knowledge, competency and QOL among such participants, compared with the usual programmes (Jorm et al. Reference Jorm, Kitchener, O'Kearney and Dear2004). The present study similarly showed that medical residents' confidence in suicide management improved, and the improvement was greater, among participants in the Gatekeeper Training Program group than among the control group receiving the usual programme, at a marginally significant level.
No differences in participants' QOL were revealed between the two groups. One possible explanation is that overload and burnout was common among the medical residents, as has been commonly reported (Dyrbye et al. Reference Dyrbye, Thomas, Massie, Power, Eacker, Harper, Durning, Moutier, Szydlo, Novotny, Sloan and Shanafelt2008), and that the intervention was too brief under such circumstances to address their QOL, which is a complex interplay of academic competency, financial status and other work characteristics (West et al. Reference West, Tan, Habermann, Sloan and Shanafelt2009; Campbell et al. Reference Campbell, Prochazka, Yamashita and Gopal2010). Improving skills to handle difficult patients, such as those with suicidality, may contribute to disentangling these factors, whereas a stand-alone training programme may have limited effect in improving QOL among medical residents. However, in the long run, receiving a suicide management programme in advance of clinical encounters may enable residents to handle emotional distress when actually faced with patients exhibiting suicidal behaviour in a better manner.
Limitations
This study did not address the assessment of emotional and behavioural change when residents encounter suicidal patients. Our ultimate goal was to change medical residents' competency, and a more behavioural assessment anchored by objective observation would have provided a stronger basis for evaluating the programme's effect. Considering current knowledge on suicide prevention measures and the complex interplay of cause and path of suicide-related factors, suicide prevention strategies should be underpinned by multi-faceted training programmes: any single programme may not be able to contribute enough to effecting a gross change. The limited observation period in the present study, during which encounters with patients with suicidality were presumably few, did not allow us to adopt these objective behavioural measures. However, it is noteworthy that the residents' confidence was increased after the intervention.
With regard to methodological issues, the CONSORT 2010 Statement (Campbell et al. Reference Campbell, Piaggio, Elbourne and Altman2012) recommended that the minimum number of clusters per arm should be at least four. However, our study had only two clusters per arm owing to practical issues with recruiting the participants. Therefore, our study might be still confounded by cluster effects.
The proportion of residents that participated at the follow-up time points was low and the number of respondents in the two groups was unbalanced. At T1, the proportion lost to follow-up was 7.7% (5/65) and 4.1% (2/49), respectively, but more than a half of the participants did not return the questionnaire at T2 and T3, and therefore some respondent and follow-up bias might be present. In the medical residency system, psychiatric rotation is allocated only for 1 month before the residents move on to training in another department. The follow-up questionnaires were sent out when the medical residents were busy adjusting to training in the new department and they might therefore have been too busy to respond to the mailed questionnaire. Several reminders of the follow-up response were sent, but resulted in very few further returns. We took the view not to send out further additional reminders so as not to give a negative impression about psychiatric training by insistently demanding a response from the participants. This dilemma, resulting in a major limitation of the study, is inherent to this kind of study.
In addition, the proportion lost to follow-up was greater in the Gatekeeper Training Program group than in the control group for reasons that are unknown. We compared the characteristics of the respondents and drop-outs at T2 (data are available upon request), but no differences were found in scores for SIRI, interest in mental health or confidence in managing people with suicidal thoughts. Thus, unknown factors might have affected the respondents' data. The results may well be different if we had a dataset for those who were lost to follow-up.
In conclusion, our study did not demonstrate that the Gatekeeper Training Program improved medical residents' competency in suicide management when measured by the SIRI-1 score, and it showed an unexpected, unfavourable effect in the short term when measured by the SIRI-2 Japanese version score. While this study failed to show that the Gatekeeper Training Program increased medical residents' competency in suicide management skills, this may be due to a mixture of contributing factors. Selection of the SIRI as the primary outcome measure is questionable because of its limited validity, which suggests that the Japanese community must reconsider and build consensus on the principles first aiders should follow when encountering persons with suicidal thoughts. Further elaboration of the programme and valid measurement of its outcome would be needed to show the programme's effectiveness.
Acknowledgements
The authors would like to express thanks to Drs Masayuki Tomita and Koichiro Watanabe and Professors Haruo Kashima, Masaru Mimura (Keio University), Shigenobu Kanba (Kyusyu University) and Ichiro Kusumi (Hokkaido University) for coordination of the study. The authors thank Professor Anthony Jorm and Ms Betty Kitchener of the University of Melbourne for their mentorship in Mental Health First Aid programme, Drs Daisuke Kawashima, Kenji Kawano and Hiroto Ito of the National Institute of Mental Health Japan for providing suggestions on the SIRI, and the Japan Young Psychiatrists' Organization (JYPO) for its contribution to coordinating the research group.
Financial Support
This research was fully supported by a Grant-in-Aid from the Japanese Ministry of Education, Culture, Science and Technology (JSPS KAKENHI Grant Number 22591296).
Conflict of Interest
The authors have no conflicts of interest.
Ethical Standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committee on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. The study protocol was approved by the institutional review board of each participating institution and the National Center of Neurology and Psychiatry. Written informed consent was obtained from all participating medical residents.
