The terrorist attacks on September 11, 2001, the catastrophic aftermath of disasters such as Hurricane Katrina in 2005, and the growing threat of pandemic influenza together raise considerable concern about the nation's public health preparedness.Reference Kade, Brinsfield, Serino, Savoia and Koh1 Many state and community preparedness efforts focus largely on detecting the potential for deadly terrorist strikes, developing disease surveillance systems, and planning for mass evacuations from disaster-pending areas.Reference Kade, Brinsfield, Serino, Savoia and Koh1 Reference Klein, Pepe, Burkle, Nagel and Swienton2 Disaster mental health (DMH) sequelae, however, must also be addressed as part of comprehensive disaster preparedness.Reference Abramson, Stehling-Ariza, Garfield and Redlener3 Reference Herrmann4 Reference Parker, Barnett, Everly and Links5 Reference Reid, Ruzycki and Haney6
Few states have developed comprehensive DMH training programs that prepare a professional mental health workforce to respond to disaster victims' psychosocial needs. Moreover, traditional mental health curricula, whether in medicine, psychology, social work, or nursing professions, fail to include disaster-related training as a core element. Project Liberty, a community-based outreach program following the September 11, 2001, terrorist attacks, was an extensive effort made in New York City and the surrounding area to train mental health professionals and paraprofessionals to provide supportive services.Reference Donahue, Lanzara, Felton, Essock and Carpinello7 Services were effective in assisting community members to regain previous levels of functioning and access mental health resources as needed.Reference Donahue, Lanzara, Felton, Essock and Carpinello7 Reference Frank, Pindyck and Donahue8 Reference Jackson, Covell and Shear9 The authors concluded that “an effective public mental health response to terrorism requires a population-based approach that anticipates a wide range of individual and collective reactions, ranging from emotional distress, to changes in behavior, to emerging psychiatric illness.”Reference Donahue, Lanzara, Felton, Essock and Carpinello7
An effective population-based approach requires broad-based training of a critical mass of individuals who are prepared to respond to disasters and other public health emergencies when they strike.Reference Parker, Barnett, Everly and Links5 Reference Aakko, Weed, Konrad and Wiesman10 Reference Hites, Lafreniere and Wingate11 Reference Subbarao, Lyznicki and Hsu12 A train-the-trainer (TTT) model is frequently used in business,Reference Merrill13 education,Reference Levine, Brett and Robinson14 and interventionsReference Becker15 Reference Coogle, Osgood and Parham16 to disseminate knowledge and skills. Master trainers teach curriculum content and the process of delivering the course to instructor-trainees, who are then charged with delivering subsequent training programs to target groups. Despite the widespread use of TTT programs for training large groups of people in a relatively brief period of time, few studies have examined the dissemination of TTT models to the target audience or their cost-effectiveness. Evidence is particularly lacking for public mental health programs including disaster mental health.Reference Gelkopf, Ryan, Cotton and Berger17
To begin to address this gap in the literature, we report on naturally occurring dissemination of a DMH New York State training program developed through a partnership between the University of Rochester Medical Center (URMC), the New York State Department of Health, and the New York State Office of Mental Health. A TTT model was used to disseminate DMH curriculum to new instructors across the state. Given the preliminary nature of our evaluation of a secondary data set, the primary research questions are (1) How successful is a TTT model in disseminating a DMH program as part of a multicommunity public health initiative? That is, do newly trained instructors proceed to train others as expected? (2) What factors predict which instructors will successfully disseminate, both in terms of predicting whether the trainings were conducted and the number of second-generation trainees ultimately trained?
METHODS
Setting and DHM Training
Beginning in 2004, the New York State Department of Health funded the Department of Psychiatry at the URMC to develop, implement, and monitor a statewide DMH training initiative. The program included the development of evidence-informed curricula to train mental health professionals to respond to the psychological consequences of disaster. Two master trainers (J.H., V.C.) delivered the 3-day DMH TTT workshop across the state. A total of eight 3-day TTT workshops were delivered to representatives of 51 counties. One master trainer (J.H.) provided ongoing technical assistance to newly trained instructors for the next 12 months. (Information on the DMH TTT program is available from the authors.)
Participants
County mental health directors selected participants from 57 counties. A total of 51 counties were represented; some counties sent more than 2 participants. Candidates were informed of the requirement to disseminate the program as part of the TTT model within 12 months of the completion of training programs held during 2005 and 2006. (One training group (n = 41) was not included in the analyses because pretraining measures were inadvertently not administered.)
Data Collection
DMH instructor participants completed surveys before and after the training. Program facilitators collected data on the number of training programs that new DMH instructors conducted 12 months after the training programs using a Web-based portal. To receive credit for attending the training, second-generation trainees in the community sites—the target group—were required to complete a survey that identified DMH instructor(s) by name and the date of training. For analyses, we credited instructors with completing trainings and the number of new trainees who received their training during the course of 12 months. Instructor pairs were each given credit for cofacilitated training. This methodology provided an objective measure of program dissemination without relying on instructor self-reports.
Measures
The pretraining questionnaire consisted of several demographic and pretraining items including sex, race, education level, and previous DMH training. A 14-item, multiple-choice knowledge about DMH (KDMH) assessment designed for this study was also administered. Items included questions about reactions to disaster, purpose of mental health intervention, and administrative structures in emergencies. The posttraining questionnaire included a second 14-item KDMH questionnaire matched for difficulty and a 14-item measure of perception of transfer of training (PTT), based on 3 subscales of the Learning Transfer System Inventory.Reference Holton, Bates and Ruona18 The PTT assesses trainees' perceptions about 3 factors that may impede transfer of training: motivation to transfer, capacity to transfer, and learning readiness to do so. PTT internal reliability for this sample is high (Cronbach α 0.85). The impact of the DMH TTT was measured by the number of trainees who received DMH training following TTT workshop training and is reported as frequency count data.
Data Analyses
Data were stored in Microsoft Excel (Microsoft, Redmond, WA) and transported into SPSS version 16 (SPSS Inc, Chicago, IL) for descriptive analysis. SAS version 9.1 (SAS Institute, Cary, NC) and Stata version 9 (StataCorp, College Station, TX) were used to perform descriptive statistical analysis, logistic analysis, and zero-truncated Poisson analysis.Reference Grogger and Carson19 Logistic regressions were used to model whether subsequent trainings were conducted by newly trained DMH instructors and variables with a P value of <.20 were entered into the full multivariate logistic regression model. For instructors who had conducted at least 1 subsequent DMH training, zero-truncated Poisson regressions were used to model the number of the subsequent trainings. Again, simple logistic regressions were conducted and variables with a P value of <.20 were entered into the full multivariate zero-truncated Poisson analysis. The URMC Research Subjects Review Board approved this study.
RESULTS
Demographics
Six participants were excluded from analyses because they were judged to be limited in the ability to disseminate the training (ie, retired, unemployed, or a student). Demographic information is presented in Table 1 for the sample (n = 134).
Demographic and Pretraining Variables (N=134)
KDMH
Paired Wilcoxon signed rank test assessed pre- to posttraining KDMH differences because the scores were not normally distributed. Results showed a significant increase in correct responses from pre- to posttraining. Participants' posttraining scores (χ2 12.42, SD 1.33) were significantly higher than baseline knowledge scores (χ2 9.43, SD 1.70), P < .0001.
PTT
Participants completed the PTT immediately following the TTT workshop. The group mean was 44.9 (SD 6.0).
Primary Analyses
We were primarily interested in learning how likely it is that newly trained instructors will conduct training programs and what factors predict successful dissemination. Eighty of the 134 participants (59.7%) conducted 1 or more training programs (training at least 1 second-generation participant). A bivariate logistic analysis was conducted to investigate the relation between the likelihood that a newly trained instructor conducted any DMH trainings during the 12-month period following a TTT workshop. All demographic data pretraining variables and total PTT were entered separately to predict the dependent variable, training, as a dichotomous variable. Variables included community size, work setting, prior DMH experience, and PTT. Results (Table 2) showed that only PTT scores significantly predicted whether a newly trained DMH instructor conducted any training in the subsequent 12-month period (P < .05). Newly trained instructors with higher PTT scores were more likely to conduct subsequent training programs. The odds ratio of conducting any subsequent training was 1.075 (confidence interval 1.006–1.148) when the PTT score increases 1 unit.
Multivariate Analysis Predicting Subsequent Training (Dichotomous) by Newly Trained Instructors (N = 134)a,b
Our second analysis (Table 3) used the same approach to predict the number of individuals trained by those DMH instructors (n = 80). A total of 559 trainees were trained in the “second generation.” The significant variables were sex and prior DMH training hours, controlling for age, community, and PTT. On average, women trained 2.6 more new trainees than did men. Instructors with more prior DMH trainings (≥17 hours) trained 4.9 more trainees than those without any prior DMH training.
Multivariate Analysis Predicting Number of Individuals Trained by Newly Trained Instructors 12 Months After Train-the-Trainer Training (N = 80)a,b
DISCUSSION
We found widespread interest to include DMH training in a comprehensive preparedness plan in New York communities. Almost every invited county sent 2 trainees to a TTT program with the understanding that the 3-day training would result in subsequent dissemination of the DMH program by the newly trained instructors to other mental health professionals in their communities. In addition, training resulted in a significant increase in knowledge about DMH for newly trained instructors. Sixty percent of TTT participants conducted DMH training programs during the subsequent 12 months, which resulted in 559-second-generation DMH trainees. Therefore, despite being chosen as “good candidates” by mental health leadership in their counties and a clear expectation that dissemination was a criterion for attending the training, 40% of trainees did not provide training to others.
To date, TTT programs have not routinely measured success in achieving the goal of widespread dissemination. The assumption is that the training will “spread” as intended and the TTT model will proliferate. Our finding that 40% of the participants in the TTT for DMH training did not use their training as intended must be measured against the zero return. Participants were absent from their workplaces for 3 days, incurring loss of productivity. The resources involved in the training itself and the food and lodging of participants add further to the calculable expense. Given this scenario and fiscal constraints across the public and private sectors, it is critical to understand the obstacles and facilitating factors for successful TTT programs in DMH. In particular, “low tech” (ie, low-cost, easily implemented, little reliance on technology) strategies to enhance outcomes could improve the cost-effectiveness of the popular model.
We did not directly assess transfer factors at the 51 participant sites; the cost to do so was beyond the scope of the training. Instead, we administered a brief self-report of participants' perceptions. Although it may seem self-evident that workplace factors are likely predictors of subsequent training programs, the present study demonstrates that trainees from a variety of settings accurately assess the likelihood that they will disseminate the DMH training. The practical implication for TTT programs is that a simple, inexpensive survey of trainee perceptions predicts successful dissemination, providing training administrators with the opportunity to intervene and increase the impact of the training. Intervention may occur at the organizational level (eg, contact supervisors about resources required for subsequent trainings or the importance of clear supervisory support in the workplace for dissemination of DMH training) or the individual level (eg, provide additional opportunities for practice and technical support for those who report low levels of readiness). An extension of this approach would be to assess trainee perceptions of transfer of training before conducting DMH TTT programs. It may be that potential trainees can provide important information about workplace or site readiness to disseminate training.
Our data indicate that participants with the most prior training trained an average of almost 5 more individuals than those participants who had no previous experience or less experience. From an adult learning perspective, new learning is most meaningful and is best retained when the learner's previous experiences are valued and integrated with new learning.Reference Knowles, Holton and Swanson20 A training program that has the dual goals of learning new content and learning the process of training others may not be reasonable for trainees without previous experience to support their transformation to “expert trainer” role. Thus, for individuals who went on to train others, previous relevant training experience provided a foundation for the DMH TTT program. The curriculum and training format presented here appears to have been most successful for participants with 2 days (or more) of previous DMH training. One low-tech strategy to enhance TTT success may be to select participants with significant previous training.
An alternate strategy would be to modify the current training. Two levels of training could be offered: a beginner's program for those with minimal previous DMH training that would focus heavily on content and process skills, and an advanced program for those with prior training, which would review content and focus more on process skills. Training methods could also be a factor in the ultimate dissemination of the DMH curriculum. One training approach is to incorporate active learning approaches. Behavioral rehearsal and role-play practice, small-group problem solving, and expert feedback have been shown to enhance active engagement and learning among adult learners in a variety of settingsReference Levine, Brett and Robinson14 and to enhance self-efficacy and readiness to transfer training to the workplace.Reference McGill21 Reference Salas and Cannon-Bowers22 Reference Stuart, Tondora and Hoge23 Moreover, these activities provide the opportunity for instructors to assess, either formally or informally, trainee knowledge and skills about the content and the process of training others.
On average, women trained 2.5 more individuals than men. We are not certain why women ultimately trained more individuals than men. Further study of how sex may be a factor in DMH training is recommended, as is replication of these findings.
There are clear limitations to this study. In terms of design, we report on a secondary data analysis of a limited number of variables in a naturally occurring implementation and dissemination of a DMH curriculum. The study lacks the scientific rigor of random assignment and systematic procedures for follow-up contact with new instructors. Primary data collection on the newly trained instructors was limited during the follow-up period to whether they were named as instructors by new trainees. Although this is an objective measure that is superior to self-report, errors, including underreporting if instructors trained after the 1-year window, may have occurred. Observational assessment of actual skills in delivering the training would be a useful measure of trainer ability and may be associated with the number of individuals subsequently trained.Reference Snyder, Reid and Stoolmiller24 Observational methods are, however, prohibitively costly for most program evaluations. Finally, our findings may not be generalizable to other DMH trainings or other TTT programs.
Despite the limitations, this is the first report on a TTT model for DMH training dissemination in the field that we are aware of that provides methods for other public health education efforts. Our findings suggest the success of TTT procedures can be measured and cost-effectiveness should be considered. The most important outcome measure is whether trainees (new instructors or their trainees) actually go on DMH missions as a consequence of participation in the training. Future studies may include longitudinal follow-up of a cohort of first- and second-generation trainees to examine how prepared these professionals were to meet the challenge of disaster victims' mental health needs.
Author Disclosures:
The author reports no conflicts of interest.
Acknowledgements Dr Cross is supported by a K23 Career Development award from the National Institute for Mental Health (NIMH; MH073615) and Dr Cerulli is supported by a K01 Career Development award from NIMH (MH75965-01). Drs Cerulli and Cross were also supported by Dr Eric Caine's National Research Grant (529442). Mr. Herrmann received funding for the project from Health Resources and Services Administration (Grant No. 15-0205-02, 15-0255-03, and 15-0285-04). The authors are grateful to Valerie Cole, who served as a master trainer for the Disaster Mental Health Program and training participants, and to Dr Elwood Holton III for allowing them to modify and use subscales of the Learning Transfer System Inventory. The authors also thank Dr Eric Caine, chair of the Department of Psychiatry at URMC, for support.
