When disasters impact a community, health care facilities may receive a surge of casualties and people searching for potentially injured loved ones and information. To adequately manage these emergency demands, professionals with expertise in a range of areas are needed to respond.Reference Rottman, Shoaf and Dorian 1 - Reference Frasca 3 However, facility staffing is sometimes a challenge because of the unavailability of usual staff due to the emergency (eg, directly impacted by the event, difficulty commuting, conflicting responsibilities) or a need for additional staff to handle the surge demand.Reference Davidson, Sekayan and Agan 4 To prepare for a potential staff shortage, hospitals offer disaster response training to employees and national teams of disaster health care volunteers (eg, Medical Reserve Corps) are coordinated to provide the needed support to maintain patient access to care.
The US Department of Veterans Affairs Veterans Health Administration (VHA) is the largest integrated health care system in the United States. Serving ~9.3 million Veterans each year at over 1700 sites of care, it is vital that individual facilities and VHA, as a whole, are prepared to continue delivering services despite a potential mass casualty surge or staffing shortage following an event. 5 The Disaster Emergency Medical Personnel System (DEMPS) is a team of employee volunteers available to deploy to disaster sites and provide clinical and non-clinical staffing assistance when local systems are overwhelmed. VHA supports DEMPS volunteers with training for deployment readiness. In 2012, a quality improvement project surveyed DEMPS volunteers on their experience with the program up to that point. Initial analyses of the resulting data evaluated factors influencing volunteers’ readiness to deploy,Reference Zagelbaum, Heslin and Stein 6 while this study focused on perceptions of and feedback on training experiences.
While the current literature emphasizes that training of health care staff and volunteers in disaster medicine and public health emergency response is integral for preparedness,Reference Jose and Dufrene 7 - Reference Ye, Stanford and Gousse 9 there is no universally accepted course curriculum or method of training.Reference Jose and Dufrene 7 , Reference Collander, Green and Millo 8 , Reference Hsu, Jenckes and Catlett 10 , Reference Miller, Scott and Issenberg 11 At the time of the survey, in 2012, the program consisted of a comprehensive curriculum with 3 modes of training delivery: (1) online independent study, (2) hands-on field exercise, and (3) face-to-face classroom lecture. The goals of this analysis were to (1) better understand DEMPS volunteers’ training mode utilization and attitudes about training, and (2) identify areas of improvement for DEMPS’ training program. The primary focus of this study was to better understand the impact of a multi-modal training program on perceptions of training.
METHODS
In 2012, all DEMPS volunteers (N=8250) were invited by e-mail to participate in an online survey to provide feedback on their DEMPS training experiences up to that point. The survey included questions about training, self-perceived preparation and readiness, levels of stress, and ideas or areas for program improvement, but did not request input on specific training content. Three reminder e-mails were sent following the initial invitation to those who had not replied. A total of 2120 (26%) DEMPS volunteers responded to the survey. Data submitted online were converted into analytic files for analysis and items were reviewed for completeness, coding, and usefulness in addressing the study aims. The study was approved by the US Department of Veterans Affairs Greater Los Angeles Health Care System Institutional Review Board as a quality improvement study.
Measures
Key variables
Key items for analysis included training mode participation and satisfaction (online, field exercises, and face-to-face), attitudes about DEMPS training, and continued engagement in DEMPS (see Figure 1 for these survey items). All satisfaction, attitude, and continued engagement questions utilized 7-point Likert scale responses ranging from most negative (eg, extremely dissatisfied) to most positive (eg, extremely satisfied), but were recoded to indicate either a positive or a neutral/negative binary response for further descriptive and regression analysis as the dependent variables. Training mode participation included the options of having completed any combination of online, field exercise, or face-to-face modes. Number of training modes completed (0, 1, 2, 3) was the key independent variable used in the analysis. For regression models, dichotomous indicators for each count value of training modes was created with 1 training mode as the reference category.
Measures This image is a visual summarization of the training-related questions evaluated by this paper, but is not a direct replication of the Disaster Emergency Medical Personnel System (DEMPS) on-line survey format. The category titles were generated by the manuscript authors.
Bivariate and regression analyses considered demographic characteristics (gender, race/ethnicity, age, and education) along with DEMPS experience (roles, deployments) as independent variables. Demographic characteristics were collected in the survey using multiple choice and short answer questions. Gender was a binary descriptor (male/female), whereas ethnicity included white, black/African American, Hispanic/Latino, Asian, American Indian/Alaska Native, and Native Hawaiian/Other Pacific Islander. Due to the small number of people within the non-white categories, categories were collapsed for multivariate analysis as white and Non-white. Age was initially measured as 1 of 6 categories, but subsequently collapsed into 3 groups: 18-40, 41-50, and 51 years or older. Level of education was derived from 6 multiple choice and 1 open-ended “other” option. These were grouped into 4 categories: less than a college degree (high school diploma, some college), 2-4-year degree (associates, nursing, bachelor’s degree), master’s degree, doctoral or medical degree. The authors then reviewed and coded “other” responses into 1 of these groups.
The respondents’ DEMPS role and participation were also analyzed as independent variables. Respondents were asked if they had deployed with DEMPS (yes/no). The length of time as a DEMPS volunteer was measured on a 4-point scale. Primary DEMPS volunteer roles collected through the survey included 43 options. Volunteer role categories were grouped as “Clinical” (eg, Doctor, Nurse, Other Clinician) and “Non-Clinical” (eg, Medical Technicians, Police Officers, Chaplains Administrative/Technical/Clerical Employees) for analysis. “Other Clinician” included roles such as Physician Assistant, Pharmacist, and Social Worker.
Survey respondents were also asked for their open-ended feedback. Short answer questions included items to assess additional training suggestions, ideas for maintaining volunteer engagement, and challenges to DEMPS participation. Most survey respondents (48%-64% depending on the question) answered these questions. The qualitative data from these questions referring to training were extracted and evaluated for frequency and content in a manner consistent with grounded theory methodology.
Data Analysis
Survey data were analyzed using Microsoft Excel, Statistical Analysis System software version 9.3 (SAS Institute, Cary, NC), and Stata 13.1 (StataCorp, College Station, TX). Initial analysis determined response rates, data quality, and sample adequacy. Descriptive and univariate analysis of the entire sample (n=2120) for categories of training modalities, frequencies of modalities used, and other basic sample characteristics helped guide subsequent analysis. χ 2 tests were used to identify associations between categorical variables. Bivariate analyses assessed the relationship between number of training modalities and respondent characteristics, satisfaction, attitudes, and engagement. Multiple variable logistic regression was used to confirm the descriptive associations between the number of training modalities completed and attitudes about training and continued engagement in DEMPS. The 7 measures (see Figure 1) were collapsed to reflect whether the respondent slightly agreed, agreed, or strongly agreed with the statement and then analyzed using the same set of covariates for each logistic regression model. Multiple logistic regression models were used to determine whether associations persisted once other factors were measured. Each of the regression models included the same set of covariates, including number of training modalities, DEMPS role, prior deployment, education, age, race/ethnicity, and gender. The analytic sample excluded responses that reported no training (n=405), as well as responses with missing data on training questions (n=70) or independent variables (n=34-61 depending on the item). The resulting analytic sample size ranged 1438-1457 survey responses, depending on the variable being assessed.
RESULTS
Participant Characteristics
Most the study sample was white and non-Hispanic (78%) and the modal age range reported was 51-60 years old. The modal level of education was less than a college degree (34%); however, when categories of higher education were combined, a large majority (66%) reported a higher education degree. Most participants had been in DEMPS for 5 years or less (79%) and had not deployed with DEMPS (83%). Respondents almost equally represented clinical (54%) and non-clinical (45%) volunteer roles. See Table 1 for detailed participant characteristics. A notable bivariate relationship within participant characteristics was that deployment experience was highly related to longer length of time as a DEMPS volunteer. For logistic regression models, length of time in DEMPS was omitted because of its high correlation with deployment experience.
Participant Characteristics
Abbreviations: HS, high school; DEMPS, Disaster Emergency Medical Personnel System.
These numbers are based on the entire sample but categories do not total to (n=2120) due to missing values.
Training Mode Participation
Most respondents (80%) indicated they had completed at least some DEMPS training at the time of the survey. Around half (51%) finished only 1 training mode, whereas 14% participated in all 3 modes. Among the entire sample, the majority (76%) completed online modules with substantially lower percentages reporting either field exercises (24%) or face-to-face training (23%). Table 2 provides more details regarding the combination of modes completed by survey participants. Bivariate analyses showed female respondents were more likely to have 1 or 3 training modes completed, and volunteers with deployment experience more frequently indicated completing 2 or 3 modes. None of the remaining sociodemographic, DEMPS role, or participation groups showed statistically significant differences in the number of training modes completed (data not shown).
Mode Participation by Type and Count
These numbers are based on the entire sample but categories do not total to (n=2120) due to missing values.
Training Satisfaction, Attitudes, and Continued Engagement
Satisfaction was universally high for those who indicated completing the specific training modalities in question, with field exercise (86%) receiving the highest overall rating. Multivariate models (Table 3) showed that white, non-Hispanic respondents were significantly more likely to be satisfied with online training and those with non-clinical roles were significantly less likely to be satisfied with face-to-face training (P-value<0.05 for both models). More than half of all participants that completed at least 1 training type responded positively on all questions regarding attitudes about DEMPS training. In addition, the majority of volunteers, regardless of whether they completed training, indicated they plan to continue to volunteer with DEMPS for at least another year (81%) and they would recommend volunteering with DEMPS to others (77%).
Training Satisfaction, Attitudes, and Continued Engagement
Dependent variables were coded as a “1” if the respondent slightly to strongly agreed with the statement and “0” otherwise. Multiple logistic regression models included dichotomous indicators for the number of Disaster Emergency Medical Personnel System (DEMPS) training modalities completed (1 training mode as reference category), clinical versus non-clinical role (clinical role as reference category), education category (less than a college degree as reference category), female gender (male as reference category), age 18-40 and 51 years or older (age 41-50 as the reference category), white, non-Hispanic (all other race/ethnicity categories as reference), and prior DEMPS deployment (not previously deployed reference category). Variables with odds ratios (OR) that were significantly different than 1.0 are reported in this table.
After controlling for demographic and participant characteristics, participation in all 3 training modes (vs those with only 1 training mode indicated) was directly related to a more positive attitude about training, plans to continue as a DEMPS volunteer, and recommending DEMPS to others (P-value<0.001 for all 7 logistic regression models). Prior DEMPS deployment was associated with reporting overall training satisfaction and with higher likelihood to recommend DEMPS to others in these regression models (P-value≤0.001 for both models). For questions on training as it related to deployment expectations and preparation, respondents aged 51 years and older were significantly more likely to respond that training provided useful information about responsibilities during deployment (P-value=0.020), whereas those with deployment experience were more likely to feel they were provided sufficient training in preparation for deployment (P-value=0.001). The regression models for these questions were also analyzed for only the subsample that had deployed; results were that age was no longer significant, but more training modes continued to be significantly related to positive agreement on these 3 questions. White, non-Hispanic ethnicity was found to be negatively associated with reporting that training was appropriate and with overall satisfaction with training (P-value<0.05 in both models). No significant findings arose for education level or gender in any of the regression models.
Open-Ended Response Findings
Respondents most often requested field exercises or live sessions when providing suggestions for additional training, with emphasis on “hands on” and “group” experiences. Some respondents noted they wanted more of this mode of education while others stated they had never had it but wanted and enjoyed it in other contexts. Suggestions for maintaining engagement ranged from general suggestions for improving DEMPS communication with volunteers to very specific ideas such as starting a monthly newsletter. Two frequent training-related suggestions were to make training more available and to hold field exercises. Although some comments were unclear regarding types of training that should be provided more frequently, many responses indicated the desire for more field exercises. Scheduling “interactive” exercises, “mock drills,” and “reenactments” with teams were all suggestions about how DEMPS could keep volunteers engaged. Some respondents noted that practice could “get volunteers involved” while “building social contact.”
When asked about DEMPS participation challenges some respondents indicated the training program in some way posed a challenge to their continued participation. Some stated the lack of availability of face-to-face training or field exercises was a problem, while others were unhappy with the online courses or did not believe online training was adequately supplemented by other training modes. These comments included references to a “lack of skill training” and “no teamwork building” with online training.
DISCUSSION
This analysis suggests that respondents were overall very satisfied with DEMPS training, had positive attitudes about the program, and that participation in multiple modes of training strengthened these associations. Determining volunteer preferences and effective teaching methods for the required skills during deployment is important to maintain the needed roster of qualified DEMPS volunteers. Past studies have found that training programs relying primarily on web-based independent courses have mixed results in improving knowledge.Reference Hsu, Jenckes and Catlett 10 , Reference Chung, Mandl and Shannon 12 In contrast, interactive training formats such as drills, tabletop exercises, and simulationsReference Collander, Green and Millo 8 , Reference Hsu, Jenckes and Catlett 10 , Reference Miller, Scott and Issenberg 11 , Reference Summerhill, Mathew and Stipho 13 - Reference Ingrassia, Ragazzoni and Tengattini 16 and multi-modal training programsReference Jose and Dufrene 7 , Reference Collander, Green and Millo 8 , Reference Ingrassia, Ragazzoni and Tengattini 16 , Reference Curran, Reid and Reis 17 can effectively increase knowledge retention and have been described as more helpful and preferred over self-study by health care professionals. To accommodate various learning styles and preferences, DEMPS’ multi-modal education program incorporated both interactive and self-study components.
Approximately 76% of respondents to the DEMPS survey participated in online independent study courses, whereas fewer attended field exercises (24%) or face-to-face training (23%). It was highly recommended that volunteers complete some of the online courses before engaging in face-to-face training or exercises, and many introductory sessions were offered in this manner. This may be 1 potential reason for higher engagement in this modality over the others. Regardless of the type of training mode completed, the majority of respondents who participated in each of the 3 were satisfied with what was provided. White, non-Hispanic respondents were more likely to be satisfied with online training. This may be because this ethnic group was more likely than non-white respondents to have completed online training, which could be related to differential access to or interest in this mode and correspondingly less satisfaction with it. In contrast, non-clinical DEMPS volunteers were less likely to be satisfied with face-to-face trainings. One potential reason for lower satisfaction with this mode could that non-clinical respondents were more likely to have less than a college degree, which may mean this group had less experience with or desire to be in a classroom lecture setting. Field exercises, however, were universally well received by this sample, with no group differences in satisfaction ratings and with extremely high overall satisfaction.
Most volunteers also had positive attitudes about the training content. Older DEMPS volunteers (older than 50 years of age) were significantly more likely to feel training provided useful information about responsibilities during deployment. In addition, respondents who deployed were more likely to feel training sufficiently prepared them for deployment. After checking the sensitivity of these models by running them as conditional upon deployment experience, age, and deployment experience were found to be no longer significant. This may be because those respondents 51 years old and above were the most likely age group to have deployment experience, showing collinearity in these models. To test this, we conducted a sensitivity analysis of our regression model specification and examined bivariate relationships between the covariates. We found that deployment experience was consistently related to overall satisfaction with DEMPS training and that multiple training mode participation was related to positive attitudes.
A potential supplement for volunteers without deployment experience is participation in a field exercise or hands on training. This was commonly requested in the open-ended sections of the survey as a way to improve training and enhance the DEMPS program in general. Some DEMPS volunteers noted in their comments that there were not enough opportunities to participate or they did not believe there were enough interactive training modes available. While this shows interest in multiple modalities of training, it may also highlight a limitation of the DEMPS training program. Field exercise and face-to-face trainings were generally offered less frequently than online courses, and aimed to accommodate individuals who were interested and available to attend them as much as possible. Administrative barriers to expanding these opportunities include cost and time, which are both more substantial for interactive training, particularly training that requires travel or time away from core duties of employment. However, for the relatively small percentage of volunteers who did complete more than 1 type of training (29%), their multi-modal training experience appeared to be very strongly related to their perceptions of not only the training but also of the DEMPS program itself.
Results indicate that volunteers who were willing and able to attend multiple types of trainings tended to be happier with the overall DEMPS training program. As positive attitudes about training in DEMPS could positively impact their readiness to deploy,Reference Zagelbaum, Heslin and Stein 6 it is important to understand factors that can influence these attitudes. In general, other studies have suggested that an ancillary benefit to any education or training program may be improved volunteer satisfaction, commitment, confidence, and willingness to continue to participate.Reference Zagelbaum, Heslin and Stein 6 , Reference Ye, Stanford and Gousse 9 , Reference Qureshi, Gershon and Conde 18 The current study has also demonstrated that more participation in training (based on number of modes) and deployment can improve volunteers’ willingness to continue in DEMPS or recommend it to others. Participants specifically noted in their written responses that training and field exercises could keep volunteers involved in the program.
Based on this study’s findings, it is plausible that offering additional opportunities for volunteers to engage in multiple training methods and hands on experience could improve attitudes about training and foster continued involvement in the volunteer program. In response to this possibility, in 2015, DEMPS program leaders redesigned the training program to incorporate aspects of all training modes (online, field exercises, face-to-face) into an interactive web-based platform that builds on the efficiency and standardization advantages of online training.Reference Rottman, Shoaf and Dorian 1 , Reference Chaput, Deluhery and Stake 15 , Reference Hsu, Jenckes and Catlett 19 Noting the concern that some computer-based training systems may lack interactive, team-based, or experiential components,Reference Rottman, Shoaf and Dorian 1 the redesigned DEMPS education program that was launched in 2015 includes features that support interaction and team learning. Online independent study continues to be included, while the addition of asynchronous webcasts and synchronous virtual reality provides a collaborative and interactive learning environment for volunteers. This appears to align with the trend of some other clinician-focused, disaster medicine programs to be multi-modal and incorporate information and communication technologies into their courses.Reference Jose and Dufrene 7 , Reference Collander, Green and Millo 8 , Reference Ingrassia, Ragazzoni and Tengattini 16 , Reference Curran, Reid and Reis 17 Ultimately, these new training components are intended to facilitate a multi-modal education program that reaches more DEMPS volunteers and provides more opportunities to engage in different training methods due to its accessibility through online delivery.
The results of the 2012 DEMPS volunteer survey were used to make important changes to the way VHA trains its volunteers for emergency deployment duties. As this was a pragmatic and operational survey, there are some design limitations including a limited response rate and the possibility that respondents had systematically different attitudes compared with non-respondents. An additional limitation is that the survey was only available online and completion was voluntary, which meant data were cross-sectional and respondent demographics may not be representative of all DEMPS volunteers. In addition, questions about reasons for participation or potential training barriers were not included in the survey, which may have provided a more nuanced explanation of volunteers’ experiences. Lastly, because data were de-identified, the research team was unable to link the analytic files to other data sources to address the generalizability of our findings to all DEMPS volunteers or VHA employees.
CONCLUSION
Fostering a blended learning environment with multi-modal training methods and opportunities to apply training could improve disaster volunteer training satisfaction and positive attitudes. Successful training programs may, in turn, encourage continued engagement in DEMPS, similar programs such as the Medical Reserve Corps, or disaster response volunteer programs (eg, Team Rubicon, American Red Cross). Future research on volunteer satisfaction and attitudes of the DEMPS web-based curriculum could provide insight into the viability of this type of training program for disaster health care and response volunteers.
Acknowledgments
This article is based upon work supported by the Department of Veterans Affairs, Veterans Health Administration, Office of Patient Care Services. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the US government.
