Improving the operational performance of disaster responders is a national priority. The importance of training for disaster preparedness and response professionals regardless of the professional discipline has been discussed for more than 20 years.Reference Ingrassia, Foletti and Djalali1–Reference Neal4 However, reports of gaps in training and preparedness persist, particularly for public health and medical professionals.Reference Atack, Parker and Rocchi5, Reference Williams, Nocera and Casteel6 The current programs are not well-aligned and do not meet all national capabilities described in Federal preparedness and response plans.
Domestically and globally, multiple governmental agencies, organizations, and universities have developed disaster-related education and training programs, Reference Jacquet, Obi, Chang and Bayram7, Reference Algaali, Djalali and Della Corte8 and the Federal Emergency Management Agency (FEMA) leads a national education program specifically for emergency management professionals. However, questions have been raised about the quality and completeness of these courses in relation to education and training design and content, including those for medical and public health professionals.Reference Daily, Padjen and Birnbaum9
The objective of this study is to characterize US-based online disaster training courses currently available to disaster medicine and public health professionals. Its purpose is to better inform policies and decision-making regarding workforce and professional development.
METHODS
Courses were identified from the 4 largest US-based inventories of publicly-available training courses available to disaster professionals in 2017, as follows:
1. Disaster Lit: resource guide for disaster medicine and public health; a National Library of Medicine (NLM) guide to disaster and public health preparedness literature and related information freely available on the Internet
2. TRAIN National Learning Network Course Catalog: a Public Health Foundation learning management system of public health learning products
3. FEMA National Preparedness Course Catalog: The Emergency Management Institute (EMI), serving as the national focal point for the development and delivery of emergency management training
4. Centers for Disease Control and Prevention (CDC) funded academic partnerships, the Preparedness and Emergency Response Learning Centers (PERLCs), online course catalog
An online search was performed for each of the 4 databases using 30 key words from 5 domains as criteria for inclusion in the study sample (Table 1).
Key Words Used in Online Searches, According to Disaster-Related Domain
Inclusion criteria included entries that contained at least 1 of the 30 key words AND: identifiable as an online education offering; or identifiable as a live training course; or unsure. For exclusion criteria, courses were excluded from analysis if they were not available to the general public, not related to disaster or emergency situations, or applicable toward an accredited academic degree. Assignments of criteria for data inclusion/exclusion and discipline were then validated by 3 individual reviewers using the same algorithmn.
The results of these 30 key word searches were aggregated into a spreadsheet database. Duplicate courses were identified and excluded. Those data collected included the course title, course description, host institution, sponsoring insitutions, target audience, and modality for delivery. Simple calculations of descriptive statistics were performed using MS Excel™ software. Assignments of level of learning, target capability, and function were performed by 2 expert reviewers. Discrepancies were then reviewed and adjudicated by a third reviewer (MK) using the same reference criteria (Appendix A).
The levels of learning categories were based on Bloom’s revised taxonomy.Reference Krathwohl10 This taxonomy is a hierarchical model that classifies educational learning objects into increasingly complex and specific cognitive levels. They describe the level of learning that participants would be expected to achieve based upon the course description and learning objectives (Table 2).
Criteria for Assigning Levels of Learning According to Bloom’s Revised Taxonomy
Target capability and function category were categorized according to a standard set of criteria from a predesignated list of 37 widely-accepted emergency management capabilities and functions (Appendix A). These capabilities and functions were derived from capabilities created by FEMA, 11 the US Veterans Administration (VA),Reference Barbera, Macintyre and Shaw12 and the CDC13 to guide US Government preparedness and response activities. Items were considered to be related to a given capability if the item description contained mention of that capability or 1 or more functions associated with that capability.
RESULTS
The searches identified a total of 4870 courses: 2252 (46.2%) from the NLM Disaster Lit database; 1759 (36.1%) from the TRAIN learning system; 707 (14.5%) from the FEMA National Preparedness Course Catalog; and 152 (3.1%) from the PERLC course catalog. After the deletion of 1208 duplicates, a total of 3662 unique online training courses were identified as meeting criteria for inclusion.
Modality
Of the 3662 courses identified, 3049 (83.2%) were available via web-based or other distance learning modalities (eg, CD-ROM and webinar); 604 (16.5%) courses were available by classroom only; 4 (0.1%) courses were available as mixed online and classroom; and 2 (0.05%) courses were reportedly available during meetings or conferences. Most modalities for content delivery involved didactic presentations. Some coursework also involved limited live group interactions, largely in the form of classes involving discussions or exercises.
Target Audience
The target audiences for the courses were generally divided into either disaster professionals (2380 courses, 65.0%) or the general public (732 courses, 20.0%), with 659 (18.0%) courses targeting both and 12.3% that did not specify a target audience, leaving 3039 courses for the professionals audience (Figure 1). Of the courses that targeted the professionals audience, 1868 (61.5%) were intended for public health professionals and 220 (7.2%) were intended specifically for health care professionals, leaving 842 (27.7%) targeted toward emergency management/preparedness professionals.
Target Audience Among US-based Disaster Training Courses.
Content
Course subject matter was categorized according to 37 target learning capabilities detailed in Appendix A. Figure 2 depicts the number of training courses identified for each target capability.
Number of US Disaster-Related Training Courses, According to Target Capability.
The number of courses ranged from 5 (0.1%) for 3 capabilities (medical material management and distribution; management and organization) to 733 courses (20.0%) related to emergency management. The mean number of courses per capability was 99 with a median of 37 courses. Half of the courses (53.1%) focused on only 3 of the capabilities: emergency management, 733 courses (20.0%); hazard-specific disaster knowledge, 662 courses (18.1%); and health care, 551 courses (15.0%). Associated functions were categorized for all 37 capabilities. Functions associated with these 3 most common capabilities are reported here.
Emergency management accounted for one-fifth of all courses. Most content (62.2%) focused on preparedness and response, whereas only 14% were associated with mitigation (8.7%), recovery (4.2%), and prevention (1.1%). Other courses in this category included emergency operations centers (13.3%), incident command systems (12.8%), and emergency management (8.5%).
In total, courses associated with chemical, biological, radiological, nuclear, and explosives (CBRNE) comprised 76.3% of all available courses involving hazard-specific knowledge: general CBRNE, 37.8%; biological, 35.1%; chemical, 3.4%. Course content related to biological hazards related mostly to Ebola and Zika viruses. Other hazard-specific topics included natural hazards (11.8%), climate change (5.4%), man-made hazards (3.9%), public health emergencies (0.9%), case study (0.9%), and active shooter (0.6%).
Of the 551 courses related to health care, 71.7% focused on 4 functions: mental health (32.8%), infectious disease (20.1%), CBRNE (9.9%), and mass casualty management (9.0%). Others included pediatrics (6.6%), health care systems (5.6%), hospital emergencies (3.2%), emergency medical services (2.9%), toxicology (2.9%), primary health care (1.4%), health care preparedness (1.3%), health physics (2.0%), mass care (0.9%), non-communicable diseases (0.5%), reproductive health (0.5%), and geriatrics (0.2%).
Level of Learning
The levels of learning were assessed by applying Bloom’s taxonomy of learning to course descriptions. Essentially, all courses (99.4%) delivered learning at a level consistent with the lowest 3 levels of learning: Remembering, 221 (6.0%); Understanding, 2866 (78.2%); and Applying, 557 (15.2%). Twelve courses (0.3%) were found to include learning objectives and descriptions consistent with Bloom’s level 4 (Analysis). There were no courses identified that related to Bloom’s level 5 (Evaluation) and only 4 courses (0.1%) related to Bloom’s level 6 (Creation).
DISCUSSION
Training and education are critical steps for preparing the professional disaster workforce. The importance of professionalizing the disaster workforce has been increasingly discussed in the professional and academic literature.Reference Ingrassia, Foletti and Djalali1, Reference Johnson, Idzerda and Baras14–Reference Schultz, Koenig, Whiteside and Murray16 This study has revealed a very large number of publicly available courses for disaster-related training for health and public health professionals in the United States, but the question is, “Do they meet the criteria for improving professionalism and national preparedness?”
This study raises a number of concerns about the existing available training in all apects of the courses studied: target audience, content, training approach, and levels of learning. The first and most practical difficulty is the sheer number of courses available and the relative paucity of descriptive course information available online. The lack of information makes it difficult for those seeking training to identify the appropriate, high-quality training to meet their education needs. The lack of guidance is compounded by “course overload.” In many subject areas, the number of courses available appears to be excessive given that the training specifically targets 6 professions only. For example, there were 1318 courses targeted for public health professionals.
Despite the large number of courses, most of the content is duplicative while simultaneously limited to a narrow range of topics. When comparing course content to 37 recognized national disaster capabilities (Appendix A), over half of all courses focused on only 3 capabilities, and 38% of all hazard-specific courses had content related to CBRNE, despite the rare nature of these potential events (as compared with 12% related to natural disasters – the most common). This results in large gaps in training related to target capabilities and the disaster hazards themselves.
Although it was not possible to make a detailed assessment of the quality of each course without taking each of them, there were indications of general issues with quality. The majority of training courses studied were not only teaching the same subjects, but also nearly all were designed to impart only a lower level of learning on Bloom’s taxonomy. These lower levels are similar to introductory or overview courses in high school or college and do not develop professional competency beyond a level of general understanding and have very limited opportunities for application of knowledge.
The target audience of the courses also appeared uncoordinated and was unequally distributed among professions, with public health accounting for more than half of all courses, leaving the health care workforce with only 6% of the total courses. An internationally accepted framework was developed to define disaster education target audiences on the basis of their level of responsibility – strategic, tactical, and operationalReference Alexander17 – but we were unable to determine the operational level of the courses from the information available on the websites.
Overall, despite the availability of literally thousands of courses, nowhere is there a clear path to the stepwise development of professional competency for health and public health professionals. Effective performance is limited by the absence of these standards. There are no measures of content completeness or certifications of professional competency. There is a need for further clarification and vetting of the disaster-related roles and responsibilities of public health, emergency management, and medical professionals. Task analyses should be used to guide development and updating of tasks, competencies, and associated learning objectives according to professional role and target audience.
All trainings appear to use a relatively narrow knowledge-based approach. Students expectedly gather knowledge in absence of also developing the closely associated skills and abilities that are known to ensure a successful performance. Most models of performance and quality management recognize that, in addition to knowledge, other qualities (eg, skills and abilities) are also required for effective performance. All three of these elements are necessary for any employee to perform their job effectively. Other than real-time, on-the-job experience, disaster professionals have few opportunities to gain, not only a deeper understanding, but also to learn, practice, and perfect more complex skills, abilities, and behaviors consistent with the demands of the profession.
Experiential and analytical knowledge is important for disaster managers, but the courses described in this study are unlikely a suitable resource for upper levels of knowledge. Opportunities for students to perform complex analysis, evaluation, and creation of original content were limited because most courses were only available as online fixed lectures without discussion groups. There were very few descriptions of the types of activities that are necessary for higher level of learning (eg, case study, problem solving, assessments, critiques, planning, teamwork, and projects). These are more efficiently delivered through more complex interpersonal group interactions, structured facilitation, and iteration in collective venues such as workshops, projects, and programs. Each training modality (lecture, case study, simulations, and so on) should be aligned with the level of taxonomy most appropriate for that individual’s role and responsibility in the disaster workplace.
More research is needed to study the effectiveness of disaster-related training courses and education. A recent review article on health-related disaster training found only 36 scientific studies in the prior 35 years that assess the methods and quality of disaster education courses.Reference Gowing, Walker, Elmer and Cummings18
Possible Solutions
There is a need to create a comprehensive strategy and standardize the educational approach across many levels. To start, a standard curriculum content focused on (evidence-based) core competencies should be developed. From that standard content allows for targeting learning according to actual task analysis and helps prioritize the scarce time resources for the busy professional who will benefit most from these courses.
There is also a need to define tiered levels of learning for curriculum content that includes progressively higher levels of learning for some managerial and technical components of the curriculum content. Bloom’s taxonomy can be useful to define measurable stages of competencies that can then be achieved through step-wise training. Use of a standard taxonomy to define measurable stages of competencies will allow for a measurable progression through step-wise learning. Each modality (lecture, case study, simulations, and so on) may be then aligned with the stage most appropriate for that individual. The need for higher levels of disaster-related training, such as experiential and simulation-based, has been well described in the education and even disaster literature.Reference Weaver, Salas and Lyons19–Reference Foletti, Ingrassia and Ragazzoni21
Competency-based models have become the norm in designing education and training strategies and courses. Such competency models have been developed for disaster managers by FEMAReference Feldmann-Jensen, Jensen and Maxwell22, Reference Blanchard23 and disaster health and public health professionals.Reference Walsh, Subbarao and Gebbie24 Using these models ensures that educational strategies map back to disaster preparedness and response priorities and help standardize training to improve the professionalization of the discipline. Authors have also suggested strategies to facilitate their alignment to practice.Reference Walsh, Altman, King and Strauss-Riggs25, Reference Schor and Altman26
FEMA has attempted to organize their disaster management training courses by competencies and target audiences, and has worked with universities to develop appropriate curriculum for higher level learning (FEMA National Preparedness, https://training.fema.gov/). FEMA offers courses through 3 mechanisms and sites: The Center for Domestic Preparedness in Anniston, Alabama, that offers on-site simulation and classroom training; the EMI online training site for state, local, and tribal emergency managers; and the National Training and Education Division, which focuses on first responders. The online course catalog (https://www.firstrespondertraining.gov/frt/npccatalog) lists over 700 courses and provides the most complete descriptions to allow learners to more easily identify appropriate training.
Similar to FEMA’s efforts for disaster managers, there is a need for a structured information and knowledge management system to manage training for US disaster health and public health professionals. Effective management of a functional curriculum in the future would be dependent upon not only standards being defined, but also measured and subsequent interventions applied to improve quality. A structured approach to information and knowledge management for the community of practice should include the following components:
1. Information Management
– Archiving: hosting and organizing training content
– Enterprise portals: aggregating sources of training in 1 access point
– Course accreditation: according to standard criteria
– Content management: hosting of libraries; subsidizing access
2. Knowledge Management
– Interactive platforms: threaded discussions, file-sharing, mixed media
– Curriculum monitoring and evaluation
– Participant evaluation, certification, and credentialing
Data derived from the structured, internal management of this system would also logically inform long-term monitoring and evaluation to include efficiency and effectiveness of the training, as well as identifying training needs and resources.
Limitations
This review was limited to publicly available online trainings cataloged in 4 databases as available during the fall of 2017. There may be additional disaster trainings that were missed in this review, but these 4 national inventories specifically exist to capture US-based courses and identified nearly 5000 courses, so that this sample may be considered representative of the current state of US disaster training.
University and college-based courses and degrees were not included in this study. These more structured, detailed, and lengthy courses may better meet the higher cognitive levels of Bloom’s taxonomy, but they are more difficult to access, costlier, and thus have a much smaller audience.
There is also potential for selection bias during categorization and assignment of learning levels, capabilities, and functions. To mimimize this, we used a battery of 3 separate reviewers but the limitations of the available course descriptions and learning objectives sometimes made identifying categories difficult. Because disaster health trainings are not typically built upon standardized capabilities or competencies, the authors used the most widely recognized capabilities as a best practice comparison for subject matter content of the training topics.
CONCLUSIONS
There are a very large number of public available courses for training disaster professionals in the United States. Although competencies, curriculum, and content have been described for disaster management, this is not true for health and public health professionals seeking disaster training and education. There are issues identified with access, content, target audience, and quality. These gaps may limit the effectiveness of the trainings to impart an appropriate level of knowledge, skills, and abilities that is requisite for competency among US disaster professionals. Adapting FEMA’s knowledge-management model to health and public health could be a solution for many of these issues.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/dmp.2019.15.
