The increasing frequency and expanding scope of disasters and public health emergencies globally have attracted many health care providers, public health workers, administrators, and other professional personnel to the emerging discipline of disaster medicine and public health preparedness.Reference Stopford1 2 Reference Rinnert, Wiggington and Pepe3 Reference Hodge, Pepe and Henning4 In responding to disasters these personnel must often work with limited resources in extremely stressful, high-risk situations in which many lives may hinge on their performance.Reference Hodge, Pepe and Henning4 Reference Carpenter5 Reference North, Tivis and McMillen6 Reference North7 Reference Curiel8 Reference Klein and Nagel9 Reference Coyle, Sapnas and Ward-Presson10 Specific competencies in disaster preparedness and response are essential to effective performance in disasters. Consequently, training curricula for various health-related disciplines increasingly incorporate education and such competencies.11 Reference Archer and Seynaeve12 Reference Uddin, Barnett, Parker, Links and Alexander13 Reference Alexander, Bandiera and Mazurik14 Reference Garfield and Al15 Reference Weiner, Irwin, Trangenstein and Gordon16 Reference Veenema17 Reference Silenas, Akins, Parrish and Edwards18 Reference Swienton19 Reference North, Hong and Pfefferbaum20 Reference Cummings, Della Corte and Cummings21 Reference Cherry and Davis22 Reference Macfarlane, Joffe and Naidoo23 24 Reference Debacker, Delooz and Corte25 To address rapidly evolving training needs, educators must continue to learn from actual disasters, identify attributes that distinguish effective from ineffective workers, and update competency frameworks and training programs accordingly.
A number of competency descriptions for health care providers, public health workers, and students has been developed using various methods of data gathering and consensus building.Reference Subbarao, Lyznicki and Hsu26 Reference Hsu, Thomas, Bass, Whyne, Kelen and Green27 Reference Gebbie and Merrill28 Reference Markenson, DiMaggio and Redlener29 Reference Parker, Barnett, Fews, Blodgett and Links30 Reference Hites, Lafreniere and Wingate31 Reference Gebbie and Qureshi32 Reference Larkin and Arnold33 Reference Larkin, Iserson and Kassutto34 35 36 37 In this exploratory study, the authors sought the perspectives of civilian out-of-hospital emergency medical care experts who were leaders of large emergency medical services (EMS) systems throughout the United States. The researchers used focus groups, a qualitative research technique, to gather data from these experts concerning the competencies of effective disaster responders and leaders.
A focus group is a specific type of group interview that relies on discussion among its participants to explore one or more topics.Reference Kitzinger38 Reference Basch39 This technique can elicit from participants spontaneously expressed opinions, ideas, and perceptions, and it can generate qualitative data for research in education and health care.Reference Basch39 Reference North, Pollio and Pfefferbaum40 Reference Eisenman, Glik, Maranon, Gonzales and Asch41 Reference Cone and Murray42 Reference Boendermaker, Schuling, Meyboom-deJong, Zwierstra and Metz43 Focus groups have been used to identify worker competencies in a number of fields. Typically, participants in these groups are people who are believed to possess important perspectives, insights, or expertise pertinent to the identification of attributes of the worker groups in question. These individuals may be expert workers, supervisors, subordinates, patients, customers, or others as deemed appropriate. For example, general practitioner trainers, trainees, and other staff members identified the attributes of general practitioner trainersReference Boendermaker, Schuling, Meyboom-deJong, Zwierstra and Metz43; expert emergency nurses identified attributes of effective triage nursesReference Cone and Murray42; and users of mental health services identified attributes of mental health nurses.Reference Rydon44
In these studies and in the general literature on workforce competencies, such attributes have included knowledge, skills, attitudes, and behaviors, as well as certain personal characteristics.Reference Cone and Murray42 Reference Boendermaker, Schuling, Meyboom-deJong, Zwierstra and Metz43 Reference Rydon44 Reference Spencer and Spencer45 Reference McClelland46 Reference McClelland47 Reference Barrick and Mount48 Reference Clayton, Lypek and Connelly49 Reference Retzlaff and Gibertini50 Reference Chidester, Helmreich, Gregorich and Geis51 Reference Rose, Fogg, Helmreich and McFadden52 Reference Helmreich, Foushee, Benson and Russini53 When describing an effective person for a particular job or role, an optimal combination of such attributes has been referred to as “the right stuff.”Reference Wolfe54 In the present study, focus groups of expert emergency response leaders with responsibility for numerous EMS and disaster response personnel identified attributes of effective disaster workers.
METHODS
Three focus groups were conducted at an annual meeting of jurisdictional medical directors for the 9-1-1 EMS systems for the majority of the nation's 30 or so largest cities. This preconference meeting was held in conjunction with the February 2006 conference called EMS State of the Science: A Gathering of Eagles.55
Subjects
Participants in the focus groups were the 24 medical directors who attended the meeting, which included 21 men and 3 women. It was known from previous meetings, medical literature, media reports, and other documentation that these participants collectively had significant on-scene experience and direct involvement in large-scale disaster management including major hurricanes, tornadoes, floods, ice storms, air and rail crashes, wildfires, earthquakes, mass explosions, chemical releases, terrorist attacks, and other catastrophes.
Data Collection
Three 8-member focus groups were conducted concurrently. Three of the authors served as the group facilitators. The facilitators asked the participants to discuss any attributes that they as experts believed distinguish “effective” versus “ineffective” disaster responders or leaders. This research was exploratory rather than theory-based, therefore, no preexisting frameworks or categories were provided to the focus groups. Rather, the information provided by the disaster experts in response to an open-ended question was considered the starting point to examine characteristics of disaster workers.
Once the discussion was launched, the role of facilitators was to monitor the discussion but not contribute to it, encourage all of the participants to enter the discussion, intervene if the discussion veered off topic by prompting participants to return to the question at hand, and answer general questions but redirect any content-specific questions back to the group. After approximately 45 minutes the discussions had run their course, and the participants indicated that they had no information to add. The facilitators called time and thanked the participants, who then returned to their meeting.
Advance approval for this research was obtained from the University of Texas Southwestern Medical Center institutional review board. All of the participants signed informed consent forms before the focus group sessions and agreed to the sessions being recorded with audio equipment. The audio recordings were subsequently transcribed by an independent transcriptionist, after which the recordings were destroyed.
Data Analysis
The focus group transcriptions initially contained 224 separate passages of text, demarcated by a change in speaker. NVivo software (QSR International (Americas) Inc, Cambridge, MA) was used to organize and interpret these data for qualitative analysis. The passages were reviewed by 1 of the present authors to identify recurring themes from which to create preliminary categories into which the passages could be coded. Two other authors reviewed the passages and eliminated a number of them as uncodable (eg, off topic, with insufficient content such as “uh-huh“), yielding 138 codable passages. The raters then independently coded a sample of the remaining passages and compared their ratings. This process led to the development of detailed category definitions with rules for inclusion/exclusion and guidelines for differentiating certain categories from others. Areas of rater disagreement were resolved by discussion and mutual agreement between the 2 raters, and category definitions were revised accordingly. Interrater reliability was assessed using κ statistics for each category and the total of all of the categories. The frequency of passages coded into each category was also calculated.
RESULTS
From a review of the 224 passages in the focus group transcriptions, 10 themes were identified to serve as categories for coding the passages: (1) Incident Command System/Disaster Training/Experience, (2) General Training/Experience, (3) Teamwork/Interpersonal, (4) Communication, (5) Cognition, (6) Problem Solving/Decision Making, (7) Adaptable/Flexible, (8) Calm/Cool, (9) Character, and (10) Performs Role. Table 1 contains detailed descriptions of these categories with illustrative examples from the focus group transcripts. The codable passages were coded to 1 or more of the categories. Figure 1 shows the percentage of the 138 passages coded into each category (due to coding of some passages into more than 1 category, the sum of percentages exceeds 100%). The percentages varied widely from 43% of the total passages (Incident Command System/Disaster Training/Experience) to 1.5% (Calm/Cool).
Frequencies of passages sorted into thematic categories.
Categories and Definitions
The overall κ statistic for interrater reliability was 0.8, achieving a recognized threshold for excellent interrater reliability. Individual κ values for the 10 categories ranged from 0.6 to 1.0, with 6 of the 10 scoring at least 0.8 (excellent) and the lowest, 0.6, reflecting good agreement.Reference Fleiss56
DISCUSSION
This study identified 10 categories of attributes of effective disaster response personnel, based on the spontaneously expressed opinions of the experts in our focus groups. The breadth of these categories suggests that a variety of attributes may contribute to effective performance of disaster response personnel. These diverse attributes also span several general domains of knowledge, skills, attitudes, behaviors, and personal characteristics, domains consistently found in the general literature on competencies. For example, a number of studies in health care provide evidence that, in addition to skills and knowledge, certain personal characteristics contribute to successful clinical performance.Reference Clayton, Lypek and Connelly49 Reference Murden, Galloway, Reid and Colwill57 Reference Shen and Comrey58 Reference McGaghie59 Reference McGaghie60 Reference Peskun, Detsky and Shandling61 Reference Crosby, Dunn, Fallacaro, Jozwiak-Shields and Maclsaac62 Reference Schneider-Kolsky, Wright and Baird63 Reference Greganti, McGaghie and Finn64 Reference Feldman and Crook65
Systematic development of a competent disaster medicine workforce requires first that worker competencies be defined and then validated. These competencies, then, could guide workforce development processes such as recruitment, assessment, selection and placement, training, and performance management. Ideally, the recruitment population should contain a sufficient number of eligible candidates who possess specific knowledge, skills, attitudes, behaviors, personal characteristics, and experience known to contribute to competence and effective performance—i.e., “the right stuff.” Selection and placement could be enhanced by screening and assessment of candidates for these competencies. Training could address identified gaps in knowledge and skills and, where appropriate, could reinforce attitudinal and behavioral components of these competencies. Managers could incorporate these competencies into their periodic review and evaluation of personnel. Workforce development plans could incorporate input based on these evaluations of worker competence.
Certain relevant personal characteristics may be enduring, even lifelong, and refractory to change. Consequently, selection and placement, using appropriate screening and assessment methods, serve to obtain workers who have the desirable personal characteristics.Reference Feldman and Crook65 There is also evidence that training that actively engages participants and provides practice with feedback can effect measurable positive changes in attitudes, behaviors, and personal reactions, as well as in other areas of personal effectiveness such as decision making, interpersonal communication, and leadership.Reference Hannigan66 Reference Helmreich67 Reference Nullmeyer and Spiker68 Reference Borrelli, Lee and Novak69 Thus, an individual who is not particularly strong in certain desired personal characteristics may, through targeted training, be able to modify certain attitudes or behaviors that would make him or her more effective.
This study had several limitations. The raw data were the expressed opinions of a purposive sample of experts. These experts were mostly male and held positions of significant scope and responsibility in large urban areas. As such, these experts may not be representative of the disaster response community at large (which would include, for example, responders not in leadership roles and those from rural areas), and therefore further validation of the expressed opinions is needed. The focus group methodology inherently limits confidentiality in the sense that each participant's opinions are shared with others in the same group, which could inhibit sharing of controversial or socially undesirable opinions. Finally, this qualitative study was intended to identify categories, not to quantify their relative importance to the participants in the focus groups; therefore, the frequency with which passages from the focus group transcripts were coded to any particular category should not be interpreted as indicating the importance of the category.
Further research is needed to develop valid descriptions of disaster worker competencies. Replication of this study may help validate the 10 categories by examining whether other experts also would identify attributes related to these categories. Further validation of the categories might include surveying a broad sample of experts from the disaster response community and having them rate the categories or rank them in order of their relative contribution to the effectiveness of disaster workers. Using multiple methods, researchers should continually seek to learn from each new disaster and to describe in detail the attributes of disaster workers who perform effectively. Competency frameworks describing disaster workers should be considered works in progress and should be reexamined regularly in light of what is learned from each new disaster. Finally, workforce development processes, including recruitment, assessment, selection and placement, education and training, and performance management, should be reviewed periodically in light of updated competency frameworks.
CONCLUSIONS
Experts in focus groups were able to identify a variety of competencies including knowledge, skills, behaviors, and personal characteristics that may distinguish effective versus ineffective disaster responders and leaders and help guide workforce planning and personnel practices such as selection and placement, training, and performance management. These competencies should be validated through further research that involves input from the disaster response community at large.
Author Disclosures:
The author reports no conflicts of interest.
Financial Disclosures: This research was supported by National Institute of Mental Health Grant MH40025 to Dr. North.
Dr. North discloses employment by VA North Texas Health Care System, Dallas, TX, USA. Points of view in this document are those of the author(s) and do not necessarily represent the official position of NIMH, the Department of Veterans Affairs, or the United States government.
Acknowledgment: The authors thank the attendees of the 2006 Gathering of Eagles conference who participated in the expert focus groups that served as the primary data for this study. We also thank Lorrie Burkhalter for transcription and other essential support provided for this project.
