Introduction
Disasters and other public health emergencies in the US often reveal deficiencies in response capacity and in overall preparedness planning to safeguard the wellbeing of citizens, particularly those residing in rural geographic areas. A special problem has been the disproportionate surge of problematic psychological (vs. physical) health problems associated with such events. The core hypothesis of the authors’ research is that a partnership among three, key stakeholder entities in the public health emergency preparedness system (PHEPS) can be an effective structure through which a two-phased/two-intervention model of community-based, public health emergency preparedness can be developed and disseminated.
Exemplifying work in the field of public health systems research, and funded by the Centers for Disease Control and Prevention (CDC), the investigations are seeking to characterize, refine, and validate a set of evidence-based interventions for addressing the problem of behavioral health surge on selected at-risk populations. Involving psychological first aid training for paraprofessionals and guided preparedness planning for communities at-large, the project is an extension and enhancement of a series of pilot projects supported by the Hospital Bioterrorism Program of the US Health Resources and Services Administration (HRSA) and awarded by the Office of Preparedness and Response of the Maryland Department of Health and Mental Hygiene. Focusing on collaborative disaster mental health projects with faith-based organizations (FBOs) and local health departments (LHDs) in Maryland, these early studies provided a foundation for an hypothesized effective approach to enhancing the capacity and capability of the public health emergency preparedness system. Early results of the psychological first aid training phase of the study have already been reported,Reference McCabe, Perry, Azur, Taylor, Bailey and Links1 This article describes the background, methods, and findings for the guided preparedness planning phase of the model.
Detailing the Problem(s)
Behavioral Health Surge
Psychological casualties greatly outnumber physical casualties following disasters and other large-scale, public health emergencies.2, Reference Norris, Friedman, Watson, Byrne, Diaz and Kaniasty3 The phenomenon of disproportionate negative effects on the mental and emotional well-being of disaster survivors is ubiquitous and applies to all events irrespective of origin, type, magnitude, or duration.Reference Green, Jacob, Lindy, Grace and Leonard4-Reference Ursano, Cerise, DeMartino, Reissman and Shear12 Compounding this general public health problem is the vulnerability of many sub-populations who, by virtue of such factors as geographic isolation, socioeconomic status, and physical or psychiatric disabilities, are susceptible to both acute and chronic post-trauma problems.Reference Smith, North, McCool and Shea13-Reference Pole, Best, Metzler and Marmar16
Local Hazard Vulnerability
The study was, and continues to be, conducted in the northern-, mid-, and eastern shore areas of Maryland, a region where formal hazard vulnerability analyses routinely identify threats having a relatively high probability of occurrence, particularly flooding (flash/riverine and tidal/coastal) occasioned by hurricanes and storms. Such public health emergencies represent threats not only to property, but also to human health because of risks related to injuries, sanitation, water supply contamination, and disease outbreaks—all of which portend disruption of emotional equilibrium and mental health status among affected residents. Given recent worldwide attention and concern about avian/swine pandemic influenza outbreaks, as well as possible intentional acts of spreading bacteria, viruses or natural toxins that can cause sickness or death to many people at once, the categories of high concern in these hazard vulnerability reports may well be expanded in future analyses.
Absence of Formal Community Disaster Plans
Despite the risk liability identified in these hazard assessments, community disaster preparedness plans, particularly plans to mitigate the effects of psychological trauma on vulnerable rural populations, tend to be nonexistent or underdeveloped. Moreover, national planning approaches and guidance documents are likely to be overwhelming in their form, content, and scope. Although laudable efforts have been made to customize the US Department of Health and Human Services’ Mental Health All Hazards Disaster Planning Guidance for lay communities,Reference Koenig17 without the aid of expert support, the process of developing viable disaster plans remains a daunting challenge for the average individual, organization, or community.
Toward a Solution: Planning Collaborations Within the Public Health System
Government/Community/Academic Partnerships
Building the capacity and capability to protect citizens in the face of periodic crises is a formidable task for local health departments and state/province public health officials. Such a challenge is unlikely to be met without collaborative prearrangements among key stakeholders who can jointly plan for emergency response and recovery needs for persons most likely to be affected adversely by catastrophic events. Preparedness planning is likely to be most effective if it enhances existing community partnerships and engages community partners. One example of a vital, but suboptimally utilized, resource for pre-disaster collaborative planning is the faith-based organization (FBO). A study of partnerships with local health departments in Wisconsin found that faith organizations tend to be engaged when issues require broad community attention. Noteworthy contributors to the effectiveness of such partnerships were: having financial support; a broad array of partners; and sufficient time for partnerships to succeed.Reference Zahner and Corrado18, Reference Zahner and Kaiser19
Faith communities also would appear to be an especially valuable resource for enhancing community response to behavioral health surges, but their true potential likely would be realized only within the context of formal relationships with government agencies whose missions relate to emergency preparedness. Such alliances not only can provide linkages for systematic activation and deployment of volunteer responders, but also serve as mechanisms through which formal, mutually beneficial preparedness planning efforts can be pursued. However, the full potential of government/faith partnerships is likely to remain dormant without a third, appropriately-qualified agent to catalyze, coordinate, and guide such joint ventures. In the present study, this role was served by an academic health center (AHC).
The Project: A Systems-Based, Three-Party Collaborative Model
Based on the foregoing rationale, a study was conducted involving multiple local health departments (LHDs) and FBOs, and one AHC. The study extends to a rural region academic/faith pilot work begun earlier in the Baltimore metropolitan area.Reference McCabe, Mosley and Gwon20-Reference McCabe, Mosley, Gwon and Kaminsky22 The purpose of the present study was to build on this systems-based approach by extending, characterizing, and creating an evidence base for the effectiveness of the LHD/FBO/AHC partnership model. Project partners are developing and evaluating a two-phased/two-intervention program. As a follow up to the report on the Phase I work (developing and evaluating a paraprofessional model of psychological first aid training of individual prospective-responders),Reference McCabe, Perry, Azur, Taylor, Bailey and Links1 this report focuses on the Phase II intervention, ie, disaster mental health preparedness planning with teams of LHD leaders and FBO/community representatives.
Methods
Participants
Local health departments, faith-based organizations, academic health centers, and individuals participated in the study.
Local Health Departments
Government partners were the “Emergency Planners,” or “Emergency Coordinators,” of four local health departments in the upper-, mid-, and eastern shores of the state of Maryland. The average population size of the health jurisdictions (counties) in the target region is 33,846 (range: 19,983 to 99,506). There is an overrepresentation of the elderly in this region (19.5% compared with 11.6% in the state). One LHD partner (author CP) served as an interface between the AHC faculty and LHD partners.
Faith-Based Organizations
Community partners were clergy- and lay leaders of 100 predominantly Christian faith organizations in the same health jurisdictions.
Academic Health Centers
The academic component of the collaboration was represented by selected faculty members and administrators of multiple offices, departments, and centers within the Johns Hopkins Bloomberg School of Public Health, the Johns Hopkins School of Medicine, and the Johns Hopkins Hospital and Health System. To encourage and reward community participation, funding for purchase of emergency equipment and materials was allocated to LHDs for disbursement to participating FBOs. Following participation in the first phase of training, faith leaders received customized “go-kits” for their organizations. These kits were comprised of emergency tools and supplies, hand-crank radio receivers, citizen-band (CB) radios, walkie-talkies, flashlights, blankets, water, and hand-sanitizers.
Individual Trainees
Individual participants were: (a) clergy; (b) lay leaders of selected ministries; and (c) adult community members at-large, all of whom were recruited through outreach efforts conducted by FBO and LHD leadership. Recruitment approaches included newspaper ads, announcements at Sunday worship services, electronic- and postal service mail, bulletin inserts, radio spots, snowballing approaches (ie, person-to-person networking). Individual participants received personal/family emergency kits with items related to weather (emergency poncho and blanket), hydration (drinking water), signaling (whistle with neck cord, light stick), germ protection (hand sanitizer-pack, face mask), and first aid equipment (American Red Cross Mini First Aid Kit, five types of bandages, antiseptic cleansing wipes, antibiotic ointment; etc).
The Intervention Curriculum: Guided Preparedness Planning
The study intervention, designated Guided Preparedness Planning (GPP), involved one training session followed by two technical assistance (TA) workshops. The one-day session was held with all participants, using a professional continuing medical education/continuing education unit format, with a combination of didactic and experiential teaching methods. The didactic portion depended heavily upon a Microsoft PowerPoint-based lecture, while the latter involved group consideration of disaster vignettes focused on at-risk population types, with discussion to encourage practical application of plan content. The second part of the GPP intervention was comprised of technical assistance workshops, held after the full-day training sessions, in which emergency planners from each LHD met with multiple planning teams (each with two or three persons and representing a given FBO) to further develop the drafts of their parish disaster plans. The TA workshops were co-led and facilitated by Johns Hopkins University faculty members.
The curriculum and plan development involved establishing the parameters for a comprehensive but practical disaster planning template. Criteria for the latter were: (1) adoption of an “all-hazards” orientation; (2) identification of the key functions needing to be performed in emergency contexts, and which persons in the target communities have qualifications that fit with those responsibilities; (3) priority focus on mental and behavioral health surge issues; (4) special attention to vulnerable populations; (5) sensitivity to socio-cultural issues, including rural residence; and (6) the least possible respondent burden to maximize likelihood of adherence to the planning protocol and overall project requirements.
Formal plan content was organized in modules that corresponded to a disaster plan guidance provided by Maryland's Office of Preparedness and Response, that included: (1) Background and Assumptions; (2) Target Community (the delineation thereof); (3) Roles and Responsibilities (the Incident Command System); (4) Operations and Response; (5) Communications; (6) Preparedness Tools and Resources; and (7) Plan Review and Evaluation.
A planning template/workbook was used with modules linked to the PowerPoint slide presentation, with appropriate pauses built into the presentation to allow the teams of trainees to direct their plan-completion efforts to corresponding sections in the workbook. As a central part of the planning process, participants were guided through a Strength/Weakness/Opportunity/Threat (SWOT) analysis, whereby they identified their (internal) community strengths and weaknesses, as well as their (external) opportunities and threats.
The primary trainer was one of the co-authors (HG) who serves as the Administrator of the Office of Emergency Management of the Johns Hopkins Hospital and Health System. His presentations were augmented by co-trainers who served as content experts with specialized knowledge in hospital/community relations, emergency preparedness challenges for county governments, special needs populations, academic/government relations, and behavioral health issues.
Primary Outcome Constructs and Measures
Two main categories of outcome were targeted in the study: (1) feasibility of the partnership concept and (2) effectiveness of the GPP intervention.
Three types of evidence were selected from which to infer feasibility of the partnership model, corresponding to the idiomatic expression, “ready, willing, and able,” a conceptual framework currently unifying the initiatives at the Johns Hopkins Preparedness and Response Research Center.Reference McCabe, Barnett, Taylor and Links23 In the order in which the associated activities were implemented, these criteria were: (1) the willingness of LHDs and FBOs to respond to the AHC-initiated project concept, as measured by verbal agreement to participate in the study, followed by a signed letter of collaborative intent; (2) the readiness of LHDs and FBOs to respond (and commit resources) to the project concept, as measured by attendance of qualified individuals, in person or by teleconferencing mode, at ≥80% of all monthly Partnership Steering Committee meetings; (3) the ability to respond to the project concept (and recruitment responsibilities), as measured by successful outreach of LHD partners to FBOs, and the subsequent successful outreach of FBOs to community members with participation in GPP workshops of at least organizations and 15 individuals per jurisdiction.
Evaluation of the GPP curriculum effectiveness involved a combination of self-report and behavioral indices in three domains of hypothesized learning: (1) knowledge, beliefs, and perceptions, as measured by participant reports of understanding disaster mental health concepts and disaster plan content; (2) abilities and skills, as measured by individual-participant reports of enhanced confidence (perceived “self-efficacy”) in their ability to execute planning strategies and techniques; and (3) attitudes (about the program), as measured by participants’ reports of satisfaction with achievement of program goals, with the level of trainer/trainee interaction, and participants’ perception of overall program value.
Other Outcome Constructs
Assessment of two additional outcome categories was contingent upon positive findings relative to feasibility and effectiveness. These categories of inquiry were model sustainability and scalability. The respective questions for these categories were: (1) Will the leaders of local health departments attach sufficient value to their brief collaboration with faith-based organizations to be willing to strengthen and sustain relationships beyond the term of the project?; and (2) Can portable tools and materials be developed to facilitate the model being taken to scale?
The criteria selected to assess the LHD's willingness to sustain and advance joint planning relationships with FBO partners was their willingness and ability to develop ideas about how to strengthen and sustain the new preparedness relationships. The operational criteria to assess the above criteria were the evidence of brainstorming and the submission of written plans from each LHD partner with specific suggestions for nurturing and expanding relationships with faith communities in their jurisdictions. Three discrete, post-project actions planned by each LHD were set as the minimum threshold to denote “willingness” and “ability” to generate ideas to sustain preparedness relationships.
To bring the GPP intervention and model to a scale that has genuine public health implications, an aim of the study was to develop and validate materials and operations to support application of the model in other health jurisdictions, ideally for online use.
A post-test was employed to evaluate all outcome aims and objectives. With regard to intervention effectiveness, an effort was made to have a proxy for pre-/post-training changes by using item-wording that required respondents to compare their post-test knowledge with their pre-test knowledge. For example: “Compared to what I knew before today's training, I am much more aware of the content of an ideal community disaster plan.”
Data Analysis
Simple frequency counts (eg, the number of meetings attended by organizational partners) and descriptive statistics (eg, percentage-based demographic characteristics of individual participants) were the metrics used to characterize outcomes on key dependent/criterion variables. The number of Likert-item endorsements of “Agree” and “Strongly Agree” were aggregated from each participant's evaluation form and summarized as percentages of total possible endorsements of all participants.
Results
Demographic characteristics of the study population, by participating county, are summarized in Table 1. In each county, the majority of participants were white females; approximately one-third of the participants were African-American. Only trainees who fully completed the evaluation forms were counted as final project participants, which explains in part the difference between the 210 individuals recruited to the study (Table 2) and the 178 reported in Table 1. Additionally, some participant attrition (n = 15) is attributable to Latino trainees who, although receiving real-time English-to-Spanish translation of GPP training, did not have access to Spanish-language versions of the evaluation forms.
Demographic Characteristics of Faith Participants, by County. Numbers may not sum to the total due to information omitted by participants when completing demographic data forms and/or to rounding.
Number of Individuals and Faith-Based Organizations Participating in Planning Workshops, by County
Willingness of Partners to Respond to the Project Concept
Four of the five LHDs (80%) who were formally invited to participate in the project provided affirmative verbal responses about project participation during a conference call arranged by the LHD/AHC coordinator, and they subsequently signed letters of collaborative intent.
Readiness of Partners to Respond to the Project Concept
Representatives or designees of AHCs and LHDs attended all meetings of a partnership steering committee. Monthly steering committee meetings were hosted by AHC and LHD partners on an alternating schedule. A community advisory committee was established in one county, composed of faith leaders who met on a bimonthly schedule with the LHD Emergency Coordinator (author CP), who served as the coordinating nexus between AHC faculty and LHD representatives. A primary function of the advisory committee was to brainstorm and execute ideas for recruiting trainees from the community.
Ability to Respond to Project Concept
Partners from LHDs and FBOs were successful in carrying out the recruitment requirements of the project, as indicated by attendance of a minimum of 10 FBOs and 15 individuals per jurisdiction (counties) at GPP workshops (Table 2). A total of 100 FBOs from four counties (range 11 to 41 per county) participated in this study.
Effectiveness of the GPP Intervention
Program evaluations reflected increased understanding of core content categories of community disaster mental health plans. Table 3 describes the distribution of responses to the items. Participants strongly believed (98% agreed or strongly agreed) that they gained a better understanding of disaster mental health concepts, including better understanding of plan content (eg, incident command structures, key leadership roles, and the importance of partnerships).
Distribution of Participants Reporting a Post-Project “Better Understanding of Knowledge and Skills Required to Create a Disaster Mental Health Plan.” Percentages were calculated using only item-responders as the denominator; participants left some items blank.
Individual participants reported enhanced confidence (perceived self-efficacy) in their ability to execute disaster planning strategies and techniques.
Participant evaluations of the overall program were uniformly favorable. Table 4 describes the distribution of responses to each program satisfaction variable. Depending upon the evaluation item, between 93% and 98% of the participants agreed or strongly agreed that program objectives had been achieved, core planning concepts were learned, and the course was a valuable experience.
Overall Participant Evaluation of GPP Training Program. Percentages were calculated using only item-responders as the denominator.
All LHD partners met the criterion of producing a minimum of three ideas for nurturing their new relationships with the community and for reaching out to new communities. The mean number of suggestions generated was six (Range: 3-8). No effort was made to judge the quality of the ideas, a representative list of which is provided in Table 5.
Representative List of Ideas Generated by Local Health Departments to Sustain Disaster Planning Partnerships with Faith Community Leaders
The investigators have developed, and are in the process of validating, a portfolio of enduring materials to promote portability, dissemination, uptake, and overall translational impact of the GPP model beyond the geographic locations and participating populations in the immediate study. Letters of introduction, orientation brochures, participation agreements, training slides/speakers notes, planning workbooks, and assessment tools and methods are undergoing cycles of revision and iterative refinement to be compatible for direct and online administration.
Discussion
The feasibility, effectiveness, sustainability, and scalability of the model were evaluated.
Feasibility
The success of the LHD partners recruiting church leaders and community members to participate in the project is seen as a vital criterion for “proof of concept,” and is attributed to certain outreach strategies and tactics, the most productive of these were observed to be meeting with ministerial associations; e-mail messages; church-bulletin inserts; community flyers; and word-of-mouth communications following in-person presentations to clergy and lay ministerial leaders. Less effective strategies were: radio spots; postal mailings; and outreach to smaller FBOs (due to difficulty finding phone numbers and addresses).
GPP Effectiveness
Following the first one-day training session, all trainee-participants proved able to develop at least partial drafts of community disaster mental health plans on behalf of their respective faith communities. Missing information in draft plans most often related to two categories of content: (1) names of individuals to serve Incident Command System leadership roles, particularly for smaller parishes that tended to struggle most with the task; and (2) an explicit delineation of the target population for which the plan was being prepared. Both types of missing-information problems were addressed by workshop facilitators in subsequent TA sessions: the first, by emphasizing that one person could serve multiple leadership roles, and the latter, by recommending that participants think realistically about their organization's resources, and then prioritize by vulnerable populations. A separate task in the resource-appraisal process was the identification of asset surpluses that might be available for residents beyond their own parish or immediate geographical area, information considered of great prospective utility to government partners. At the end of the project (42 days after the first GPP training session was conducted), a total of 15 organizations had submitted completed disaster plans on behalf of their organizations and communities. All were generated in one county under the leadership of an especially active LHD emergency planner.
Sustainability
In addition to the targeted sustainability initiatives previously mentioned, other efforts to foster enduring informal and formal faith/government relationships were made by the academic partners from the outset of the project. For example, the figurative and literal faces of LHD leadership were promoted at every opportunity through multiple strategies, including having the emergency planner of each county welcome participants at all workshops; outline the day's training or TA workshop program; host the prepared lunches and snacks; describe the role of the LHD within the county emergency management system; present emergency preparedness kits to participants; and describe the next steps of the program. Complementing this, each participating LHD was encouraged to establish ad hoc, project-focused work groups for trainee recruitment. One community advisory committee has a mission with a multi-jurisdictional scope that is continuing to advance county, cross-county, and regional planning under the leadership of the LHDs. Collectively, the efforts have led to new alliances, both vertical (between FBOs and LHDs) and horizontal (among FBOs), through which mutual aid agreements, formalized by executed memoranda of understanding, document intentions to continue joint-planning efforts.
Scalability
Once GPP training materials are further refined and validated, they will be available for local, regional, and national application through the CDC-supported Johns Hopkins Center for Public Health Preparedness and the Johns Hopkins Preparedness and Emergency Response Research Center via open access to the Johns Hopkins Training Management System (TRAMS, http://www.jhsph.edu/preparedness/training/online/mentalhealth_trainings). Prospective participants will be able to access content on an as-needed basis, or users can deploy trainers to sites where they can facilitate sessions using prerecorded training material and lectures housed in TRAMS. Handouts and other resources also will be made available for use through TRAMS and the Johns Hopkins Open Courseware system (http://ocw.jhsph.edu/Topics.cfm?topic_id=16). Trainers will be able to launch the training, and provide guidance to trainees through a question and answer section, and other potential exercises or activities. The provision of training materials in PDF and CD-ROM formats for in-person workshops, along with reprints of current and prospective articles describing the project, curricula, source of support, etc., will be available for distribution to enhance portability of the program.
Limitations
Ideally, randomizing participants to a control condition in which disaster planning training was not provided would have allowed greater confidence in concluding that GPP alone was responsible for the positive outcomes. A social desirability effect (ie, behaving in a way that one believes others desire, in the absence of intrinsic motivation to do so) or other concurrent events unrelated to the intervention can confound results in one-group studies.Reference Cook and Campbell24 Other considerations, such as gratifying interpersonal transactions associated with the activity, may have contributed to nonspecific, positive assessments of program value. However, the development of content-intensive draft disaster plans as the behavioral outcome measure would seem to mitigate this concern, as the development of such plans requires significant work (ie, requires more than would likely be perceived as “worth it” were the only motivators social desirability, gratifying interactions, or other non-specific rewards).
Because the study sample was comprised exclusively of members of rural Christian denominations, the outcomes may not necessarily be generalizable to community populations living in urban or suburban settings, practicing different faiths, or otherwise possessing relevant characteristics significantly different from those of the study sample, although the researchers have no actual evidence to suggest the results are not generalizable. The observation that the majority of community participants was female raises the question of whether the gender imbalance was unique to the study sample, potentially diminishing the relevance of findings to other faith communities, or whether the gender imbalance reflects more ubiquitous differences in how women and men relate to their religions. In this regard, a recent nationwide survey25 conducted among a representative sample of more than 35,000 adults in the US reveals a multidimensional trend of women being more statistically prominent than men on such factors as affiliation with a religion (86% vs. 79%); certainty of belief in God or a universal spirit (77% vs. 65%); pray at least daily (66% vs. 49%); say religion is very important in their lives (63% vs. 49%); and attend worship services at least weekly (44% vs. 34%). These data suggest that participation in disaster planning on behalf of faith congregations and social communities may be one more example of what seems to be a pervasive difference in the way males and females relate to faith-specific obligations and opportunities, and do not reflect a unique, limiting feature of the study.
Next Steps
The aims of ongoing and future investigational efforts with GPP are to: (1) advance the outcomes logic model by differentiating more clearly the levels and types of impact on the public health emergency preparedness system; (2) validate planning templates, measuring instruments, and outcome metrics; (3) adopt a pre-/post-outcomes measurement schedule; (4) add multiple-choice questions to better assess acquisition of relevant knowledge, skills, and attitude constructs. (5) characterize effective partnership sustaining activities; (6) enhance model replicability by creating a manual of the GPP protocol; and (7) determine what participant-, process-, and context factors are predictors or moderators of successful plan development.
Conclusion
Study findings suggest that appropriate leaders of health departments, faith communities, and academic health centers can work effectively to execute an approach that has the potential for being a practical, effective, and widely applicable model of capacity building at multiple levels in the public mental health emergency planning. Beyond creating behavioral health surge plans for given communities, each completed plan would constitute a modular (micro) component, or community surge annex, to preexisting state government (macro) disaster plans. Additionally, the model supports and enhances Tiers 2, 3, and 4 in the “Medical Surge Capacity and Capability Management System,”Reference Knebel and Trabert26 namely, improving surge capacity and capability: (1) within a county through coalitions (Tier 2); (2) across disciplines in the county Emergency Operation Center (Tier 3); and (3) within a state or geographic region (Tier 4).
Acknowledgments
Important contributions to training or administration of the project were made by the following persons (listed alphabetically): Tavonya Chester, MSW, LGSW; William Dial, MA, CEM; Pat Fosarelli, MD, DMin; Blanche Fuhrman; Pam Jennings, MS, LCPC; Cheryl MacLaughlin, RN, MSN; Pat Rollo; Judy Strong, APRN, MSN; Jerry Truitt; Mary Walker; Dawn Wayman, MS; and Angela Young.
