Public health agencies play a key role in responding to emergencies and disasters, and these services are critical to ensuring the public's health in the aftermath of a disaster. Since September 11th, more than $6 billion has been spent to support state and local agencies in preparing for disasters, outbreaks, and public health threats.Reference Nuzzo, Mair and Franco1 Traditionally, preparedness efforts have focused on assembling emergency medical teams and first responders to improve the coordination of governmental agencies in emergency response. Equally important, but often overlooked in disasters, is the provision of key public health services.
Business continuity planning, or continuity of operations planning, involves plans to mitigate risks, reduce the impact of a disaster, and return an organization to normal operations after a disaster through risk identification, plan development, and employee training.Reference Cerullo and Cerullo2 Maintaining daily functions (preparedness or business continuity) differs from an emergency response. The focus in business continuity is to maintain normal services rather than manage emergency services. Experience has shown that up-to-date information regarding available workforce resources in a physical disaster is needed to ensure uninterrupted public health services.Reference Rebmann, Carrico and English3 During emergencies and disasters, public health decision-makers need to sustain the highest level of essential public health services including primary care, family planning, and communicable disease investigations. Continuing essential public health functions during a disaster, such as supporting temporary public health clinics, requires matching public health personnel to services and facilities under dynamic conditions.
Low-cost, easy-to-use information systems to inventory and manage public health personnel and resources under changing conditions are lacking. Also, few tools are available to assist public health professionals with business continuity and decision making during dynamic conditions such as disasters.Reference Reeder, Turner and Demiris4 To our knowledge, this project is the first technology design research project that aims to specifically support public health business continuity efforts.
In 2008, faculty at the University of Washington Center for Public Health Informatics received a request from the Community Health Services (CHS) Division, Public Health - Seattle & King County (PHSKC), for informatics assistance in developing a method of visualizing the location of clinic staff and providers in relation to existing public health facilities using the PHSKC service delivery plan, whereby the existing health clinics would be collapsed into 4 major clinic areas in the aftermath of a disaster. This report describes the development of the Continuity of Operations Data Analysis (CODA) pilot system, which was designed to assist local health agencies to maintain essential services during emergencies.
Methods
System Design
Setting
PHSKC is a large, municipal-county health agency that serves the residents of Seattle and surrounding King County, Washington. It employs more than 1100 fulltime employees and has an annual budget of nearly $210 million. During normal health department operations, PHSKC's CHS manages 10 public health centers. The proposed emergency plan for maintaining clinical services in the event of a disaster is to establish service delivery areas (SDAs) that will serve as “islands” of self-sufficiency within King County.
Business Continuity Plan
In the event of a disaster or pandemic, the 18 public health centers will be “collapsed” into fewer SDAs, depending on pandemic severity and staff-reduction levels. The SDAs will be managed from a fourth administrative site. After a disaster, the number of SDAs will be determined by the number of functioning sites reported by on-site staff.
A list of services has been created and prioritized; the services available at an SDA will depend on the job class and number of staff available. During an emergency, on-site staff report to SDA managers who have a situation assessment worksheet that includes a site attendance roster. These rosters are then “rolled up” into an all-sites assessment, which is reported from Business Continuity to the Emergency Operations Center. The incident commander of the center determines the allocation of overall county resources and reports this information to the Business Continuity managers. The managers then coordinate staff and supply logistics among public health sites within the various SDAs.
Design Principles
The following design principles were used to develop CODA:
• Participatory design and validation: Information needs were based on on-site interviews with key informants. Scenario-based designReference Carroll5 methodology was used to create scenarios of use from validated information needs. A formative evaluation of implemented scenarios was conducted using a think-aloud protocol.Reference Jaspers, Steen, van den Bos and Geenen6, Reference George7
• Freely available open-source software: CODA is built entirely from freely available software including an Apache Tomcat,8 Java9 servlet container, a MySQL database,10 and a web-based interface compatible with all major web browsers.
• High-performance server and client-side software: CODA uses Java servlets along with asynchronous JavaScript and XML (AJAX) to provide a highly responsive and interactive user experience by avoiding round trips between the web browser client and the server.
Interviews
The CODA system design is based on information system requirements identified through interviewsReference Blomberg, Giacomi, Mosher and Swenton-Wall11, Reference Ulin, Robinson and Tolley12 and meetings with business continuity plannersReference Reeder and Turner13, Reference Reeder and Demiris14 and PHSKC senior managers who manage multiple clinics (area managers). This study was approved by the University of Washington Institutional Review Board, and all participants gave their informed consent to participate in the study. Employee identifiers used in this study are simulated to protect the privacy and confidentiality of PHSKC employees.
We conducted 9 semistructured interviews with senior public health managers, including former and current area managers. The 5 current area managers represented the entire population that was employed in this position at the time of the study. Interview questions focused on communications, staff scheduling, and decision making during routine and disaster operations. Interviews were informed by the critical incident of a recent and severe December 2009 snowstorm that shut down public health operations in the metropolitan and county area.
Interviews were transcribed and analyzed for information needs using Atlas.ti software.15 Scenarios of use were based on the results of the assessment of information needs.Reference Carroll5 Information needs and scenarios were checked by all 5 current area managers and adjusted according to feedback. Selected scenarios of use were implemented in the CODA prototype.
System Description
Requirements
Area managers identified the following system components as essential for managing personnel during emergencies and disasters:
• Maintain access to individual staff member information regarding job class, job skills, home address, assigned worksite, and contact information;
• Evaluate prioritized services in the context of available staff skills and resources;
• Show which services can be delivered based on available staff and skill sets;
• Visualize location of individual staff members in relationship to functioning public health centers;
• Provide a low-cost, easy-to-operate system that can be used under disaster conditions.
Figure 1 shows the overall CODA architecture.
The Continuity of Operations Data Analysis (CODA) System. CODA comprises freely available software, including Apache Tomcat, Java servlet container, MySQL database, and a web-based interface compatible with all major web browsers.
Resource Integration
The CODA database connects essential clinic functions to job category definitions for 821 PHSKC employees in 14 clinics. Data downloaded from the PHSKC workforce human resource database and the King County jobs database link job category and clinic assignment with employee zip code and clinic location. Clinics are further grouped into SDAs for use by managers who oversee multiple clinics. The King County employee database links PHSKC job categories to essential skills, duties, licensing, and salary information to provide a comprehensive set of employee data for allocating resources. The CODA screenshot in Figure 2 shows a Google Maps16 overlay of the location of PHSKC employees by zip code and the location of 3 PHSKC clinics in the northern SDA. The clinics and the number of employees living within each ZIP code are identified.
The Continuity of Operations Data Analysis (CODA) System Locates Employees and Clinics in the Northern Service Delivery Areas (SDAs). A Google Maps overlay of the CODA system locates Community Health Services Division, Public Health - Seattle & King County (PHSKC) employees and the 3 PHSKC clinics in the northern SDA by zip code.
Resource Evaluation
CODA allows a user to drill down into the underlying employee database to evaluate resources. The screenshot in Figure 3 shows the result of querying the CODA system for those employees in the northern SDA whose essential duties include primary care operations. Employees are identified by name, zip code, city, job title, assigned clinic, SDA, and essential duties. Links are provided for additional detail of employee job descriptions.
The Continuity of Operations Data Analysis (CODA) system locates employees in the northern Service Delivery Areas (SDAs) who perform primary care (PC) operations. Querying the CODA system for employees in the northern SDA whose essential duties include PC operations shows employees identified by name, zip code, city, job title, assigned clinic, SDA, and essential duties.
The CODA database specifies the number of staff required to provide an essential service and the number of staff available within an SDA or individual clinic who are qualified to perform an essential service or services. CODA can also display output from queries about the number of staff required to provide primary care services and the number of primary care staff available in an SDA. This functionality allows managers to reallocate or combine resources or suspend operations because the staff required to provide a particular service are not available.
System Validation
We conducted a preliminary formative evaluation with 2 of the 5 area managers using a think-aloud protocolReference Jaspers, Steen, van den Bos and Geenen6, Reference George7 to validate the CODA prototype design. A think-aloud protocol is a method of usability testing in which users “think aloud” as they perform specific tasks related to the system being tested. In 2 separate sessions, additional feedback was gathered during formal demonstrations of CODA to senior administrators and program staff.
Think-Aloud Sessions
Each session was conducted by 2 researchers with 1 area manager as the participant. Each session lasted about 60 minutes and was recorded using a digital audio recorder. Step-by-step instructions for participants were written by mapping a navigation path through the CODA prototype to achieve the goal of each selected scenario of use. Sessions began with a training session to explain the features of the CODA prototype and how the interface worked. The participants became familiar with the interface and asked questions about the CODA system. The training period ended when the participants were ready to use the system. All participants were given instructions for the same 3 scenarios of use and asked to verify that the steps taken through CODA satisfied the goal for each. Participants were encouraged to comment aloud about interactions with the prototype during the session. Recordings were transcribed and analyzed for design feedback.
Validation Through Demonstration Sessions
Two demonstration sessions were held with senior administrators in emergency preparedness and administrative services as well as clinical services to introduce the system and gain feedback on its perceived usefulness and usability. A total of 9 staff members participated in these sessions. Feedback was gathered through note taking by one of the researchers attending the demonstration session.
Results
Resources and Costs
CODA was developed on a laptop computer system by a part-time programmer working a few hours a week. Estimated cost of the pilot system was less than $1000. The technologies used during the pilot are cost effective in comparison to commercial operating systems, database servers, and geographic information systems that are beyond the budget of many public health agencies. The database itself, along with the open-source Java and web-server software, is of modest size. These technologies can be installed and maintained on an inexpensive laptop computer. Much of the power of the CODA system stems from its use of the freely available Apache high-performance web-server software in conjunction with MySQL database software. Likewise, the use of AJAX to interface with Google Maps harnesses the power of the local web browser and avoids round trips between client and server.
Perceived Usefulness
Both area managers verified features of the CODA system implemented from the scenarios and provided feedback about their perceived usefulness. Both participants verified that the features were properly implemented to meet information needs and were useful. In addition, managers in Emergency Preparedness and Administrative Services provided positive feedback about the potential usefulness of the system to meet their business continuity needs. One area manager offered the following feedback:
This kind of a system is really helpful because it standardizes some things, it streamlines communication pathways…. It gives you backup in an emergency and it automates some things that we have to do by hand that will make you crazy.
A common observation of first-time CODA users who view employee location using the Google Maps interface is the widespread geographical distribution of King County personnel (Figure 4). Many PHSKC employees work in the city of Seattle but live far outside of King County, which has obvious impact for business continuity planning.
Geographical distribution of King County personnel. The Google Maps interface shows the widespread distribution of employees in relationship to the Green River flood zone (shaded area). Community Health Services Division, Public Health - Seattle & King County.
Figure 4 illustrates the usefulness of CODA for business continuity planning under a variety of emergency scenarios. It shows the Green River flood zone (shaded area), an area recently endangered by potential failure of the Howard Hansen dam on the river 50 miles west of Seattle. Interviews with King County disaster management personnel revealed that, in addition to closing South King County Public Health offices, the major problem with the flood zone is re-assigning personnel living east of Seattle owing to the disruption of traffic along Interstate 5, the main Seattle corridor.
Usability Issues
Both area managers observed that de-identified test data for employees and clinics were out-of-date and that organizational changes due to budget issues had been made since the CODA prototype was developed. In addition, both had difficulties with how the Google Maps interface dynamically resized maps when employee home zip code data extended map boundaries beyond the viewable area of the computer screen.
Suggested Features
During the think-aloud sessions, area managers suggested that contact information and home address be available from the employee list. They also asked about the possibility of a geographically visualized link to web-based cameras to assess facilities during disasters. They expressed a desire for an integrated view of current attendance data from other information systems to manage staff rosters during normal or emergency operations. In addition, 1 area manager requested filtering the data by job position. The following activities were suggested as additional features CODA might support and alternative ways the system might be used:
• To assist a person filling in during normal or emergency operations for an area manager who did not have the area managers’ expert knowledge;
• To offer an easily available view of interpreters by language and location to manage interpretation capacity during normal or emergency operations;
• To support strategic planning for new programs or grant applications during normal operations.
Discussion
Challenges and Innovations
The functions of local public health emergency operation centers during disasters are coordination, communication, resource management, and information collaboration, analysis, and dissemination.Reference Burkle17 State and local health agencies must fully understand whether resources are in place to support operational capacity and to enable them to continue services and work in conjunction with other health care organizations during emergencies.Reference Burkle17 CODA is a novel system built with open-source technologies and free, highly responsive, and interactive web 2.0 technologies, such as Google Maps and AJAX, to merge workforce and location information from multiple data sources. These technologies offer new opportunities for public health agencies to quickly evaluate and allocate resources by leveraging visualization abilities previously available only in commercial geographic information systems.Reference Yi, Hoskins and Hillringhouse18, Reference Scotch, Yip and Cheung19
CODA is presented as a “proof of concept” and is not intended as a commercial product, although further expansion is possible, depending on additional funding. The CODA system was developed to demonstrate how to build a flexible software system using freely available software on inexpensive computer hardware to fulfill an unmet need in a large local public health agency. CODA or similar systems could be constructed using information technology (IT) resources in local public health agencies. The CODA system was developed using a basic laptop computer at minimal cost ($1000). An experienced IT employee could initially construct, maintain, operate, and update the databases as a pilot project. Once the CODA system is installed and tested, fewer and less-expensive resources should be required for maintenance. Alternatives to CODA are commercial geographic information systems software that can carry licensing and maintenance costs of tens of thousands of dollars and require the use of trained software engineers and systems administrators.
The CODA design and implementation pilot attempted to address the identified needs of actual public health managers in small iterations. These efforts required fewer software and database resources due to a more focused effort. CODA is simple and flexible enough to interface with existing geographic information systems, but this would require customization to accommodate diverse data sources. CODA currently requires manually importing data from diverse systems. A public health agency that wanted to customize the CODA system to fulfill local needs and interface with their own custom IT systems would need to identify local resources for the effort.
CODA software and database system components were selected for the ability to securely update underlying databases. The CODA pilot system was initially loaded and manually updated using a tab-delimited text file. Automated updating of CODA databases will require further development to integrate CODA with local IT resources. Regular, secure downloads from different databases for employee and clinic information will be necessary when deployed in a production environment.
CODA's Google Maps visualization requires a connection to the Internet; CODA does not currently support a stand-alone visualization tool such as Microsoft MapPoint. The CODA web interface functions with or without the Google Maps visualization component. In the event of Internet connectivity interruption, it continues to allow users to evaluate staffing requirements and resources until connectivity is resumed.
Based on user feedback, we would recommend for strategic planning for new programs or grant applications that personnel requirements including job qualifications and costs be linked using information from CODA's PHSKC workforce database. In addition, CODA could show a table with an employee's experience and certifications that indicate his or her ability to perform a particular public health function. Moreover, an individual employee could personally update information about skill set and availability to maintain a current database.
One of the challenges for the CODA database-backed web browser system is maintaining updated employee and clinic information. The health department payroll database was used because it provided the most frequently updated data on individual workers’ home location and job classification. While not publically available, payroll databases are an accurate source of workforce data, as noted in a recent report.20 Not adopting standardized job classification systems greatly interferes with integration of workforce data across regions,whereas their adoption would greatly facilitate local and national public health workforce data integration and aggregation efforts.
Conclusions
Local health agencies are seeking information tools that will assist with workforce decision making in times of disaster. Based on input from local health agency personnel, we developed a low-cost, easily portable, visualization tool to assist with matching personnel, clinics, and services under dynamic conditions. Initial user feedback suggests that CODA can help public health area managers determine the availability of staff required to deliver services during routine and emergency operations.
This study highlights the benefits of linking practitioners to information system design and soliciting their feedback as part of an iterative design process. Our preliminary validation of the CODA system indicates the value of iteratively describing and validating needs and designs through interviews and evaluating the prototype in a public health context. More extensive evaluation with a larger number of end users will be required as the system is modified.
Public health practitioners work in resource-constrained environments and have little time to spare for efforts that do not produce tangible results. A results-oriented approach is important because it creates a design conversation between informaticians and public health practitioners. Our validation efforts raised design issues and generated new features by making the public health professionals partners in the efforts to support their work through new technology.
Future system developments will include enhancements to integrate environmental data such as traffic barriers, potential flood plains, and infection outbreaks. In addition, we plan to expand the continuity of operations needs assessment to other health department divisions such as Maternal and Child Health and Environmental Health. Although CODA is a stand-alone system, it can also function as a web server for other computers. Recent developments in network coverage through cellular telephones would allow a CODA laptop to serve as a wireless hub for other CODA clients.
Funding and Support
This work was supported in part by the Centers for Disease Control and Prevention grant No. 5P01TP000297. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention. Additional support was provided by the National Library of Medicine Medical Informatics Training Grant T15LM007442 and National Institute of Nursing Research Training Grant T32NR007106 (Dr. Reeder).
Acknowledgments
The employees of Community Health Services, Public Health - Seattle & King County, provided input and feedback in the development of the CODA system.
