Introduction
Building on the lessons learned from the recent Ebola outbreak response in West Africa (2015), the World Health Organization (WHO; Geneva, Switzerland) undertook a process of internal reform to ensure a better and faster reaction to any future humanitarian crises.1 This reform embraces a number of quality improvements in the global health emergency workforce, which has been defined as “unprepared” to operate within the complexity of humanitarian scenarios.Reference Van Hoving, Wallis, Docrat and De Vries2–Reference Nuland4
The challenges of clinical practice in resource-limited environments are abundant.Reference Iserson, Biros and James Holliman5,Reference Birch and Miller6 The increasing presence of young and unexperienced physicians in humanitarian missions, with most of them lacking formal training before deployment to disaster and humanitarian settings, can lead to serious danger to the health of the local populations.Reference Birch and Miller6 In most cases, new providers from developed countries tend to import their home habits and practices, which may lead to further resource depletion in health systems already at the edge of collapse.Reference Iserson, Biros and James Holliman5
In an exploratory study among 200 health providers, Slepski, et alReference Slepski7 found that an “abrupt change or transition from their everyday practice worlds” was often mentioned as one of the main issues when carrying out humanitarian work. Indeed, the need for prior training and a period of acclimatization to understand the new physical and social environment, as well as to adapt one’s competencies to the needs and values of local populations, is well-recognised.Reference Iserson, Biros and James Holliman5,Reference Slepski7,Reference Ripoll Gallardo, Ingrassia and Ragazzoni8
Until now, studies reporting on the preparedness of health providers in low-resource settings remain predominantly narrative, anecdotal, or published as grey literature.Reference Slepski7 While it is conceivable that professionals may display a poorer level of performance when working outside their comfort zone, scientific evidence supporting this assertion is lacking. As the availability of quantitative data may contribute to underline the importance of proper training before deployment, the primary objective of this pilot study was to compare medical residents’ overall performance in High- versus Low-Resource Scenarios (HRS vs LRS, respectively). The secondary objective was to compare auxiliary management skills, such as leadership, in both scenarios. It was hypothesized that residents’ overall performance and all crisis resource management (CRM) skills would decrease in LRS when compared to HRS.
Materials and Methods
This manuscript adheres to the applicable TRENDS guidelines.
Ethical Clearance
This study was approved by the Institutional Review Board “Comitato Etico Interaziendale Azienda Ospedaliera Maggiore della Carità di Novara” (date: February 24, 2016; study code: UPO.2015.4.10), and written informed consent was obtained from all subjects.
Participants
Ten senior residents (fifth-year post-graduate) of the Maggiore Hospital School of Medicine (Novara, NO, Italy) were included in the study; all participants were Italian. Residents had no prior simulation experience, had never received training in CRM, and had no previous experience in humanitarian missions.
Study Design
This study used a randomized crossover experimental design. Every resident acted as the lead physician during the management of two identical postpartum hemorrhages from uterus atony in an HRS and an LRS (Table 1). The study was conducted over two days during April 2017, and residents were randomly allocated into two intervention groups using assignment envelops; on the first day, five residents were exposed to HRS and then to LRS, while on the second day, the remaining five residents began first with LRS and followed with HRS.
Description of the Context, Facilities, and Resources Available in HRS and LRS.
Abbreviations: HRS, High-Resource Scenario; LRS, Low-Resource Scenario; OT, operating theater.
a Virtual facilities.
In order to ensure that any lack of basic knowledge in managing emergency cases was not the cause of a difference in residents’ performance, participants were asked to study the following learning material two weeks before the study was conducted: Advanced Life Support (ALS), Obstetric Emergencies (OE), and Advanced Trauma Life Support (ATLS). In addition, prior to entering the scenarios, all residents participated in a tutorial session that included a seminar on ALS, OE, and ATLS topics, as well as a simulation tutorial where the functioning of the mannequin and the simulator room was presented. Participants were allowed a 15-minute familiarization with the final setting (without the patient) of both scenarios.
Simulation Environment
The equipment, drugs, and personnel in HRS mirrored the operating theater (OT) of the researchers’ tertiary teaching hospital; LRS reflected and average OT in a developing country and was equipped according to the basic international standards of anesthesia.Reference Merry, Cooper, Soyannwo, Wilson and Eichhorn9 In LRS, the number and level of competency of national staff was based on authors’ personal experience in field missions. Actors complied with the behavior and dress code of the country where the scenario was based, for instance, speaking poor Italian.
The mannequin NOELLE (S550 Maternal Simulator; Gaumard Scientific; Miami, Florida USA) was used in both cases and was operated by a simulator technician. All actors were trained to standardize their performances and give pre-established responses to resident’s actions or inactions, or in case of failure to recognize key events. The coordination of actors to standardize responses was performed by another simulation instructor through a handheld radio (Midland G9 PMR446; Midland Radio Corp.; Kansas City, Missouri USA). The progression of all cases was observed and directed from an adjacent control room through a two-way mirror.
Since postpartum hemorrhage remains the leading cause of pregnancy-related mortality,Reference AbouZahr10 and the rate of maternal death is disproportionately high in resource-constrained environments,Reference Prata, Passano, Rowen, Bell, Walsh and Potts11 this condition was deemed worthy of special attention on the basis of the authors’ cumulative experience. The scenario was based on a real-life case, and the sequence of events and cues were reviewed for realism and timing by two expert anesthetists with valuable experience in humanitarian missions.
Both scenarios were played for a maximum of 20 minutes, and before each, residents received a situational report with a detailed explanation of the context (data not shown but available upon request). In both cases, the scenario started with a phone call reporting the presence of an emergency in OT; initially, clinical events in both scenarios were identical and progressed on a pre-defined fashion according to the cue system described by Kim, et al.Reference Kim, Neilipovitz, Cardinal, Chiu and Clinch12 This ensured residents’ exposure to all critical events in both scenarios.
Evaluation Tool
Since CRM skills are paramount to patient safety in anaesthesia,Reference Howard, Gaba, Fish, Yang and Sarnquist13 and their evaluation reflects professionals’ performance during emergencies,Reference Kim, Neilipovitz, Cardinal, Chiu and Clinch12 the Italian translation (TIGR)Reference Franc, Verde, Gallardo, Carenzo and Ingrassia14 of the Ottawa Global Rating Scale (Ottawa GRS) was used.Reference Kim, Neilipovitz, Cardinal, Chiu and Clinch12 Residents’ performances were videotaped and then rated by an external and independent evaluator. To ensure anonymity and confidentiality during the entire process, all students were assigned a tracking number that was reported on all evaluation forms and videotape labels.
Statistical Analysis
This study was a two-treatment crossover design (AB, BA) with possible carryover effect. As previous studies have shown no difference in Type I and Type II error rates when five-point Likert scales are analyzed using the t-test versus Mann-Whitney-Wilcoxen test, and the TIGR scale uses a 7-point semi-anchored design, the decision was made to analyze the ratings as continuous variables. The primary objective - difference in “overall performance” between the HRS and LRS groups - was evaluated by Student’s t-test. For the secondary objective - comparison of the five specific scoring fields - Student’s t-test was again used. However, as the analysis involves five tests, P values were adjusted by the Schaffer-Holm stepwise procedure to control for the family-wise error rate. As a significant learning (carryover) effect was possible after each participants’ first simulation, a linear mixed effect model was to be fitted in the event that the direction of effect was different between the participants who completed the LRS first and those who completed the HRS first. This would allow separation of this carryover effect from the true difference between the two scenarios. P values of less than .05 were considered significant for all statistical tests.
Results
Among the ten participants, six were females and four were males. The median age was 31 years. The hypothesis was confirmed for both objectives of this study. Overall performance score for the HRS simulations was 5.2, while overall performance score was 2.3 for LRS (Figure 1). The mean effect size for the overall score was 2.9 (95% CI, 1.7–4.0; P <.001). A significant effect was also found for all fields of the TIGR in LRS when compared to HRS (Table 2). As the effect was the same direction for all participants, separate analysis of the carryover effect was not performed.
Box-Plot Difference in Residents’ Overall Technical (Medical) Performance between High-Resource (High) and Low-Resource (Low) Environments.
Difference in Residents’ Scores Across the Different Fields of the TIGR (Non-Technical Skills)
Abbreviations: HRS, High-Resource Scenario; LRS, Low-Resource Scenario; TIGR, Italian translation of the Ottawa Global Rating Scale.
Discussion
This is the first article comparing young doctors’ performances in an HRS versus an LRS setting. These results show an overall decrease in residents’ performances when exposed to an unfamiliar environment despite an exhaustive preparatory process. While a decrease in technical skill is to be expected, the reduction in non-technical skills (eg, leadership, problem solving, or resource utilization) is a finding that deserves special attention. This study suggests that untrained physicians in low-resources environments may experience a considerable setback to their interpersonal skills when deployed in humanitarian missions, which may endanger the health of local populations. Accordingly, these findings draw attention to the compelling need for humanitarian workers to gain new competencies before deployment in low-resource settings, which has already been highlighted by previous studies.Reference Burkle3,Reference Ripoll Gallardo, Ingrassia and Ragazzoni8 The WHO affirmed:
It is vitally important that people deployed to work on an emergency response have training and skills at the appropriate level and of the necessary quality (…). It is dangerous to individuals and to the collective response work to deploy people with inadequate experience or a lack of expertise with the hazard involved, without training and mentorship.1
Yet, typical training programs do not prepare professionals for the adversities in developing countries.Reference Iserson, Biros and James Holliman5,Reference Birch and Miller6 It is the authors’ hope that this study will send a warning about the need of additional preparedness for inexperienced staff before first deployment.
Limitations
This study has several limitations. First, due to the limited availability of staff, only ten Italian residents could be included. Accordingly, this should be regarded as a pilot study and the results cannot be further generalized. Secondly, this study was conducted in a simulated, rather than actual, working environment. Third, this study included the opinion of only one evaluator.
Conclusions
This pilot study suggests that untrained physicians in low-resources environments may experience a considerable setback not only to their professional performance, but also to their interpersonal skills, when deployed ill-prepared to humanitarian missions. Consequently, this may endanger the health of local populations.
Acknowledgements
The authors thank all actors involved in the simulated scenarios: Ilaria Sguazzotti, MD; Miriam Lamboglia, MD; and Elisabetta Vaianella and Virginia Deregibus, midwives. The authors also thank Pier Luigi Ingrassia, MD-PhD and director of the simulation center SIMNOVA of the Università del Piemonte Orientale, for lending the simulation facilities. This work is the result of a thesis submitted in partial fulfilment of the requirements for the degree of Master in Science of Disaster Medicine (European Master in Disaster Medicine).
Conflicts of interest
none
