Introduction
Terrorism and natural catastrophes have made disaster preparedness a critical issue. Children are likely victims in disasters, due to their physiological and psycho-developmental needs, which are different from and more vulnerable than those of adults.Reference Gnauck, Nufer, LaValley, Crandall, Craig and Wilson-Ramirez 1 - Reference Waisman, Aharonson-Daniel, Mor, Amir and Peleg 3 For example, children have a shorter trachea and smaller lungs, which makes their airway harder to secure and maintain against respiratory distress, and children may not be able to verbalize their pain and symptoms, especially younger children.Reference Partridge, Proano and Marcozzi 2 Although susceptibilities to physical and psychological injuries during disasters are higher among children, there are gaps in health care systems regarding pediatric disaster preparedness.Reference Gausche-Hill 4 , 5 In recognizing pediatric issues in disasters, local efforts have led to the development of in-hospital training.Reference Behar, Upperman, Ramirez, Dorey and Nager 6
Preparing for children’s needs in disasters pertains to all in-hospital medical personnel who provide direct patient care, such as physicians, nurses, mental health practitioners, and prehospital workers.Reference Cicero and Baum 7 However, non-medical hospital personnel should likewise be trained to contribute integrative efforts that help protect children from further harm.Reference Cicero and Baum 8 , 9 These personnel include all others working at the hospital since they indirectly affect pediatric care, such as housekeeping service workers, maintenance engineers, and administrative staff. 9
The aim of this research study was to better understand how medical and non-medical hospital personnel interface with online training regarding pediatric disaster preparedness. In this research study, changes in knowledge acquisition of pediatric disaster preparedness were examined within and between medical and non-medical personnel of a children’s hospital who all participated in an online training course. Research and evaluation of training courses on pediatric disaster preparedness are necessary before a nation-wide curriculum can be designed and implemented.Reference Ablah, Tinius and Konda 10
Methods
This research study was part of a larger, federally-funded project. The Institutional Review Board at Children’s Hospital Los Angeles (Los Angeles, California USA) approved the study.
Setting and Participants
The research setting was an academic children’s hospital that serves a diverse, urban population in a large, metropolitan area. In 2008, a multi-disciplinary team within the Pediatric Disaster Resource and Training Center at Children’s Hospital Los Angeles developed an online training course, using the Developing A CUrriculuM (DACUM) methodology (ie, storyboarding to explicate a worker’s duties, tasks, knowledge, skills, tools, and so on). 11 The course is composed of five modules: planning, triage, age-specific care, disaster management, and hospital emergency code response. Specifically, the planning module covers different types of disasters, pediatric vulnerabilities, patient flow, staff identification, response team development, supplies, and language services; the triage module covers different patient acuity levels and the nuances of triaging children during disasters; the age-specific care module covers patient care considerations (eg, nutrition) by group (ie, newborn, infant, toddler, preschool, primary school, or adolescent); the disaster management module covers patient safety, transport, family reunification, infection control, and evacuation; and lastly, the hospital emergency code response module covers the hospital’s disaster policy.
Since 2009, the hospital’s Offices of Human Resources and Medical Staff have mandated hospital-wide completion of this course. All medical personnel (eg, physicians, nurses, and nurse practitioners) and non-medical personnel (eg, housekeeping service workers, maintenance engineers, and administrative staff) may repeat any module for any number of times.
Data Analysis
Archival course data from July 2009 through August 2012 were analyzed in IBM Statistical Package for the Social Sciences (IBM SPSS) Version 23 (IBM Corporation; Armonk, New York USA) using linear growth curve multi-level modeling.Reference Curran and Bauer 12 , Reference Shek and Ma 13 This analytic approach was conducted five times to investigate each of the five modules. In analyzing each module, the outcome variable was module total score (0 to 100 points), the Level 1 (within participants) variable was attempt (any module could be repeated), the Level 2 (between participants) variable was role in the hospital (medical or non-medical), and the cross-level interaction was attempt by role (to test whether medical and non-medical personnel had significantly different average rates of change in module total score per attempt). Compared to repeated measures analysis of variance, growth curve multi-level modeling is a more flexible analytical approach because it does not require units to be uniform (j participants uniformly have i repeated measures), and it uses all available information from the dataset.Reference Curran and Bauer 12 , Reference Shek and Ma 13 There is no ideal sample size for multi-level modeling, but having more units per level is recommended.Reference Gelman and Hill 14
Model Testing
For each module, linear growth curve multi-level modeling was applied through a sequence of models.Reference Curran and Bauer 12 , Reference Shek and Ma 13 This analytical approach began with an unconditional means model as the null model, which contained the fixed and random effects of the intercept (average module total score upon first attempt and the variance of that average) and generated the intra-class correlation (ICC) indicating the amount of outcome variance existing between participants. If the ICC was substantial, then the approach proceeded to an unconditional linear growth model, which added the fixed and random effects of attempt (slope) to test whether module total score significantly changed, on average, across attempts. Next in the approach was a conditional linear growth model, which added the fixed effect of role in the hospital, a time-invariant covariate, to test whether medical and non-medical personnel significantly differed, on average, in how they scored initially and through additional attempts. Role was a time-invariant covariate because participants did not switch roles while attempting the modules, and it was therefore analyzed as a fixed effect only. Last in the approach was a conditional linear growth model with cross-level interaction to test whether medical and non-medical personnel had significantly different slopes (different average rates of change in module total score per attempt). Table 1 summarizes how linear growth curve multi-level modeling was applied in this research study.
Linear Growth Curve Multi-Level Modeling Approach
Abbreviation: ICC, intra-class correlation.
Model Specifications
Since module total score was a continuous outcome, the estimation procedure chosen for each model was maximum likelihood.Reference Gelman and Hill 14 By default, the covariance matrix type for each unconditional means model (null model) was identity. In subsequent models, the covariance matrix type was set to unstructured to most flexibly model the module dataset; the unstructured type allows the variances of the intercept and slope to freely covary.Reference Hayes 15 However, when a model failed to converge, the matrix type was changed to variance components, which is less flexible because it assumes that the variances of the intercept and slope do not covary.Reference Hayes 15
Model Comparisons
To determine the best fitting model for each module dataset, model comparisons were conducted through the X 2 difference test, which compared the deviances (-2 log likelihood values) between models and indicated that the model with a comparatively smaller value showed better fit.Reference Hayes 15
Results
A total of 44,115 module attempts by 5,773 participants (3,686 medical personnel and 2,087 non-medical personnel) were analyzed. As shown in Table 2, the ICC for each module indicated a substantial amount of outcome variance existing between participants.
Participants, Attempts, and ICC Per Module
Abbreviation: ICC, intra-class correlation.
Linear Growth Curve Multi-Level Modeling Results
Table 3 shows the model testing, specifications, and comparisons for all five modules. Table 4 details the final (best fitting) models. The average module total score upon first attempt across all participants ranged from 60.28 to 80.11 points (out of a total of 100 points possible), and participants significantly varied in how they initially scored. The conditional linear growth model showed the best fit for the planning, triage, and age-specific care modules. For the planning module, the results indicated that on average, medical personnel initially scored higher than non-medical personnel (average difference was 16.42 points), but they did not differ in slope (average rate of score change per attempt across all participants was 1.84 points). For the triage module, the results indicated that on average, medical personnel initially scored higher than non-medical personnel (average difference was 13.25 points), but they did not differ in slope (average rate of score change per attempt across all participants was 0.97 points). For the age-specific care module, the results indicated that on average, medical personnel initially scored higher than non-medical personnel (average difference was 15.27 points), but they did not differ in slope (average rate of score change per attempt across all participants was 0.59 points).
Linear Growth Curve Multi-Level Modeling Results
Final Model Details Per Module
a Oriented toward medical personnel.
b P≤.001.
c P<.05.
d P<.01.
The conditional linear growth model with cross-level interaction showed the best fit for the disaster management module. The results indicated that on average, non-medical personnel initially scored lower than medical personnel (average difference was 12.62 points), but they increased per attempt by 3.77 points, while medical personnel did not improve their total scores through additional attempts.
Following the null model in the hospital emergency code response module, convergence was an issue. It was resolved when model testing proceeded without the random effect of attempt (without slope variance), which was found to be non-significant (P=.26) in the unconditional linear growth model with the covariance matrix type set to variance components. The subsequent conditional linear growth model assuming no slope variance showed better fit, but it was surpassed by the conditional linear growth model with cross-level interaction assuming no slope variance as the final model. These results indicated that on average, non-medical personnel initially scored lower than medical personnel (average difference was 10.95 points), but they increased per attempt by 6.40 points, while medical personnel did not improve their total scores through additional attempts.
Discussion
A better understanding of how hospital personnel interface with online training for pediatric disaster preparedness was the aim of this research study. Accordingly, this study examined changes in knowledge acquisition of key content areas (planning, triage, age-specific care, disaster management, and hospital emergency code response) within and between medical and non-medical personnel of Children’s Hospital Los Angeles who all participated in an online training course. The results altogether suggest that medical and non-medical hospital personnel alike can acquire knowledge of pediatric disaster preparedness through an online course, and that key content can be reinforced or improved through successive training.
Specifically, the study found that medical personnel scored higher than non-medical personnel initially and through additional attempts in the planning, triage, and age-specific care modules of the online training course, but they did not significantly differ in their average rates of score change per attempt. The implication from this pattern of findings is that even if non-medical personnel begin the course knowing little about pediatric disaster preparedness, their knowledge acquisition rate can still match that of medical personnel. This implication parallels a previously published study in which disaster response knowledge increased for community citizens and professional emergency responders who all participated in the same disaster drill exercises.Reference Perry 16
Furthermore, the present study also found that scores in the disaster management and hospital emergency code response modules improved through additional attempts by non-medical personnel, but not by medical personnel. This pattern of findings implies that a ceiling effect can emerge when providing training in certain areas of disaster preparedness in which personnel may already be knowledgeable.Reference Perry 16 Regarding disaster management and hospital emergency code response, medical personnel may not necessarily need as much training as non-medical personnel.
Limitations
This study adds to research and evaluation of training courses for pediatric disaster preparedness, which are necessary before a nation-wide curriculum can be designed and implemented.Reference Ablah, Tinius and Konda 10 However, the limitations of this study should be noted. First, knowledge acquisition of pediatric disaster preparedness was assumed to transpire linearly over successive attempts of the online training course modules. Other patterns of knowledge acquisition (non-linear) are possible, which may be examined in future research. Second, knowledge may not necessarily translate into effective behavior. Future research and evaluation should explore not only how medical and non-medical hospital personnel interface with pediatric disaster preparedness training, but also how they apply their acquired knowledge to disaster drills.
Conclusion
This research study found that on average, medical personnel initially scored higher than non-medical personnel in the planning, triage, and age-specific care modules of an online training course for pediatric disaster preparedness, but they did not differ in their rates of score change in these three modules. This study also found that on average, non-medical personnel initially scored lower than medical personnel in the disaster management and hospital emergency code response modules, but they had higher rates of score change in these two modules and their scores improved through additional attempts.
Moreover, medical and non-medical hospital personnel alike can acquire knowledge of pediatric disaster preparedness. Even among personnel whose official roles do not involve providing direct medical care, their work at the hospital still indirectly affects patients. Thus, both medical and non-medical personnel should be trained to contribute integrative efforts that help protect children from further harm during disasters. Acquiring preparatory knowledge is necessarily a critical first step for all hospital personnel. Key content can be reinforced or improved through successive training in an online course.
