For several decades, there has been a growing interest in teaching disaster medicine—a practical and comprehensive discipline—to medical students. Reference Wunderlich, Ragazzoni and Ingrassia1-Reference Wiesner, Kappler, Shuster, DeLuca, Ott and Glasser3 This interest in disaster management is understandable given the frequency and impact of natural and man-made disasters. Reference Afzali and Viggers4-Reference Hai6 A key point of difficulties associated with teaching medical students disaster medicine is how to effectively care for patient surge at once. Reference Chan, Mercuri and Van Dewark7 Therefore, schools are increasingly trying to determine the best method with which to teach medical students how to simultaneously manage patient surge in a disaster scenario with limited resources. Reference Tsoy, Sneath and Rempel8,Reference Kobayashi, Shapiro, Gutman and Jay9 In response to this growing interest, we established an optional course—‘‘Multi-patient Management in Disaster Medicine’’—at Sichuan University for fourth-year medical students who have completed almost all other clinical courses.
Traditionally, disaster medicine has been taught using lectures and/or simulation exercises. Reference Sarin, Cattamanchi, Alqahtani, Aljohani, Keim and Ciottone10 Although lectures require less funding and fewer teaching resources than simulations, they are often more boring and abstract; hence, lectures may not be the best way to teach disaster medicine. Although simulation exercises may be a good method with which to help medical students master the knowledge and skills of disaster medicine, they require considerable funding and a great amount of teaching resources (such as longer instructional time periods, more teachers or instructors, considerable preparation, and the need for more consumables). Reference Curtis, Trang, Chason and Biddinger11 Therefore, like lectures, simulation exercises may not be an ideal way to teach disaster medicine to large classes of medical students.
Recently, game-based learning has received increased attention as an interactive teaching method. Reference Akl, Pretorius and Sackett12 To date, some immersive game-based computerized applications based on virtual reality (VR) technologies have been applied in disaster medicine education; previous studies validate the efficacy of these applications. Reference Knight, Carley and Tregunna13,Reference Mohan, Fischhoff and Angus14 When they work with a curriculum, these immersive applications can facilitate easier and more enjoyable learner-centered educational experiences. However, the VR equipment required for such pedagogies renders the practicality and usability of immersive game-based computer applications in the classroom disputable. Despite that, Menin, et al. Reference Menin, Torchelsen and Nedel15 argue that such an application might make this education accessible to a much larger user population while facilitating a more comfortable learning experience. While Menin, et al. get the ball rolling in with these findings, minimal research has been done on the effectiveness of game-based pedagogies for teaching disaster medicine. This study responds to this gap in the scholarly archives by exploring the effectiveness of game-based pedagogies for teaching medical students disaster management in comparison to the traditional lecture format. More specifically, after teaching medical students a patient surge management course using a lecture format in 2017, we designed a game-based computer application for teaching the topic and applied it to our course in 2018 (Figure 1). Reference Zihan16 We then compared the test scores and feedback results of each iteration of the course to explore the effectiveness of the application for teaching patient surge management.
Game-Based Learning in Sichuan University.
METHODS
Participants
After approval by the ethics review board, the study was carried out in the West China School of Medicine at Sichuan University. In 2017, before our application was complete, we taught our disaster medicine course using a lecture format. The class consisted of 63 third-grade medical students who voluntarily joined this course and were termed ‘the lecture group.’ Then, when our application was complete and ready for use in 2018, 68 third-grade medical students voluntarily enrolled in the course—we termed these students ‘the game group.’
Research Tools
Pre-Test, Post-Test, and Final Test
All medical students were tested on 3 separate occasions: (1) a ‘pre-test’ before the educational intervention, (2) a ‘post-test’ following the intervention, and (3) a ‘final test’ at the end of the term (6 weeks after the intervention). Each test consisted of 20 multiple-choice questions (0.5 points per item) that were designed to assess student mastery of multi-patient disaster medicine. In order to ensure comparability among the 3 tests, 7 questions (out of 20) were the same, 7 questions were identical in content but were re-described, and another 6 questions were totally different while maintaining a similar level of difficulty.
Questionnaire
In order to investigate medical students’ opinion of the lecture and the application during the class, a survey consisting of 5 items was conducted at the end of the course:
-
1) I was motivated to learn in this course.
-
2) I gained much knowledge from this class.
-
3) I liked the teaching method in this course.
-
4) The disaster medicine course was interesting.
-
5) All in all, I enjoyed this course.
Students were asked to rate each item using a 5-point likert scale in which “1” meant “strongly disagree,” and “5” meant “strongly agree.”
The Educational Game
For our purpose, it is helpful to note that we developed our own application because no digital games on disaster medicine existed at the time of the study. In a previous study, Reference Zihan16 we used the application as a teaching instrument. In this study, all medical students were divided into study teams of approximately 10 members who participated in the game as 1 unit (i.e., 1 player). The instructors offered each team a set of study materials including 39 mass casualty patient cases. The game itself was turn-based and involved a number of missions that each group needed to complete, including (1) triaging all patients, (2) providing initial medical care, (3) determining the order of transit to the hospital, (4) arranging hospital beds, and (5) rationally allocating hospital resources (including the appropriate assignment of patients to the operating room, intensive care unit, and surgical ward). In each round, students would engage in team discussions to choose 1 suitable patient to triage, treat, refer, etc. The player’s dissimilar operations in the game influenced the patient’s outcome (i.e., relieved, aggravated, or dead), which the game’s settings automatically judged. After each round, the game would calculate the number of patients who experienced each different outcome and present the rates on its interface with different colors for each outcome (Figure 2). The teams would then go to the podium to summarize the round and share the lessons they learned by completing it. Ultimately, the team that treated the most patients and had the fewest dead patients won the game, an achievement the teacher praised.
The Serious Game Interface.
Procedure
The study was conducted in 2017 (the lecture group) and 2018 (the game group) over 6 classes of 1 hour each. This section outlines their nuances. In the lecture group, before each class, students were provided with a series of references, including those related to triage, mass casualty management, the evaluation and treatment of trauma patients, and surge capacity and capability. Students were required to preview this content before each class. Students completed a pre-test before the lecture to investigate prior knowledge, a post-test immediately after the lecture, and a final test at the end of the term (6 weeks after the course ended). The game group was given a lesson on general principles through a 1-hour lecture and then received the same reference materials as the lecture group, which they were also required to preview before each class. Subsequently, 4 class hours were taught with the application. The last class hour was used for a course summary and questions and answers. The game group completed the same pre-test, post-test, and final test as the lecture group. At the end of the final test, a brief survey on the student’s opinion of teaching methods was distributed and completed (Figure 3).
General Overview of Study Procedure.
Data Analysis
All data were listed using the SPSS 20.0 software (IBM Corp, Armonk, NY) for statistical analysis. Only students who participated in all 3 tests were included in the analysis. Of the students, 2 were absent in the follow-up 6 weeks later and were therefore excluded from the analysis. Measured data with normal distribution were described by mean ± standard deviation. Measured data with non-normal distribution were described by medians [25% quartiles, 75% quartiles]. Counting data were constructed using composition ratios. The pre-, post-, and final test scores of the groups were compared using ANOVA for repeated measurements. The trend line of each group with indicators at different time points was plotted. The difference was considered to be statistically significant at an alpha level of p < 0.05.
RESULTS
Of the 131 medical students were enrolled in the study, 79 were female (60.3%) and the mean age was 23.4 ± 1.1 years. Table 1 lists the baseline of both groups.
Baseline of Both the Game Group and the Lecture Group
There was no significant difference between the game group and the lecture group in baseline level, including gender, age, and pre-test.
There was no significant difference between the game group and the lecture group in baseline level characteristics, including gender, age, and pre-test results. Knowledge scores were submitted to a 2x3 mixed design ANOVA in which ‘group’ served as the between—subjects variable, and ‘time of measurement’ (i.e., before, immediately after, and a week after) served as the within-subjects variable. Mauchly’s test indicated that the assumption of sphericity had been violated (χ2 = 120.99, p = 0.000). In response, degrees of freedom were corrected using Greenhouse-Geisser estimates of sphericity (ϵ = 0.621). Statistically significant results revealed a main effect of time of measurement (F = 199.49, p = 0.000) and an interaction between group and time of measurement (F = 3.24, p = 0.041). To investigate the interaction, we analyzed the main effects by separately testing the effects of time of measurement for each group and the effects of each group at each time of measurement. Pairwise comparisons using Bonferroni tests and a series of 1-way ANOVAs with repeated measures were performed to separately test the effects of time of measurement for each group.
In the lecture group (Figure 4), the mean pre-test knowledge score was 4.4 ± 1.8, the mean post-test score was 7.8 ± 1.2, and the mean final test score was 6.2 ± 1.5. Repeated measures of ANOVA with a Greenhouse-Geisser correction (Mauchly’s test χ Reference Pfenninger, Domres, Stahl, Bauer, Houser and Himmelseher2 = 53.4, p = 0.000) revealed a statistically significant difference (F = 100.134, p = 0.000). All 3 pairwise comparisons with Bonferroni tests were also significant (Table 2). Meanwhile, in the game-based computer application group (Figure 4), the mean pre-test knowledge score was 4.5 ± 1.8, the mean post-test score was 7.6 ± 1.2, and the mean final test score was 6.8 ± 1.3. Repeated measures of ANOVA with Greenhouse-Geisser correction (Mauchly’s test χ Reference Pfenninger, Domres, Stahl, Bauer, Houser and Himmelseher2 = 87.6, p = 0.000) revealed a statistically significant difference (F = 105.1, p = 0.000). All 3 pairwise comparisons with Bonferroni tests were also significant (Table 2).
Score Curves of the Game Group and the Lecture Group During Pre-Test, Post-Test, and Final Test Phases.
Pairwise Comparisons to Three Time Points
*p < 0.05. There was significant difference between the pairs.
In order to test group effects separately at each point of time of measurement, a between-subjects ANOVA was performed—there was no statistical difference between the 2 groups at pre-test and post-test measurement times. A significant difference, however, was observed in the final test (F = 5.718, p = 0.018) and the game group demonstrated higher knowledge retention (6.8 ± 1.3) than the lecture group (6.2 ± 1.5).
Results of Medical Student Views on Game-Based and Lecture-Based Learning
We compared the ratings of the 2 iterations of the course that both groups of medical students articulated in the questionnaires. While the results did not reveal any significant difference in perceived knowledge acquisition (p > 0.05) between the groups, the game group gave higher scores than the lecture group to motivation, teaching method preference, course interest, and enjoyment (p < 0.05). Table 3 details these results.
The Results of the Investigation of Medical Student Opinions
*p < 0.05. There was significant difference between the game group and the lecture group.
DISCUSSION
The aim of this study was to investigate the effectiveness of a game-based computer application as a tool for teaching disaster medicine and more specifically, patient surge management to medical students with a focus on learning and enjoyment. Comparing the experience medical students had with the same course taught by lecture and with the application, we found that the game group’s knowledge of disaster medicine improved immediately after the course. Ultimately, we found that teaching patient surge management with the application enabled medical students to gain more knowledge in disaster medicine. Our study thus suggests that a game-based computer application based on the curricular objectives of disaster medicine is an effective pedagogical method.
Our study is similar to Drummond’s, Reference Drummond, Monnier, Tesnière and Hadchouel17 which applies an educational game to teach students about asthma. Meanwhile Greci’s study introduces a novel, Reference Greci, Ramloll and Hurst18 immersive 3D game for training students how best to manage patient surges in a disaster and suggests that such a game-based method is well-suited to team-based, reflective practice and to learning best practices for managing patient surges in a disaster. In addition, we also found that 6 weeks after the last class, the game group retained more knowledge than the lecture group, a finding similar to Chittaro and Buttussi’s, Reference Chittaro and Buttussi19 whose study not only focused on assessing learning immediately after a class but also on long-term knowledge retention. Notably, the subjective and physiological measurements in their study show that students found it more engaging and exciting to learn with a game than with a lecture—such a meaningful level of engagement explains why the students that used the game retained more knowledge than those who learned by lecture. For our purposes, it is helpful to note that in this study our active learning (i.e., game-based learning) emerges as a more effective alternative to the passive learning that is characterized by lecture-based instruction.
Due to the fact that game-based learning enables medical students to learn about disaster medicine and multi-patient management by engaging in the processes of deduction and discussion, it has an advantage over simulated situations. More specifically, game-based learning strategies which place active participation and interaction at the center of the learning experience engage students far more effectively than lectures. Reference Curtis, Trang, Chason and Biddinger11 Using game-based computer applications in the classroom will thus provide an opportunity for learners to apply acquired knowledge and experiment with and get feedback in the form of consequences or rewards, which can help them readily engage in learning experiences in the safety of the ‘virtual world.’ Reference Englund, Gustafsson and Gallego20
Another important difference between both methods in our study was that the game group had higher feedback grades than the lecture group. This may result from the fun inherent to game-based learning which may have made the classes more interesting for and attractive to students. Along these lines, Sward’s study demonstrated that the willingness to continue learning was higher among students in an educational game group than that those in a traditional learning group. Reference Sward, Richardson, Kendrick and Maloney21 In addition, Buijs-Spanjers’s study indicates that the distinguishing feature of educational games may be that students enjoy themselves more when they are studying than when they are not. Reference Buijs-Spanjers, Hegge, Jansen, Hoogendoorn and de Rooij22 All in all, educational games can be enjoyable and motivating ways to encourage knowledge acquisition and retention in students because they involve group interactions, competition, and fun. Reference Middeke, Anders, Schuelper, Raupach and Schuelper23
LIMITATIONS
First, our experimental design is a cohort study, not a randomized group study and moreover, since participants volunteered to enroll in our disaster medicine course, they were already interested in the topic of study indicating a slight bias. To determine whether or not the course will be equally effective for other medical students will thus require further research, especially that which uses a randomized sample. Second, our participants were all part of our institution—the West China School of Medicine at Sichuan University. Applying our game in different institutions may yield different results due to the differences in teacher abilities and medical student characteristics. Moreover, because our study used a very specific game-based computer application (our own) to teach disaster medicine, prudence is required when extrapolating these results to analyze other game-based computer applications. Lastly, it is important to note that several students in the game group were confused at the beginning of the course because they had to change their learning strategies to adapt to learning with the application.
CONCLUSION
Our success with our application suggests that such games may be effective tools with which to teach disaster medicine (especially patient surge management). Against the lecture format, this pedagogical strategy enabled medical students to master and retain knowledge for a longer period of time while also enjoying the learning process. As an effective and enjoyable pedagogical strategy, game-based learning may well encourage life-long learning, the key goal of countless curricula.
Acknowledgements
This work was financially supported by the Special Science Foundation for Public Interest of Ministry of Health (201302003) and Science Foundation of Sichuan Health Commission (150169). The authors would like to acknowledge John Nyland, Doctor of Physical therapy (DPT) EdD from Spalding University, for his assistance with English language editing.
Conflicts of Interest
The authors declare that there are no conflicts of interest.
