Introduction
In recent years, recommendations from the Shape of Training ReportReference Greenaway1 have meant further emphasis has been placed on the training of broader-based emergency-safe ENT surgeons with the knowledge and skills to treat a variety of acute ENT conditions. Of these skills, navigating the temporal bone anatomy during mastoid surgery represents a challenging yet essential part of the surgical repertoire.
Surgeons in training must develop the fine dexterity required to accurately manipulate a drill within the mastoid bone, visualised down the field of view of an operating microscope. Emergency conditions such as sub-periosteal abscesses complicating acute mastoiditis are uncommon, but are associated with high-morbidity.Reference Migirov, Yakirevitch and Kronenberg2 Cortical mastoidectomy involves drilling through the cortex of the mastoid, exenterating air cells, thus externalising any infection. It represents a ‘gold standard’ treatment for sub-periosteal abscesses, superior to simple incision and drainage.Reference Psarommatis, Giannakopoulos, Theodorou, Voudouris, Carabinos and Tsakanikos3,Reference Taylor and Berkowitz4 The new August 2021 otolaryngology curriculum expects trainees to be competent to perform this procedure to the level of a day-one consultant by the end of phase 2 of training.Reference Davis, Spraggs and Murray5
Despite the need for experience in temporal bone drilling, opportunities to practice are limited. Over the past three decades, post-graduate surgical training within the UK has seen several major changes: Calman reforms,Reference Calman, Temple, Naysmith, Cairncross and Bennett6 European Working Time Directive legislationReference Benes7 and the introduction of Modernising Medical Careers.Reference Neville8 More recently, the aerosol-generating nature of temporal bone drillingReference Mick and Murphy9 has meant the coronavirus disease 2019 (Covid-19) pandemic further limited training opportunities in theatre for ENT trainees.Reference Lion, McClenaghan, Hall, Mackinnon and Navaratnam10 Virtual reality models such as the Voxel-Man TempoSurg simulator offer safe, accessible alternatives for learning temporal bone anatomyReference George and De11 and temporal bone drilling.Reference Arora, Khemani, Tolley, Singh, Budge and Varela12,Reference Nash, Sykes, Majithia, Arora, Singh and Khemani13 Simulation may provide more frequent access to learning opportunities than alternatives such as cadaveric temporal bone drilling, thus enhancing learningReference Aussedat, Venail, Marx, Boullaud and Bakhos14 and enabling deliberate practice.Reference Ericsson, Krampe and Tesch-Römer15
Interweaving competition into surgical training exercises may offer additional benefits. In laparoscopic surgery, competition has been shown to significantly increase the surgical efficiency as measured by the number of movements and instrument path length,Reference Hashimoto, Gomez, Beyer-Berjot, Khajuria, Williams and Darzi16 and has been used to achieve post-graduate fundamental proficiency standards by undergraduates.Reference Leraas, Cox, Bendersky, Sprinkle, Gilmore and Gunasingha17 Furthermore, gamification through competition reportedly improves the overall engagement and enjoyment of traditionally challenging areas of the undergraduate curriculum, such as anatomy.Reference Janssen, Shaw, Goodyear, Kerfoot and Bryce18 It can also stimulate participants to consider ENT as a career choice.Reference Killick, Amnolsingh and Chu19
Surgical competitions have potential inherent psychosocial benefits, for example surrounding well-being, the importance of which has become particularly apparent during the Covid-19 pandemic.Reference Ford and Cooper20 In addition, they facilitate successful establishment and integration of new learners into communities of practice by, for example, promoting broad participation, providing the opportunity for focus and being held in one location at one point in time.Reference Wenger21
To our knowledge, only one previous Voxel-Man TempoSurg simulator competition, limited to core trainees, has been reported in the literature.Reference Killick, Amnolsingh and Chu19 Our competition represents the largest temporal bone competition utilising the Voxel-Man TempoSurg simulator described in the current literature open to all training grades and equivalent non-training grades. Our study aimed to identify the trainee-perceived educational benefits of a virtual reality temporal bone drilling competition in the context of ENT training.
Materials and methods
Recruitment
All 35 ENT higher surgical trainees (ST3–ST8), 6 ENT ST1/2 run-through trainees and 6 CT1/2 ENT-themed trainees, as well as staff-grade equivalents within the West Midlands Local Education Training Board region were invited to participate in a free virtual reality temporal bone competition via email. Competition places were limited to 18 in total, with places allocated on a first-come, first-served basis.
Pre-competition data collection
Competition candidates were emailed a pre-competition electronic survey using Google Forms (Appendix A), to be completed before the competition date. The questions asked about prior experience with mastoid surgery, candidates’ confidence performing a cortical mastoidectomy, previous experience with ENT simulators and ENT competition experience. Answers were scored on a 10-point Likert scale.
Competition details
The competition was held at the regional ENT Simulation Dry Laboratory, Queen Elizabeth Hospital, Birmingham, over two days, with participants competing on either day one or day two. Competitors were divided into junior (grades up to and including ST3) and senior (more senior than ST3) groups.
Participants were set two temporal bone drilling tasks on the Voxel-Man TempoSurg simulator lasting 30 and 90 minutes, respectively. The first was focused on drilling under time pressure, whilst the second was focused on demonstrating anatomy. Trainees were independently rated on both their surgical technique and final outcome by a judging panel consisting of three experts (consultant otologists). Raters used the previously validated modified Welling,Reference Andersen, Cayé-Thomasen and Sørensen22 StanfordReference Francis and Niparko23 and Intercollegiate Surgical Curriculum ProgrammeReference Davis, Spraggs and Murray5 scales to calculate the overall combined competition score and identify winners (Appendix B).
After completing both drilling tasks, trainees additionally each received verbal feedback from the faculty lasting up to 10 minutes. Prizes were available for the first- and second-place scorers in the junior and senior groups in each category. One ‘Commendation’ was awarded at the judges’ discretion.
Post-competition data collection
After the competition, before prizes were announced, all participants were asked to complete a post-competition questionnaire using Google Forms (see Appendix A). Participants were asked about their experience using the simulator and of the competition in general, and how the competitive element might have affected their preparation or performance on the day, again using a 10-point Likert scale (1 = negative, 10 = positive).
Statistical analysis
Google Sheets was used to calculate basic summary data (mean, range and standard deviation). R was used to perform the Shapiro–Wilk test for normality and statistical tests.24 The appropriate test was selected from Pearson's correlation coefficient, Spearman's rank test and the two-sample paired student's t-test. The one-sample Wilcoxon test or the one-sample student's t-test was used where the 10-point Likert scale was two-tailed (μ was assumed to be 5.5, significance p < 0.025). Microsoft Excel (2023) was used to calculate the one-sample student's t-test.
Results and analysis
Participants
Eighteen trainees (12 male, 6 female) enrolled in the competition, including 3 CT2, 3 ST3, 2 junior trust grades (Senior House Officer level equivalent), 5 ST5, 3 ST6, 1 ST8 and 1 senior trust grade (Registrar level equivalent). All agreed to participate in this study.
Previous experience
Baseline relevant operative experience and course attendance are reported in Table 1. Seven participants had not performed any cortical mastoidectomies previously. Eight participants had not performed any mastoid explorations previously. Confidence in performing a cortical mastoidectomy was self-rated as a mean of 4.83 ± 2.41 on a 10-point Likert scale (1 = extremely unconfident, 10 = extremely confident). Three participants had not attended any cadaveric bone courses previously and two had no previous experience on the virtual reality temporal bone simulator.
Summary of competitors’ previous baseline experience (n = 18)
Four participants (22 per cent) had used ENT simulators before, including plastic models for laryngoscopy, tracheostomy, nasal packing and balloon sinus dilatation. One participant had used a sheep larynx model for tracheostomy. Four participants (22 per cent) had previously taken part in surgery-related competitions, mostly essay-writing competitions (Table 2).
Summary of competitor's previous experience of surgical competitions
Competition and motivation
The most cited reason for choosing to attend the competition was for learning or education, followed by a desire to compete (Figure 1).
The cited motivators for attending the virtual reality temporal bone simulation competition reported by candidates, organised by frequency of reason given. Each candidate could cite more than one reason for attending.
The perceived influence of the competition as a motivator to practice or read around temporal bone surgery is described in Table 3. Thirteen participants (72 per cent) believed that a competitive element motivates more practice and/or reading compared with a non-competitive event (mean 7.06 ± 2.32, t = 2.85, p = 0.011).
Influence of competition on learning activity as reported by candidates
* 1 = negative motivator; 10 = positive motivator
Fourteen participants (78 per cent) reported undertaking some preparation for the competition. Of these, seven prepared via practice on the Voxel-Man TempoSurg simulator, seven did reading around the topic and three watched videos of the procedure or attended theatre. Four participants did more than one type of preparation.
All participants (100 per cent) claimed they would take further action as a result of taking part in this competition. Eleven participants (61 per cent) planned to undertake practice on the simulator or in the operative setting, and 9 (50 per cent) planned to do further reading around the topic.
Competition and training perceptions
Participants overall agreed that ‘surgical simulation improves intra-operative performance’ (mean 8.28 ± 1.78, V = 169, p < 0.001). All 18 participants rated the competition as educational (mean 8.22 ± 1.17, t = 2.41, p = 0.0278). Participants rated their agreement with the statement ‘the competitive element made [the event] more educational’ a mean of 6.67 ± 2.06 (V = 132, p = 0.0416), with 15 participants (83 per cent) agreeing with this statement.
Seventeen trainees (94 per cent) believed simulation with the Voxel-Man TempoSurg simulator improved intra-operative performance in cortical mastoidectomy, mastoid surgery and understanding of temporal bone anatomy (Table 4). Participants’ self-rated confidence in performing cortical mastoidectomy improved from a pre-competition mean of 4.83 ± 2.41 to 5.72 ± 2.22 post-competition (t = 3.49, p = 0.003).
Candidates’ perceptions towards the benefits of the Voxel-Man TempoSurg simulator
* 1 = strongly disagree; 10 = strongly agree
Seventeen participants (94 per cent) agreed that ‘Competitions are useful within the ENT training programme’ (mean 8.00 ± 1.46, t = 7.29, p < 0.001), but only 6 (33 per cent) agreed they ‘should be made essential’ (mean 3.83 ± 2.83, t = −2.5, p = 0.0231).
Seventeen participants (94.4 per cent) rated the competition as ‘fun’ and quantified this with a mean of 8.06 ± 1.21 (t = 8.95, p = <0.001) on a 10-point Likert scale (1 = extremely not fun, 10 = extremely fun). Thirteen participants (72 per cent) agreed that ‘the competitive element made it more fun’ (mean 6.83 ± 1.92, t = 2.95, p < 0.001). The most enjoyable elements of the competition were deemed to be peer or social interaction (mentioned 10 times), interaction with faculty (reported 5 times) and food (reported 3 times). Seventeen participants (94 per cent) stated they would attend another competition similar to this one (mean 8.5 ± 1.54, V = 169, p < 0.001).
Discussion
The results from this study suggest that trainee participants perceived the virtual reality temporal bone competition to be foremost a highly valuable educational tool. Their perceptions of simulation-based learning parallel closely previously described key benefits, including the opportunity for feedback, deliberate practice, mastery learning and curriculum integration.Reference McGaghie, Issenberg, Petrusa and Scalese25 The most cited motivator for attending the competition in our study was for the learning potential, rather than a desire to win or compete.
All participants agreed the competition was educational, and 83 per cent agreed the competitive element made it more so. The competition encouraged preparative work in most trainees (78 per cent), including reading, simulator practice or watching videos and/or surgery, and further encouraged participants to take ongoing action after the competition. All trainees reported they would take further action as a result of the competition. However, the influence of the competition to motivate these activities did not reach statistical significance, suggesting that the external motivation provided by a virtual reality temporal bone competition alone is insufficient to bring a strong effect.
Trainees, like most adult learners, are likely to be driven more by intrinsic motivators,Reference Knowles, Holton, Swanson, Swanson and Robinson26 thus viewing the competition more as a formative exercise. In contrast, in a larger-scale national Canadian cardiothoracic surgical competition, competitors did see the competition as a significant motivator for increased use of simulation devices.Reference Mokadam, Lee, Vaporciyan, Walker, Cerfolio and Hermsen27 Top performers had accrued many more hours of surgical simulation practice and studying the curriculum compared with those who did not advance in the competition. Elsewhere, the introduction of competitive game mechanics through an elimination tournament for urology residents greatly improved engagement with simulator-based training outside of the tournament.Reference Kerfoot and Kissane28
Virtual reality temporal bone competitions may be of particular educational benefit for more junior trainees. Eight candidates (44 per cent) had not performed any cortical mastoidectomies previously. For such trainees, an interactive virtual reality interface can help address some of the most challenging aspects of temporal bone anatomy in a safe environment. Novices were faster at identifying anatomical landmarks and showed faster improvement when evaluating the three-dimensional virtual reality temporal bone compared with more traditional cross-sectional evaluation.Reference Timonen, Dietz, Linder, Lehtimäki, Löppönen and Elomaa29
In our study, seventeen trainees (94 per cent) reported improved anatomical understanding of the temporal bone and believed this to be useful for future temporal bone surgery. This is supported by a significant post-competition improvement in self-rated confidence regarding performing cortical mastoidectomy (mean from 4.83 ± 2.41 to 5.72 ± 2.22, t = 3.49, p = 0.00277). We expect that the feedback of experts after the drilling exercise highlighted specific development areas in trainees’ techniques. Vygotsky described the importance of this social interaction in focusing learning in the zone of proximal development.Reference Vygotsky and Cole30 The competition format provides opportunities for verbal guidance, affirmation and evidence of performance attainments necessary for cultivating the concept of self-efficacy highlighted by Bandura.Reference Bandura31
Trainees perceived the social aspect of the competition format to carry great value. Interaction with peers and faculty was cited as a particular highlight, making it not only ‘more fun’ overall, but also more educational. The competition format aligns closely with Lave and Wenger's concept of communities of practice.Reference Wenger21 These represent forums for trainees to discuss challenges, collaborate on problems and engage with experts.Reference McCoy, Lewis and Dalton32 For more junior trainees, such activities help to strengthen the sense of belonging in the ENT and regional trainee community. This aspect was particularly lacking during the Covid-19 pandemic. For more senior trainees, formative assessment gives a valuable opportunity for benchmarking against peers and reflection, and guides the direction of future learning or practice.Reference Epstein33
Competitions can improve engagement with ENT simulation training in general. Despite a variety of ENT simulators being described in the literature,Reference Pankhania, Pelly, Bowyer, Shanmugathas and Wali34 only 4 trainees (22 per cent) had reported using other types of ENT simulation models previously in their training. This surprisingly low figure may be explained by recall bias or variations in individual definitions of ‘simulation models’.
In the West Midlands, trainees are expected to attend at least one regional cadaveric temporal bone course per year. Recently, optional regional virtual reality temporal bone simulator courses have started to be offered to trainees in a newly established dry laboratories to supplement their existing training. It was encouraging that prior to the competition 16 trainees (89 per cent) had already used Voxel-Man TempoSurg simulator as part of this facility, spending an estimated mean of 7.22 ± 4.86 hours in total on the simulator per trainee.
In our study, most participants (94 per cent) felt competitions would be a useful addition to the ENT training programme, although most (67 per cent) felt they should not be mandated. There is much appetite for virtual reality temporal bone competitions given that almost all the trainees (94 per cent) stated they would attend a similar one in future.
We recommend surgical simulation competitions to complement traditional simulation training techniques, promote social learning opportunities and reward achievement. However, it must be recognised that competitions in isolation cannot be expected to improve trainee engagement in simulation.Reference McCreery, El-Beheiry and Schlachta35 Newer methods of training should incorporate suitable technologies that reflect the technology literacy of newer generations.Reference McCoy, Lewis and Dalton32,Reference Greenhalgh36 Combining competition and simulation technologies is especially suited to improve engagement with challenging topics such as anatomy and histology,Reference Janssen, Shaw, Goodyear, Kerfoot and Bryce18 and has already been used in several surgical specialties including general surgery,Reference McCreery, El-Beheiry and Schlachta35 cardiothoracicsReference Mokadam, Lee, Vaporciyan, Walker, Cerfolio and Hermsen27 and urology.Reference Kerfoot and Kissane28
ENT competitions, especially those utilising virtual reality technology, such as the one reported here, have a potential role in introducing and improving operative knowledge and skill acquisition through the uptake of simulation training at a time when the Covid-19 pandemic limited many clinical opportunities.Reference Hope, Reilly, Griffiths, Lund and Humes37
Limitations
While all West Midlands trainees were invited to take part in the competition, only 18 of the 47 trainees attended. These trainees are likely to have already formed overall positive opinions regarding simulation, competitions or both, reflecting a positive selection bias. Like any training tool or method, it must be recognised that different trainees will have different perceptions and preferences towards their learning. Individual trainees are likely to accrue different benefits from a virtual reality temporal bone competition. This competition, however, represents the largest virtual reality temporal bone competition in the published literature to date. Although only West Midlands trainees were included, we do not believe this limits the generalisability of our conclusions.
Conclusion
ENT trainees had an overall positive perception towards the educational benefits of a virtual reality temporal bone competition, believing the competition to be educationally valuable, fun and social. Whilst trainees have a healthy appetite for such competitions, it is generally felt they should not be mandated.
From the trainers’ perspective, we encourage the use of competitions, particularly using virtual reality temporal bone simulation model, as a supplement to traditional ENT training to engage trainees in simulation, stimulate learning, provide opportunities for social learning and networking, and recognise trainee achievements.
• Detailed knowledge of the anatomy of the temporal bone is essential for surgical management of complicated mastoiditis
• Newer methods of practice, such as virtual reality simulation, allow trainees to practice temporal bone drilling in a safe and controlled environment, complementing existing training
• The competition format can transform drilling simulation into an engaging and social learning activity, but is relatively underutilised in ENT
• ENT trainees have a positive perception of the educational benefits of virtual reality temporal bone competitions, believing them to be educational, fun and social
• Whilst trainees have a healthy appetite for such competitions, it is generally felt they should not be mandated within training programmes
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0022215124000070
Competing interests
None declared
