Introduction
Otological surgery training has its foundations in the surgical skills laboratory, where a degree of competence in mastoidectomy is acquired before progressing to supervised operative experience. Most training programmes stipulate a minimum number of dissections to be undertaken on cadaveric temporal bones before performing in the operating theatre, based on the idea that repetitive practice develops skill and exposes the surgeon to a range of anatomical variation.Reference Stefanidis 1 – Reference Gallagher, Ritter, Champion, Higgins, Fried and Moses 3 Cadaveric dissection is currently the ‘gold standard’ for otological training,Reference Flint, Haughey, Lund, Niparko, Richardson and Robbins 4 – Reference Snow, Wackym and Ballenger 6 and this inherently provides exposure to anatomical variation. However, cadavers are becoming increasingly difficult to source, and the infrastructure to support their use is costly. Furthermore, it is impossible to guarantee that trainees will be exposed to all the important anatomical variations.
Alternatives to cadaveric bone drilling that offer the opportunity to introduce anatomical variation are emerging, such as three-dimensional (3D) printed synthetic modelsReference Awad, Ahmed, Taghi, Ghufoor, Wareing and Patel 7 and virtual reality surgical simulators.Reference Gallagher, Ritter, Champion, Higgins, Fried and Moses 3 , Reference Agus, Giachetti, Gobbetti, Zanetti, Zorcolo and John 8 – Reference Piromchai 12 Exposure to anatomical variation is intuitively accepted as an important part of training, and the literature on expertise development provides strong support for this. One difference between expert and novice surgeons is their use of recognition-primed decision-makingReference Klein 13 when faced with complex, unusual or pressured situations. The literature on clinical decision-making suggests that expert diagnosticians are able to quickly recognise critical cues (from the patient and context), generate a limited number of alternative options and practise forward reasoning, which leads to an optimal diagnosis.Reference Norman and Ericsson 14 – Reference Norman 16 As Norman suggests, ‘any expert has acquired her expertise in part by working through many examples that can now serve as a rich source of analogies to permit efficient problem solving’.Reference Norman and Ericsson 14 When applied to surgical training, these findings strongly suggest that providing surgical trainees with a range of anatomically varied cases will enhance the development of their surgical knowledge, ability and skill.Reference Crochet, Aggarwal, Dubb, Ziprin, Rajaretnam and Grantcharov 17 , Reference Francis 18
Traditional surgical training relies on the natural variation present in cadaveric cases. On the other hand, simulated training environments present a novel opportunity to embed anatomical variation into training programmes and ensure consistent exposure to these variations. However, in typical circumstances, synthetic or virtual reality models provide repeated practice on the same temporal bone.Reference Awad, Ahmed, Taghi, Ghufoor, Wareing and Patel 7 Course directors using these training tools must choose to introduce anatomical variation, and there is a need now to understand how important this is to training outcomes.Reference Carr 19 Similarly, this information is likely to be of significance to developers of training tools that provide suites of anatomical models. To address this gap in knowledge, we conducted a randomised controlled trial investigating the importance of exposing surgical trainees to anatomical variation.
Materials and methods
Participants
The sample size was calculated based on data from a previous study,Reference Piromchai, Kasemsiri, Wijewickrema, Ioannou, Kennedy and O'Leary 20 under the hypothesis that participants who train on multiple models will, upon completion of the study, attain improvement of 16 points in total score. A sample size of 5.5 participants per arm allows for 90 per cent power with a type 1 error of 5 per cent.
Following approval of the study protocol by the Royal Victorian Eye and Ear Hospital Human Research and Ethics Committee, 11 otolaryngology residents were recruited on a voluntary basis. The residents were accredited trainees in the Royal Australasian College of Surgeons’ Surgical Education and Training Program (Otolaryngology Head and Neck Surgery).
The residents were stratified into two groups according to their level of knowledge and prior experience. Stratification was undertaken according to number of years in the residency programme, into early trainees (first and second year residents) or more experienced residents (third to fifth years). Participants in each stratification level were randomised to either a single training model (n = 5) or multiple training models (n = 6) using a computer-generated randomisation list with balanced blocks of two or four. A participant flow diagram is shown in Figure 1.
Participant flow diagram.
Test platform
The University of Melbourne temporal bone surgery simulator was used in this study. The face and content validity of this simulator have been demonstrated in previous work.Reference Zhao, Kennedy, Yukawa, Pyman and O'Leary 21 , Reference Zhao, Kennedy, Yukawa, Pyman and O'Leary 22 The system comprises two Intel® Xeon® W3565 3.2 GHz processors with 12 GB main memory, an nVidia® Quadro 4000 graphics card, and a 24-inch liquid crystal display 3D monitor. The nVidia 3D Vision Kit is used to provide 3D stereoscopic rendering of the temporal bone. The user interacts with the virtual temporal bone using a Sensable Phantom® Desktop haptic device, which simulates the tactile feedback of a surgical drill.
We developed eight new temporal bone models for the above simulator from micro-computed tomography with a voxel resolution of 96 × 96 × 96 µm. Anatomical structures were segmented manually, and then rendered in 3D. The specific anatomical features presented were the facial nerve, chorda tympani, malleus, incus, stapes, cochlea, semi-circular canal, dura mater, stapedius tendon, basilar membrane, round window membrane and sigmoid sinus.
Experimental procedure
Participants were trained to use the simulator system efficiently before initiating the study. They were instructed to perform the cochlear implant surgery procedure according to the standard textbooks used as part of the Surgical Education and Training programme in Otolaryngology Head and Neck Surgery of the Royal Australasian College of Surgeons.Reference Flint, Haughey, Lund, Niparko, Richardson and Robbins 4 , Reference Paparella 5
Both groups of residents attended four weekly training sessions, performing cortical mastoidectomy, posterior tympanotomy and cochleostomy on the simulator. Participants performed two procedures per session. For the first three weeks, one group was given six different temporal bone models to train on, while the other group performed the surgery on the same temporal bone six times. Participants randomised to the single model group were informed that they would be seeing the same anatomy six times prior to joining the study, to minimise the chance of early withdrawals because of boredom or disinterest.
In the final week of the study, all participants operated on two temporal bones they had not seen before. These bones consisted of one normal temporal bone, and one challenging temporal bone with an anteriorly located sigmoid sinus and otosclerotic overgrowth completely obliterating the round window membrane. Each procedure was recorded using screen capture software on the virtual reality system. Additionally, the simulator software recorded a variety of performance measures, such as time on task, path length and injuries to vital structures.
Upon completion of the study, videos of the last two procedures were de-identified and assessed independently by two consultant otolaryngologists (who were blinded to the test allocation of participants). To minimise bias, video segments were not isolated and the length of each procedural stage was not shortened. The evaluator needed to watch the video from start to end.
The evaluators scored performances using the validated assessment tool we developed in our previous work for evaluating competency in cochlear implant surgery.Reference Piromchai, Kasemsiri, Wijewickrema, Ioannou, Kennedy and O'Leary 20 This tool comprises two parts: a global rating scale to rate the overall surgical skills relevant to this surgical procedure, and a task-based checklist to evaluate competence in preparing the surgical approach for cochlear implantation (Appendices 1 and 2). The scope of the assessment was the surgical drilling associated with cochlear implantation, specifically the cortical mastoidectomy, facial recess and cochleostomy.
Statistical analysis
Assessor agreement was evaluated using the Spearman correlation. Chi-square tests were used to test for differences in proportions. Performance of the single and multiple model groups was compared across bones based on summed scores (primary analysis) and separately for each of the two specimens (subgroup analysis). For each comparison, a histogram of the data was created and normality was evaluated using the Kolmogorov–Smirnov test. If a normal distribution was assumed, two-sample t-tests were conducted. The Mann–Whitney rank sum test was used where data did not meet normality assumptions.
Results
Six residents were randomly allocated to the multiple model group and five to the single model group (Table I). The two groups were balanced in terms of their prior skill level. The overall inter-rater correlation between the two assessors was 0.44 (p < 0.001).
Group characteristics
*n = 5; † n = 6. ‡Two-sample t-test; **chi-square test
The sum of global rating scale indices was not statistically significantly different between the two groups (mean difference = 4.06, 95 per cent confidence interval (CI) = −0.84 to 8.97, p = 0.09). The multiple model trainees displayed significantly better use of the otological drill on the more challenging specimen (Table II).
Comparison of global rating scores by group for the two test specimens
*Statistically significant (p < 0.05; two-sample t-test; normal distribution). CI = confidence interval
The sum of task-based checklist indices between the two groups was not statistically significantly different (mean difference = 3.98, 95 per cent CI = −1.55 to 9.50, p = 0.14). The multiple model participants performed significantly better in preserving the facial nerve on the challenging specimen and in visualising the round window niche on the normal specimen (p < 0.05) (Table III).
Comparison of task-based checklist scores by group for the two test specimens
*Statistically significant (p < 0.05; two-sample t-test; normal distribution). CI = confidence interval; EAC = external auditory canal
Automated simulator-generated metrics revealed that the multiple model group caused fewer injuries to the incus (mean difference = 1.5, 95 per cent CI = 0.0 to 3.0, p = 0.037) and facial nerve (mean difference = 3.5, 95 per cent CI = 0.0 to 12.0, p = 0.037). On the more challenging specimen, multiple model participants were significantly more likely to reach the round window membrane (single model = 1 out of 5, multiple model = 6 out of 6, chi-square = 4.48, p = 0.034).
Discussion
The results of this study indicate that practice on multiple bones led to improved surgical performance across several domains relating to both general surgical technique and aspects specific to the cochlear implantation procedure.Reference Mangus, Rivas, Tsai, Haynes and Roland 23 Although null hypothesis testing for the summative differences between group means did not yield statistically significant results, residents who trained on multiple bones were better at performing some of the more difficult aspects of the procedure. This group was better at preserving important anatomical structures such as the facial nerve and incus. Moreover, they exhibited better drill handling and exposure of the facial nerve and round window membrane when presented with a challenging specimen. This is consistent with the findings of Malik et al., who noted that differentiation of skill is better delineated with more challenging procedures such as a facial recess approach.Reference Malik, Varela, Park, Masood, Laeeq and Bhatti 24
-
• Simulated training environments enable anatomical variation in training programmes and ensure consistent exposure to these variations
-
• The results presented here suggest that exposure to a range of varied anatomical cases is beneficial for surgical competency
The results presented here suggest that exposure to a range of varied anatomical cases is beneficial in achieving surgical competency, as it provides surgeons with a rich library of past experiences that can be drawn upon when they encounter novel cases.Reference Stefanidis 1 – Reference Gallagher, Ritter, Champion, Higgins, Fried and Moses 3 These results support the introduction of anatomical variability into virtual reality surgical simulations. Virtual reality simulation has advantages for surgical training, including the presentation of the same training experience to all participants, and ensuring that all trainees can experience specific cases, or anatomical variants. Our study highlights that particular care needs to be exercised when developing training programmes within simulation environments; namely, these should introduce sufficient anatomical variability in the training exercises.
Conclusion
Exposure to multiple anatomical specimens in a virtual reality temporal bone simulation led to better identification of, and less injury to, the incus and facial nerve. The results presented here suggest that exposure to a range of varied anatomical cases is beneficial in achieving surgical competency.
GLOBAL RATING SCALE FOR COCHLEAR IMPLANTATION COMPETENCY
TASK-BASED CHECKLIST FOR COCHLEAR IMPLANTATION COMPETENCY
