Introduction
Simulation-based training is rapidly becoming a vital aspect of surgical training. It refers to a structured educational approach that uses simulators and simulation models to replicate surgical scenarios and procedures in a controlled environment.Reference Pietersen, Bjerrum, Tolsgaard, Konge and Andersen1 This provides surgical trainees with a safe and immersive platform to improve their skills.Reference Aydın, Ahmed, Abe, Raison, Van Hemelrijck and Garmo2 Simulation-based training encompasses various modalities, including virtual reality, computer-based and anatomical models (e.g. three-dimensional (3D) models), cadaveric simulators, box trainers and robot-assisted surgical simulators.Reference de Montbrun and MacRae3 Simulation training provides many benefits, including risk mitigation, ability to provide repetition, immediate and objective feedback, transferability of skills to other areas of surgery and a standardised method of training for all trainees.Reference Agha and Fowler4
Otolaryngology has always been a highly specialised field that requires aptitude in various surgical methods, for example endoscopic surgery, open surgery and microsurgery. In the subspecialised area of rhinology there are many barriers to training that leave trainees feeling less experienced and potentially under confident.5,Reference Oremule, Khwaja and Saleh6 For example, the complexity of rhinological procedures requires a high level of precision and anatomical knowledge, and therefore senior trainers may be hesitant to allow trainees to perform procedures independently without prior training. In addition, with increasing constraints on operating theatre space and availability of theatre staff, the National Health Service is seeing fewer rhinology index procedures being listed, for example septorhinoplasty, and therefore there are fewer training opportunities.Reference Balai, Jolly, Bhamra, Osborne and Barraclough7 As an adjunct to this, system pressures are leading to limited time on surgical cases and therefore if a trainee is operating, efficiency and proficiency in these procedures is essential to keep lists running smoothly.Reference Ashmore8
As simulation models and techniques grow in their popularity, and new models are frequently being introduced, it becomes imperative to assess the effectiveness and validity of these training tools. In 2017, Musbahi et al. reviewed the current status of simulation in otolaryngology and found there was a limited number of high-validity otolaryngology simulators.Reference Musbahi, Aydin, Al Omran, Skilbeck and Ahmed9 This systematic review aims to provide an up-to-date assessment of literature specifically focusing on validated rhinology simulators and their impact on trainees in otolaryngology.
Methods
Search strategy
PubMed, Embase, Cochrane and Google scholar (first 10 pages) were searched for studies conducted between July 2012 and July 2022. Search terms using Boolean operators included ‘ENT OR Otolaryngology OR Rhinology OR Nose’ AND ‘Simulation OR Simulator’. A total of 2092 articles were generated in our initial search. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (‘PRISMA’) protocol (Figure 1) was used to deduce the final list of articles for review. Duplicates and non-English papers were excluded first. Titles and abstracts were then reviewed for relevance.
The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (‘PRISMA’) flowchart of identification of eligible studies.
Selection criteria
Table 1 lists the inclusion and exclusion criteria.
Study inclusion and exclusion criteria
Two independent reviewers sought references to establish studies that met the inclusion criteria to reduce the risk of bias. Subsequently, the PRISMA protocol was used to finalise a list of studies (see Figure 1).
Analysis and validity
The studies to be included were analysed for outcomes including procedure simulated, types of simulators used, participant outcomes from use of the simulation model, whether validity testing was used and level of fidelity. A level of evidence and a level of recommendation using the Oxford Centre of Evidenced-based Medicine classification, adapted for education, were given to each study that conducted validity testing.10,Reference Carter, Schijven, Aggarwal, Grantcharov, Francis and Hanna11
Results
In total, 2092 articles were identified using the search criteria. Of those, 42 met the inclusion criteria and were included in the review (see Figure 1).
Within these 42 articles, 6 main types of simulator were described (see Figure 2): cadaveric human, 3D printed model, virtual reality and augmented reality, animal models and physical models. There was overlap as some articles reviewed more than one simulator or procedure. Twenty-six simulation studies had conducted at least one validation study.
Types of simulators used in the articles included in the study.
The most simulated procedure was endoscopic sinus surgery (n = 30), followed by skull base and cerebrospinal fluid (CSF) leak repair (n = 12) and management of epistaxis and sphenopalatine artery ligation (n = 7). Septoplasty and septorhinoplasty were also covered (n = 5) (see Figure 3).
Distribution of skills simulated in the articles included in the study.
Endoscopic sinus surgery
A total of 30 studies described the use of an endoscopic sinus surgery simulator. The most commonly tested simulators were 3D printed models (n = 12),Reference Chang, Lin, Bowe, Bunegin, Weitzel and McMains12–Reference Ahmad, Citardi, Luong and Yao23 followed by physical models (n = 4),Reference Cao, Feintuch, Feintuch, Tran, Senior and Yang24–Reference Lindquist, Leach, Simpson and Antisdel27 virtual reality and augmented reality (n = 6)Reference Richards, Done, Barber, Jain, Son and Chang20,Reference Barber, Jain, Son and Chang21,Reference Dharmawardana, Ruthenbeck, Woods, Elmiyeh, Diment and Ooi28–Reference Fried, Sadoughi, Gibber, Jacobs, Lebowitz and Ross31 and cadaveric models (n = 5).Reference Stephenson, Farquhar, Masood, Capra, Kimple and Ebert32–Reference Harbison, Dunlap, Humphreys and Davis36 Animal models were tested in three studies.Reference Sadeghnejad, Farahmand, Vossoughi, Moradi and Mousa Sadr Hosseini37–Reference Touska, Awad and Tolley39 Eleven studies performed validity testing for endoscopic sinus surgery simulators,Reference Chang, Lin, Bowe, Bunegin, Weitzel and McMains12–Reference Gillanders, McHugh, Lacy and Thornton19,Reference Chan, Siewerdsen, Vescan, Daly, Prisman and Irish22,Reference Harbison, Johnson, Miller, Sardesai and Davis25,Reference Dharmawardana, Ruthenbeck, Woods, Elmiyeh, Diment and Ooi28,Reference Fried, Sadoughi, Gibber, Jacobs, Lebowitz and Ross31,Reference Awad, Touska, Arora, Ziprin, Darzi and Tolley38 and of these the most common type of validity testing was face, content and construct validity. All studies that performed validity testing (see Table 2) found that their model had confirmation of validity.
Summary of studies conducting validity testing, including the level of fidelity, level of evidence (LoE) and level of recommendation (LoR) based on the Oxford Centre of Evidenced-based Medicine classification10
3D = three-dimensional; n/a = the study did not comment; Y = Yes
All studies asked participants to complete tasks relating to basic functional endoscopic sinus surgery (FESS) (identification of anatomy, examination under anaesthesia, middle meatal antrostomy and ethmoidectomy). The exception being of one study which used virtual reality technology (CardinalSim) to simulate more complex procedures, for example an endoscopic endonasal approach to an inverted papilloma of the maxillary sinus.Reference Won, Hwang, Lim, Cho, Paek and Losorelli29 Most 3D and virtual reality models demonstrated medium or high fidelity (n = 13) and four out of five virtual reality simulators used haptic devices for tactile feedback.Reference Richards, Done, Barber, Jain, Son and Chang20,Reference Dharmawardana, Ruthenbeck, Woods, Elmiyeh, Diment and Ooi28–Reference Kim, Kim, Park and Kim30 Virtual reality was the most costly simulation method, with the most expensive set up costing US$15 000.Reference Dharmawardana, Ruthenbeck, Woods, Elmiyeh, Diment and Ooi28
Finally, all studies showed that their simulation method improved outcomes amongst trainees. The levels of trainee tested amongst the studies included medical students (n = 5), interns (n = 2), junior residents and registrars (n = 8), senior residents and registrars (n = 14), and fellows and consultants and attending (n = 15). When consultants and attendings were included in studies, this was primarily used to assess validity. Most studies compared either a control group and a simulator group, or a before and after simulation measure of performance (e.g. the time to complete the task and dexterity and instrument handling). Most studies also asked for survey feedback from participants following their participation. All simulators for endoscopic sinus surgery showed either an improvement in measured outcomes following simulation or positive survey feedback.
Skull base
Twelve studies looked at the use of simulation in training for skull base surgery. Of these studies, 42 per cent (n = 5) performed validity testing,Reference Awad, Touska, Arora, Ziprin, Darzi and Tolley38,Reference AlQahtani, Albathi, Castelnuovo and Alfawwaz40–Reference Tai, Wang, Joseph, Wang, Sullivan and McKean43 with face and content validity being the most tested. All six of these studies confirmed their models had validity. Three-dimensional printed models (n = 4)Reference Chan, Siewerdsen, Vescan, Daly, Prisman and Irish22,Reference Hsieh, Cervenka, Dedhia, Strong and Steele42–Reference London, Rangel, VanKoevering, Zhang, Powell and Prevedello44 and human cadaver models (n = 4)Reference AlQahtani, Albathi, Castelnuovo and Alfawwaz40,Reference Shen, Hur, Zhang, Minneti, Pham and Wrobel41,Reference Dias, Gebhard, Mtui, Anand and Schwartz45,Reference AlQahtani, Albathi, Alhammad and Alrabie46 were the most common simulators used. Animal models (n = 2)Reference Awad, Touska, Arora, Ziprin, Darzi and Tolley38,Reference Touska, Awad and Tolley39 and virtual reality and augmented reality (n = 2)Reference Won, Hwang, Lim, Cho, Paek and Losorelli29,Reference Kim, Kim, Park and Kim30 were also tested.
The most popular skills covered by the skull base simulation models included CSF leak and skull base repair (n = 4),Reference Awad, Touska, Arora, Ziprin, Darzi and Tolley38–Reference AlQahtani, Albathi, Castelnuovo and Alfawwaz40,Reference AlQahtani, Albathi, Alhammad and Alrabie46 skeletisation of the internal carotid arteries (n = 2)Reference Shen, Hur, Zhang, Minneti, Pham and Wrobel41,Reference London, Rangel, VanKoevering, Zhang, Powell and Prevedello44 and sella turcica (n = 2).Reference Tai, Wang, Joseph, Wang, Sullivan and McKean43,Reference London, Rangel, VanKoevering, Zhang, Powell and Prevedello44 Models to simulate CSF leak and skull base repair were only performed on either a human cadaver or an animal model. Both human cadaver models were created by AlQahtani et al.,Reference AlQahtani, Albathi, Castelnuovo and Alfawwaz40,Reference AlQahtani, Albathi, Alhammad and Alrabie46 with an intradural catheter with fluorescein dye used to simulate CSF leak. Their initial study in 2018 did not validate the use of fluroscein dye, but in 2021 they performed validation studies using a five-point Likert scale questionnaire, a global rating scale of operative performance and a specific skull base reconstruction checklist to confirm the face, content and construct validity of their model. Other skills covered included management of an internal carotid artery injury using a human cadaver modelReference Shen, Hur, Zhang, Minneti, Pham and Wrobel41 and clivus ablation, transpterygoid and transclival approaches using 3D printed models. Similar to the FESS simulation studies, all studies testing 3D models and virtual reality simulation used haptic feedback to improve participant experience and assessment.
All studies apart from AlQahtani et al., Chan et al. and Won et al.Reference Chan, Siewerdsen, Vescan, Daly, Prisman and Irish22,Reference Won, Hwang, Lim, Cho, Paek and Losorelli29,Reference AlQahtani, Albathi, Alhammad and Alrabie46 appraised their simulation models using either neurosurgical or otolaryngology trainees. Consultants were mostly included for validity testing. Most studies carried out post-simulation surveys that showed that the participants found models to be anatomically similar to real life and useful for their training in skull base surgery.
Epistaxis management
Seven studies simulated epistaxis management, of which 57 per cent (4 out of 7 studies)Reference Bright, Varghese and Kurien13,Reference Zhuo, Lei, Yulin, Wentao, Shuangxia and Chao14,Reference Gillanders, McHugh, Lacy and Thornton19,Reference Awad, Touska, Arora, Ziprin, Darzi and Tolley38 performed validity testing. Face, content and construct were the most common validity tests. Three-dimensional printed models were the most commonly used (43 per cent),Reference Bright, Varghese and Kurien13,Reference Zhuo, Lei, Yulin, Wentao, Shuangxia and Chao14,Reference Gillanders, McHugh, Lacy and Thornton19 followed by animal models (29 per cent),Reference Awad, Touska, Arora, Ziprin, Darzi and Tolley38,Reference Touska, Awad and Tolley39 human cadavers (14 per cent)Reference Cervenka, Hsieh, Lin and Bewley47 and physical models (14 per cent).Reference Malekzadeh, Deutsch and Malloy48
Only one paper focused primarily on simulating epistaxis management,Reference Bright, Varghese and Kurien13 with all other studies incorporating it into their review of other tasks. Three studies used simulation to allow candidates to practice sphenopalatine artery ligationReference Gillanders, McHugh, Lacy and Thornton19,Reference Awad, Touska, Arora, Ziprin, Darzi and Tolley38,Reference Touska, Awad and Tolley39 and four studies looked at the non-surgical management of epistaxis, for example nasal packing.Reference Bright, Varghese and Kurien13,Reference Zhuo, Lei, Yulin, Wentao, Shuangxia and Chao14,Reference Cervenka, Hsieh, Lin and Bewley47,Reference Malekzadeh, Deutsch and Malloy48 Gillanders et al. were the only group to use the PHACON sinonasal simulator (Phacon GmbH, Leipzig, Germany).Reference Gillanders, McHugh, Lacy and Thornton19 This is a 3D printed model used primarily to mimic endoscopic sinus surgery. In this study, authors commented on its use for simulating sphenopalatine artery ligation, but trainees felt that simulated bleeding was required to improve its likeness to reality. The trainees felt studies using animal models to simulate sphenopalatine artery ligation offered better anatomical similarities.
All studies found that participants were more confident in managing epistaxis following the use of their stimulator, although training for non-surgical management of epistaxis was understandably more beneficial for training medical students and junior trainees.
Septoplasty and septorhinoplasty
Septoplasty simulation was reviewed in five studies and septorhinoplasty in one. Animal models were used to simulate septoplasty in two studies,Reference Awad, Touska, Arora, Ziprin, Darzi and Tolley38,Reference Touska, Awad and Tolley39 two studies used 3D printed modelsReference Gillanders, McHugh, Lacy and Thornton19,Reference AlReefi, Nguyen, Mongeau, Haq, Boyanapalli and Hafeez49 and one used a physical model.Reference Rosenbaum, Faba, Varas and Andrade50 A 3D printed model was also used in the study aiming to simulate septorhinoplasty.Reference Ho, Goldfarb, Moayer, Nwagu, Ganti and Krein51 All but one study for septoplasty performed validation studies, but the septorhinoplasty study did not test the validity of the model.
Two articles looked at septoplasty simulation alone,Reference AlReefi, Nguyen, Mongeau, Haq, Boyanapalli and Hafeez49,Reference Rosenbaum, Faba, Varas and Andrade50 whereas three studies looked at septoplasty in combination with other procedures. The study by AlReefi et al. claimed to be the first to create and validate a septoplasty training model.Reference AlReefi, Nguyen, Mongeau, Haq, Boyanapalli and Hafeez49 This study used a 3D printed model and compared performance between professionals, senior residents and junior residents to validate the model. In a post-simulation questionnaire all participants agreed that the model was realistic. In 2022, a physical model was created by Rosenbaum et al. which used pigs’ ears and a wooden frame.Reference Rosenbaum, Faba, Varas and Andrade50 The study was validated and 100 per cent of participants said they found it useful for their training.
Ho et al. created a low-cost 3D printed model to help trainees perform nasal osteotomy as part of a septorhinoplasty procedure,Reference Ho, Goldfarb, Moayer, Nwagu, Ganti and Krein51 and claimed it to be the first of its kind. Professionals tested the model and found it was useful, but no validation studies were performed.
Discussion
Simulation-based training in rhinology has escalated in the past five years to become an approach to address the challenges in training faced by otolaryngology trainees.Reference Rosenbaum, Faba, Varas and Andrade50 This systematic review aimed to provide an up-to-date assessment of the effectiveness and validity of various simulation models in enhancing the skills and confidence of rhinology trainees.
Since the review conducted by Musbahi et al. in 2017, there has been a significant increase in the number of simulation models being employed to train otolaryngology trainees in rhinology.Reference Musbahi, Aydin, Al Omran, Skilbeck and Ahmed9 This has been especially apparent since the coronavirus disease 2019 pandemic. As shown by this review, nearly 50 per cent (n = 20) of studies(Reference Bright, Varghese and Kurien13–Reference Yoshiyasu, Chang, Bunegin, Lin, Aden and Prihoda16, Reference Suzuki, Miyaji, Watanabe, Suzuki, Matoba and Nakazono18–Reference Richards, Done, Barber, Jain, Son and Chang20, Reference Ahmad, Citardi, Luong and Yao23–Reference Cao, Feintuch, Feintuch, Tran, Senior and Yang24,Reference Lindquist, Leach, Simpson and Antisdel27,Reference Kim, Kim, Park and Kim30,Reference Stephenson, Farquhar, Masood, Capra, Kimple and Ebert32,Reference ten Dam, Helder, van der Laan, Feijen and Korsten-Meijer34–Reference Dell'Era, Garzaro, Carenzo, Ingrassia and Aluffi Valletti35,Reference Sadeghnejad, Farahmand, Vossoughi, Moradi and Mousa Sadr Hosseini37,Reference AlQahtani, Albathi, Castelnuovo and Alfawwaz40,Reference London, Rangel, VanKoevering, Zhang, Powell and Prevedello44,Reference Cervenka, Hsieh, Lin and Bewley47, Reference Rosenbaum, Faba, Varas and Andrade50–Reference Ho, Goldfarb, Moayer, Nwagu, Ganti and Krein51) were published in or after 2019 and many comment on the need for improved access to out-of-theatre training. These models encompass 3D printed models, virtual reality systems, physical models, human cadavers and animal models. Three-dimensional printed models were the most validated and were found to offer a cost-effective, medium to high fidelity option to rhinology simulation. Virtual reality systems, whilst expensive, provided a highly immersive and interactive learning environment.
Simulation training in the studies reviewed proved to be beneficial to all the trainees tested. Trainees found that simulation-based training demonstrated improved skills and increased confidence, as evidenced by reduced task completion times, enhanced dexterity and high survey scores. One notable observation, however, is that many of the simulation models tended to favour junior trainees. Most studies tested simulation models that focused on simpler procedures and pathologies, for example basic FESS or epistaxis management. More experienced clinicians commented that they would benefit from simulation of more complex surgical procedures.
Simulation-based training appears to be a valuable complement to traditional methods of training such as observation and hands-on experience. Its advantages include a controlled learning environment, risk mitigation and immediate feedback.Reference Agha and Fowler4 However, the synergy between both simulation and traditional methods should be further explored to optimise the training curriculum for otolaryngology trainees. Further studies are therefore needed to investigate long-term skill retention, patient outcomes and the impact of simulation on surgical proficiency.
There could be significant expense to enhancing surgical training with simulation. As the availability of better simulation models is increasing, the cost of simulation models will most likely decrease. Hence, as financial constraints continue to impact our healthcare systems, education departments will begin to look for economically viable and well-tested models, factors which are essential for their widespread adoption in training programmes.Reference Maloney and Haines52
• Simulation-based training has become increasingly popular over the past 10 years but is still in the early stages of development and assessment
• Both high- and low-fidelity, as well as high- and low-cost, models have been found to enhance trainees’ surgical competency and confidence in rhinology-based procedures
• There are currently no systematic reviews available assessing the impact of simulation-based training in rhinology
• Simulation-based training is important in rhinology because we are seeing a reduction in overall training opportunities
• Selection of an appropriate simulation model, coupled with rigorous validation and cost-effectiveness studies, is needed to ensure the integration of simulation-based training into training programmes
This review has several limitations, including the variation in the quality of evidence because it encompasses studies with different methodologies and levels of accuracy. In addition, the scope of this review may be limited by studies that were not highlighted with our search methods, and there may be further studies missed by restricting our search to only English language studies and those conducted within the last decade. Finally, it is challenging to eliminate the potential bias created by these studies, for example most articles were written by simulation creators with a potential vested interest or by clinicians with little to no previous simulation experience.
Conclusion
Improved training through simulation models may lead to more confident and proficient rhinology trainees, ultimately benefiting patient care. Simulation-based training is undoubtedly becoming more established with advancing technology and availability of resources.Reference Cardoso, Suyambu, Iqbal, Cortes Jaimes, Amin and Sikto53 As a result, we hope to see reduced risk to patients, increased efficiency and better management of rhinology pathologies. However, appropriate simulation models, coupled with rigorous validation and cost-effectiveness studies, are needed to ensure their integration into training programmes.
Competing interests
None declared
