Introduction
In the UK, surgical training is challenged on a number of fronts, including the decline of the ‘firm’ model, increased emphasis on service provision and, more recently, redeployment, and the cancellation of elective surgery and teaching courses as a result of the coronavirus disease 2019 (Covid-19) pandemic. Simulation is touted by many as a solution to enable surgical trainees to develop skills independently from clinical practice in the operating theatre.
Simulation in ENT surgery, as with many other surgical specialties, is undergoing a renaissance. Models have been developed to simulate a variety of procedures, including temporal bone surgery, endoscopic sinus surgery, myringotomy and middle-ear surgery, as well as laryngeal and throat surgery.Reference Musbahi, Aydin, Al Omran, Skilbeck and Ahmed1 These models may be incorporated into structured ‘boot camp’ style courses, which have been shown to increase procedural confidence.Reference Chin, Chin, Roth, Rotenberg and Fung2,Reference Hogg, Kinshuck, Littley, Lau, Tandon and Lancaster3 However, only 32.9 per cent of ENT trainees report access to simulation skills facilities.Reference Milburn, Khera, Hornby, Malone and Fitzgerald4
Some commercial simulation models can retail for anywhere up to $20 000, a cost which is prohibitive for the average surgical department.Reference Bennett, Wilmot, Reddy and Alderson5 Other sources of simulation training, such as craft courses, which are frequently a prerequisite for application to higher training, may cost a trainee upwards of £9105 over the duration of their training.Reference O'Callaghan, Mohan, Sharrock, Gokani, Fitzgerald and Williams6 Furthermore, short courses are no replacement for the countless consecutive hours of training required for mastery of practical skills.
In response to this, a number of low-cost models for surgical procedures across a variety of surgical specialties have been developed, to allow for continued independent practical training.Reference Pelly, Shanmugathas, Bowyer, Wali and Pankhania7–Reference Gravante and Venditti9 Some studies describing validation of these models are documented in the literature.
This article systematically reviews the available models for low-cost simulation in ENT surgery. Each model is discussed in terms of its cost, materials and ease of construction, and is reviewed for attempts to demonstrate its educational value and validity. Educational value was defined as an attempt to demonstrate that the simulation model produced an improvement in trainees’ performance. Validity was assessed in terms of face, content and construct validity.
Materials and methods
A systematic review of the literature was performed in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses (‘PRISMA’) guidelines (Figure 1). The search string used was: ‘(ENT OR otorhinolaryngology) AND (simulation or simulator) AND model AND low-cost’. The literature search was performed by two of the authors independently (TP and RP) using PubMed, Medline, Embase, Cochrane and Google Scholar databases. Relevant references from the reviewed studies were also included and analysed.
Preferred Reporting Items for Systematic Reviews and Meta-Analyses (‘PRISMA’) flowchart demonstrating article selection. VR = virtual reality; 3D = three-dimensional
Studies were excluded if the model's components were not felt to be readily available online or in UK shops. All virtual reality simulation tools and models generated by three-dimensional (3D) printers were excluded given their relative expense. The authors defined ‘low-cost’ as being less than $150; studies describing models with costs greater than this were subsequently excluded. This limit was set as it was felt to be a reasonable cost that a trainee might be able to afford without departmental financial support. Conference abstracts were excluded as they lacked sufficient detail for critical analysis.
The included studies were reviewed by authors with the use of a standardised proforma. The data extracted included the following parameters: total cost, model type (animal, synthetic or other), ease of construction, number of study participants and their grade or experience, educational value, and face, construct and content validity. Costs were converted to US dollars for ease of comparison.
Results
Otology simulators
Malekzadeh et al. developed a low-cost model for myringotomy and tube placement, constructed from a 3 ml syringe, cut to resemble the ear canal, and a glove (representing the tympanic membrane) placed inside a cardboard glove box.Reference Malekzadeh, Hanna, Wilson, Pehlivanova and Milmoe10 The model was validated using 20 first-year surgical trainees randomly assigned to didactic teaching or simulation model teaching. Assessment was carried out using the Objective Structured Assessment of Technical Skills model. Mean Global Rating Scale scores at six weeks were higher (7 vs 4.7) and time improvement was greater for the simulation model group in comparison to the control group. However, the final assessment was made on the same trainer to which the skills group had previously been exposed. Improvements therefore could be related to improvement on the specific trainer, and may not translate to an improvement in skills on real patients. This is particularly relevant in the absence of any face validity assessment, and indeed the model appears to be low-fidelity.
Molin et al. developed a similar synthetic model constructed from a ‘bocce’ ball, with a cylindrical working shaft drilled into it, and a slot for paper tympanic membranes mounted on slides.Reference Molin, Chiu, Liba and Isaacson11 This came to a total cost of $35. The Global Rating Scale scores of the 10 medical student participants increased after 30 minutes of instruction with the model, with time to myringotomy tube placement reduced after practice. However, there was no control group, and the sample size was small. In addition, there was no formal assessment of face or content validity.
Dedmon et al. described a synthetic ‘task trainer’ style model constructed from a polyvinyl chloride (PVC) housing with interchangeable low-fidelity skills models.Reference Dedmon, O'Connell, Kozin, Remenschneider, Barber and Lee12 Experience of the model was assessed by seven participants, ranging from novices to experienced operators. No formal attempt was made to assess educational impact, although all participants felt that their skills had improved. The model demonstrated good construct validity, with time taken for tasks such as reverse needle threading reduced for experts (92 seconds) in comparison to novices (160 seconds). The model was viewed favourably in terms of suitability as a simulation tool, suggesting content validity; however, it is unclear to what extent this positive evaluation came from novices or experts. Eighty-six per cent agreed that the orientation of the model was accurate, although it appears low-fidelity.
Chiesa Estomba,Reference Chiesa Estomba, Melendez Garcia, Hamdam Zavarce and Betances Reinoso13 and Owa and Farrell,Reference Owa and Farrell14 described simple low-cost synthetic models for myringotomy and grommet insertion. Their models are developed from materials readily available in hospital; they are easy to construct and offer reproducible training. However, no attempt was made to validate the educational benefit of these models.
Owa et al.Reference Owa, Gbejuade and Giddings15 and Mathews et al.Reference Mathews, Hetzler and Hilsinger16 both describe simple and inexpensive models for creating confined spaces to practice artificial stapes manipulation. The models have not been validated for use in training, but with the scarcity of otosclerosis patients now being operated on, their potential benefit is apparent.
Rhinology simulators
Malekzadeh et al. (2011) described the development of a synthetic ‘low-cost sinus surgery task trainer’, which can be used to simulate five specific tasks: ethmoidectomy, superior ethmoidal tissue removal, frontoethmoidal recess exploration, targeted injections, and uncinectomy with maxillary antrostomy.Reference Malekzadeh, Pfisterer, Wilson, Na and Steehler17
A follow-up study by the same group in 2012 extensively evaluated its face, construct and content validity.Reference Steehler, Pfisterer, Na, Hesham, Pehlivanova and Malekzadeh18 Their study population of 77 individuals (10 attending otolaryngologists, 15 residents and 52 medical students) were recorded using the simulation model and assessed by 2 blinded attending otolaryngologists using the validated Global Rating Scale tool. The experienced cohort outperformed their novice counterparts across the board (p < 0.0001 for all metrics). Face and content validity were assessed using a post-study survey of 10 local attending otolaryngologists on a Likert scale. Whilst 80 per cent of respondents indicated that the model ‘correlates with the essential skills needed for endoscopic sinus surgery’, there was a lack of consensus regarding its face validity, with a mean score of 3.1 given for the ability of the model to ‘mimic actual sinus anatomy’.
These authors subsequently evaluated the model as an educational tool.Reference Steehler, Chu, Na, Pfisterer, Hesham and Malekzadeh19 They recruited 52 medical students with no prior experience of sinus surgery who were then enrolled in a training programme using the task trainer. Performance was evaluated by two blinded experts. Post-training performance was reported as ‘significantly better’ using the Global Rating Scale tool (p < 0.001).
Overall, this model appears to be genuinely affordable, and has demonstrated good content and construct validity. Costing estimates, however, do not include endoscopic instruments, and its face validity remains questionable. In terms of educational impact, the authors have demonstrated that training with the model improves performance on the model itself; however, its impact on real-life performance in sinus surgery has not been evaluated.
Airway and throat simulators
One paper, by Rinaldi et al. (2019), described the development and use of a synthetic simulator for barbed snore surgery, for the treatment of obstructive sleep apnoea syndrome.Reference Rinaldi, Costantino, Moffa, Baptista, Sabatino and Casale20 These authors developed a model consisting of a resin skull with a triple-layered silicon soft palate adhered with silicone rubber, for the cost of $19.25 per unit. Although Rinaldi et al. provided photographs detailing the simulated procedure in various stages, they made no attempt to formally validate their model or demonstrate the objective educational impact. The representation of soft tissues is extremely limited (to the soft palate only), although the authors claim that this provides better appreciation of important bony landmarks.
Richtsmeier (2002) outlined the development and construction of a low-cost model for simulating endoscopic staple-assisted oesophagodiverticulostomy for treatment of Zenker's diverticulum.Reference Richtsmeier21 Constructed from latex gloves, PVC piping and wooden boarding, the author estimated an average cost of $7.32 per unit for non-consumables. Whilst construction was estimated to be inexpensive, with clear instructions provided, the recommended consumable components were prohibitively expensive, with the EndoStitch™ suturing device and Ethicon stapler estimated at $449.96. Twelve ENT surgeons were surveyed following the performance of oesophagodiverticulostomy with a questionnaire assessing validity. In terms of content validity, 11 of the 12 participants reported improved confidence with oesophagodiverticulostomy after using the model; however, no expert or independent opinion was described. The author states that the majority of participants ‘thought the simulator provided reasonable approximation of the endoscopic view’; however, this was not quantified. No attempt was made to assess construct validity.
Laryngeal simulators
Zambricki et al. demonstrated a model for phonomicrosurgery, constructed for less than $20 from a grape embedded in gelatine, although it required access to a microscope and microlaryngeal instruments.Reference Zambricki, Bergeron, DiRenzo and Sung22 Assessments with 30 trainees of varying experience demonstrated improved pre-test to post-test scores using the model. Construct validity was demonstrated in some tasks, with senior residents found to be superior to the other groups in terms of: positioning of the microscope (p < 0.001), vocal fold injection (p < 0.05), preservation of the remaining tissue (p < 0.002) and time taken to complete all tasks (p < 0.003). Face validity was also demonstrated. When asked if the model was a realistic representation of microlaryngeal surgery, the average response score was 7.5 out of 10. When asked if the phonomicrosurgery was a realistic representation of a vocal fold injection, the average response score was 6.1 out of 10.
A similar model was developed by Holliday et al., using grapefruit.Reference Holliday, Bones, Malekzadeh and Grant23 Sixteen otolaryngologists with varying levels of experience assessed the model for content and face validity. In terms of face validity, 100 per cent of the participants scored the simulator at least 4 out of 5 across a number of exercises. An assessment of content validity was made, with at least 70 per cent of surgeons agreeing that the model helped to develop dexterity and general skills related to the excision of lesions. Nevertheless, this model, whilst well validated, appeared relatively complex to assemble and required access to a laryngoscope. Although participants were in agreement regarding the model's educational value, there was no formal assessment of this.
A mixed animal and synthetic model for laryngotracheal reconstruction was developed by Kavanagh and Murray.Reference Kavanagh and Murray24 This model was inexpensive, with materials costing less than $10, and could be constructed in less than 2 minutes. The model was assessed by 2 ENT surgeons, as well as 12 participants including residents and advanced practitioners. The model scored favourably in some areas, such as tissue manipulation and ability to place sutures; however, face, content and construct validity were not formally assessed.
Another study described a model for transcervical laryngeal injection,Reference Cabrera-Muffly, Clary and Abaza25 constructed from common household objects costing less than $1. Educational value was assessed in terms of participant ‘confidence’; however, face validity was assessed by the authors of the paper only.
Quinsy and tonsil simulators
Bunting et al. produced a synthetic model for the simulation of peritonsillar abscess drainage.Reference Bunting, Wilson, Malloy and Malekzadeh26 The reported cost was less than $5, although it required the use of a resuscitation mannequin face which comes at extra expense. The model was not demonstrated to have educational impact, although all 19 ENT registrars who participated in the simulation felt the model ‘would be a useful tool for residency training’. They also scored it highly in terms of anatomical and physical accuracy, suggesting face validity. Construct validity was not assessed.
Street et al. developed a synthetic model for carrying out tonsil ties.Reference Street, Beech and Jennings27 The cost was low (less than $5), although it does require the use of a trainer shoe and replacement of the inner sole with each use. The model was easy to construct, but was not formally assessed by any users for face, construct or content validity.
Ross et al.Reference Ross, Jaiswal and Jones28 utilised a potato crisp tube and Wasson et al.Reference Wasson, De Zoysa and Stephens29 used a tissue box to create easy-to-construct, low-cost (less than $5) tonsil tie simulators. There was no formal assessment of educational value, or of construct, face or content validity, and these trainers are only hypothesised by the authors to develop technique.
Discussion
This systematic review of the literature has shown that there are numerous low-cost simulation models in otorhinolaryngology that encompass a broad range of surgical techniques. A summary of the low-cost ENT simulation models is shown in Table 1.Reference Malekzadeh, Hanna, Wilson, Pehlivanova and Milmoe10–Reference Zambricki, Bergeron, DiRenzo and Sung22,Reference Kavanagh and Murray24–Reference Wasson, De Zoysa and Stephens29 Several of these models have demonstrated good validity across several areas, as well as educational value. Malekzadeh's sinus surgery bench model is one such example (Figure 2).Reference Malekzadeh, Pfisterer, Wilson, Na and Steehler17 This model is well validated, low-cost and has an educational impact. Unfortunately, this model appears to be an exception, as the vast majority of simulation models rely on informal or partial assessments of validity. Often only weak attempts were made to assess educational value, or no attempt was made at all. In addition, the reliability of these simulation models may be questioned, where construction of individual models by different surgical trainees may vary.
Summary of low-cost ENT simulation models
Malekzadeh's low-cost sinus surgery simulator.Reference Malekzadeh, Pfisterer, Wilson, Na and Steehler17 (a) Completed model ready for use. (b) Labelled sinonasal cavity of the fully constructed sinus surgery task trainer. (c) Sinus task trainer covered with mask. This was the best performing simulator across validity parameters and demonstrated educational value. PDS = polydioxanone suture
The concept of ‘validity’ in terms of surgical simulation has been scrutinised in a systematic review performed by Schout et al.Reference Schout, Hendrikx, Scheele, Bemelmans and Scherpbier30 This review suggested that assessments of validity in surgical simulation are generally unreliable. This is reflected within our sampled literature, where the definitions of the different types of validity were poorly defined and methodology varied significantly between studies. We agree with Schout and colleagues that there is a need for a consensus opinion on methodologies to be employed when validating simulation models. Identifying robust methodology for validity assessments of simulation models will be essential in order to allow governing bodies to advocate their use in the training of surgeons, alongside more traditional educational methods.
In the Covid-19 era, where operating experience may be limited for surgical trainees, it is clear there is a need to adapt from traditional methods of surgical education. Simulation courses are becoming increasingly difficult to access and are often cancelled at short notice. As a result, inexpensive simulation models that trainees can construct for themselves and use independently, outside of the workplace, are increasingly valuable.
One of the common misconceptions among trainees and senior surgeons alike is the concern that self-constructed models do not offer the same level of face validity as more expensive models, and therefore have less educational impact. However, there is some evidence to suggest that low-fidelity simulation is as equally effective as high-fidelity simulation in medical training.Reference Diederich, Mahnken, Rigler, Williamson, Tarver and Sharpe31–Reference Matsumoto, Hamstra, Radomski and Cusimano33 Indeed, one study investigating medical student performance during simulation on a high-fidelity or a low-fidelity mannequin suggested that practice with a high-fidelity simulation model may lead to overconfidence – a finding which is concerning when applied to surgical trainees.Reference Massoth, Roder, Ohlenburg, Hessler, Zarbock and Popping34
The advantages of low-cost simulation lie in the minimal investment in terms of expense or construction. Thus, in poorly validated models, equivocal or small gains in surgical skills do not come at a significant cost to the trainee. This is advantageous for trainees in early stages of their training, who may lack access to high-fidelity surgical equipment, or in developing countries.
There are a number of limitations to this review. Although the literature search was repeated by multiple authors, it may be possible that some viable simulation models were not reviewed. The ‘low cost’, defined as less than $150, may still be out of reach for some trainees or for surgeons from lower income backgrounds. There is also the argument that surgical training should never rely on the surgeon's ability to provide themselves with training opportunities. The low-cost models presented here may exclude some well validated simulation models that would be worth the extra expense. Any simulation models that used 3D printing and virtual reality simulators were excluded, as they were felt to be inaccessible for the majority of UK surgical trainees. However, for otorhinolaryngology trainees fortunate enough to have easy access to these tools, they may provide more effective simulation than some of the low-fidelity bench models described in this article.
This review has demonstrated that there are a plethora of low-cost simulation models for practising otolaryngological procedures that may be of use to the surgeon in training. However, there is a need for further scrutiny in terms of validity and educational value before many of these low-cost models can be recommended.
Competing interests
None declared
