Introduction
The General Medical Council (GMC) states that all new doctors beginning an ENT surgery rotation must be offered an induction that includes the teaching of core procedures.Reference Jaques1 Such procedures include flexible nasendoscopy (FNE) and peritonsillar abscess drainage, which are routinely performed by ENT on-call doctors. The FNE involves passing a flexible nasoendoscope, which has a light and camera, through the nasal cavity and towards the larynx; it plays a crucial role in assessment of the upper aerodigestive tract. Peritonsillar abscesses are the most common deep neck space infection to present in the emergency department. Management of the latter includes identifying the maximal area of fluctuance above the upper pole of the tonsil, and removal of the pus by either aspiration or incision and drainage.Reference Gupta and McDowell2 Both procedures are considered low risk, but they require a comprehensive understanding of the anatomy and surgical technique to be successful.
Post-graduate doctors working within ENT can vary in surgical experience, from ENT-themed core surgical trainees to non-themed core surgical trainees, general practice trainees and foundation doctors. A national survey of ENT juniors providing night on-call cover found that 68 per cent had no prior ENT experience and 42 per cent were not confident with managing ENT emergencies.Reference Biswas, Rafferty and Jassar3 Therefore, a robust induction is required to improve confidence in ENT-specific skills.Reference Kucheria, Bastianpillai, Khan and Acharya4
However, the coronavirus disease 2019 (Covid-19) pandemic has placed significant pressure on medical education, and junior doctor training has been affected.Reference Seifman, Fuzzard, To and Nestel5 More precisely, surgical trainees have faced less clinical exposure and reduced surgical case volume.Reference Seifman, Fuzzard, To and Nestel5,Reference Luu, Yver, Douglas, Tasche, Thakkar and Rajasekaran6 Consequently, trainees are searching for alternative teaching methods to develop their surgical skills.
Rapp et al. found that surgical trainees were most commonly using YouTube as their educational resource to prepare for a procedure.Reference Rapp, Healy, Charlton, Keith, Rosenbaum and Kapadia7 YouTube is an easily accessible video source that can be accessed from any location. However, YouTube content is not chosen for its educational value but rather for its popularity, as well as several other criteria such as view count, comments, and likes or dislikes.Reference Koya, Bhatia, Hsu and Bhatia8
Previous studies have applied the Laparoscopic Surgery Video Educational Guidelines (‘LAP-VEGaS’) to evaluate the educational quality of YouTube videos. These Guidelines were created by a multidisciplinary committee with the objective of appraising surgical videos submitted for publication and presentation.Reference Celentano, Browning, Hitchins, Giglio and Coleman9–Reference Rossler, Lahner, Schebesta, Chiari and Piochl11 Since then, the tool has also been used to assess the quality of YouTube videos on otolaryngology procedures that include: thyroidectomy, parathyroidectomy, endoscopic endonasal approaches and neck dissection. Therefore, we sought to critically evaluate the educational quality of YouTube videos demonstrating FNE and peritonsillar abscess drainage by applying the Laparoscopic Surgery Video Educational Guidelines framework.
Materials and methods
Video selection
YouTube's video sharing platform (https://www.youtube.com) was searched on 13 March 2022 using the following key words and phrases: ‘flexible nasendoscopy’, ‘flexible nasal endoscopy’, ‘flexible nasolaryngoscopy’, ‘flexible nasopharyngoscopy’, ‘flexible nasopharyngolaryngoscopy’, ‘quinsy drainage’, ‘peritonsillar abscess drainage’, ‘peritonsillar abscess aspiration’, ‘peritonsillar abscess treatment’, ‘quinsy aspiration’, ‘peritonsillar abscess incision and drainage’ and ‘peritonsillar abscess needle aspiration’. The results for each search query were screened based on ‘relevance’ and the first 10 eligible videos were selected. Exclusion criteria included patient testimony, non-instructional videos and presentations without video footage of the procedure. Furthermore, videos that demonstrated procedures performed under general anaesthetic, rigid endoscopic approaches or abscess tonsillectomy were also excluded. The remaining videos were compiled, and duplicates were removed. This video selection pathway is shown in Figure 1.
Flow chart outlining video selection pathway and review process.
Video evaluation
The following characteristics were extracted for each included video: title, uniform resource locator (‘URL’), publication date, country of practice, view count, video length, comments, likes, and presence of audio and/or written commentary. The videos were then evaluated using the Laparoscopic Surgery Video Educational Guidelines checklist.Reference Celentano, Smart, Cahill, Spinelli, Giglio and McGrath12 Each video was marked against a checklist consisting of nine criteria, graded from 0 (not present in video) to 2 (extensively present in video). Final scores are tallied and divided into three categories defined by the checklist (low = 0–6, medium = 7–12, high = 13–18). This process was performed independently by two otolaryngology core trainees with full membership of the Royal College of Surgeons of England (HD and BVT).
Statistical analysis
If not otherwise specified, descriptive statistics were presented as absolute number with percentage, or mean with standard deviation (SD). Cohen's kappa co-efficient was used to calculate inter-rater agreement between the two otolaryngology trainees. Correlations between video characteristics (view count, video age, video length, number of comments and number of ‘likes’) and the overall Laparoscopic Surgery Video Educational Guidelines score were calculated using Pearson's product-moment correlation co-efficient. A p-value of less than 0.05 was considered statistically significant. Statistical analyses were performed using Microsoft Excel for Mac software (version 16.57; Microsoft, Redmond, Washington, USA).
Results
Video characteristics
Our search yielded a total of 27 unique videos that met the inclusion criteria for analysis, including 18 for FNE and 9 for peritonsillar abscess drainage. The characteristics of the 27 selected videos are reported in Table 1. Overall, 11 videos (41 per cent) were produced in the USA, and 8 videos (30 per cent) were produced in the UK. The remaining videos were produced in Canada (n = 1), China (n = 1), India (n = 3), Malaysia (n = 2) and the Netherlands (n = 1). Across all videos, the mean view count was 147 366 (SD = 342 876), with a range of 26 to 1 767 444 views. Videos were uploaded between 2010 and 2022, with a mean age of 4.19 years (SD = 3.28). The mean video length was 5.0 minutes (SD = 2.9). Videos received a mean of 36 comments (SD = 55) and a mean of 458 ‘likes’ (SD = 577). Twenty-three videos (85 per cent) provided audio and/or written commentary.
Video characteristics of the 27 selected videos
Hr:min:sec = hours:minutes:seconds
Video quality assessment
The Laparoscopic Surgery Video Educational Guidelines assessment (individual criteria and total scores) for both FNE and peritonsillar abscess drainage videos are presented in Tables 2 and 3, respectively. In summary, the mean total Laparoscopic Surgery Video Educational Guidelines scores for FNE and peritonsillar abscess drainage videos were 10.3 (SD = 3.1) and 11.7 (SD = 4.6), respectively, out of a possible 18 points. Only seven videos (26 per cent) were assessed as high quality, with the majority (n = 18; 67 per cent) designated as medium quality. Inter-rater reliability was moderate for both FNE (weighted Cohen's kappa co-efficient of 0.66) and peritonsillar abscess drainage videos (weighted Cohen's kappa co-efficient of 0.61).
Flexible nasoendoscopy video quality evaluation using 2020 LAP-VEGaS tool*
Item 1: Authors and institution information. Title of the video including name of the procedure and pathology treated. Item 2: Formal presentation of the case, including patient details and imaging, indication for surgery, co-morbidities, and previous surgery. Patient anonymity is maintained. Item 3: Position of patient, access ports, extraction site and surgical team. Item 4: The surgical procedure is presented in a standardised step-by-step fashion. Item 5: The intra-operative findings are clearly demonstrated, with constant reference to the anatomy. Item 6: Relevant outcomes of the procedure are presented, including operating time, post-operative morbidity and histology when appropriate. Item 7: Additional graphic aid is included, such as diagrams, snapshots and photographs, to demonstrate anatomical landmarks, relevant or unexpected finding, or to present additional educational content. Item 8: Audio or written commentary in English language is provided. Item 9: The image quality is appropriate, with a constant clear view of the operating field. The video is fluent with appropriate speed.
* Flexible nasoendoscopy videos, n = 18. LAP-VEGaS = Laparoscopic Surgery Video Educational Guidelines; SD = standard deviation
Peritonsillar abscess drainage video quality evaluation using 2020 LAP-VEGaS tool*
Item 1: Authors and institution information. Title of the video including name of the procedure and pathology treated. Item 2: Formal presentation of the case, including patient details and imaging, indication for surgery, co-morbidities, and previous surgery. Patient anonymity is maintained. Item 3: Position of patient, access ports, extraction site and surgical team. Item 4: The surgical procedure is presented in a standardised step-by-step fashion. Item 5: The intra-operative findings are clearly demonstrated, with constant reference to the anatomy. Item 6: Relevant outcomes of the procedure are presented, including operating time, post-operative morbidity and histology when appropriate. Item 7: Additional graphic aid is included, such as diagrams, snapshots and photographs, to demonstrate anatomical landmarks, relevant or unexpected finding, or to present additional educational content. Item 8: Audio or written commentary in English language is provided. Item 9: The image quality is appropriate, with a constant clear view of the operating field. The video is fluent with appropriate speed.
* Peritonsillar abscess drainage videos, n = 9. LAP-VEGaS = Laparoscopic Surgery Video Educational Guidelines; SD = standard deviation
For FNE videos, there was a high positive correlation between video length and total Laparoscopic Surgery Video Educational Guidelines score (r = 0.62; p = 0.01). For peritonsillar abscess drainage videos, there was a very high negative correlation between video age and total Laparoscopic Surgery Video Educational Guidelines score (r = −0.81; p = 0.01). The full results from the Pearson's correlation analysis between video characteristics and total quality scores are provided in Table 4.
Video characteristics associated with overall video quality
*n = 18; †n = 9. ‡Indicates significant p-value
Discussion
The importance of YouTube as a platform for surgical education is likely to rise in light of the Covid-19 pandemic causing disruptions to elective care and medical teaching.Reference Al-Jabir, Kerwan, Nicola, Alsafi, Khan and Sohrabi13 Therefore, it is imperative that videos available on platforms such as YouTube are assessed by the surgical community before approval or recommendation for use in surgical education. Without verification or peer review of published videos by suitably qualified experts, there is a danger that new ENT doctors may refer to low-quality material purveying incorrect or potentially dangerous procedural techniques, putting patient safety at risk.
The Laparoscopic Surgery Video Educational Guidelines scores showed that 18 (67 per cent) of the 27 videos were of medium quality and only 7 (26 per cent) were of high quality. Previous studies performing Laparoscopic Surgery Video Educational Guidelines assessment of neck dissection, thyroidectomy and parathyroidectomy YouTube videos reported similar findings, with the majority being of medium quality.Reference Luu, Yver, Douglas, Tasche, Thakkar and Rajasekaran6,Reference Chorath, Luu, Douglas, Yver, Thakkar and Tasche14 We also analysed the relationship between the characteristics of each video and its educational quality. Traditional markers of popularity, such as total view count, number of comments and number of likes, were not predictors of high-quality videos. For videos relating to FNE, we found that those of a longer duration were scored as higher quality. For videos relating to peritonsillar abscess drainage, we found that videos published on the platform more recently were scored as higher quality. In comparison, Chorath et al. noted a low correlation between overall quality and dislikes or likes of thyroidectomy and parathyroidectomy videos, which emphasises the difficulty of outlining educational metrics from specific video characteristics.Reference Chorath, Luu, Douglas, Yver, Thakkar and Tasche14 The search algorithms used to populate ‘relevant’ videos on YouTube are unknown, but highlight the dissonance between targeting viewership and ensuring educational significance. Therefore, considering the growing popularity of YouTube as an educational tool, a greater effort should be undertaken to understand the search algorithms applied by the platform.
One significant limitation of YouTube is that videos are freely uploaded without ethical approval, and with no evaluation of the video author or the content. Video authors range from surgical instrumentation companies, marketing their products, to surgeons promoting their practice. Therefore, the content available online may be promoted for alternative purposes to education, such as patient awareness, marketing or entertainment.Reference Luu, Yver, Douglas, Tasche, Thakkar and Rajasekaran6 Learners can provide feedback through the comments or likes feature, although this is optional and does not constitute rigorous peer review. As a result, learners must independently appraise the educational value of online videos. Another limitation is the difficulty of optimally recording operations that are performed within a narrow field of vision. This was a common theme for peritonsillar abscess drainage videos, which consist of tiny incisions in a small space with suboptimal lighting.Reference Nair, Kamal, Dave, Mishra, Reddy and Della Rocca15
There are also limitations to using the Laparoscopic Surgery Video Educational Guidelines assessment tool, which was primarily created for assessing the suitability of videos being submitted to conferences and academic presentations. Most YouTube videos provided author information, indication for procedure, step-by-step process, and audio or written commentary. The remaining criteria were less relevant to FNE and peritonsillar abscess procedures, which meant they are unlikely to score full marks. For example, both procedures are frequently performed without pre-operative investigations, and similarly, the outcomes from either procedure, such as operating time or post-operative morbidity, are not commonly measured. Furthermore, the checklist was designed for English-speaking learners, which means videos narrated in an alternative language are penalised. One important criterion that was regularly overlooked is patient positioning, which is pivotal to achieving the optimal field of vision for peritonsillar abscess drainage.
Most UK medical students receive less than two weeks’ ENT exposure and therefore have little experience of the specialty upon graduation.Reference Mace and Narula16 However, on joining an ENT department, the GMC requires doctors to undergo an induction programme, which teaches the practical and procedural skills required for the emergency patient take. A nationwide survey of first on-call ENT doctors reported that many struggled to perform ENT procedures independently and found being on-call a stressful experience.Reference Biswas, Rafferty and Jassar3 Consequently, there have been suggestions to improve the ENT induction programme, most notably a national ENT boot camp to mitigate the regional variation in standards.Reference Smith, Trinidade and Tysome17 Surgical educational videos may serve as a useful adjunct, and research has proven their direct educational value in training.Reference Hayden, Seagull and Reddy18 For example, Russell et al. showed that trainees had a better understanding of FNE after watching a narrated educational video of the procedure.Reference Russell, Brook, Platt, Grillone, Aliphas and Noordzij19 Considering that video-based learning can be accessed from any location, it could provide a more economical and effective method for delivering an induction programme.
There is already an online ENT induction programme for junior doctors provided by ENT UK, a professional membership body that represents ENT surgery in the UK. The organisation has developed the resource over several years, and it is freely accessible. The website contains a catalogue of videos that reflect the core knowledge and practical skills required for a junior doctor starting an ENT placement.20 There are teaching videos for FNE but not for peritonsillar abscess drainage. Other platforms for surgical video-based learning include online video publications such as: The Journal of Visualized Surgery, The Journal of Medical Insight, The Journal of Visualized Experiments and CSurgeries.Reference Torre, Joe and Lee21 We propose that all new surgical education videos teaching FNE and peritonsillar abscess drainage should be developed with reference to the Laparoscopic Surgery Video Educational Guidelines checklist and be offered to peer review, before being uploaded onto online platforms for dissemination.
• Most junior doctors starting a new placement in ENT attend an induction programme, which includes learning flexible nasendoscopy (FNE) and peritonsillar abscess drainage
• Currently, this induction is not standardised; however, ENT UK have created an online resource bank with videos teaching different procedures
• Another learning resource is YouTube, which has become increasingly popular since the coronavirus disease 2019 pandemic
• The Laparoscopic Surgery Video Educational Guidelines video assessment tool has been used to evaluate the educational quality of surgical videos uploaded to YouTube
• This assessment tool was applied to YouTube videos demonstrating FNE and peritonsillar abscess drainage; our analysis showed limited high educational quality videos
• YouTube may provide a useful adjunct for ENT induction, and can be accessed remotely
Conclusion
Doctors in post-graduate training are increasingly referring to online videos as their preferred means of learning surgical procedures. This is the first study to critically evaluate the educational quality of YouTube videos demonstrating FNE and peritonsillar abscess drainage. Overall, the educational quality of videos varies significantly, and this is likely attributable to the lack of content peer review on the platform. Looking towards the future, the Covid-19 pandemic has manifested an urgent need for online platforms that contain a library of high-quality educational videos, with the aim of supporting or, in part, replacing traditionally didactic elements of ENT induction.
Competing interests
None declared
