Introduction
Thyroidectomy and parathyroidectomy are common operations that are typically performed by otolaryngologists, surgical oncologists, endocrine surgeons and general surgeons alike.Reference Haugen1–Reference Miccoli and Bakkar3 Although these procedures are generally viewed as safe, they require a thorough understanding of the anatomy and meticulous surgical technique. The margin for error is very slim, which makes the operating room a less than ideal place for the initial development of these skills.
Studies have shown that adult learning is optimised when the trainees are actively engaged both physically and mentally.Reference Sullivan4–Reference Kieu, Stroud, Huang, Smith, Spychal and Hunter-Smith6 In the case of surgery, this is best achieved if the learners enter the operation with a baseline understanding of the anatomy and the steps of the procedure. As a result, web-based platforms have evolved as important tools in helping residents and students build their medical knowledge and procedural familiarity.Reference Pilieci, Salim, Heffernan, Itani and Khadaroo7–Reference Mota, Carvalho, Carvalho-Dias, João Costa, Correia-Pinto and Lima9
A recent study from the University of Iowa reported that YouTube was the most common and preferred resource to prepare for surgery.Reference Rapp, Healy, Charlton, Keith, Rosenbaum and Kapadia10 These videos are useful and effective training tools because of their ability to display the surgeon's perspective, identify major steps and identify critical anatomy. However, several reports have also highlighted the inherent limitations of preparing for surgery with YouTube.Reference Derakhshan, Lee, Bhama, Barbarite and Shaye11–Reference Chapman, Weaver, Sheikh, MacCormick and Poole14 Because it is an open access platform, anyone with an account can post content with little oversight over the quality of material. Furthermore, the YouTube algorithm utilises metrics including view counts, ‘likes’ and comments to determine popularity, which does not necessarily translate to quality and veracity.Reference Koya, Bhatia, Hsu and Bhatia15 We thus sought to critically evaluate the educational value of these videos to prepare for thyroidectomy and parathyroidectomy surgery.
Materials and methods
Video selection
The top 10 videos by relevance and view count on YouTube's video sharing platform (Alphabet, Mountain View, CA, USA) were identified using the following search terms on 1 September 2020: thyroidectomy, total thyroidectomy, subtotal thyroidectomy, hemithyroidectomy, thyroid gland surgery, partial thyroidectomy, thyroid lobectomy, hemithyroidectomy with isthmusectomy, parathyroidectomy, parathyroid gland surgery and parathyroid adenoma removal.
Exclusion criteria comprised videos that demonstrated patient testimonials, surgical animations and cadaveric dissection. Videos that employed a robotic, endoscopic or non-cervical approach for these surgical procedures were also excluded.
The following characteristics were collected for each video: title, uniform resource locator (‘URL’), publication date, view count, video length, likes, dislikes, surgeon specialty (otolaryngology or general surgery trained), country of practice, US-board certification of surgeon and presence of industry involvement. The specialty of the surgeon was determined by Google search engine inquiry of the surgeon's name. Academic or private practice affiliation was determined by their practice website.
Assessment of video quality
Included videos were independently evaluated by two fellowship-trained head and neck surgeons and two otolaryngology residents. Each video was graded according to the Laparoscopic Surgery Video Educational Guidelines scoring checklist (Table 1).Reference Celentano, Smart, Cahill, Spinelli, Giglio and McGrath16,Reference Celentano, Smart, McGrath, Cahill, Spinelli and Obermair17 The Laparoscopic Surgery Video Educational Guidelines checklist is a validated, peer-reviewed video assessment tool developed by a multispecialty committee with the goal of appraising surgical videos submitted for presentation and publication. This tool has been used to evaluate the educational value of YouTube videos depicting several operations including neck dissection, sleeve gastrectomy and laparoscopic pyeloplasty.Reference Chapman, Weaver, Sheikh, MacCormick and Poole14,Reference Luu, Yver, Douglas, Tasche, Thakkar and Rajasekaran18,Reference Haslam and Seideman19 It includes nine criteria graded from 0 (item not present in video) to 2 (item extensively presented in video). The final scores were tallied and categorised according to low, medium or high quality (low: 0–6, medium: 7–12, high: 13–18). Additionally, we wanted to focus on the audiovisual and technical quality of the surgical procedures themselves, so we performed a modified subgroup analysis according to items 3,4,5,8 and 9 and corresponding modified 10-point scale (low: 0–3, medium: 4–7 and high: 8–10).
Components of Laparoscopic Surgery Video Educational Guidelines tool
Statistical analysis
Statistical analysis was performed using Microsoft Excel® spreadsheet software. Inter-rater reliability between residents and attending surgeons was calculated according to Cohen's kappa coefficient. Two sample t-test and linear regression analysis were performed to determine the video characteristics that were associated with improved video quality for both the total and subgroup scores.
Results
Video characteristics
Our search yielded a total of 65 videos including 48 for thyroidectomy and 17 for parathyroidectomy. The characteristics for each video are reported in Table 2 and summarised in Table 3. Among all videos, there were 4 590 468 views, with a range of 196 to 1 102 629. The average age of videos was 5.44 years, with a range from 0.47 years to 11.22 years. The videos averaged 200 likes (range, 0–2013) and 26 dislikes (range, 0–530). The mean length of the videos was 12.7 minutes; the shortest video was 2.78 minutes and the longest was more than 1 hour. Overall, 15 videos (23 per cent) were made in the USA and 29 (45 per cent) were made in Asia. The remaining videos were made in Europe, South America, Australia and New Zealand, and Africa. The videos were produced predominately by surgical oncologists (30 videos, 46 per cent). Five videos were industry sponsored. Only 38 videos (58 per cent) had audio narration, and 15 videos (31 per cent) had associated subtitles.
Video characteristics of the 65 selected videos
MIHLAT = minimally invasive headlight–loupe assisted thyroidectomy
Summarised characteristics for selected videos
SD = standard deviation
Quality of videos
The comprehensive assessment and subgroup analysis using Laparoscopic Surgery Video Educational Guidelines scores are shown in Table 3. The mean (± standard deviation) comprehensive Laparoscopic Surgery Video Educational Guidelines score was 8.58 ± 3.85, with a mean modified score of 5.67 ± 2.40. The majority of videos (43 per cent) were deemed medium quality, with 39 per cent of videos classified as low quality and 18 per cent classified as high quality (see Table 1 in the supplementary material, available on The Journal of Laryngology & Otology website). There was no difference in video quality between thyroidectomy and parathyroidectomy videos (p > 0.05). Inter-rater reliability was good for both attending surgeons (Cohen's kappa coefficient of 0.42) and residents (Cohen's kappa coefficient of 0.47).
Assessment of video features
An assessment of factors associated with overall video quality is shown in Table 4. Briefly, among the total and subgroup analysis, there was improved video quality for videos made by US and non-US based physicians (p < 0.01 and p < 0.01, respectively). Additionally, the videos produced by otolaryngologists scored 3.38 points higher (95 per cent confidence interval (CI): 1.41 to 5.36; p < 0.01) on total Laparoscopic Surgery Video Educational Guidelines scores and 2.28 points higher (95 per cent CI: 1.25 to 3.32; p < 0.01) on modified Laparoscopic Surgery Video Educational Guidelines scores. There was a mild positive correlation for video age (R2 = 0.08) and like to dislike ratio (R2 = 0.05) on total Laparoscopic Surgery Video Educational Guidelines scores but no correlation with view count (R2 = −0.01). The presence of audiovisual commentary was associated with a positive correlation on total Laparoscopic Surgery Video Educational Guidelines scores (R2 = 0.38).
Factors associated with overall video quality
*Statistically significant; †characteristics only assessed in thyroid videos
Discussion
In this study, we assessed the educational value and overall quality of YouTube videos on thyroidectomy and parathyroidectomy. Among the large number of videos on these procedures, we identified 65 videos with greater than 4.5 million views in total from academic medical centres and private practices across the world. The popularity of YouTube as a source of surgical education is increasing, and video-based education may be even more important during the coronavirus 2019 pandemic considering that real-time operative experiences for elective cases may be diminished.Reference Coe, Jogerst, Sell, Cassidy, Eurboonyanun and Gee20–Reference Chick, Clifton, Peace, Propper, Hale and Alseidi22 As surgical trainees continue to use YouTube videos as a tool for surgical preparation, it is important for the surgical community to critically evaluate this open-access resource for its educational value.
Using the Laparoscopic Surgery Video Educational Guidelines scoring system, we noted that there was a range in video quality. Of the 65 videos, 28 were deemed to be medium quality and 25 to be low quality. Only 18 per cent (12 videos) were considered to be high quality. There was no difference in quality for videos presenting thyroidectomy and parathyroidectomy. The heterogeneity seen with this analysis is consistent with prior publications evaluating YouTube videos of surgical operations. In one study reviewing YouTube videos for neck dissections, the authors noted that only 8 per cent of videos were considered high quality.Reference Luu, Yver, Douglas, Tasche, Thakkar and Rajasekaran18 Similarly, 62 per cent of instructional videos for facelift surgery failed to describe key tenets of a successful operation, including identification of facial nerve or appropriate facelift incisions.Reference Derakhshan, Lee, Bhama, Barbarite and Shaye11 These shortcomings reinforce the concerns about the educational quality of surgical videos on YouTube and the need for trainees to implement discretion when using this resource.
• Most YouTube videos demonstrating thyroidectomy and parathyroidectomy operations are insufficient and considerably variable in educational quality
• Videos produced by otolaryngologists and US-based physicians scored higher than those from non-otolaryngology and non-US based physicians
• The presence of audiovisual commentary is correlated with higher educational value
We also sought to identify specific video characteristics that could influence the overall quality of these videos. Our analysis showed that narrated audio and annotated subtitles were moderately positively correlated with Laparoscopic Surgery Video Educational Guidelines scores. Interestingly, likes to dislikes ratio had only a mild correlation, and characteristics such as total view counts and age did not affect the overall quality of videos. Additionally, the YouTube search results were not organised by educational quality, but rather based on metrics of social media attention. These findings are in line with several other studies assessing surgical videos on YouTube and highlight the apparent discordance between a video's popularity and educational merit.Reference de'Angelis, Gavriilidis, Martínez-Pérez, Genova, Notarnicola and Reitano13,Reference Lee, Seo and Hong23 Indeed, for students still interested in augmenting their learning with YouTube, one consideration would be to identify those videos that are targeted, use audiovisual elements and highlight important concepts because these factors have been shown to improve learning and maximise benefit from educational videos.Reference Brame24
Quite surprisingly, our analysis demonstrated that videos uploaded by otolaryngologists and US-based physicians outperformed videos produced by non-otolaryngology trained and non-US physicians. However, this finding must be viewed with caution and may reflect differing practice trends for thyroid and parathyroid surgery across the world. For example, in the USA, many otolaryngologists perform these operations, whereas in most European and Asian countries, these procedures continue to be performed largely by general surgeons.Reference Zbären, Shah and Randolph2 In the end, we believe that surgical expertise should not be based upon educational background, but rather upon training, skill and experience and that any surgeon with the knowledge and resources to record these operations should be capable of making high-quality educational videos.
YouTube is not an accredited medical resource; therefore, any individual or organisation is capable of uploading content. Because of this, one of the clear challenges for trainees using this resource to supplement their education is the lack of transparent peer review and quality control.Reference Frongia, Mehrabi, Fonouni, Rennert, Golriz and Günther25 Particularly for thyroidectomy videos, several of the operations were performed by private practice surgeons with the videos aimed at marketing their surgical practice. Furthermore, some videos were sponsored by surgical instrumentation companies promoting their products. Notably, this analysis showed that there was no difference in educational quality between industry-sponsored and non-industry sponsored videos. These results are in contrast with studies evaluating YouTube videos for laparoscopic fundoplication, which showed that industrial-source videos were of higher educational value secondary to enhanced recording equipment and post-processing software.Reference Frongia, Mehrabi, Fonouni, Rennert, Golriz and Günther25
There are several challenges when recording operations like thyroidectomy and parathyroidectomy because of the narrow field of dissection. Issues with visibility are common because of small incisions and operating fields, excess bleeding, multiple retractors and presence of shadows.Reference Nair, Kamal, Dave, Mishra, Reddy and Rocca26 Furthermore, these operations are dynamic and require constant repositioning and manoeuvring, which can be disorienting for individuals early in their training. These issues were reflected by the persistently low scores seen with both the Laparoscopic Surgery Video Educational Guidelines and modified scores. Some of the filming options utilised in these videos included endoscopes, microscopes and camcorders. However, we noticed that videos that attached a camera over the surgeon's heads exhibited the best quality. In fact, other studies have shown that using a GoPro® attached to the surgeon's head is an effective way to record other surgical procedures in the head and neck region because minute structures can be well visualised and the post-processing software allows the user to rotate the image if the surgeon tilts their head.Reference Nair, Kamal, Dave, Mishra, Reddy and Rocca26 Nevertheless, future studies are warranted to determine the ideal recording strategy for small, delicate surgical procedures like these.
There are several limitations to this study. The Laparoscopic Surgery Video Educational Guidelines instrument was initially designed to assess videos made for conferences and academic presentations; however, we have repurposed and adapted the tool to assess YouTube videos. Although the checklist does provide a standardised framework to validate and evaluate surgical videos, some aspects of the guidelines may not be as relevant or useful for certain procedures.Reference Chapman, Weaver, Sheikh, MacCormick and Poole14 In this study for instance, ultrasound is the most common pre-operative imaging modality, which is often not included in YouTube videos as they are difficult to interpret and incorporate. The scoring system penalises videos that fail to include pre-operative imaging. To address this concern, we performed a modified subgroup analysis to focus on the specific audiovisual and technical aspects of the procedures. The videos included in this analysis were not limited to the English language; this is a potential source of bias since one of the components of the Laparoscopic Surgery Video Educational Guidelines tool is ‘audio/written commentary in English language is provided’. Additionally, several videos were sponsored by surgical instrumentation companies attempting to promote their product, which may make it difficult to compare to videos produced specifically with an educational intent using this tool. Although we did perform a comprehensive review of the platform, videos are constantly being uploaded and removed, so our search results may not be easily replicated. Furthermore, a surgeon's intention for uploading the videos may not have been for the purpose of education, which can skew the perceived appropriateness for the video's use by trainees as an educational tool. Lastly, our analysis consisted of videos only uploaded to YouTube, and videos from academic societies and privately hosted domains were not included. Videos from those sources may be of higher overall quality, but these resources may not be readily available to the public according to a country's browsing preference.
Conclusion
YouTube has become the preferred resource by trainees to learn and prepare for surgical cases. As such, this is the first study to critically evaluate the educational quality of thyroidectomy and parathyroidectomy videos from this online platform. Although some videos were considered to be of high quality, we demonstrated that most of the video content was insufficient and considerably variable in educational quality. Moving forward, we advise trainees to be cautious when using this open-access resource to supplement their surgical education. Future efforts should be directed towards increasing the number of high-quality videos on this platform or making available repositories of high-quality surgical videos that trainees can freely use, similar to those hosted by this journal on Vimeo (New York, USA) video hosting service.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0022215121004096.
Competing interests
None declared
