Introduction
Video-based learning has become an integral part of surgical training. Evidence shows that the inclusion of scholastic videos into surgical education increases surgical proficiency.Reference Mao, Teichroeb, Lee, Wong, Pang and Pleass1,Reference Mota, Carvalho, Carvalho-Dias, Joao Costa, Correia-Pinto and Lima2 The field of otorhinolaryngology offers an ideal setting for the implementation of surgical videos as an educational resource. Many otorhinolaryngology procedures are performed with technological devices that are capable of video recording, and intra-operative video recording is almost a routine procedure for endoscopic and microscopic surgical procedures.
One of the major considerations in using surgical videos as an educational tool is the data quality. The most preferred video-based platform among the surgeons is YouTube, and anyone can upload videos to that platform without a peer review.Reference Rapp, Healy, Charlton, Keith, Rosenbaum and Kapadia3 This results in a large amount of video data without quality control. Several studies have assessed the informative quality of videos on YouTube for patients, but the evaluation of videos for professionals is limited.Reference Ward, Bavier, Warren, Yan and Paskhover4–Reference Malik, Heywood, O'Connor, Baker, Marshall and Beasley6 Luu et al. evaluated the educational value of neck dissection videos, Chorath et al. applied the same methodology for thyroidectomy and parathyroidectomy surgical procedures and Wu et al. evaluated the cholesteatoma surgical YouTube videos.Reference Luu, Yver, Douglas, Tasche, Thakkar and Rajasekaran7–Reference Wu, Knoll, Bouhadjer, Remenschneider and Kozin9 They all reported that most of the video content was insufficient, and efforts should be directed towards increasing the education quality.
To increase the quality of video-based education, video content must be prepared according to a quality assessment tool. The mentioned studies used the Laparoscopic Surgery Video Educational Guidelines (LAP-VEGaS) to appraise the educational quality.Reference Celentano, Smart, McGrath, Cahill, Spinelli and Obermair10 This guideline was designed to assess the laparoscopic surgery videos. Therefore, some elements may be less relevant to otorhinolaryngological surgical procedures and may not include organ-specific recommendations. To achieve an otorhinolaryngology-specific methodology for video quality, YO-IFOS published a consensus paper. An international consensus committee agreed on 28 propositions, thus establishing the “Instructional Videos in Otorhinolaryngology by YO-IFOS (IVORY) guideline” for educational surgery videos in otorhinolaryngology.Reference Simon, Peer, Michel, Bruce, Cherkes and Denoyelle11 Based on this guideline, Mayer et al. introduced a grading system for educational videos and applied it to parotidectomy videos on YouTube.Reference Mayer, Isik, Nachtsheim, Wolber, Hansen and Grosheva12
In the presented study, we applied the IVORY grading system (IVORY-GS) to stapedotomy videos on YouTube. Stapedotomy is a highly skilled surgery among the otological procedures. The margin for error is very slim and requires a thorough understanding of the anatomy and surgical technique. Therefore, the quality of educational materials is important, and no study has evaluated the stapedotomy videos to date. Herein, this study aimed to analyse the quality of stapedotomy videos as an educational tool and to test the feasibility of the IVORY-GS in an otological procedure.
Materials and methods
The study was deemed exempt from review by the institutional ethics committee as it pertains to publicly accessible videos. We used the search terms “stapedectomy”, “stapedotomy” and “otosclerosis surgery” in YouTube's search engine in October 2023. Only videos that demonstrated stapedotomy surgical procedures were included. Cadaver dissections, patient interviews, doctor briefing sessions, animations and lectures were excluded. The following information was collected for each video: video length, upload date, view count and likes. Dislikes could not be counted because the dislike counts are no longer publicly available. Video publishers were determined as individuals or institutions, and the country of origin for the videos was noted. Additionally, the surgical approach, laser utilisation, audiological and radiological examination findings and the presence of complications were recorded.
The included videos were evaluated according to the IVORY-GS with some modifications.Reference Mayer, Isik, Nachtsheim, Wolber, Hansen and Grosheva12 Ethics, technical aspects, case presentation and surgical procedure sections were scored with the exclusion of the following four items:
Item 9: “When available, peer review of the video is recommended before publication, assessing the procedure (e.g., scientific validity, safety) as well as the quality of educational editing.” Because of the lack of a formalised peer review system for uploading videos to YouTube, we could not determine whether a video had been adequately peer reviewed or whether any peer review process had occurred.
Item 12: “Relevant preoperative workup should be shown. Imagery should be explained by arrows/overlays.” Pre-operative workups in stapes surgery were considered as audiological tests and radiography and, in the organ-specific section, these items were scored independently. We excluded item 12 to avoid duplication.
Item 15: “If an endoscope is used, angle, diameter, and length should be specified.” We noted whether an endoscope was used in the videos, but since the study set consisted of both endoscopic and microscopic surgical procedures, we did not include item 15 in the scoring system.
Item 17: “Relevant pathology shown during the film should be identified and named. A picture of the specimen (with ruler) may be included if applicable.” This item was not applicable, because there is no pathological specimen in almost all stapes surgical procedures.
The organ-specific section was redesigned for stapedectomy. Two recommendations specific to otology in the IVORY guidelines were modified and scored. Additionally, two items listed below were added:
Item 19: “It is recommended to specify the device used in a footplate perforation procedure.”
Item 20: “It is recommended to specify the piston prosthesis type.”
According to the Ivory guidelines, if the video met the requirements, it received 2 points; if it partially met the requirements, it received 1 point; and if it did not meet the requirements, it received 0 points. Our grading system consisted of 20 items with a maximum of 40 points (Appendix). The total score was calculated in five categories from A to F following the USA academic grading system.13
Two experienced otorhinolaryngologists (S.Y., four years of specialist; A.Ö., 11 years of specialist) independently rated the videos according to the prepared guidelines. In case of discrepancies in scores, a consensus was reached after discussion.
Results and analysis
Characteristics of the videos
Our search yielded 67 stapes surgery videos. The videos were uploaded between 2010 and 2023; the newest video was uploaded 3 months ago, and the oldest video was uploaded 13 years ago. The mean video length was 10.7 minutes, ranging from 1 minute to 1:17 hours. Twelve of the videos were uploaded by an institution, and the remaining ones were posted by individuals. The surgical approach was trans canal in 62 videos and endaural in five videos. A surgical microscope was used in 31 cases, and 33 cases were performed under endoscopic view. Both devices were used in two cases. Twenty-four surgical procedures were laser assisted. Video characteristics are listed in Table 1.
Descripted data for the stapedotomy videos
The country of origin of videos was listed as Turkey (33%), USA (31%) and India (8%), respectively. The distribution of videos according to countries is shown in Figure 1.
The distribution of videos according to countries
IVORY grading system
The maximum achievable score in the grading system was 44 points. The mean IVORY-GS total score was 21.8 (±6.7). The highest total score was 39 and the lowest was 10. Table 2 shows the distribution of total scores according to the US academic grading system from A to F.
Distribution of stapedotomy videos according to grades and total IVORY grading system scores
The total score showed a significantly positive correlation with like counts. No significant correlation was found between the total score and view count. Video age also was not correlated with IVORY-GS scores. A negative correlation was revealed between the total score and the video length, but it was statistically non-significant (Table 3).
Correlation between IVORY grading system scores and video metrics
Video scores were higher in the ethics part followed by technical aspects, organ-specific part and surgical procedures. Case presentation items got the lowest scores in total. The score disturbance according to the items is given in Table 4.
Score disturbance according to the items
Statistical analysis
Statistical Package for the Social Sciences (SPSS) software version 26 was used for statistical analysis. The distribution of data was tested using the Shapiro–Wilk test. Spearman's correlation coefficient was performed to measure the strength of correlation between non-normally distributed metric variables. The Mann–Whitney U test was used to compare differences between independent groups for metric, non-normally distributed variables. The results were evaluated at the 95% confidence interval, and the significance level was p < 0.05.
Discussion
In this study, we have evaluated for the first time the educational quality of stapes surgery videos published on YouTube. YouTube is not an official education platform, but frequently is used as an educational video source among trainees.Reference Shabli, Heuermann, Leffers, Kriesche, Abrams and Yilmaz14 Considering this, the educational quality of YouTube videos becomes important for surgery education. In the otorhinolaryngology field, neck dissection, parotidectomy, cholesteatoma surgical procedures, thyroid and parathyroid surgical procedures and endoscopic endonasal approaches are the surgical videos previously analysed for their educational quality.Reference Luu, Yver, Douglas, Tasche, Thakkar and Rajasekaran7–Reference Wu, Knoll, Bouhadjer, Remenschneider and Kozin9,Reference Mayer, Isik, Nachtsheim, Wolber, Hansen and Grosheva12,Reference De La Torre, Joe and Lee15 Findings from all these studies revealed that videos have low educational standards. The result for stapes surgery videos is not different from the previous studies. Among the 67 stapes surgery videos, only 14 videos showed higher scores, which refers to medium to high educational grade; the remaining videos got the lower grades. These results reinforce concerns about the educational quality of surgical videos on YouTube and highlight the need for guidelines on improving video quality.
One of the most used guidelines for video evaluation is the LAP-VEGaS classification. It was published to provide consensus recommendations on how to prepare a laparoscopic surgical video for educational purposes. Although it was designed for laparoscopic surgical procedures, it has found widespread use in several studies.Reference Chorath, Luu, Douglas, Yver, Thakkar and Tasche8,Reference Wu, Knoll, Bouhadjer, Remenschneider and Kozin9,Reference De La Torre, Joe and Lee15,Reference Sunba, Levin, Wu and Campisi16 Recently, the IVORY guideline was published to establish consensus recommendations for the surgical videos in otorhinolaryngology, and Mayer et al. modified the recommendations into a grading system.Reference Mayer, Isik, Nachtsheim, Wolber, Hansen and Grosheva12 In our study, we chose to analyse the videos according to the IVORY-GS due to its specificity to the otorhinolaryngology field. The IVORY-GS evaluates the videos from five main aspects: ethics, technical aspects, case presentation, surgical procedure and organ-specific parts. In the study, the lowest mean scores were obtained in the case presentation part, which included recommendations about the title page and the medical history. Relevant medical history, age, sex, ear side, institution, surgeon name and country were not mentioned in most of the videos, which resulted in lower scores. These are important details for a comprehensive presentation but may be less beneficial for surgical technique training. The IVORY-GS ranks each recommendation with the same score. Therefore, a revision may be needed to prioritise the surgical procedure. Another suggestion we have for the guideline is regarding the organ-specific part. The original IVORY guideline makes two recommendations for otological surgical procedures; one is for pre-operative audiometry and imaging and the other for the type of approach. As stapes surgery represents a complex surgical procedure, these recommendations may not be comprehensive enough. We needed to add recommendations to represent the crucial aspects of the surgery. In the future, the recommendations in this section can be detailed according to the surgical procedure.
Among all the recommendations, items 16 and 17 in the organ-specific section, received the lowest score. Pre-operative CT scans and pre- and post-operative audiograms were rarely mentioned in the evaluated videos. Videos were mainly focused on the surgical steps; pre-operative and post-operative work-ups were the missing parts. Therefore, we advise the authors to pay particular attention to these parts in future videos.
The analysis of the video characteristics demonstrated a significant positive correlation between the total IVORY-GS score and the like counts, a finding similar to previous studies in the literature.Reference Chorath, Luu, Douglas, Yver, Thakkar and Tasche8,Reference Mayer, Isik, Nachtsheim, Wolber, Hansen and Grosheva12 This result indicates that YouTube viewers showed higher admiration to videos that have higher educational quality. But interestingly, view counts did not correlate with IVORY-GS scores. The possible reason for this result may be the YouTube's video publishing algorithms. The YouTube algorithm makes recommendations for each user according to their watch metrics and content relevance. Therefore, rather than educational quality, the medical background of viewers and other personalised metrics may influence the popularity of a video.
• Most YouTube videos demonstrating stapedotomy surgical procedures are insufficient.
• In most of the videos, there is a deficiency in the content related to the pre-operative and post-operative work-ups.
• The IVORY grading system is a comprehensive, suitable guideline to prepare high-quality videos in the otorhinolaryngology field.
Our study has several limitations. First, despite the comprehensive search of the platform, videos are being constantly uploaded and removed, and it makes the study data not repeatable. Second, YouTube is not an official education platform, and there is no obligation for the content on YouTube to be educational. Authors can upload their videos for several other objectives, such as marketing, entertainment or simply for archiving. In this study, we evaluated the educational quality of videos that were not uploaded for educational purposes. Moreover, it is quite possible that the authors were not aware of the IVORY recommendations when uploading their videos, because it was published recently. This means, we evaluated authors based on criteria of which they were unaware. However, considering the popularity of YouTube among trainees, we can conclude that improving the quality of videos on YouTube will help improve the quality of surgical education.
Conclusion
This is the first study to evaluate the educational quality of stapedotomy videos on YouTube. Despite YouTube's popularity as a learning tool, the educational quality of most of the videos was insufficient. Pre-operative and post-operative work-ups were not included enough in the videos. Future efforts should focus on improving the educational quality of videos using an objective quality assessment. We found that the IVORY-GS is a comprehensive, suitable guideline for authors to prepare high-quality videos in the otorhinolaryngology field. However, the grading system may require modifications based on the type of surgery.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0022215124001014.
Funding
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Competing interests
The authors declare none.
