Introduction
Adenoidectomy, surgical removal of the adenoids (nasopharyngeal tonsils), is one of the most common surgical procedures in children in Western countries.Reference Johnston, Mahadevan and Douglas1,Reference van den Aardweg, Schilder, Herkert, Boonacker and Rovers2 There are a number of indications for this operation including fluid in the middle ear (otitis media with effusion) and sleep disordered breathing with or without obstructive sleep apnoea.Reference Ramos, Mukerji and Pine3
The need adequately to relay healthcare information to patients, highlighted again in the 2015 Montgomery versus Lanarkshire case, is greater than ever because of an ever-increasing complexity of healthcare and also the desire of our patients to be better informed. There is, however, limited understanding of this operation and its indication amongst patients and parents of paediatric patients; this is partly attributed to its concealed anatomical location.Reference Strocker and Shapiro4 Over the past decade, there has been an exponential growth in access to the internet with as many as 80 per cent of internet users using the internet to access health-related information.5–7 Up to 98 per cent of parents in Canada have reported using the internet to search for health information about their child.Reference Pehora, Gajaria, Stoute, Fracassa, Serebale-O'Sullivan and Matava8 The internet can be conveniently accessed and offers a perceived sense of anonymity and confidentiality when accessing information on sensitive topics.Reference Powell, Inglis, Ronnie and Large9
However, the internet is a non-regulated resource, and therefore it is difficult to control what information patients access and the quality, reliability and readability of this information.Reference Fahy, Hardikar, Fox and Mackay10–Reference Alsaiari, Joury, Aljuaid, Wazzan and Pines13 Patients can find it difficult to navigate and critically evaluate health information, with many being unable to distinguish between high- and low-quality information.Reference Norman and Skinner14,Reference Cline and Haynes15 Up to 85 per cent of people access online healthcare information by simple search of key words using search engines such as Google®, which has more than 92 per cent of the market share.Reference Pehora, Gajaria, Stoute, Fracassa, Serebale-O'Sullivan and Matava8,16,Reference Ybarra and Suman17
To date, there has been no study that has examined the quality of online information regarding adenoidectomy. The aim of this study was to assess the appropriateness of online information on the surgical procedure of adenoidectomy.
Materials and methods
The search term ‘adenoid’ was entered into Google search engine in July 2020. This term was selected as Google uses automatic stemming of the search word to include variations such as ‘adenoidectomy’ and ‘adenoid hypertrophy’. All websites in the English language were included. It has been shown that most internet users visit fewer than 25 websites from the search results,Reference Eysenbach and Kohler18 so to ensure a comprehensive search was conducted, the first 5 pages (50 results) were reviewed. Websites were excluded from further analysis if they were in non-written format, contained irrelevant information, represented duplication or were inaccessible.
The following information was retrieved for each website and recorded in a standardised proforma: authorship and type of media (medical, physician, academic, commercial, discussion, media and social media). Three validated and commonly used readability assessment tools were used to assess the readability of each website: The Flesch Reading Ease test, the Flesch–Kincaid grade level and the Gunning Fog Index. The Flesch Reading Ease test assigns a score of 0–100 for each website with higher score corresponding to being easier to read.Reference Flesch19 The Flesch–Kincaid grade level corresponds to the US reading grade level with worsening readability as Flesch–Kincaid grade level increases.Reference Flesch20 The Gunning Fog Index provides an estimate of the number of years of formal education required to be able to read and comprehend the text on the first reading.Reference Gunning21 This information is summarised in Table 1. Microsoft Word® word processing software was used to calculate the Flesch Reading Ease and Flesch–Kincaid grade level scores, and an online calculator (gunning-fog-index.com) was used to measure the Gunning Fog Index.22
Summary and interpretation of the readability and reliability tests
For each website, the quality of the content was assessed using three validated methods: the Discern instrument,Reference Charnock, Shepperd, Needham and Gann23 the Journal of the American Medical Association benchmark criteriaReference Silberg, Lundberg and Musacchio24 and the Health on the Net Foundation code certification. The Discern instrument is a 16-item standardised set of criteria for evaluating the quality of health information written for the public, such as online information on treatment choices.Reference Charnock and Shepperd25,Reference Weil, Bojanowski, Jamart, Gustin and Leveque26 Each website receives a score of 16 to 80, with a higher score indicating a higher quality and reliability. The Journal of the American Medical Association benchmark uses four criteria to examine websites: the authorship, source of the information, the date of update, and the disclosure of ownership or conflict of interest.Reference Silberg, Lundberg and Musacchio24 One mark is awarded for the satisfaction of each criterion. The Health on the Net foundation was founded in 1996 as a non-profit organisation that aims to protect the public from misleading health information.27 It does so by providing the Health on the Net Foundation code certificate to websites that meet quality and reliability standards. In this study, each website was examined for the presence of the Health on the Net Foundation code certificate.
For comparison, the online information on adenoidectomy from four International Otolaryngology Societies was evaluated: American Academy of Otolaryngology Head and Neck Surgery (AAO-HNS),28 the British Association of Otorhinolaryngology – Head and Neck Surgery (ENT UK),29 the Canadian Society of Otolaryngology Head and Neck Surgery,30 and the Royal Australasian College of Surgeons.31 These were analysed for both readability and reliability in the same fashion as those found using the Google search.
Based on the type of media, the websites were divided into two groups: group one (assumed to have a higher quality and reliability) were medical, physician and academic types and group two (assumed to have a higher risk of bias) were media, social media, commercial and discussion websites. Student t-tests and Mann–Whitney U tests were used to determine statistical differences between the groups based on the type of media (group 1 and group 2) and between those that displayed the Health on the Net Foundation code certification and those that did not. Statistical analysis was performed using Microsoft Excel® statistical software. The statistical significance was defined as a p-value less than 0.05.
Results
Following review of each website, 9 websites were excluded from further analysis (6 had irrelevant information, 1 was inaccessible, 1 was a non-written format and 1 was the duplicate of a website already included) leaving a total of 41 websites for analysis. None of the websites included in the analysis were New Zealand based websites (where the search was conducted). The year of publication or update was present on 31 websites (75.6 per cent) with the median year of publication or update being 2018 (range, 1997–2019). This included 22 medical, 7 commercial, 4 physician, 4 academic, 2 media, 1 social media and 1 discussion website.
The mean Flesch Reading Ease score was 48.1 (± standard deviation (SD), 19.0) corresponding to the ‘difficult to read’ category. The mean Flesch–Kincaid grade level was 10.8 (± SD, 3.3) with 40 (97.6 per cent) websites scoring above the recommended level of 6. The mean Gunning Fog Index was 13.7 (± SD, 3.1) corresponding to university reading level.
The mean Discern score was 43 (± SD, 11.8) which corresponds to the ‘fair’ category. Eleven websites (26.8 per cent) had poor quality (Discern score of 27–38) and 5 websites (12.2 per cent) had very poor-quality content (Discern score of 15–26). The median Journal of the American Medical Association benchmark criteria score was 2, with 10 websites (24.4 per cent) scoring only 1 out of 4, and 7 websites (17.1 per cent) gaining the highest mark of 4 out of 4. Of the included websites, 8 (19.5 per cent) displayed the Health on the Net Foundation code certification.
When comparing the group of websites that displayed the Health on the Net Foundation code certification and those that did not, no statistically significant difference was observed between the two groups in any of the recorded variables (shown in Table 2). Similarly, no statistically significant difference existed between any of the variables recorded when comparing the presumptive low risk of bias websites (group 1) with those websites with presumptive higher risk of bias (group 2).
Comparison of readability and quality variables of the websites included in the study* based on display of Health on the Net Foundation code certificate
* n = 41; †n = 8; ‡n = 33. SD = standard deviation
The overview of readability of quality scores from AAO-HNS, ENT UK, the Royal Australasian College of Surgeons and the Canadian Society of Otolaryngology Head and Neck Surgery is outlined in Table 3. None of these websites displayed the Health on the Net Foundation code certification. Although comparative statistics were not used, the ENT UK resource appears to represent the best of the studied resources for the parameters considered.
Overview of the patient information from International Otolaryngology Societies
Discussion
Despite adenoidectomy being a common operation in otolaryngology, its indications, technical details and post-operative care are not well understood amongst patients. Therefore, many patients and parents are likely to depend on online information as a major source of their information. This study has found the readability and quality of online information on adenoidectomy to be largely inadequate. Patient information provided by the surgical training bodies examined, with the exception of ENT UK, also had less than desired readability and quality scores.
This study found that most of the websites analysed were difficult to read, with the Flesch–Kincaid grade level being above 10th grade. Integral to health literacy is the individual's ability to read, understand and use the information gained to make decisions, consent and follow instructions for treatment. Regardless of the quality of the patient information, if it is not readable for the majority of the targeted audience, then it has failed to inform. The American Medical Association recommends that the readability of patient information material should be no greater than sixth grade reading level.Reference Weiss32 Readability improves when the information is written at or below the individual's reading level, particularly when the reader's interest in the topic is low.33 Another method for improving readability is use of illustrations which, when used correctly, can exceed language and numeracy barriers.Reference Pratt and Searles34 Other simple measures that can help improve the readability of a document include using simpler terms and shorter sentences.Reference Weiss32,Reference Eltorai, Han, Truntzer and Daniels35 However, there is a note of caution: oversimplifying may make it difficult to convey accurate information.Reference Weiss32 This highlights the need for striking the right balance between simple information and accurate information.
The mean Discern instrument score for the websites examined was 43, corresponding to fair quality with only 12 websites (29 per cent) with good or excellent quality material. Similarly, only 7 websites (17 per cent) fulfilled all 4 Journal of the American Medical Association score criteria. When comparing websites that displayed the Health on the Net Foundation code and those that did not, we found no difference between the Discern instrument scores or Journal of the American Medical Association scores (shown in Table 2). Similarly, when comparing media group 1 (assumed to have higher quality) with media group 2, we found no statistically significant difference in Discern instrument score or Journal of the American Medical Association score. In other words, even if patients screen multiple websites and only focus on websites which are assumed to have a lower risk of bias (e.g. medical websites), the quality of the information does not change. Fortunately, healthcare professionals remain the primary and the most trusted source of information for patients.36,37 This places the healthcare professional in a privileged position to educate on the health concern at hand. Our results highlight that it is imperative for healthcare professionals to guide adenoidectomy patients and their families to high quality online information. Based on the results of this study, the quality of online information, relating to what is a common surgical procedure, is inadequate.
ENT UK was found to have the most readable patient information on adenoidectomy amongst the International Otolaryngology Societies examined (shown in Table 3). This website also appeared in the first 50 results from the Google search. In this study, the ENT UK website was amongst the top three most readable websites across all the three assessment tools examined. In addition, the information from ENT UK and Royal Australasian College of Surgeons was considered to be of good quality based on their Discern scores. The information from the Canadian Society of Otolaryngology Head and Neck Surgery was found to have the lowest readability scores and also lowest Discern instrument score. Although the quality of the information provided by the International Otolaryngology Societies was largely of fair or good standard, with the exception of the online information from ENT UK, they were difficult to read.
• Adenoidectomy is one of the most common surgical procedures in children in Western countries
• There is limited understanding of this operation amongst patients and parents
• This is the first study to assess the readability and reliability of online information about adenoidectomy
• The readability and quality of online patient information on adenoidectomy is inadequate
• Patients are vulnerable to making decisions based on misinformation or poorly understood information
• Surgeons should help direct patients to information that is both readable and reliable
This study has some limitations. The number of websites examined was relatively small, raising the possibility of a type II error. However, even if there is a difference between websites, this study suggests that this difference is so small that it has little to no real-world impact. The Google search engine was used in this study as it has been reported as the dominant search engine. However, other search engines may conceivably have produced different results. Google also uses search algorithms based on search location; however, it is noted that no local websites were retrieved in our search. This study examined English language websites, with English being a language spoken by an estimated third of the world's population. The results presented here may not be representative of all the online health information on adenoidectomy.Reference Crystal38 In this study, we used three formulas to assess readability. However, none of these formulas directly measure comprehension.Reference Kauchak and Leroy39 The authors also acknowledge that there are many factors that influence patients' ability to read and comprehend health information, such as formal education, socioeconomic status, language barriers, intellectual ability and cultural beliefs. Finally, the result of this study reflects the information available at the time of the Google search. However, the internet is always evolving with more information constantly being added and the existing information being regularly updated.
Conclusion
In conclusion, the majority of patients have and will continue to use the internet to access health-related information. However, the readability and quality of online patient information on adenoidectomy is inadequate. This makes our patients vulnerable to making decisions based on misinformation, poorly understood information or both. These data challenge surgeons to engage with patients' online reading and to recommend appropriate resources regarding adenoidectomy. In the longer-term, these data also challenge us as a specialty to improve the accessibility, readability and quality of resources that we make available to patients.
Acknowledgements
The authors are developing an open access internet resource for ENT patients with the assistance of a charitable donation from which no pecuniary benefit will be obtained. This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Competing interests
None declared
