Introduction
Glue ear, or otitis media with effusion, affects 20 per cent of two-year olds, decreasing to 8 per cent by age seven to eight years.Reference Simpson, Thomas, van der Linden, Macmillan, van der Wouden and Butler1 Children with hearing loss, speech delay, or high-risk conditions such as Down's syndrome or cleft palate, are referred to ENT services for further management.
Shared decision-making in chronic illness has been shown to improve patient satisfaction and adherence to treatment plans.Reference Joosten, DeFuentes-Merillas, de Weert, Sensky, van der Staak and de Jong2, Reference Zhang, Terry and McHorney3 Written patient information acts as an adjunct to the patient–clinician encounter; however, there has been a movement from paper-based leaflets to web-based information. Furthermore, the general public is increasingly accessing the internet for health information and using this information in their decision-making process. Seventy-three per cent of the UK population accessed the internet every day in 2013, with the rate of mobile phone data use doubling from 24 to 53 per cent between 2010 and 2013.4 Parents of children with glue ear are invariably ‘internet savvy’, and this corresponds with an internet usage of over 80 per cent for those aged 25–44 years. The percentage of people seeking health-related information online in the UK increased from 18 to 43 per cent between 2007 and 2013.4
Health literacy is a strong predictor of health outcome. As 1 in 6 adults' literacy is below that of an 11-year-old, it is important to adapt to variation in reading and comprehension skills.Reference Zhang, Terry and McHorney3, 5 The use of readability scores enables the medical community to structure and direct literature at a wider range of patients, and, importantly, it ensures that the average patient can comprehend its content.
Readability reflects the ease with which a document can be read and understood. The concept of readability emerged from the need to match appropriate reading material to students in order to advance their reading level in an educational environment. With the advent of computers, various formulas were developed to rank literature's degree of readability.
Kincaid and colleagues developed the reading-ease formula to improve the readability of military technical manuals in order to reduce human error.6 The Flesch–Kincaid reading-ease test takes account of the word count, sentence count and number of syllables to formulate a score indicative of the text's comprehension difficulty (Table I). Long sentences and polysyllabic words significantly affect the score.
Flesch–kincaid score categorisation
The Discern instrument is a standardised quality index for consumer health information assessment.Reference Charnock, Shepperd, Needham and Gann7 However, it does not rate accuracy of content. It consists of 16 questions, with scores ranging from 1–5. The questions address the following: the presence of the article's objectives and whether these are achieved; relevance to the topic and the sources of information used to compile the article; the degree of bias and reference to uncertainty within the literature; benefits and risks of all treatments available, effects on quality of life, and risks of a decision not to treat; and the emphasis on shared decision-making and recommended resources for readers.
This study aimed to objectively evaluate the current information available to the general public related to glue ear on the World Wide Web using the tools outlined above.
Materials and methods
The term ‘glue ear’ was entered into the three most commonly used internet search engines: Google, Bing and Yahoo. The search engines were selected based upon combined data collected by the two leading website traffic analysis companies – Compete (Boston, Massachusetts, USA) and Quantcast (San Francisco, California, USA) – on the 26th November 2013.8
The first 20 pages of results from each search engine were analysed. We manually excluded duplicates and websites not related to glue ear. The first 400 words of each page were used to calculate readability using the Flesch–Kincaid score. The Flesch–Kincaid scores were then converted to a score out of 80 to enable direct comparison with Discern scores.
The Discern instrument is subjective, and uses a Likert scale with response options 1, 3 and 5, labelled ‘No’, ‘Partially’ and ‘Yes’ respectively. Each website was analysed separately by two ENT trainees and an average of each score was used.
The Discern scoring system is applicable in any language; however, the Flesch–Kincaid score is for English-language content only as it involves a syllable counter.
Results
As the Flesch–Kincaid score is computer-generated, it was only necessary to investigate the inter-rater reliability for the Discern scores obtained. Cohen's linearly weighted kappa showed fair inter-rater reliability (κ = 0.296, p = 0.0012).Reference Ademiluyi, Rees and Sheard9, Reference Landis and Koch10 However, Spearman correlation was strong (rs = 0.721, estimated p < 0.001). This suggests that the relative rating of websites was more reliable than the absolute score.
There was overlap between the top 20 websites presented by each search engine. The combined number of websites for all 3 search engines was 27. The search results for Bing and Yahoo were identical. The Flesch–Kincaid score and Discern score was calculated for each website. The mean Discern score was added to the Flesch–Kincaid score to generate a total score, with a maximum possible value of 160 (Table II). The top three websites, in terms of total score, were: Webmd.boots.com, Bupa.co.uk and Patient.co.uk.
Overall flesch–kincaid scores and discern scores for each website ranked by total score
Websites not accompanied by a footnote symbol contain no marketing or advertising. *Reflects advertising websites. †Denotes marketing websites. ‡Denotes the highest score in each column. **Uniform Resource Locator (URL) for the former main website of the British Tinnitus Association.
The websites with the highest Flesch–Kincaid score were: Ican.org.uk, Londonentsurgeon.co.uk and Webmd.boots.com. The charity ican.org.uk website came 13th on the Google search results list and scored moderately in the Discern assessment; however, this website has a narrow spectrum of information, related to speech and language therapy only. A marketing website for ENT services, Londonentsurgeons.co.uk, scored highly in terms of reading ease, but again Discern scores were moderate as a result of the lack of information about the benefits and risks of treatments and its failure to reference the information given.
The websites with the highest Discern scores were: Netdoctor.com, Patient.co.uk and Bupa.co.uk. These websites scored highly in all sections of the Discern instrument; they provide broad information about all aspects of glue ear and its treatment.
The mean Flesch–Kincaid score was 49.7 (range, 25.8–65.7), which reflects readability suitable for 11–15-year-olds. Wikipedia (En.wikipedia.org) and the British Medical Journal (Bmj.com) websites had the lowest readability scores. Nine of the 27 websites (33 per cent) had a readability score appropriate for graduate level entry.
The mean Discern score was 57 (range, 26–75). Three of the 27 websites (11 per cent) had a low Discern score (less than 2 out of 5) (i.e. Deafnessresearch.org.uk, Wellcome.ac.uk and Netmums.com), which indicates that they should not be recommended as sources of information to make treatment choices. Of the 27 websites, only 11 (40 per cent) scored above 4; such scores indicate that the publication is likely to be of good quality and may therefore aid treatment choices.
The 27 websites were categorised into 3 groups according to their competing interests (Table II): websites that exist primarily to market a product or service; websites that contain advertisements, but do not market a particular product or service; and websites that have no apparent marketing or advertising. Forty-five per cent of the websites have competing interests. The mean scores for websites in different categories were calculated (Table III). The readability and Discern scores were lowest for websites with no competing interests, higher for advertising websites and highest for marketing websites. This is reflected in the total score (Table III).
Comparison of mean scores across website types
*Maximum score of 180
Nevertheless, statistical testing with Spearman correlation did not reveal any significant correlation between the website category and Flesch–Kincaid score, Discern score or total score. The distribution of website scores can be seen in Figure 1.
Comparison of Flesch–Kincaid scores and Discern scores for different types of website. Squares signify websites with no advertising or marketing, circles denote advertising websites, and triangles reflect marketing websites.
Discussion
Non-profit groups and clinicians have a duty to ensure that disadvantaged groups such as those on a low income, the elderly and the disabled have equal access to health information.Reference Kuk11 The Department of Health and Human Services initiated a campaign to improve health literacy in the USA. Their ‘key stones’ are ‘everyone has the right to health information that helps them make informed decisions’ and ‘health services should be delivered in ways that are understandable and beneficial to health, longevity, and quality of life’.12 Patient information documents can be further improved by gathering data on patient literacy, to allow us to direct health literature and become responsive to individual patient needs.Reference Hasnain-Wynia13
The quality and readability of the available health information pertaining to glue ear varied widely in this study. Only 4 of 27 websites were rated highly for both readability (Flesch–Kincaid) and consumer health information quality (Discern). Patient.co.uk and Webmd.boots.com feature in the top three websites and are peer-reviewed. Netdoctor.com is peer-reviewed but had a lower readability score, whereas Bupa.co.uk had an equivalent Discern score but a higher readability score.
Sites with the highest Discern score make use of diagrams and audio-visual information to communicate to the reader. There is transparency with regards to the authors, editing dates and references.
The relevant Patient.co.uk webpage is easy to navigate, with a summary in a question format that has links to the area of question. There is also a row of tabs with the titles ‘support’, ‘decision aids’, ‘clinical trials’ and ‘discuss’. Webmd.boots.com is written and edited by the British Medical Journal group. It uses clearly marked headings to divide a topic, and it is comprehensive, citing national guidelines. It does not cover the operative risks, nor does it discuss the long-term effects of not undergoing treatment. There are six adverts per page. Bupa.co.uk has a summary of contents at the beginning of the page, with direct Uniform Resource Locator (‘URL’) links to the relevant area of the page. It has a 2-minute animated video explaining the pathophysiology of glue ear, and diagrams. It uses tabs at the top of the page to navigate to a ‘FAQs’ (frequently asked questions) section and ‘Resources’ section, where links to supporting charities are given. It has no adverts.
Chat forums such as those at Mumsnet.com and Dailymail.co.uk had high readability but poor Discern scores. These websites may attract readers with poorer reading and comprehension skills.
Websites that contain advertising or marketing had higher readability and Discern scores. There are multiple possible reasons for this, such as more funding for website development or reduced readability barriers to attract more customers. The marketing websites did not always declare their competing interests, and this may add bias to the quality and breadth of information presented.
Goslin and Elhassan found that the use of validated instruments highlighted poor quality literature, thereby avoiding misinformation.Reference Goslin and Elhassan14 McKearney and McKearney used the Discern instrument and Flesch–Kincaid readability test to assess information pertaining to surgical intervention for glue ear.Reference McKearney and McKearney15 They reported mean readability and Discern scores of 49.4 and 38.5 respectively; in our study, these scores were 49.7 and 57.0 respectively. Their mean scores may be different as they only evaluated information regarding ventilation tubes in treating glue ear. However, the general low scores in their study do echo our findings and reflect a concerning poor level of relevant, high quality and easily readable health information on the internet. The McKearney and McKearney study is similar in design to our study, and despite addressing patient education material on the internet regarding ventilation tubes, it does not specifically address the overall condition being treated. Clearly, it is important that patients are informed regarding the disease process being treated and the treatment options available to them.
Swartz investigated the readability of eight printed patient leaflets pertaining to glue ear. The author reported that patient satisfaction did not change with the reading age of the material.Reference Swartz16 However, as the reading level increases above that recommended by the US Department of Health and Human Services, patient comprehension decreases.12 This may cause a risk to health; for instance, a patient may choose a treatment irrespective of its risks as no alternative was offered. Risks associated with misinformation are hard to quantify, as although there are specific case reports of harm secondary to internet-based information, these are uncommon. However, this may be due to lack of reporting or other bias.Reference Crocco, Villasis-Keever and Jadad17
Search engines play a role in directing patients towards appropriate websites pertaining to their illness. Larner showed that 40% of web-based literature accessed by patients was not specifically pertinent to their diagnosis.Reference Larner18 The younger population who are more computer literate will have skills to search and identify websites, whereas the older population may require help from their physician to access information. The usability of a website interface will further impact on a patient's ability to use that website, and to identify features such as outbound links, search capabilities and information sources.Reference Eysenbach and Köhler19
• Shared clinician–patient decision-making improves patient satisfaction and clinical outcome
• Patients are increasingly well informed regarding health conditions as a result of web-based health information
• To enable decision-making, clinicians must empower patients with information
• A clinician's ability to critically review information is central to choosing good literature for patients
• Clinicians must appreciate variation in patients’ reading abilities, and appraise web-based article readability
• This enables patients to have suitable literature on their medical problems
The study limitations include the fact that the top 20 websites are subject to change. The subjectivity of the Discern instrument lies in the structure of the Likert scales, which has response options 1, 3 and 5 labelled ‘No’, ‘Partially’ and ‘Yes’ respectively; numbers 2 and 4 are not labelled, and this makes it harder to distinguish which individual score is most appropriate. Polysyllabic and monosyllabic words may skew the Flesch–Kincaid score, but other tools such as Coleman–Liau Index and the automated readability index do not use syllable counters and can therefore be used reliably for most European languages. Finally, the quality of audio-visual information is not assessed, and these tools may appeal to patients with different styles of learning.
Conclusion
There is significant variation in the quality of information available to patients on the World Wide Web. Investment in websites from commercial sources through marketing or advertising was not shown to affect the quality of information provided. However, most websites scored highly in terms of either readability (Flesch–Kincaid) or consumer health information quality (Discern), but rarely both.
In order to encourage shared decision-making, clinicians should understand the role of quality and bias in effecting patient choice, and use validated instruments such as the Flesch–Kincaid reading-ease test and Discern instrument to assess and recommend or write literature.
