Introduction
Thyroidectomy is a commonly performed operation in the UK: 14 149 such procedures were performed in English National Health Service (NHS) trusts between 2008 and 2009.1 Complications can occur, and patients are often informed of these in the out-patient clinic as well as on the day of surgery. How much of this information the patient retains is questionable.Reference Godwin2 However, the World Wide Web offers a wealth of information for patients with unanswered questions.
In this study, we reviewed the content of the most commonly accessed websites, using a validated healthcare website assessment tool known as the Lida instrument.3 This tool assesses a website's accessibility, usability and reliability.
We also assessed the readability of these websites, which on the whole are aimed at the general public. A recent report by the UK Department for Education and Skills found that one in six adults has a literacy age below that of an 11-year-old.4 This study used the Flesch Reading Ease Score,Reference Flesch5 a tool commonly used to assess how difficult a reading passage is to understand.
The information available on the World Wide Web may not always be of a reliable quality, or wholly accurate.Reference Silberg, Lundberg and Musacchio6 The motivation of those running a website may not always be to educate impartially, and health information websites can often present biased, inaccurate content.Reference Chalmers7 The internet, being a free market, is often used to promote products, which may or may not be presented in a context of healthcare, health treatments or healthcare providers. Therefore, the most popular website may not contain the most scientific or the most impartial content; in some cases, it may just be the website which is able to provide and fund the largest advertising platform.
Search engines remain the commonest method of finding information on the World Wide Web. Neilson's Net rating8 suggests that more than 90 per cent of all search questions fielded in the UK are through only five search engines: Google, Yahoo, MSN, AOL and Ask Jeeves. Most web users are likely to visit the highest ranking website; however, the ranking does not compare in any way to a website's reliability or relevance.
Methods
This study involved searching the World Wide Web using the key word ‘thyroidectomy’. This search was performed using the five most commonly used search engines – Google, Yahoo, MSN, AOL and Ask Jeeves – during March 2010.8 We did not change any of the default settings or use any ‘plug-ins’ or advanced search settings. Searches were not limited to the UK, and accessed websites written in all languages. The Lida website validation tool (version 1.2) was used to assess the accessibility, usability and reliability of the first 50 websites found by each of the five search engines. Website readability was also assessed using the Flesch Reading Ease Score.
Accessibility
This part of the Lida tool mainly assesses whether web users can access the website. It also checks whether the website meets legal accessibility requirements, including W3C and Bobby standards. (This aspect of the Lida tool was accessed using an online tool available from the Minerva website).3 The accessibility part of the Lida tool also checks whether websites are available without registration, login or subscription, and whether the websites work on all commonly used browsers (tested with Internet Explorer, Firefox, Opera and Safari) and platforms (tested with Windows and Macintosh operating systems).
The maximum possible score for website accessibility is 63.
Usability
This part of the Lida tool primarily addresses whether users can access the information they need. Clarity, consistency, functionality and ‘engagability’ are assessed. Clarity implies a well designed website which is clear and which ‘signposts’ relevant information. Consistency of website design and layout help users learn how the site works. Functionality refers to whether the website has appropriate browsing tools and search engines, and also whether users need to download other plug-ins to use the site. Engagability assesses whether the site is interactive (e.g. is there scope for feedback?), whether it can be personalised and whether other media are used.
The maximum possible score for usability is 54.
Reliability
This part of the Lida tool assesses whether the website provides comprehensive, relevant and unbiased information. Aspects assessed include whether the site is regularly updated, whether it responds to current events (known as ‘currency’), whether there are any conflicts of interest in producing the information, and whether the information has been gathered in a robust, unbiased manner.
The maximum possible score for reliability is 51.
Lida scores were considered low if less than 50 per cent, moderate if between 50 and 90 per cent, and high if greater than 90 per cent.
Readability
The Flesch Reading Ease Score was used to test website readability. To illustrate the scoring system, the average Flesch Reading Ease Score for comic strips, the Reader's Digest, the Wall Street Journal and the Harvard Law Review are 92, 65, 43 and 32, respectively (lower numbers indicate passages which are harder to read).9 This is an easily reproducible and consistent score.
The main text of each website was transferred by ‘cut and paste’ into the Microsoft Word 2004 software application, and the grammar checker function used to formulate the Flesch Reading Ease Score. Possible scores range from 0 to 100. The Flesch Reading Ease Score was calculated using the average sentence length (ASL) and the average number of syllables per word (ASW), as follows:
All the information from the different websites was combined into a Microsoft Excel spreadsheet, and the data analysed.
Results
We found 250 websites, using the five search engines and the key word ‘thyroidectomy’. Of these, 147 websites were excluded as they were repetitions or had restricted access.
Therefore, 103 websites underwent analysis. All the websites analysed were in English. The majority (72) were created in the USA, 28 in the UK, and one each in Australia, Canada and Poland. Six of the websites were subsections of NHS trust websites. Others were produced by private healthcare providers, medical journals (including the British Medical Journal), registered charities, healthcare societies (including the Royal College of Surgeons and the British Thyroid Foundation) and medical universities. Online medical (i.e. E-medicine) and non-medical (i.e. Wikipedia) information databases were also included.
The results are summarised in Tables I and II.
Lida and flesch reading ease score data
*Mean score as percentage of highest possible score. SD = standard deviation; FRES = Flesch Reading Ease Score
Distribution of websites by lida score
Data represent numbers of websites. *≥90%; †between 50 and 90%; ‡<50%. Max = maximum; mod = moderate; prodn = production; suppl = supplement
Accessibility
The mean accessibility score was 48 (±6.29 standard deviation (SD)), representing 76.2 per cent of a possible score of 63; the score range was 27–59. Five websites scored more than 90 per cent for accessibility. One website scored below 50 per cent. All websites were accessible using the Windows and Macintosh operating system platforms and the four web browsers we tested. One website required paid registration for access, and seven websites required a pdf (portable document format) file reader to be accessed.
Usability
The mean usability score was 36 (±8.13 SD), representing 66.7 per cent of a possible score of 54; the score range was 21–50. Eleven websites scored below 50 per cent on clarity, consistency, functionality and engagability. These websites were poorly designed and not very ‘user-friendly’. Five of these 11 websites were pdf files, and provided a very rigid and unyielding experience for the user. One website from the e-Medicine group (from WebMD)11 scored in the high category (i.e. over 90 per cent) for usability; however, its overall Lida score (79.9 per cent) was not above 90 per cent.
Reliability
The mean reliability score was 21 (±9.07 SD), representing 41.2 per cent of a possible score of 51; the score range was 4–38. Sixty-seven of the websites had low scores (i.e. less than 50 per cent), and none had high scores. The main reasons for this were failure to disclose sponsorship, insufficiently robust data collection, and poor currency (i.e. lack of regular updates).
Overall Lida score
The overall mean Lida score for the 103 websites analysed was 105 (±16.59 SD), representing 62.0 per cent of a possible overall score of 168; the overall score range was 70–136. Interestingly, none of the websites achieved a high score; 90 had a moderate score and the remaining 13 had low scores.
Flesch Reading Ease Score
The mean Flesch Reading Ease Score was 43.9 (±17.52 SD); the range was 2.6–77.6. Websites with higher Flesch Reading Ease Scores usually had good usability scores; however, this did not translate into good overall Lida scores.
Two websites performed consistently well in all tests. The first was from the Cancer Research UK group,12 and had accessibility, usability, reliability and Flesch Reading Ease scores of 92, 89, 55 and 64.2 per cent, respectively. The second was from the Royal College of Surgeons of England group,13 with scores of 84, 80, 63 and 65.5 per cent, respectively.
Discussion
The internet has become a source of second opinion for many patients. The wealth of information available on the World Wide Web is subject to the same analysis as information supplied via all other medical media. Medical information has become increasingly easy to access; however, this information is not regulated, due to the nature of the World Wide Web.Reference Wyatt14
The most popular method of information-gathering on the World Wide Web is via search engines. However, internet website rankings are not always indicative of information quality. Highly ranked websites are most likely to be visited first, and therefore may possibly influence patients' attitudes the most. Using Google, currently the most popular search engine, the first page of search results (ie; 10 websites) had among it's sites three run by private healthcare providers. One of these websites promoted robotic (Da Vinci®) thyroidectomy as first-line treatment.
Usability was deemed vital when considering ease of use by patients. Our study findings show that the majority of websites were not particularly well designed or user-friendly. The E-medicine database scored well in this category (79.9 per cent). In contrast, five of the six websites associated with NHS trusts were uploaded pdf files, and hence scored very poorly on usability: their mean usability score was 51.5 per cent (with three websites scoring below 50 per cent), and their mean overall Lida score was consequently not good (53.9 per cent). They were no more beneficial than a printed paper leaflet, and had all the associated disadvantages (e.g. lack of interactivity and currency). However, not all websites provided poor information quality, with the Cancer Research UK and Royal College of Surgeons of England groups scoring consistently highly for all sections of the Lida tool.
Overall readability results (assessed using Flesch Reading Ease Scores) were highly variable, and were not indicative of Lida scores. Interestingly, the website scoring the highest Flesch Reading Ease Score was in fact written by a patient.
Other studies have also investigated healthcare information available over the internet. For example, Impicciatore et al. Reference Impicciatore, Pandolfini, Casells and Bonati15 assessed the information available for managing childhood fever at home. They could only find a few websites with accurate medical information, and advised an urgent review of medical information available on the World Wide Web.
To provide an efficient service for their patients, it is imperative that clinicians are aware of the information available to them.Reference Sheppard, Charnock and Gann16 The current study shows that there is a wealth of literature available via the internet; however, much of it is provided by companies seeking commercial gain, and many websites have conflicts of interest. It must be recognised that, while clinicians may be able to distinguish between well referenced and inaccurate literature, patients may not. In order to ensure better patient education, and hence a more efficient service, accurate medical referencing is vital, whether such endorsement comes from an appointed body or society, or from individual, impartial clinicians advising their patients. Eysenbach and DiepgenReference Eysenbach and Diepgen17 have suggested using labelling and filtering technology such as PICS (platform for internet content selection), which could supply healthcare professionals and consumers with labels to help them separate valuable internet health information from dubious material.
