Introduction
The Delphi technique is an established method of ‘needs analysis’ in curriculum planning. It consists of a series of sequential questionnaires or ‘rounds’, interspersed by controlled feedback that seeks to gain the most reliable consensus of opinion of a group of ‘experts’.Reference Linstone and Turoff1 The Delphi process has been used in other specialties with limited undergraduate or postgraduate training (e.g. dermatology) to identify generic, clinically relevant content.Reference Clayton, Perera and Burge2
The aim of this study was to develop an educational website (www.enttheatre.com) that showed common ENT operations and emergencies, in order to help improve the basic surgical ENT knowledge of medical students and junior doctors.
Method
Ethical approval was obtained from the North Staffordshire Research and Ethics Committee.
First-round Delphi process
An e-mail was sent to the following ‘experts’ to invite them to complete the first-round questionnaire online (Appendix 1): all 35 ENT specialist registrars in the West Midlands Deanery; all 15 ENT consultants who were departmental-lead educational supervisors in the West Midlands; 30 general practitioners in the North Staffordshire Primary Care Trust; all 15 ENT-themed core-training doctors in the West Midlands Deanery; and 30 final-year medical students from Keele Medical School.
A five-point Likert scale was used for the responses to the first questionnaire (i.e. ‘not important’, ‘fairly unimportant’, ‘undecided’, ‘fairly important’ or ‘very important’).
A threshold of 50 per cent consensus was used to determine the contents of the second questionnaire, for those items that had a response of ‘very important’ or ‘fairly important’. Any suggestions made in the first-round responses were also added as new items in the second-round questionnaire. This was sent to all respondents who had replied to round one. A polite reminder e-mail was sent after four weeks if no response had been received.
Second-round Delphi process
Six weeks after the first-round questionnaire had been sent, responses were analysed and the second questionnaire was compiled (Appendix 2). This was e-mailed to all respondents from the first round of the questionnaire. There were three response options for each item: ‘definitely yes’, ‘no – it is not necessary’ and ‘not sure’.
Results
The response rate to the round one Delphi questionnaire was 49 per cent (61/125 respondents). Respondents comprised 12 medical students, four specialist trainee year one and two doctors, 10 general practitioners, 10 ENT consultants and 25 ENT registrars.
The response rate to round two of the Delphi questionnaire was 92 per cent (56/61 respondents). The non-responders comprised two medical students, two ENT registrars and one ENT consultant.
Each questionnaire in both rounds of the Delphi survey had three main sections: ‘Observing an operation’, ‘Knowing about complications’ and ‘ENT emergencies’.
Observing an operation
The following operations had a 50 per cent consensus of ‘very important’ or ‘important’ in the round-one Delphi process: adenoidectomy, tonsillectomy, grommet insertion, septoplasty, nasal polypectomy and thyroid surgery.
Other operations suggested by respondents in round one included pituitary surgery, cochlear implantation, tracheostomy, oesophagoscopy, panendoscopy, lymph node biopsy, facial plastic procedures, mastoid surgery and rhinoplasty.
In round two of the Delphi process, only mastoid surgery was added to the round one selected operations because of agreed consensus within the group.
Knowing about complications
Round one results with a 50 per cent consensus of ‘important’ or ‘very important’ included adenoidectomy, tonsillectomy, grommet insertion, septoplasty, nasal polypectomy and thyroid surgery. Respondents in round one suggested that mastoid surgery should be included. There were no further additions to this list in the round-two Delphi process.
ENT emergencies
Round one results with a 50 per cent consensus of ‘important’ or ‘very important’ included management of a foreign body in the ear or nose, epistaxis, tonsillitis, quinsy, and otitis externa.
Other ENT emergencies suggested by respondents included cerebrospinal fluid leakage, tracheostomy, bleeding from the ear, periorbital cellulitis, sudden-onset hearing loss, Bell's palsy, acute otitis media, ‘glue ear’ and penetrating neck trauma. However, none of these suggested emergencies had an agreed consensus in round two.
Table I summarises the respondents' consensus on which ENT topics medical students and junior doctors should know about, following the two-round Delphi process.
Discussion
Despite the prevalence of ENT conditions, undergraduate otolaryngology training has declined over recent years, with some UK medical schools having no formal otolaryngology teaching.Reference Mace and Narula3
In 2009, Chawdhary et al. surveyed 389 final year medical students from a UK medical school.Reference Chawdhary, Ho and Minhas4 Of the 80 per cent that responded, only 28 per cent felt that they were adequately prepared to handle common, routine ENT complaints.
In 2006, Sharma et al. performed a telephone survey of 20 doctors working in different accident and emergency (A&E) departments in the UK.Reference Sharma, Machen, Clarke and Howard5 Ninety per cent felt their undergraduate ENT teaching was directly beneficial to working in A&E; however, 75 per cent felt they had not received enough undergraduate ENT teaching. Forty-five per cent had received no postgraduate ENT teaching while working in A&E.
In 2007, Clamp et al. performed a postal survey of 500 general practitioners in south-west England.Reference Clamp, Gunasekaran, Pothier and Saunders6 They found that 75 per cent of respondents felt their undergraduate otolaryngology experience was inadequate, and only a quarter of them had received specialty-led tuition in an otolaryngology hospital post.
A similar study was performed in the Trent region of England, which highlighted the limited otolaryngology experience of general practitioners at both undergraduate and postgraduate level.Reference Veitch, Lewis and Gibbin7 The authors found that 85 per cent of the 220 general practitioners surveyed felt they would benefit from further ENT training. Only one in three general practitioners had received formal postgraduate training in ENT, and approximately half had not been trained in ENT examination at undergraduate level.
In view of the above literature showing limited otolaryngology experience and the perceived need for both undergraduate and postgraduate doctors to improve their knowledge, it was important to determine clinically relevant learning content for our proposed website.
In order to achieve this, educational learning theory was applied. The website content was developed using a combination of ‘top down’ and ‘bottom up’ approaches. These approaches have previously been used in curriculum development.Reference Bull and Mattick8
The traditional top down approach is employed by an organisational body of experts. These experts decide the content of their educational programme based upon their expertise and experience; it reflects important topics that they believe a student should know about.
A bottom up educational approach considers the learner's ideas and expectations when developing the content of an educational programme. This approach can sometimes identify areas that are not considered when employing a more traditional curriculum development approach.
The content of our proposed website was developed using a Delphi questionnaire distributed to ENT consultants, ENT registrars and general practitioners as well as to medical students and junior doctors. In this way, we hoped to employ both the top down and the bottom up educational approaches.
• Undergraduate and postgraduate otolaryngology training is limited
• Medical students and junior doctors believe they need to improve their basic otolaryngology knowledge
• The Delphi technique was used to obtain a consensus opinion on clinically relevant content for an ENT education website
• This identified the basic ENT knowledge that respondents believed medical students and junior doctors should have
The Delphi process we employed was purposive, in that the respondents surveyed reflected those directly involved in delivering ENT training, as well as the intended recipients of ENT training. The two-round Delphi process used identified content thought to be important by both experienced clinicians and the target audience (i.e. medical students and junior doctors).
The questionnaire response rates in this study were consistent with those reported by similar studies using online questionnaires.Reference Sax, Gilmartin and Bryant9
Our findings represent a consensus opinion on the basic ENT surgical knowledge with which medical students and junior doctors should be familiar.
Appendix 1. Round-one Delphi questionnaire
Observing an operation
(1) How important do you feel it is that a medical student or foundation doctor should have observed at least one of the following elective ENT operations at some point during their undergraduate or postgraduate education, regardless of what their final career specialty may be? (For each operation, respondents were asked to select either ‘Not important’, ‘Fairly unimportant’, ‘Undecided’, ‘Fairly important’ or ‘Very important’.)
– Adenoidectomy
– Tonsillectomy
– Grommet insertion
– Myringoplasty
– Ossiculoplasty
– Stapedectomy
– Septoplasty
– Nasal polypectomy
– Inferior turbinate reduction
– Parotid gland removal
– Submandibular gland removal
– Thyroid surgery
– Laryngectomy
– Neck dissection
(2) Please list any other elective ENT operations, other than those listed above, that you feel it is important for a medical student or foundation doctor to have observed:
Knowing about complications
(3) How important do you feel it is that a medical student or foundation doctor should be aware of potential complications for each of the following operations? (For each operation, respondents were asked to select either ‘Not important’, ‘Fairly unimportant’, ‘Undecided’, ‘Fairly important’ or ‘Very important’.)
– Adenoidectomy
– Tonsillectomy
– Grommet insertion
– Myringoplasty
– Ossiculoplasty
– Stapedectomy
– Septoplasty
– Nasal polypectomy
– Inferior turbinate reduction
– Parotid gland removal
– Submandibular gland removal
– Thyroid surgery
– Laryngectomy
– Neck dissection
ENT emergencies
(4) How important do you think it is that a medical student or foundation doctor should be aware of the general principles of management of the following ENT emergencies? (For each operation, respondents were asked to select either ‘Not important’, ‘Fairly unimportant’, ‘Undecided’, ‘Fairly important’ or ‘Very important’.)
– Foreign body in the ear
– Foreign body in the nose
– Epistaxis
– Food bolus with or without bone
– Tonsillitis
– Quinsy
– Stridor
– Otitis externa
(5) Please list any other ENT emergencies, other than those listed above, that you feel it is important for a medical student or foundation doctor to know about:
Your position
(6) What is your position?
– ENT consultant
– ENT registrar or equivalent
– Foundation doctor or ST1 or ST2
– General practitioner
– Medical student
– Other (please specify)
Appendix 2. Round-two Delphi questionnaire
Observing an operation
(1) Do you feel that a medical student or foundation doctor should have observed at least one of the following ENT operations at some point during their undergraduate or postgraduate education, regardless of what their final career specialty may be? (For each operation, respondents were asked to select either ‘Definitely yes’, ‘No, it is not necessary’ or ‘Not sure’.)
– Adenoidectomy
– Tonsillectomy
– Grommet insertion
– Nasal polypectomy
– Thyroid surgery
– Stapedectomy
– Pituitary surgery
– Cochlear implant
– Tracheostomy
– Panendoscopy or oesophagoscopy
– Microlaryngoscopy
– Lymph node biopsy
– Facial plastic procedure
– Mastoid operation
– Rhinoplasty
Knowing about complications
(2) Do you feel that a medical student or foundation doctor should be aware of the potential complications of each of the following operations, regardless of their career intentions? (For each operation, respondents were asked to select either ‘Definitely yes’, ‘No, it is not necessary’ or ‘Not sure’.)
– Adenoidectomy
– Tonsillectomy
– Grommet insertion
– Septoplasty
– Nasal polypectomy
– Parotid surgery
– Submandibular gland removal
– Thyroid surgery
– Laryngectomy
– Neck dissection
ENT emergencies
(3) Do you feel that a medical student or foundation doctor should be aware of the general principles of management of the following ENT emergencies, regardless of their career intentions? (For each operation, respondents were asked to select either ‘Definitely yes’, ‘No, it is not necessary’ or ‘Not sure’.)
– Foreign body in the ear
– Foreign body in the nose
– Epistaxis
– Food bolus (with or without bone)
– Tonsillitis
– Quinsy
– Stridor
– Otitis externa
– Cerebrospinal fluid leakage from nose or ear
– Periorbital cellulitis
– Sudden-onset sensorineural hearing loss
– Bell's palsy
– Penetrating neck trauma
Your position
(4) What is your position?
– ENT consultant
– ENT registrar or equivalent
– Foundation year doctor or ST1 or ST2
– General practitioner
– Medical student
– Other (please specify)
