Introduction
Effective communication with patients with severe hearing loss can pose a challenge in healthcare settings. Even when healthcare professionals use ‘deaf-aware’ communication strategies to aid lip-reading, studies suggest that patients still miss essential information during the consultation.Reference Ebert and Heckerling1, Reference Reeves and Kokoruwe2 Patients with congenital deafness tend to prefer using sign language, but those with progressive or recent hearing loss may not have this skill.Reference Reeves and Kokoruwe2, Reference Middleton, Turner, Bitner-Glindzicz, Lewis, Richards and Clarke3 Communication via writing can be useful and is the most common technique employed by doctors.Reference Ebert and Heckerling1
Patients with severe to profound hearing loss may benefit from cochlear implants, especially when external hearing aids fail to provide adequate benefit. Assessing patients for cochlear implantation requires a detailed evaluation of the hearing impairment, medical history, and psychological and social factors. Patients are counselled and educated about the risks and benefits, and post-operative rehabilitation. Effective communication is required at all stages of the process. In the UK, cochlear implants in adults are approved for profoundly deaf patients (with a minimum of 90 dB HL loss at 2 and 4 kHz) with aided speech recognition (Bamford–Kowal–Bench test) scores of less than 50 per cent at 70 dB SPL.4 By definition, these patients find verbal communication difficult, even with well-fitted and tuned acoustic hearing aids.
In our clinics, we use a range of communication aids, including sign language interpreters and visual aids such as written text. Many of these patients are post-lingually deaf and do not use sign language. When written text is required, we favour typing conversations onto a desktop computer using word-processing applications. This allows clear text to be displayed on a monitor, and the font size can be increased as required.
In some situations, access to interpreters or computers is not possible. This includes many clinical areas such as emergency departments, wards, operating theatres and post-operative recovery areas. In these situations, we have had anecdotal success using automated speech recognition software on smartphones. This allows users to dictate a conversation directly onto the smartphone. Using mobile data communication, this is quickly converted to text that can be read from the screen. In order to more formally assess the feasibility of this technology, we conducted a study of its speed and accuracy compared with typing and writing.
Materials and methods
In this prospective study, volunteer medical students and doctors were enrolled. These subjects were given a standard set of six sentences which reflected a typical conversation in the recovery room or ward following cochlear implant surgery (Appendix 1). The subjects were asked to replicate this text by writing the information down using pen and paper, typing it into a computer word processor, and dictating it into a smartphone running an automated speech recognition software application. For computer typing, we used Word 2003 software (Microsoft, Redmond, Washington, USA). Automated speech recognition was performed using the free Dragon Dictation application (Nuance, Burlington, Massachusetts, USA) running on an Apple iPhone 4 (Apple, Cupertino, California, USA). The iPhone had third generation (‘3G’) network connectivity.
Each method of transcription was timed and the final result measured for accuracy. Timing did not include turning on the computer and logging on as it would normally be on for the duration of the clinic; however, it did include unlocking the computer and opening Word. Similarly, the timing of the dictation included unlocking the iPhone and opening the application. Handwriting was assessed for legibility by two independent readers. The accuracy of the typed and dictated information was scored in terms of the percentage of words correct.
Results
Thirty subjects, 11 doctors and 19 medical students, were recruited.
The mean times and accuracy values for the three transcription methods are shown in Figures 1 and 2. The time and accuracy for each method were compared using the paired samples t-test (SPSS Statistics software, version 19; IBM, Armonk, New York, USA).
Mean time taken for each transcription method (vertical lines represent 95 per cent confidence intervals).
Mean accuracy for each transcription method (vertical lines represent 95 per cent confidence intervals).
The mean time for smartphone dictation (17.8 seconds, 95 per cent confidence interval (CI) = 17.0–18.7) was significantly faster than writing (59.2 seconds, 95 per cent CI = 56.6–61.7) or typing (44 seconds, 95 per cent CI = 41.0–47.1) (p < 0.001). Smartphone dictation was less accurate than the other methods (dictation – 92.5 per cent words correct, 95 per cent CI = 89.8–95.1; writing – 99.5 per cent words correct, 95 per cent CI = 99.1–99.9; and typing – 99.8 per cent words correct, 95 per cent CI = 99.5–100.1) (p < 0.001).
Digital dictation
The accuracy of smartphone dictation appeared to improve with time spent dictating (Spearman's rank correlation coefficient = 0.655; p < 0.01) (Figure 3).
Correlation between time taken to dictate and accuracy of dictation.
Students versus doctors
Students were faster than doctors at digital dictation (mean time of 16.7 vs 19.7 seconds respectively, p < 0.001), but less accurate (90.3 vs 96.4 per cent words correct respectively; p = 0.004 Mann–Whitney U test).
Discussion
This pilot study is the first to show that the use of smartphone automated speech recognition software may be feasible in a clinical setting.
A review of published studies identified five papers that examined the use of visual text communication methods in the clinical setting.Reference Hinman, Lupton, Leeb, Avestruz, Gilmore and Paul5–Reference Bottrill9 Hinman et al. showed that comprehension was significantly better in deaf patients who read text generated by automatic speech recognition software (alone, with listening, or with listening and lip-reading) compared to listening alone.Reference Hinman, Lupton, Leeb, Avestruz, Gilmore and Paul5 However, this was a small study of eight patients with a range of mild to profound hearing losses. These findings have since been confirmed by Zekveld et al. in a study in which subjects with no hearing impairment demonstrated better speech comprehension against background noise when speech was accompanied by text generated by automated speech recognition software.Reference Zekveld, Kramer, Kessens, Vlaming and Houtgast6
Three papers have described the use of automated speech recognition software for administration in out-patient clinics. Klann and Szolovits used Dragon Naturally Speaking automated speech recognition software (Nuance) to produce consultation records.Reference Klann and Szolovits7 However, the software worked better when the user wore a headset, which may not be acceptable during a consultation. One group studied the use of automated speech recognition software for creating paediatric out-patient clinic letters.Reference Issenmann and Haffer8 They found that training the software and correcting errors took double the physician time in comparison to using a handheld tape recorder, although the letters were available the same day, in comparison to one week using the standard practice. Bottrill described his use of the same software for producing ENT out-patient letters.Reference Bottrill9 He found it freed up his secretary's time to check the letters for accuracy, and sped up the turnaround time; however, it took time to train the system.
We did not find any studies of automated speech recognition software used to communicate with patients directly, although the possibility has been described.Reference Fellbaum and Koroupetroglou10
Our study shows that smartphone automated speech recognition is significantly faster than writing or typing. Although the results for dictation were not quite as accurate, the error rate was only 7 per cent on average. In our experience with this technology, patients experience little difficulty understanding text with this low frequency of errors. There is significant redundancy in most language, which means that patients can understand the meaning of sentences even when some words are incorrect. Several studies have shown this to be the case, particularly for deaf patients.Reference Hinman, Lupton, Leeb, Avestruz, Gilmore and Paul5, Reference Zekveld, Kramer, Kessens, Vlaming and Houtgast6 Accuracy can be improved by slower dictation.
There are factors that may limit the use of this technology in clinical settings. We used the free Dragon Dictation application for the iPhone. This requires a connection to the internet via third generation, fourth generation (‘4G’) or Wi-Fi™ to process speech. Not all clinical areas have cellular reception, and many institutions do not provide wireless internet access for staff. The benefit of uploading the speech to the network is that the application learns different accents and vocabulary, and continuously improves. We anticipate that accuracy will increase as the technology improves. Another drawback is that the size of smartphone display limits the size of text displayed. Patients with poor eyesight may be better served by tablet computers.
Despite these limitations, we have found this technology to be a useful aid in communicating with patients in the out-patient clinic and recovery ward.
Acknowledgements
Thank you to the medical students and doctors who gave their time to be subjects.
Appendix 1. Post-operative questions and information giving
How do you feel?
The operation went very well.
Do you have any pain? Do you feel dizzy or sick?
Can you close your eyes tightly and show me your teeth?
Do you have any questions?
