Outpatient antibiotic misuse is widespread in the United States and has been associated with several patient harms, including Clostridioides difficile infections, adverse drug reactions, and rising rates of antibiotic resistance.Reference Marston, Dixon, Knisely, Palmore and Fauci1,Reference Dantes, Mu and Hicks2 Recent estimates suggest that ~30% of the >200 million outpatient antibiotic prescriptions in the United States each year may be inappropriate.Reference Chua, Fischer and Linder3–Reference Fleming-Dutra, Hersh and Shapiro5
Although outpatient antibiotic misuse is common, it remains difficult to identify and study. Prior research has relied on billing claims data or clinic surveys, which may be limited by inaccurate coding, unreliable clinical documentation, and long delays between data collection and analysis.Reference Chua, Fischer and Linder3,Reference Fleming-Dutra, Hersh and Shapiro5 Additionally, these methods focus only on provider behaviors and do not capture the misuse of nonprescribed antibiotics, which occurs frequently but has not been well studied.Reference Grigoryan, Germanos and Zoorob6 Novel patient-centered approaches are therefore needed to more quickly and accurately characterize inappropriate outpatient antibiotic use.
In this study, we describe the use of Twitter data, natural language processing and deep learning to identify self-reported episodes of antibiotic misuse in the United States.
Methods
Unique English language Tweets describing outpatient antibiotic use in the United States from March 2018 to March 2019 were aggregated via the Twitter developer platform. A search query was designed to find Tweets likely to describe outpatient antibiotic use while excluding retweets and some Tweets describing appropriate antibiotic use (see Appendix A for search details).
Included Tweets were deidentified and then labelled by an infectious diseases physician as either describing possible recent antibiotic misuse or not describing misuse. Possible misuse was defined as antibiotic use for bronchitis, asthma, any viral infection (including influenza and the common cold), cough without pneumonia, or any use of nonprescribed antibiotics. Antibiotic use for any other indication, including any inpatient use, was not considered misuse. Antibiotic prescriptions for sinusitis, otitis media, or pharyngitis were not considered misuse because appropriate use for these indications could not be reliably determined. Tweets not describing any instance of recent antibiotic use were also labelled as not misuse. Examples of labelled Tweets are presented in Appendix B.
The geographic location of each Tweet was collected using geotagging data when available. If a Tweet was not geotagged, then the location was derived from the user’s profile data.
Text from the labelled Tweets was converted into numerical vectors and randomly divided into training, validation and test sets consisting of 80%, 10%, and 10% of the data, respectively. Training and validation set vectors were used to train a long short-term memory (LSTM) recurrent neural network using learned word embeddings. Model features were derived only from the text of each Tweet, and no other input data were used. Model performance was assessed using the test set. Analyses were performed in Python software using the scikit-learn, NLTK, Keras and TensorFlow libraries.
Results
We included 9,323 Tweets in this study, of which 1,664 (18%) were labeled as describing possible outpatient antibiotic misuse. Of these, 55 described the use of nonprescribed antibiotics. Locations were determined from user profile data for 91% of included Tweets, and from geotagging data for the remaining 9%. Antibiotic misuse was identified across a broad geographic distribution (Fig. 1). The deep-learning algorithm correctly identified Tweets describing possible antibiotic misuse in the test set with an accuracy score of 0.87 (95% confidence interval, 0.85–0.90) and area under the receiver operator characteristic curve of 0.91.
Geographic distribution of episodes of possible antibiotic misuse.
Discussion
In this study, we have demonstrated that a deep-learning algorithm using Twitter data and natural language processing can accurately identify episodes of recent outpatient antibiotic misuse in the United States. This approach provides insight into the timing and location of antibiotic misuse, and it captures the misuse of nonprescribed antibiotics, which is not easily identified by other methods.
These findings have several possible applications. This approach can be used to study prior trends in antibiotic misuse and to prospectively monitor Twitter data to identify antibiotic misuse in real time. One attractive application of this work, to identify and automatically alert users who may be describing antibiotic misuse, is prohibited by Twitter policy and is therefore not possible.7 However, real-time monitoring of Twitter data could be applied to many other valuable tasks, for example, to track emerging trends in antibiotic use or to actively gauge the effect of interventions designed to reduce misuse.
This study has several limitations. The study population was limited to Twitter users who chose to Tweet about antibiotic use, and only to English language Tweets. An initial search query was used to compile the Twitter data, which was an essential step but may have caused some relevant Tweets to be excluded. Additionally, the design of the initial search query may affect study reproducibility because the algorithm would be expected to perform differently on different data sets.
Although standard definitions of antibiotic misuse were used, the content of some Tweets was ambiguous, which might also affect reproducibility. Because all episodes of antibiotic use for sinusitis, otitis media, and pharyngitis were not considered misuse, some cases of misuse for these indications were likely labelled as not misuse. Finally, although precise geolocation data were used whenever possible, the locations of most Tweets were derived from users’ profile data, which may be less accurate.
Despite its limitations, this study shows that Twitter data and deep learning can be used to rapidly and accurately identify outpatient antibiotic misuse. This work represents a novel, patient-centered approach to better understanding the pervasive and harmful overuse of outpatient antibiotics in the United States.
Acknowledgments
None.
Financial support
No financial support was provided relevant to this article.
Conflicts of interest
All authors report no conflicts of interest relevant to this article.
Appendix A. Twitter Search Query Used to Compile the Data Set
SEARCH_TERM = ‘(antibiotics OR antibiotic OR levaquin OR levofloxacin OR avelox OR moxifloxacin OR cipro OR ciprofloxacin OR z-pak OR azithromycin OR amoxicillin OR augmentin OR keflex OR cefuroxime OR ceftin OR cefaclor OR cefpodoxime OR vantin OR cefdinir OR omnicef OR bactrim OR clindamycin) (fever OR cough OR bronchitis OR flu OR virus OR asthma OR throat) -RT -sinus -vet -cat -dog -abscess -strep lang:en’
Appendix B. Examples of Tweets Labeled as Possible Misuse and Not Misuse
Tweets labelled as possible misuse:
1. “Anybody have any home remedies or super good meds they know of for cold/flu? I’ve been sick for 3 weeks. Went to the Dr and got antibiotics today but need to feel better ASAP”
2. “I’ve had bronchitis since Thursday. It appears to be viral because the antibiotic did nothing.”
3. “I had the flu 102 had some leftover antibiotics, started taking them last night, this morning fever broke, feeling a lot better.”
4. “The doctor NEVER wants to give me antibiotics. That’s the ONLY THING THAT WORKS. Ugh. I got some from my neighbor. Took a pill this morning and feel better already.”
5. “Viral bronchitis is my diagnosis this afternoon. Cough suppressant and inhaler in hand as well as a prescription for antibiotics.”
Tweets labelled as not misuse:
1. “If you have a cold, antibiotics won’t help you feel better. Instead, try sipping warm liquids such as bone broth, hot tea, or soup.”
2. “On antibiotics. For my horrible throat infection.”
3. “An update: the doctors are pretty sure it’s pneumonia, so they’ve started heavy antibiotics and are keeping her at the hospital for observation.”
4. “#Antibiotics Tied to Longer Hospital Stays for #Asthma #pophealth #medicalevidence”
5. “This ear infection is going on 2 weeks with me dealing with about 80% hearing loss in left ear. After 1 week on Amoxicillin (with no results), doc changed me to Amox-Clav which I’ve been [taking] since Monday.”
