Noninfectious syndromes mimicking bacterial pneumonia are a leading cause of unnecessary antibiotic use in hospitalized patients.Reference Fridkin, Baggs and Fagan1 Existing approaches to mitigating unnecessary antibiotic use in pneumonia have focused largely on minimizing excessive treatment durations for presumptive cases.Reference Cole, Stark and Hodge2,Reference Bond, Boutlis, Yeo and Miyakis3 Because pneumonia is a clinical diagnosis with a wide spectrum of manifestations, a challenge to antimicrobial stewardship programs (ASPs) is effectively identifying potentially misdiagnosed patients in whom antimicrobial treatment for bacterial pneumonia is unnecessary, without formal bedside clinical evaluation.
Abnormalities on chest imaging are foundational to the diagnosis of pneumonia, yet a substantial number of hospitalized patients presumptively diagnosed with pneumonia lack compelling radiographic disease features.Reference Burton, Price and Barr4,Reference Marrie and Huang5 Simple text abstraction from radiographic imaging reports represents a potentially inexpensive, automated, and practical method to identify patients with pneumonia misdiagnosis. The purpose of this study was to assess whether the absence of certain radiographic keywords in chest radiography interpretations could accurately predict the likelihood of inappropriate treatment for misdiagnosed pneumonia in hospitalized patients.
Methods
This was a retrospective cohort study utilizing an existing database of patients ≥18 years of age hospitalized between 2008 and 2017 at Michigan Medicine with a presumptive diagnosis of pneumonia,Reference Albin, Henig and Patel6 defined as growth of a pathogenic organism from respiratory culture, receipt of ≥5 days of pneumonia-specific antibiotics (according to Infectious Diseases Society of America (IDSA) and American Thoracic Society (ATS) guidelines)Reference Kalil, Metersky and Klompas7 and chest imaging (radiograph or computed tomography) within 48 hours of culture order date. All cases of presumptive pneumonia and associated imaging were reviewed by an infectious diseases physician to evaluate for pneumonia misdiagnosis, defined as (1) a case not fulfilling IDSA/ATS diagnostic criteria of a “new lung infiltrate plus clinical evidence that the infiltrate is of infectious origin, which includes new onset of fever, purulent sputum, leukocytosis and decline in oxygenation”Reference Kalil, Metersky and Klompas7 and (2) a case in which the reviewing clinician recommended against antibiotics for treatment of pneumonia. Equivocal cases were adjudicated by a blinded panel of infectious diseases physicians. Waiver of informed consent for this study was approved by the institutional review board at the University of Michigan.
Participants were grouped into those with prespecified keywords (“infection,” “pneumonia,” “consolidation,” “opacity(ies),” “opacification,” “infiltrate(s),” “air space,” assuming such words were not preceded by “no” or “no evidence of”) present in chest radiography interpretations and those without prespecified keywords present. Selected clinical variables at the time of presumptive pneumonia diagnosis (ie, temperature >38.3°C or <36.1°C, white blood cell count >12 or <4 × 103 cells/µL, new sustained oxygen requirement, and documentation of sputum purulence or increased suctioning requirements) were abstracted from the medical record, as were relevant demographic and comorbidity data.
The purpose of this study was to evaluate the role of radiographic keywords in ruling out bacterial pneumonia, and the primary outcomes for this study were sensitivity, negative predictive value (NPV) and negative likelihood ratio (LR) of radiographic keywords alone and in conjunction with selected clinical criteria for the diagnosis of pneumonia, using physician case and imaging review as the gold standard reference. Specificity and positive predictive values (PPV) were calculated as secondary outcomes. Categorical variables were compared using the Fisher exact test and continuous variables were compared using the t test. Statistical analyses were performed in SAS version 9.4 software (SAS Institute, Cary, NC).
Results
Of 705 cases reviewed, prespecified keywords in chest imaging were present in 640 (90.8%) cases and were absent in 65 (9.2%) cases. Age, sex, and prevalence of specific comorbidities were similar between those with and those without radiographic keywords present. Patients with positive radiographic keywords had higher summative Charlson comorbidity index scores (6.75) than patients without radiographic keywords (4.86; P < .01) (Supplementary Table 1 online).
Diagnostic Test Characteristics of Radiographic Keywordsa and Selected Clinical Variables in the Diagnosis of Pneumonia
Note. WBC, white blood cell count >12 or <4 ×103 cells/µL; Temp, temperature >38.3°C or <36.1°C. O2, new sustained oxygen requirement; Sputum, documentation of sputum purulence or increased suctioning requirements in electronic medical record; Negative LR, negative likelihood ratio; CI, confidence interval.
a Keywords: “infection,” “pneumonia,” “consolidation,” “opacity (ies),” “opacification,” “infiltrate (s),” “airspace,” assuming such words were not preceded by “no” or “no evidence of” in chest radiography interpretations within 48 hours of positive respiratory culture.
Overall, 242 cases reviewed (34.3%) met the criteria for pneumonia misdiagnosis, whereas 462 (65.7%) of cases met criteria for a valid diagnosis of pneumonia warranting antibiotic treatment. Prespecified keywords were present in 450 (97.2%) of patients with a valid diagnosis of pneumonia and 190 (78.5%) with pneumonia misdiagnosis (P < .01). Radiographic keywords had a sensitivity of 97.2% (95% confidence interval [CI], 95.7%–98.7%), a NPV of 80.0% (95% CI, 70.3%–89.7%) in the diagnosis of pneumonia. Coupling select criteria (fever, leukocytosis, and/or new oxygen requirement) with radiographic keywords improved the NPV compared to radiographic keywords alone (Fig. 1 and Table 1).
Negative likelihood ratios for the absence of radiographic keywords and selected clinical variables for the diagnosis of bacterial pneumonia. Keywords: absence of “infection,” “pneumonia,” “consolidation,” “opacity(ies),” “opacification,” “infiltrate(s),” “airspace,” assuming such words were not preceded by “no” or “no evidence of” in chest radiography interpretations within 48 hours of positive respiratory culture. Note. WBC, absence of white blood cell count > 12 or < 4 x 103 cells/uL; temp, absence of temperature >38.3°C or <36.1°C; O2, absence of new sustained oxygen requirement; sputum, absence of documentation of sputum purulence or increased suctioning requirements in electronic medical record.
Discussion
In this study of hospitalized patients with positive respiratory cultures receiving antibiotics for treatment of presumptive pneumonia, the absence of specific keywords from radiologists’ chest-imaging interpretations was associated with a very low likelihood of pneumonia, particularly when combined with absence of select clinical criteria. Although a prior study has successfully employed radiographic keyword abstraction for influenza-related pneumonia surveillance,Reference Bramley, Chaves and Dawood8 this is the first study to apply radiographic keyword abstraction methods to identification of bacterial pneumonia misdiagnosis. Unlike natural language processing algorithms and machine-learning approaches to radiographic data analysis that are presently impractical for routine clinical and stewardship use,Reference Annarumma, Withey, Bakewell, Pesce, Goh and Montana9,Reference Liu, Clark and Mendoza10 keyword abstraction is a relatively simple and economical process to implement within electronic medical records. This type of strategy may be used to target patients for further assessment by ASP personnel regarding antibiotic discontinuation and de-escalation opportunities. Similarly, this strategy may assist infection prevention programs in refining local pneumonia surveillance efforts.
This was a single-center study, and patterns of radiologist reporting verbiage may not be generalizable to other institutions. We also utilized a reference gold standard of physician review rather than histopathologic confirmation to ascertain pneumonia diagnosis. Lastly, we used an existing cohort of patients with a high pretest probability of pneumonia (specifically, a database of patients with positive respiratory cultures who received multiple days of antibiotics), limiting the generalizability of the results to patients with a lower pretest probability of pneumonia. However, given the robust negative predicative value of keyword abstraction in ruling out pneumonia in this pneumonia-enriched cohort, application of this methodology to patients with a lower disease prevalence is expected to improve its negative predictive value.Reference Albin, Henig and Patel6
These data support radiographic keyword abstraction as a viable method to identify patients with misdiagnosed pneumonia. The methodology and findings reported here can be utilized to design stewardship interventions aimed at reducing inappropriate antibiotic use for misdiagnosed pneumonia.
Acknowledgments
The authors are indebted to Arina Bierdz from the Data Office for Clinical & Translation Research at the University of Michigan for data extraction.
Financial support
No financial support was provided relevant to this article.
Conflicts of interest
All authors report no conflicts of interest relevant to this article.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/ice.2020.1417
