Discussion

In this study, we demonstrated the application of an objective framework to capture vancomycin “never events” (NEs). The methodology, when applied at separate hospitals using local data capture strategies, resulted in comparable performance characteristics across systems. This approach is important because contemporary tracking methodologies, such as the National Healthcare Safety Network Antimicrobial Use module, provide mechanisms to compare and benchmark hospital- and unit-specific antimicrobial consumption for acute-care centers in the United States; however, they do not classify appropriateness of individual antimicrobial use. ⁷ Although a PPV of 34% in our study may seem low at first, this methodology is designed to function as a screening test for very low probability events (ie, 0.5%–2.2% of vancomycin courses). Thus, electronic flagging of potential NEs using our methodology would be expected to generate 1 true positive NE for every 3 electronically identified NEs. Furthermore, the excellent NPV objectively ruled out patients not experiencing an NE. Overall, clinicians performing antimicrobial stewardship would only need to review ~15 potential NE instances per 1,000 vancomycin treatment days to find the ~5 true NEs.

Creating guided approaches to safely use antibiotics is important because ~55% of hospitalized patients receive ≥1 dose of antibiotic therapy, and 20% of patients receiving antibiotics for ≥24 hours experience ≥1 adverse drug reaction (ADR). ^{Reference Baggs, Fridkin, Pollack, Srinivasan and Jernigan8,Reference Tamma, Avdic and Li9} We selected vancomycin as a prototype because it is commonly used nosocomially and can cause downstream ADRs. ^{Reference Branch-Elliman, O’Brien, Strymish, Itani, Wyatt and Gupta10} Because much vancomycin use is empiric (~69.9% of vancomycin use at NMH was empiric, data not shown) and understanding the appropriateness of use has traditionally required time-consuming chart reviews, it is important for stewards to efficiently and accurately screen for potential harms. Using our methodology based on readily retrievable clinical data, we were able to comprehensively capture these NEs with relatively high sensitivity and NPV.

Although differences existed in EHR data structures and presented challenges to our initial data acquisition, our analytical framework eventually converged (Supplementary Fig. 2 online). These results demonstrate the flexibility of locally adaptive strategy with broad utility across different EHR platforms. A recent study attempted to quantify inappropriate antibiotic use in the hospital. Denny et al ^{Reference Denny, Gartside and Alcorn11} found that 32.7% of antibiotic prescriptions were inappropriate in an emergency room setting. Our study differed in that an electronic data capture strategy was used and we were able to circumvent time-intensive manual chart reviews for all patients. Nevertheless, there is an urgent need to address antibiotic inappropriateness globally.

Our study was limited by a single antibiotic and 2 specific types of NEs, which constituted only a proportion of all inappropriate antibiotic use. We did not consider thresholds for antibiotic initiation or durations of therapy. Another limitation was the relatively low PPV (~35%), driven by the low prevalence of NEs; however, NEs are clearly inappropriate uses of vancomycin. If 3 potential NEs required manual chart review to stop 1 NE, this would represent an efficient safety strategy in antimicrobial stewardship efforts. Our methodology has broad implications for future applications and may be automated in real time.