Family medicine providers are well positioned to promote responsible antimicrobial use as frontline prescribers for common infections in acute, transitional, and ambulatory settings. Reference Wiskirchen, Summa and Perrin1,Reference Roberts and Hicks2 Successful antimicrobial stewardship (AMS) strategies for ensuring appropriate antibiotic use on family medicine services include, but are not limited to, audit and feedback, restriction, cooperative guideline development, peer comparison, and academic detailing. Reference Barlam, Cosgrove and Abbo3 Often, end points related to antimicrobial consumption, appropriateness, duration of therapy, occurrence of adverse event are examined as the primary assessment of AMS interventions in family medicine services. However, it is difficult to fully assess the outcomes associated with AMS-related interventions given that events of greatest interest to clinicians, such as severe morbidity and mortality, are infrequent and that a large number of patients are needed to adequately power each comparison.
We now have an opportunity to examine the collective outcomes important to clinicians and patients in a one analysis through using desirability of outcome ranking (DOOR). Reference McGregor and Furuno4,Reference Evans, Rubin and Follmann5 DOOR, an ordinal scale, ranks the collective clinical outcome of each case using multiple end points to determine the likelihood of an improved intervention. Reference Evans, Rubin and Follmann5 Although DOOR analyses have traditionally been applied post hoc to results from randomized trials, they may also be useful in nonrandomized, retrospective designs to examine the benefits of rigorous, well-conducted AMS interventions. Reference van Duin, Lok and Earley6,Reference Cabellos, Pelegrin and Benavent7 The purpose of this study was to determine the global impact of a multifaceted AMS initiative comprised of behavioral interventions and academic detailing.
Methods
Design
In this quasi-experimental study, we evaluated hospitalized adults on a family medicine ward between January 2017 and June 2018 (Fig. 1). Henry Ford Hospital is an 877-bed tertiary-care medical center with dedicated AMS staffing from an infectious diseases physician, an infectious diseases pharmacist, and trainees responsible for audit and feedback and consultation services during the study period. The primary objective was to evaluate global outcomes for patients admitted to the 20-bed academic family medicine ward before and after an AMS initiative.
Quasiexperimental study design.
In the study, 2 groups were compared: the preintervention group (January 2017–June 2018) and the postintervention group (January 2018–June 2018), along with a parallel control arm of patients who were managed by providers who did not receive the intervention. Participants eligible for inclusion were ≥18 years of age and received at least 1 day of antimicrobial for any of the following infections diagnosed by an attending physician: pneumonia, upper respiratory tract infection, acute exacerbation of chronic pulmonary disease, pyelonephritis, complicated urinary tract infection, cystitis, wound infections, and purulent and nonpurulent cellulitis. Exclusion criteria were pregnancy, neutropenia, transfer to or from an outside institution, hospice, and leaving against medical advice. Patients were also excluded if they had been diagnosed with complex infections such as osteomyelitis, necrotizing fasciitis, diabetic foot infection, empyema, undrained abscesses, or cavitating or necrotizing pneumonia.
Intervention
In January 2018, a feedback structure was implemented for family medicine prescribers to describe unit-specific metrics and outcomes related to infectious diseases. Each month, the following interventions were provided by AMS staff:
-
Existing handheld guidelines and antimicrobial adverse event infographic were disseminated to rotating family medicine teams in 1-hour sessions. These tools highlighted health-system guidance for duration of therapy and likelihood of specific adverse drug events (ADEs) for each antimicrobial.
-
An AMS “report card” was reviewed in the first week of each month, which compared quinolone, cephalosporin, piperacillin/tazobactam, vancomycin, and narrow-spectrum oral antibiotic days per 1,000 patient days present between the family medicine ward and the internal medicine ward.
-
A dashboard of unit capacity, length of therapy for pneumonia and asymptomatic bacteriuria, incidence of C. difficile testing and positivity, and number of patients developing acute kidney injury while on vancomycin (Appendix 1 online).
-
On the third week of each month, positive feedback was provided to the department and rounding team via successful patient cases highlighting AMS in common infections and AMS objectives (Appendix 1 online).
-
An exit survey was sent to the family medicine department to evaluate provider impressions related to the intervention (Appendix 1 online).
Patient data
Microsoft SQL software (Microsoft, Redmond, WA) and Epic software (Epic, Verona, WI) were used to identify and extract encounters for hospitalized adults with ICD-10 codes for infections of interest. Data elements included demographics, prescriber type, antimicrobial consumption, electronic health record (EHR) utilization, hospitalization duration, unplanned revisits, readmissions, and microbiologic data. Demographic characteristics were age, sex, race, clinical status on admission evaluated by SIRS criteria, risk for multidrug resistance, and comorbid conditions outlined by the Charlson comorbidity index (CCI). Reference Comstedt, Storgaard and Lassen8,Reference Charlson, Szatrowski, Peterson and Gold9 Risks for multidrug resistance were prior colonization in the previous 12 months, nonambulatory status, immunocompromised status, and recent receipt of antibiotic, chemotherapy, dialysis, or inpatient hospital care in the previous 90 days. Reference Shindo, Ito and Kobayashi10 Immunocompromised status was defined as patients with metastatic cancer, hematopoietic stem-cell transplant, solid-organ transplant, or active receipt immunosuppressive medications. Reference Rubin, Levin and Ljungman11
Definitions for “antimicrobial days” and methods for calculating duration of therapy were in alignment with guidance from the Centers for Disease Control and Prevention (CDC). Reference van Santen, Edwards and Webb12 Definitions for multidrug resistance were also adapted from the CDC. 13 Prescriber type was collected as attending, fellow, resident, or midlevel practitioner. Utilization of EHR tools such as chart documentation and input of predetermined medication stop dates was collected for each case. Documentation for antimicrobial management in each progress note was audited for consistency of antimicrobial selection, durations, and dosages. Unit census was measured as the weekly mean capacity out of total available beds.
Study outcomes
Clinical resolution, defined as no additional or modified antimicrobial therapy due to poor clinical response, was assessed at discharge and at follow-up (up to 30 days after discharge) when available. Patients who died in the hospital or who required escalation or prolonging of antimicrobials due to persistent fever, leukocytosis, or clinical instability were considered to have clinical failure. Reference Kaye, Vazquez, Mathers and D’aikos14
Potential ADEs were evaluated using the Naranjo algorithm when applicable. ADEs were classified as gastrointestinal (nausea, vomiting, diarrhea without laxative administration), hematologic, hepatobiliary, renal, neurologic, dermatologic, cardiac, and anaphylaxis up to 30-days after completing the antimicrobial course. Reference Tamma, Avdic, Li, Dzintars and Cosgrove15 Severe events were C. difficile (CDI), multidrug-resistant organism (MDRO) isolation from any site, anaphylaxis, rhabdomyolysis, renal failure or loss (RIFLE criteria), drug-induced hematologic toxicity, cardiac events leading to intervention, and severe cutaneous reactions and were also evaluated up to 90 days. 13
Antimicrobial optimization was defined using classifications for appropriateness published by Spivak et al. Reference Spivak, Cosgrove and Srinivasan16 Patients receiving ≥2 days of antimicrobial therapy (both inpatient and outpatient) that were not optimal were considered to have a nonoptimal course. This cutoff was selected to allow a deviance of 1 day given the limitations using calendar days to count total antimicrobial days. A nonoptimized antibiotic day was classified as “unnecessary,” “inappropriate,” or “suboptimal” (Appendix 1 online). Reference Spivak, Cosgrove and Srinivasan16 “Unnecessary” days were defined as indications not requiring antimicrobials, prescription for duration of therapy beyond the clinical indication, or redundant antimicrobial activity. Reference Spivak, Cosgrove and Srinivasan16 An “inappropriate” day classification was administration that was not concordant with institutional practice guidelines or the targeted pathogen was resistant. “Suboptimal” classifications had antimicrobial days with excessively broad spectrum continued 24 hours after culture finalization, or therapy that was not modified based on renal function or ability to take oral medication when using highly bioavailable intravenous antimicrobials (quinolones, clindamycin, metronidazole, doxycycline). Clinical resolution, ADEs, and antimicrobial optimization were adjudicated by at least 1 nonblinded infectious disease pharmacist and/or family medicine physician (N.J.M., S.L.D., R.V., or B.R.). The electronic medication administration record was used for inpatient days, and discharge summaries were used for outpatient days.
Desirability of outcome ranking
The primary outcome was likelihood of better ranking (DOOR) between the preintervention and postintervention groups. DOOR was weighted by mortality, clinical response, ADEs, and nonoptimal antimicrobial use, as follows (Table 1). Death within 30 days from the end of therapy was the worst possible outcome; treatment success without adverse events; and an optimized antimicrobial course was defined as the “best possible clinical outcome.” Reference Evans, Rubin and Follmann5 Although the antimicrobial course was dichotomized as optimal (<2 days or ≥2 days of nonoptimal antibiotic days, respectively) in the primary analysis, additional sensitivity analyses were conducted to measure nonoptimal antimicrobial exposure as ordinal (≤1 day, 2–4 days, and ≥5 days) and continuous variables (0–n days) (Appendix 1 online). To examine the potential effects of maturation and regression to the mean, a parallel group was ascertained from an internal medicine ward, which did not receive the intervention.
Primary Desirability of Outcome Ranking (DOOR) Composition
* Best possible outcome: Clinical resolution, no adverse drug event, optimal antibiotic course.
Statistical methods
A sample size was calculated under the presumption that the AMS intervention increased the probability of having a better DOOR (60%) than the preintervention group (40%). Reference Evans, Rubin and Follmann5 At a 2-sided alpha of 0.05, and 90% power, 180 subjects were required for each comparison (G*Power version 3.1 software). Descriptive and demographic data between the preintervention and postintervention family medicine groups were compared using the χ2 test and the Fisher exact test for categorical end points and the Mann-Whitney U test for continuous and ordinal end points. Odds ratios (ORs) were calculated in univariate analyses for predictors of clinical failure, ADEs, and readmissions.
In the primary analysis, the probability of a better DOOR was deemed significantly different if the probability was >50% in the intervention group, without the 95% confidence interval crossing 50%. Logistic regression and inverse probability treatment weight (IPTW) were used to predict “best possible outcome” determined by DOOR. Covariates to control for the best possible outcome were determined a priori based on previous literature and clinically significant findings in unadjusted analyses (Appendix 1 online). Statistics were calculated with SPSS version 24 software (IBM, Armonk, NY).
Results
Patient characteristics
The 626 subjects were screened for inclusion on family medicine and internal medicine wards. Patients were excluded for the following reason: complicated infections, no receipt of antibiotic therapy, discharge to hospice, transferred or left against medical advice, and on antibiotic therapy at the time of admission (Fig. 1). In the family medicine ward, 256 patients (129 preintervention and 127 postintervention) were included and 117 patients were included in a parallel nonintervention arm. Demographics and infection types at time of admission were similar (Table 2). The hospital census was lower during the preintervention period but with no differences in average length of stay. Most of the population were women (56%), and electronic medical record (EMR) coding for race identified 67% as black, 17% as white, 3% as Hispanic, 1% as Asian, and the remainder as unknown.
Patient Demographics and Baseline Characteristics
Note. IQR, interquartile range; SD, standard deviation; UTI, urinary tract infection; AECOPD, acute exacerbation of chronic obstructive pulmonary disease; MDRO, multidrug-resistant organism.
* P<.05.
Clinical outcomes
Clinical resolution at time of discharge was 91.5% vs 96.1% between the preintervention and postintervention groups, and 80.9% versus 88.2% at follow-up (Table 3). Readmission and unplanned revisit events at 30 days were not significantly different. The most common ADEs were renal (9.3%), gastrointestinal (7.4%), cardiac (5.1%), and isolation of MDRO (4.3%). Gastrointestinal ADEs and testing for CDI were higher in the preintervention family medicine group. There were no differences in readmissions, clinical resolution, or ADEs between groups in the parallel control. In the patients with available follow-up from healthcare visits, the development of ADEs (unadjusted OR, 2.74; 95% CI, 1.43–5.28) and receipt of a nonoptimal antimicrobial course (unadjusted OR, 2.35; 95% CI, 1.17-4.72) were associated with clinical failure.
Clinical Resolution, Readmissions, and Adverse Events
Note. ADE, adverse drug event; MDRO, multidrug-resistant organism.
* P < .05.
The total optimal antimicrobial days increased in the postintervention group, while number of nonoptimal days was reduced (Fig. 2). This was driven by a reduction in prolonged duration of therapy (40.3% vs 22.0%; P = .002) and treatment of asymptomatic bacteriuria (16.3% vs 6.3%; P = .012). Decreases in inappropriate and suboptimal antimicrobial classifications were driven by reductions in antimicrobial selection nonconcordant with hospital guidelines (17.8 vs 10.2%; P = .081), and continuation of highly bioavailable intravenous antibiotic when patient was able to tolerate oral administration (10.9 vs 2.4%; P = .006). More than one-third of patients (36.4%) in the preintervention group received ≥5 days of nonoptimal therapy compared to only 7.1% following the intervention. No significant differences in utilization of antibiotic classes following the intervention were observed, with the exception of a decrease in the use of antipseudomonal agents (34.9% vs 22.8%; P = .033) and fluoroquinolones (35.7% vs 22.8%; P = .024). Uptake of inpatient electronic stop-date entry also increased from 8.5% to 34.6% (P < .001). In the parallel control arm, there were no significant differences in unnecessary, inappropriate, or suboptimal antimicrobial days.
Prevalence of antimicrobial optimization on the FAM ward before (O1–O6, January 2017–June 2017) and after (O7–O12, January 2018–June 2018) the intervention.
DOOR
Of the 373 patients evaluated, 116 experienced the best possible outcome. Patients in the family medicine postintervention group were more likely to have a better global outcome compared to the preintervention group (62.0%; 95% CI, 59.6–67.1). The intervention was independently associated with the best possible outcome while hospital length of stay and Charlson comorbidity index ≥2 were negative predictors (Table 4). The intervention remained an independent predictor of best possible outcome after controlling for other covariates in inverse probability treatment weighting (IPTW) (Adj OR, 2.86; 95% CI, 1.38–5.92) (Appendix 1 online). In a sensitivity analysis, 3 scales for nonoptimal antimicrobial utilization (dichotomous, ordinal, and continuous) in the DOOR analyses remained significantly improved in the postintervention family medicine arm for each approach (Appendix 1 online). The parallel postintervention group was not more likely to have a better global outcome compared to the control preintervention group (48.5%; 95% CI, 44.6–51.9). The results of the exit survey were reflective of the program’s impact: most providers found the interventions to be helpful, with appropriate frequency and content, and they were interested in continuing to receive the interventions and feedback (Appendix 1 online).
Logistic Regression to Predict “Best Possible Outcome”
Note. OR, odds ratio; CI, confidence interval; MDRO, multidrug-resistant organism. Hosmer and Lemeshow test, 0.604, omnibus test, <0.001.
Discussion
Family medicine and AMS partnered in a multimodal program including behavioral and educational interventions. In addition to supporting clinicians with prescribing tools, academic detailing, and positive deviance (Appendix 1 online), direct reporting of unit-based family medicine stewardship metrics improved patient care.
As health systems continue to accumulate antimicrobial consumption data, sharing performance metrics with end users represents a practical initiative for promoting inpatient AMS. Reference Wu, Webb and O’Leary17 Beyond reporting antimicrobial consumption, incorporating communication of relevant outcomes (eg, C. difficile or AKI) in a timely, transparent process can increase the value and uptake of interventions. Stakeholder engagement was fundamental in the success of our initiative; family medicine clinicians were dedicated partners for growth in AMS. Reference Hurst, Child, Pearce, Palmer, Todd and Parker18 Feedback and interventions were targeted toward the team and department rather than individuals for feasibility and promotion of positive feedback. All handheld tools, reports, cases, and end points were also developed in a collaborative effort between the family medicine and pharmacy departments. This effort was sustained when measured over 6 months in the postintervention period (Fig. 2).
When evaluating AMS programs, end points are typically measured in isolation—DOOR was advantageous in this family medicine initiative because of its ability to analyze the most important end points for common infectious diseases as a composite. The individual end points that represent DOOR should be clinically significant outcomes and should not overrepresent the intervention in the ranking. Reference Molina and Cisneros19 We used focus groups consisting of infectious diseases pharmacists and family medicine physicians to determine which outcome measures would be most impactful: clinical resolution, ADEs, and antimicrobial optimizations. Classifications of inappropriate, unnecessary, and suboptimal therapy all have been associated with poorer outcomes. Reference Spivak, Cosgrove and Srinivasan16 McCabe et al Reference McCabe, Kirchner, Zhang, Daley and Fisman20 found that using guideline-concordant recommendations for pneumonia led to reduced mortality and length of stay. Patients on inappropriate therapy for resistant pathogens are widely expected to experience failure. Unnecessary antibiotics are associated with harm; each additional antimicrobial day is associated with increased risk of ADEs. Reference Tamma, Avdic, Li, Dzintars and Cosgrove15,Reference Branch-Elliman, O’Brien, Strymish, Itani, Wyatt and Gupta21-Reference Cai, Mazzoli and Mondaini23 Only 10 subjects were classified with a nonoptimal course based on suboptimal classification alone, which was related to continued use of intravenous therapy with a highly bioavailable agent and failure to de-escalate following culture results. Although these suboptimal exposures have been associated with CDI, ADEs, and vascular access complications, this population may represent a lower risk of harm than those receiving inappropriate or unnecessary therapy. Reference Seddon, Bookstaver and Justo24,Reference Keller, Williams and Gavgani25 When this subset was excluded in a sensitivity analyses, the probability of having a better outcome remained improved. Further validation should be pursued for DOOR with common infectious diseases to be standardized as a valuable end point in AMS.
Traditionally, DOOR has not been used to measure the impact of interventions in AMS, but it has been useful for highlighting improved management strategies for common infectious indications. Lodise et al Reference Lodise, Rosenkranz and Finnemeyer26 found that patients exposed to lower vancomycin area under the curve (AUC) ranges for methicillin-resistant Staphylococcus aureus (MRSA) bacteremia had achieved better global outcomes than higher AUC exposures, which was largely driven by reduced acute kidney injury. In a similar analysis, ceftazidime-avibactam was more likely to have improved efficacy and safety outcomes evaluated using DOOR than colistin in a cohort with carbapenem-resistant Enterobacteriaceae infections. Reference van Duin, Lok and Earley6 In less complex disease states, such as those our population, there is less contribution of death and serious ADEs to the global outcome given the reduced severity of illness and use of toxic antimicrobials. Celestin et al Reference Celestin, Odom and Angelidou27 retrospectively applied DOOR-RADAR to the STOP-IT trial, in which surgical patients with intra-abdominal infections were categorized based on recovery, adverse events, extra-abdominal infections, recurrent infections, and death. Further ranking was then applied using total antibiotic days. Reference Celestin, Odom and Angelidou27 We used a modified strategy that classified rankings based on clinical resolution, ADEs, and nonoptimal antibiotic days. Reference Celestin, Odom and Angelidou27 Application with total duration of therapy would not have been ideal in our study given the diversity of infection types and different durations that are prescribed. With stringent and universal definitions for optimized antimicrobial use, this scale can potentially serve as an improved AMS metric to evaluate interventions across heterogeneous groups.
Quasi-experimental designs have inherent limitations such as retrospective data adjudication, maturation, regression to the mean, and the Hawthorne effect. Reference Harris, Bradham, Baumgarten, Zuckerman, Fink and Perencevich28 To control for population differences and potential confounders, logistic regression and IPTW analyses were conducted by dichotomizing DOOR as the best possible outcome, or worse. The intervention was independently associated with 3 times the odds of achieving the best possible outcome. Optimal prescribing improved leading up to the intervention period, which may be a result of maturation during the academic training calendar (Fig. 2). However, no other changes to the AMS model or antimicrobial usage guidelines were noted during the study period. Additionally, there was no difference in the likelihood of a better DOOR in the sensitivity analysis between the preintervention and postintervention periods in the parallel control group, which did not receive the AMS intervention. Although the interventions required time-intensive planning, education, collaboration, and reporting, the improvements in outcomes, safety, and prescribing support the implementation of this program on a broader scale.
Better patient outcomes and antimicrobial usage were observed in hospitalized adults with urinary, respiratory, and skin and/or soft-tissue infections following an initiative that included feedback of AMS metrics, prescribing tools, and positive-deviance case vignettes. Patients receiving optimized antimicrobial regimens were less likely to experience clinical failure. Analysis with DOOR was a valuable for evaluating global outcomes with antimicrobial stewardship interventions in a real-world settings.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/ice.2020.288
Acknowledgments
None.
Financial support
No financial support was provided relevant to this article.
Conflicts of interest
All authors report no conflicts of interest relevant to this article.
