Antibiotic resistance is a problem that is primarily a consequence of widespread antibiotic use and misuse. A Centers for Disease Control and Prevention (CDC) report estimated that 2.87 million antibiotic-resistant infections occur in the United States annually. 1 Antibiotic-resistant infections increase US healthcare costs, including an additional $20 billion in direct costs and $35 billion in indirect costs (calculated in 2008 dollars) annually.
Antibiotic stewardship is the systematic effort to measure and improve how antibiotics are prescribed to mitigate the development of resistance and other harmful effects of antibiotic overuse. To promote optimal antibiotic use, guidelines for antibiotic stewardship programs (ASPs) in multiple practice settings have been published. Reference Barlam, Cosgrove and Abbo2,3 To improve outpatient prescribing, the CDC created the Core Elements for Outpatient Antibiotic Stewardship. Reference Sanchez, Fleming-Dutra, Roberts and Hicks4 The Joint Commission published antibiotic stewardship requirements for ambulatory healthcare organizations. 5 Delivery of antibiotic stewardship requires a commitment of economic resources. Although cost-effectiveness analyses (CEAs) of antibiotic stewardship have been conducted in inpatient settings, only 1 CEA has been conducted in the outpatient setting Reference Naylor, Zhu, Hulscher, Holmes, Ahmad and Robotham6–Reference Gong, Zangwill, Hay, Meeker and Doctor8 despite evidence that most antibiotics are prescribed to outpatients. 5 Acute respiratory tract infections (ARIs) are common outpatient diagnoses for which antibiotics are prescribed, and up to 50% are prescribed inappropriately. Reference Fleming-Dutra, Hersh and Shapiro9
Beginning in September 2017, an antibiotic stewardship intervention based on the Core Elements framework was launched at 10 outpatient Veterans Healthcare Administration (VHA) sites. Reference Madaras-Kelly, Hostler and Townsend10 The intervention used a provider-directed audit feedback and academic detailing approach to promote appropriate diagnosis and treatment of ARIs. Academic detailing consists of noncommercial, peer-to-peer communication using reinforcement techniques to facilitate change in prescribing practices. Audit feedback refers to providing clinicians with a summary of performance on healthcare indicators over time for the purpose of improving performance. Reference Avorn and Soumerai11,12 We evaluated the cost-effectiveness of the intervention compared to usual care, using a decision analytic model.
Methods
Decision analytic model
An economic simulation model was developed from the health-system perspective for patients presenting to an outpatient setting diagnosed with an ARI. Target ARI diagnoses included acute sinusitis, pharyngitis, bronchitis, and other viral upper respiratory tract infections not otherwise specified (URI-NOS). To limit the analysis to visits for uncomplicated ARIs, patients with pre-existing conditions (ie, chronic pulmonary disease, immunosuppression, chronic sinusitis or pharyngitis) or an ARI within the previous 30 days were excluded. Clinical champions within sites with facilitation by VHA Medical Center antibiotic stewards, delivered provider-level audit feedback at 2–3-month intervals for 12 months coupled with an initial academic detailing visit. A full description of the study intervention and results are available. Reference Madaras-Kelly, Hostler and Townsend10 We compared the cost and effectiveness of the intervention versus usual care (ie, no intervention). The effectiveness measure was quality-adjusted life-years (QALYs), a metric that encompasses both duration and quality of life. Costs included those related to treatment of ARIs and associated care, and cost incurred to implement the intervention. Model parameter inputs were obtained from published literature and from study data obtained from patient visits for ARIs in the intervention and control sites.
Patients enter the model by having an outpatient visit with a diagnosis of an ARI (Fig. 1). Next, patient visits are classified as having an antibiotic prescribed or not. Based on the specific ARI diagnosis and patient characteristics, this treatment is classified as appropriate or not. Appropriate therapy for acute bronchitis or URI-NOS was defined as symptomatic management (no antibiotics prescribed), whereas for pharyngitis prescription of a guideline recommended antibiotic (penicillin or amoxicillin except in cases of penicillin allergy) was considered appropriate if the patient had a positive group A Streptococcus test. Reference Shulman, Bisno and Clegg13 Appropriate therapy for sinusitis was based on prescription of a guideline recommended antibiotic; amoxicillin or amoxicillin/clavulanate except in cases of penicillin allergy. Reference Chow, Benninger and Brook14,Reference Rosenfeld, Piccirillo and Chandrasekhar15 After the initial visit, patients could present for follow-up which would require additional resources. For example, patients might experience poor resolution, worsening of symptoms, secondary infections, or an adverse drug event (ADE) requiring return visits or hospital admission. The model was programmed using TreeAge Pro 2018 software (TreeAge Software, Williamstown, MA).
Decision analytic model. Note. ADE, adverse drug event; ARI, acute respiratory tract infection.
Input parameters
The movement of hypothetical patients through the model is governed by probability input parameters. Probabilities included those for antibiotic treatment appropriateness, antibiotic ADEs, hospitalization, and return ARI visits, which were estimated based on a prior analysis of the intervention outcomes (Table 1). Reference Madaras-Kelly, Hostler and Townsend10 Estimates of the intervention effect on the occurrence of these events were obtained from difference-in-differences (DD) analyses, which minimized the potential time-varying external effects from the overall trend in probabilities. Reference Madaras-Kelly, Hostler and Townsend10
Probability Input Parameters for the Decision Analytic Model
Note. ARI, acute respiratory tract infections.
a Probabilities were estimated using the results from the difference-in-difference analyses. Reference Sanchez, Fleming-Dutra, Roberts and Hicks4
b Definitions of appropriate therapy: Acute bronchitis or upper respiratory tract infections not otherwise specified (URI-NOS), no antibiotic prescribed. Sinusitis: aminopenicillin, or in the case of a penicillin allergy, doxycycline, or a respiratory tract fluoroquinolone. Pharyngitis: aggregate of no antibiotic therapy for patients with a negative group A strep test or culture (or test not performed), or penicillin/amoxicillin for a positive test, except in cases where penicillin allergy was identified and cephalexin or clindamycin was considered appropriate.
c 30-d hospitalization includes non–ARI-related cases.
For DD analyses, control sites were selected in a 4:1 ratio from VHA sites that were unlikely to have received a similar ARI intervention and matched with intervention sites. Reference Madaras-Kelly, Hostler and Townsend10,Reference Madaras-Kelly, Hruza and Pontefract16 The preimplementation period was from October 2014 through September 2017, and the intervention implementation occurred between September 2017 and January 2018. The postimplementation observation period spanned October 2017 through March 2019 depending on the site implementation date. All intervention sites continued the intervention for at least 12 months. The DD models adjusted for time trend, baseline antibiotic prescription rates, month of year, patient age and temperature, and provider type. We assumed that 76% of patients who were not given antibiotic were treated with symptomatic therapy based upon a national utilization review of outpatient ARI management. Reference Bohan, Madaras-Kelly and Pontefract17,Reference Pontefract, Burke, Cunningham and Goetz18
In total, 16,712 and 51,275 patient visits in the intervention and control sites were evaluated, respectively. Patients had a median age of mid-fifties and were primarily male. To estimate the probability input parameters for the intervention cohort, we applied the relative changes from the DD analyses to the values of preimplementation period for the usual care cohort. Antibiotic prescribing was significantly lower for the intervention sites compared to the usual care sites (62% vs 74%) while the rate of appropriate antibiotic prescribing was higher for the intervention compared to the usual care (51% vs 41%) (Table 1). The rates of 14-day ADEs were 1.9% for the intervention and 1.7% for the control. The rates of hospitalization were 1.6% for the intervention and 1.9% for the control. The rates of 30-day ARI-related return visits were 10.4% for the intervention and 9.8% for the control. Reference Madaras-Kelly, Hostler and Townsend10
Intervention cost was estimated using data provided by the National VHA Academic Detailing Service for a similar ARI-focused campaign. Reference Madaras-Kelly, Hruza and Pontefract16 Costs included those associated with the development of the audit feedback and academic detailing materials and electronically captured time for personnel delivering the campaign. The total cost for intervention material development was estimated at $94,052 including costs to make the ARI campaign education tools, build an electronic graphical interface (ie, dashboard) to generate audit feedback reports and SharePoint site and to coach local and regional staff to implement the campaign. Additionally, time spent on intervention implementation at the 10 sites was electronically documented by provider type for clinic champions, their surrogates, and antibiotic stewards. The cost of intervention implementation time was estimated to be $39,551 from 2016 to 2018, using the median wage of each provider type from the US Bureau of Labor Statistics 2018. 19 The total estimated 2-year intervention cost was $133,604 in 2018 US dollars (Table 2). Although the postimplementation observation period of this study was 1 year, total 2-year estimated cost was used because most costs were related to development of intervention tools prior to implementation rather than labor costs to deliver the intervention.
Cost of Care and Utility Input Parameters for the Decision Analytic Model
Note. ARI, acute respiratory tract infections; ADE, adverse drug effect; VHA, Veterans Healthcare Administration; CMS, Centers for Medicare & Medicaid Services.
a Mild ADE, ADE not requiring hospitalization.
b Severe ADE, ADE requiring hospitalization.
Cost and utility parameters for ARI treatment, ADEs, and healthcare utilization were obtained from the published literature (Table 2). Costs of antibiotic and symptomatic therapy treatment were obtained for ARI using the Centers for Medicare & Medicaid Services physician fee schedule or VHA federal supply schedule. Reference Gong, Zangwill, Hay, Meeker and Doctor8,Reference Llop, Tuttle, Tillotson, LaPlante and File20 Hospitalization costs were estimated from studies evaluating ASPs for patients with suspected sepsis or lower respiratory tract infections, which occurred infrequently but were the most commonly encountered admitting diagnoses during the intervention. Reference Mewes, Pulia and Mansour21,Reference Balk, Kadri, Cao, Robinson, Lipkin and Bozzette22 Costs for mild complications not requiring hospitalization were obtained from a study on the economic burden of non–influenza-related viral respiratory tract infection including sinusitis. Reference Fendrick, Monto, Nightengale and Sarnes23 We assumed that the cost of a return clinic visit was the same as the initial clinic visit.
Effectiveness was measured in QALYs, and utility was measured on a scale in which 1.0 represented a state of perfect health and 0.0 represented death. Base utility for ARI without complications or additional utilization was estimated at 0.87. Reference Luo, Johnson, Shaw, Feeny and Coons24 Disutility of antibiotic use, symptomatic therapy, mild or severe ADE from antibiotics, inpatient complications, ED visit for infection, and hospitalization for severe infection were determined from previously published studies. Reference Bergus, Weber, Ernst and Ernst25–Reference Johnson, Banzhaf and Desvousges28 Disutility of a return ARI visit was assumed to be same as disutility of symptomatic treatment.
All costs were adjusted to 2018 US dollars using the personal consumption expenditures price index for healthcare services.
Sensitivity analysis
Point estimates for each input parameter were used in the base-case analysis. Next, we performed a one-way sensitivity analysis in which we varied the number of patients seen at the intervention sites given that most of the intervention cost was fixed (ie, the total intervention cost was the same regardless of the number of patients seen). In an additional one-way sensitivity analysis, we varied the intervention cost from 50% to 100% of the base-case value. We also performed a one-way threshold analysis to identify the intervention cost for which the incremental cost-effectiveness ratio (ICER) exceeded the willingness-to-pay (WTP) threshold of $100,000 per QALY.
A probabilistic sensitivity analysis (PSA) was performed to assess the impact of uncertainty in all probabilities, utility, and cost parameters simultaneously using 1,000,000 (1,000 trials of 1,000 hypothetical patients each) simulated patients. The PSA parameter values were based on random draws from a distribution. Probabilities and utilities were assumed to follow either a beta distribution or a triangular distribution, and cost parameters were assumed to follow a gamma distribution. Reference Briggs29
Results
In the base-case analysis, mean total costs per patient were higher for the usual care strategy ($565) compared to the antibiotic stewardship strategy ($528) (Table 3). The usual care strategy yielded 0.868 versus 0.869 QALYs for the intervention strategy. Overall, the antibiotic stewardship intervention was $37 (7%) less costly and yielded an additional 0.001 QALYs (1.2%) per patient compared to the usual care. These results suggest that the antibiotic stewardship intervention was the dominant treatment strategy because it produced slightly more QALYs at a slightly lower cost compared to usual care.
Base-Case Cost-Effectiveness Analysis Results
Note. ICER, incremental cost-effectiveness ratio. QALY, quality-adjusted life year.
Major differences between the intervention and the usual care strategies were the rate of antibiotics prescribed and appropriateness of the treatment; however, the difference in the rate of hospitalization between 2 strategies was the factor that led the biggest cost difference in the results because of the high cost of hospitalization if incurred. Although minimal, QALYs were gained by fewer antibiotics prescribed under the intervention, compared to the usual care.
Threshold analyses demonstrated that the intervention strategy would be dominant (ie, both less costly and more effective) as long as the intervention cost was <$346,923 or the total number of patient visits under the intervention strategy was at least 1,282. In our study, we estimated the intervention cost to be $133,604, and 3,273 patient visits were included. Because the threshold values for these inputs were not close to the actual values, our study results were not sensitive to the intervention cost. Additionally, the per-person mean cost savings could increase from $37 to $63 if the intervention cost was reduced by 50%.
The rate of hospitalization played a significant role in the overall cost incurred in each strategy. The rate of hospitalization was 1.6% for the intervention verses 1.9% for the usual care strategy in the base-case model. One-way sensitivity analysis demonstrated that the intervention strategy was no longer cost-effective at a WTP of $100,000 per QALY, when the rate of hospitalization for the intervention was ≥1.8%.
Finally, our PSA suggested that the results were robust to variation in the base-case values of probability, utility, and cost parameter values. The ASP intervention was dominant in 63% and cost-effective at a WTP of $100,000 per QALY in 67% of the 1,000,000 Monte Carlo iterations (Fig. 2).
Scatter plot of results from probabilistic sensitivity analysis. Note. WTP, willingness-to-pay; QALY, quality-adjusted life year.
Discussion
We assessed the cost-effectiveness of Core Element implementation to improve ARI management in an outpatient setting. To our knowledge, this is the second CEA on an antibiotic stewardship outpatient-focused intervention using US real-world data including cost to implement the intervention. Reference Gong, Zangwill, Hay, Meeker and Doctor8 Our analysis suggests that implementation of the outpatient provider-directed audit feedback and academic detailing intervention based on the CDC Core Elements yielded cost savings. For uncomplicated ARI patients, the intervention was safe and was associated with lower costs of $37 per patient without affecting quality of life. The cost savings were mainly driven by reductions in antibiotic prescribing and hospitalization after an ARI visit where most the cost difference resulted from reduced inpatient cost.
These findings were robust with regard to variation in complication rates or other probability parameters, utility or disutility of each health state and utilization, and treatment costs, except for the rate of hospitalization. One-way sensitivity analysis demonstrated that the intervention approach is a dominant strategy as long as the implementation cost is less than $346,923. We applied a conservative measure of intervention cost using the full 2-year implementation cost of $133,604 from a nearly identical National VHA Academic Detailing Service for the ARI-focused campaign. Most of the intervention costs were related to development costs prior to implementation rather than labor costs to deliver the intervention during the 12-month observation period. A longer observation period of the intervention potentially achieves further cost savings because most costs are fixed. Institutions interested in developing similar interventions might realize further reductions in start-up costs by utilizing similar intervention resources. Reference Madaras-Kelly, Hostler and Townsend10,Reference Yadav, Meeker and Mistry30 Likewise, the number of patients impacted by the intervention is an important factor to determine cost-effectiveness because the fixed costs per patient decrease as the number of patients increases. The intervention sites provided 3,273 patient visits, which was >2.5-fold the cost-effectiveness threshold. These findings suggest that the more patients who receive care with the intervention, the greater cost-effectiveness benefit. Finally, our results were fairly sensitive to the rate of hospitalization, which was a key factor of cost saving in terms of its magnitude. Although the DD analysis in the original study identified a small but significant reduction in hospitalization between usual care and the intervention, the reasons for this effect are unknown. Possible explanations include enhanced diligence in applying respiratory tract–related diagnostic and treatment criteria, unmeasured differences in patient comorbidity, and differences in practice across sites. We conclude that the intervention compared to usual care is the preferred strategy for patients with uncomplicated ARI visits conditional on the decreased chance of hospitalization because of the intervention. These findings may provide decision makers with a comparable benchmark for evaluating the antibiotic stewardship intervention.
Cost-effectiveness evidence for ASP is limited, especially for the outpatient setting. Recent studies on the cost-effectiveness of ASP suggest that they may provide health economic benefits in inpatient settings. Reference Naylor, Zhu, Hulscher, Holmes, Ahmad and Robotham6,Reference Ibrahim, Maruan, Mohd Khairy, Hong, Dali and Neoh7 Although the magnitude of benefit varied significantly across studies, ranging from <$100 to >$3,000 per patient for sepsis and lower respiratory tract infection, Reference Llop, Tuttle, Tillotson, LaPlante and File20,Reference Voermans, Mewes, Broyles and Steuten31,Reference Collins, Brockhaus, Sim, Suneja and Malani32 or bloodstream infection. Reference Scheetz, Bolon, Postelnick, Noskin and Lee33,Reference Pliakos, Andreatos, Shehadeh, Ziakas and Mylonakis34 Most analyses did not include intervention cost, Reference Scheetz, Bolon, Postelnick, Noskin and Lee33 which might have resulted in overestimation of benefit. A similar outpatient stewardship study to ours demonstrated a cost saving of $5–$6 per person and an increased utility of ∼0.05 depending on the types of behavioral economic interventions for ARIs. Reference Gong, Zangwill, Hay, Meeker and Doctor8 This report was in line with our findings, although the intervention cost was not incorporated in the model. Furthermore, the outpatient provider-directed audit feedback and academic detailing intervention was a population-based approach which differs from the individual patient audit feedback approach utilized in most inpatient settings. Although the latter requires limited fixed costs to implement, the labor to maintain patient-level review of antibiotic prescribing is more intensive than the audit feedback and academic detailing approach which requires only periodic intervention. Further economic comparisons of the 2 interventional approaches are needed.
Study limitations should be considered when interpreting the generalizability of our results. The data underlying the input parameters were obtained from literature and retrospective samples obtained within VHA sites. The one-way sensitivity analyses and PSA ensured that the results were fairly robust to the uncertainties and variations of the input parameters; however, there could still be publication bias in treatment costs or utility values that could affect the results. The use of VHA CDW data to populate the input probabilities may limit the generalizability of our results to other healthcare systems due to the skewed population distribution toward older males. However, 2020 changes to ARI-related Healthcare Effectiveness Data and Information Set measures with similar patient exclusion criteria and antibiotic appropriateness parameters as our intervention apply to all adults. 35 Additionally, this study was limited to patients with uncomplicated ARIs. Although it excludes patients presenting with ARIs that had select pre-existing conditions, we expect that the potential impact of a more broad-scale antibiotic stewardship intervention is even greater.
One of the most important goals of antibiotic stewardship is to limit the development of antibiotic resistance because infections caused by antibiotic resistant pathogens lead to higher morbidity, mortality, and cost. 1 Although we did not include antibiotic resistance in our model, it is likely that this omission favors the intervention strategy meaning that our findings of dominance for the intervention are conservative. Finally, a key variable with potential to impact study results was our estimate of the intervention cost. Although we did not have estimates of the actual cost of the intervention of interest in our study, we could use detailed data from a nearly identical intervention as a proxy measure for this input. We also conducted a sensitivity analysis of our model with a wide range of values for this parameter and found that the intervention only ceased to be dominant when this cost was excessively high (nearly $350,000).
In conclusion, we found that an outpatient provider-directed audit feedback and academic detailing antibiotic stewardship intervention based on the Core Elements was a cost-effective strategy for ARIs compared to usual care in VHA sites. By integrating information about health outcomes and healthcare costs, this CEA can help in evaluating the value of ASP intervention for clinical decision making. The results of this CEA may support the implementation of ASPs in the VHA outpatient setting. Further research in non-VHA healthcare systems is needed to confirm the cost-effectiveness of ASPs for other populations.
Acknowledgments
We recognize the dedication of clinician participants, site champions, and site investigators at the Durham, Eastern Kansas, Salt Lake City and Greater Los Angeles VA Medical Center as well as the National VA Academic Detailing Service for their commitment to providing quality care for US veterans. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Department of Veterans Affairs or the Centers for Disease Control and Prevention (CDC).
Financial support
This work was supported in part with resources and use of the US Department of Veterans Affairs and was funded by the CDC (Safety and Healthcare Epidemiology Prevention Research Development contract no. 200-2011-47039).
Conflicts of interest
All authors report no conflicts of interest relevant to this article.
