Clostridioides difficile, the most common pathogen to cause healthcare-associated infections, is estimated to result in nearly 500,000 infections associated with ∼29,000 deaths and >US$1.5 billion annually in the United States. Reference Magill, Edwards and Bamberg1–Reference Guh, Mu and Winston3 Accurate and timely diagnosis of C. difficile infection (CDI) is key to allowing for both early initiation of treatment and implementation of infection control measures. However, C. difficile tests (CDTs) are imperfect. Concerns related to decreased sensitivity of toxin-based assays previously led many hospitals to transition to more sensitive polymerase chain reaction (PCR) assays, resulting in a significant increase in observed cases of CDI on the basis of testing alone. Reference Moehring, Lofgren and Anderson4 Subsequent studies showed that PCR and similar tests were not specific for active disease and that diagnostic stewardship was needed to avoid overdiagnosis of colonized patients with borderline symptoms or alternative causes of symptoms. Reference Polage, Gyorke and Kennedy5 Overdiagnosis may lead to unnecessary treatment, which in turn may result in drug toxicity, antimicrobial resistance, persistent perturbation of the patients’ microbiome, patient anxiety, and increased healthcare costs. Furthermore, Centers for Medicare & Medicaid Services (CMS) sets benchmarks for hospitals regarding hospital-onset (HO) CDI, and poor performing hospitals may receive financial penalties through the Value-Based Purchasing Program. 6
To avoid overtesting patients who are colonized but unlikely to have clinical CDI, clinicians are advised to use clinical evidence when determining whether to test for C. difficile. Reference McDonald, Gerding and Johnson7 Multiple recent studies have demonstrated that clinical decision support incorporated into the electronic health record (EHR) can aid in reducing inappropriate C. difficile testing and overall hospital-onset CDI rate. Reference Cook, Nichols, Coogan, Opera and DeHart8–Reference White, Hamilton, Pegues, Hanish and Umscheid14 However, additional research has noted that clinicians often disregard such guidance and rely instead on clinical judgement and personal beliefs about CDI. Reference Friedland, Brown, Glick, Lusby, Lemkin and Leekha10,Reference Kara, Tahir, Snyderman, Brinkman, Fadel and Dbeibo15,Reference Kinlay and Sandora16
We implemented a series of clinical decision support interventions in the EHR from January 2018 to July 2019 with the goal of increasing appropriateness of C. difficile testing across our 3-hospital health system. The final EHR change was coupled with a laboratory policy to cancel tests when specified patient parameters were not met. Additionally, 1 hospital in our system independently implemented a policy requiring administrative approval to proceed with any CDT after hospital day 3. We sought to understand the relative impact of each successive change on rates of C. difficile testing and HO-CDI.
Methods
Setting
The Duke Health System includes 3 hospitals located in Raleigh and Durham, North Carolina. Hospital A is a 957-bed tertiary-care academic hospital that provides specialty care, including adult and pediatric bone marrow transplant, oncology, and solid-organ transplant care. Hospitals B and C are 354-bed and 175-bed academic-affiliated community hospitals providing medical and surgical care to adult patients from the surrounding community. General surgery trainees provide inpatient services at all 3 hospitals, whereas internal medicine trainees provide inpatient services at hospitals A and B only. All 3 hospitals share the same electronic medical record (Epic Systems, Verona, WI).
Microbiology laboratory practices
The CDTs were performed using standalone qualitative real-time PCR (Xpert C.difficile/Epi, Cepheid, Sunnyvale, CA) following the manufacturer’s instructions for the entire study period. Baseline laboratory protocols rejected formed stool specimens and limited repeated testing within 7 days of a previous negative test and 14 days of a previous positive test.
EHR changes
Three EHR changes aimed at improving the appropriateness of CDTs ordered for inpatients were made in an iterative fashion (Fig. 1). No changes were made to ordering processes in outpatient or emergency room environments.
-
1) Beginning January 2018, clinicians entering CDT orders for patients who received 1 or more doses of a laxative/stool softener medication in the preceding 24 hours received an alert prompting the clinician to remove the CDT order, discontinue the medication, and observe for persistence of diarrhea before proceeding with testing (Table 1; Fig. 2A). Clinicians retained the ability to override the alert if they believed that a CDT was clinically indicated.
-
2) Beginning October 2018, CDT orders that were > 24 hours old were automatically canceled if no stool specimen had been collected. The goal of this change was to decrease performing CDTs on patients without clinically significant diarrhea.
-
3) In July 2019, contextual rule-driven order questions were embedded in the CDT orders for patients who were hospitalized >3 days. These questions required providers to justify proceeding with testing in the following scenarios: (1) the patient had received laxatives within the past 24 hours and had an abnormal white blood cell count (WBC) or (2) the patient did not have documentation of clinically significant diarrhea, defined as ≥3 bowel movements or >600 mL recorded stool output, in the preceding 24 hours (Fig. 2B). CDTs were flagged in the EHR at the time the order was placed and were canceled by laboratory personnel in the following scenarios unless the ordering provider contacted the laboratory to request an exception to policy: (1) the patient was a child <1 year of age or (2) the patient had received laxatives within 24 hours and had a normal WBC (Fig. 2C).
Common Medications Included in the Clostridioides difficile Test Order Laxative Alert
Timeline of quality improvement activities and implementation of electronic health record (EHR) changes aimed to improve the appropriateness of C. difficile testing at Duke Health.
Clinician view of laxative alert and C. difficile test orders. (A) Laxative alert. (B) Contextual rule-driven order question requiring additional clinician justification due to insufficient documentation of diarrhea. (C) Contextual rule-driven order question indicating laboratory cancellation due to recent laxative use and normal white blood cell count.
Education on CDT appropriateness and the rationale for the EHR changes was provided to clinicians through memos and in-person presentations such as grand rounds and department meetings throughout the implementation period.
Other interventions
Hospital C independently implemented a “gatekeeper” function in July 2018, requiring either infectious diseases consultant or chief medical officer approval for a CDT after hospital day 3.
Reduction of HO-CDI was a health-system–wide priority. Throughout the study period, all 3 hospitals emphasized infection prevention measures such as hand hygiene, use of contact enteric precautions for the duration of hospitalization of CDI patients, cleaning and disinfection of shared equipment and high-touch surfaces in the patient care environment, and enhanced terminal bleach disinfection with adjunctive ultraviolet (UV)-C light for hospital rooms of patients known to have CDI. However, no major changes to existing policies and procedures in these areas were implemented during this time. Work to improve antimicrobial stewardship was ongoing throughout the study period.
Data collection
We defined 4 study phases: baseline (January 2017–December 2017), period 1 (January 2018–September 2018), period 2 (October 2018–June 2019), and period 3 (July 2019–December 2019). Rates of C. difficile testing performed among patients admitted to the hospital were calculated as CDTs per 1,000 patient days. Hospital-onset (HO) CDI was defined as a CDI laboratory-event occurring on or after hospital day 4, in accordance with the Centers for Disease Control and Prevention (CDC) National Healthcare Safety Network (NHSN) definition. Rates of HO-CDI were calculated as cases per 10,000 patient days.
EHR reports were generated to determine the number of times the laxative alert fired and how users interacted with the alert. These reports included patient demographics and time stamps for each time the alert fired. Because users would trigger the laxative alert multiple times within a single patient testing decision (eg, alert initially accepted and later overridden) or would cancel orders after the alert was overridden, we performed chart reviews on a subset of test orders placed during months 2, 9, and 18 of our intervention to correlate laxative alerts with test order, completion, and result.
Statistical methods
The impacts of the 3 EHR interventions on longitudinal trends of inpatient C. difficile testing and HO-CDI rate were analyzed using interrupted time series using generalized, linear, mixed-effects models to account for pseudoreplication resulting from repeated measures taken over time within each hospital. The gatekeeper intervention at hospital C was included as an important confounder. C. difficile testing and HO-CDI incidence rates were each modeled with a Poisson function using patient days as an offset term to account for variation in hospital census. Models were constructed using the GlmmTMB package in R version 3.5.3 software (https://cran.r-project.org/), and all graphs were created using ggplot2. Both hospital and time were included as random-effects terms. Each model included fixed-effects terms for time (baseline trend), level change in incidence rate following intervention, and slope (trend) change in incidence rate following intervention. Effect measure modification by hospital was assessed by creating interaction terms between intervention and hospital site; likelihood ratio test P < .05 was considered significant. Data were assessed for evidence of zero inflation and overdispersion. Inspection of QQ plots was performed to confirm the appropriate probability distribution. Error structures were compared using the Aikike information criterion (AIC). Residuals were plotted for each model to assure homoscedasticity and fitted versus actual plots constructed as a final visual inspection for each model’s accuracy.
Institutional review board approval
This study was approved by the Duke University Health System Institutional Review Board (no. Pro00106235).
Results
Impact on testing rates
Across all 3 hospitals, 13,806 CDI tests were performed over 1,295,686 patient days during the 3-year study period. The rate of tests performed per 1,000 patient days decreased from 12.8 in the baseline period to 7.5 in period 3 (Table 2). The CDT rate was already decreasing during the baseline period (annual change in IR, 0.79; 95% CI, 0.72–0.87; P < .01) and did not decline further with any of the EHR interventions (Fig. 3). EHR intervention 3 was temporally associated with a further immediate decrease in CDT rate (IRR, 0.93; 95% CI, 0.86–1.00; P = .05). Hospital C’s gatekeeper intervention was associated with both level reduction (IRR, 0.50; 95% CI, 0.42–0.60; P < .01) and trend reduction in C. difficile testing relative to the baseline period (annual change in IR, 0.91; 95% CI, 0.85–0.98; P = .02).
Rates of Clostridioides difficile Tests Ordered per 1,000 Patient Days by Phase of Intervention: Duke Health January 1, 2017, through December 31, 2019
a Tests per 1,000 patient days.
Interrupted time-series analysis of monthly rates of C. difficile tests by electronic health record intervention period at Duke Health, January 2017–December 2019. Intervention 1: Laxative alert. Intervention 2: Auto-cancellation of uncollected samples after 24 hours. Intervention 3: Contextual rule-driven order questions linked to laboratory test cancellation. Gatekeeper intervention: (hospital C only) chief medical officer or infectious diseases consultant approval required for all test orders after hospital day 3. Hospital A: 957-bed academic hospital. Hospital B: 354-bed academic-affiliated community hospital. Hospital C: 175-bed academic-affiliated community hospital.
User acceptance of the laxative alert
The laxative alert fired for 132 patients during month 2 of the laxative alert deployment compared to 83 in month 9 and 76 in month 18. At all 3 time points, however, the acceptance of the alert was low, and providers chose to override the alert and proceed down the test-ordering pathway 74%, 76%, and 83% of the time, respectively. Of 385 CDTs, 88 (22%) were performed on patients who had received laxatives within 24 hours of test order in month 2 compared to 42 (13%) of 322 tests in month 9 and 47 (15%) of 319 tests in month 18.
Impact on C. difficile rates
The overall rate of HO-CDI decreased from 9.5 cases per 10,000 patient days in the baseline period to 6.8 cases per 10,000 patient days in period 3 (Table 3). There was significant interaction between hospital and EHR intervention 1 (laxative alert) on HO-CDI rates, and we elected to create separate models for hospitals A and B versus hospital C to better characterize the differences (Fig. 4A). At hospitals A and B, HO-CDI rates were not significantly trending downward in the baseline period (annual change in IR, 0.86; 95% CI, 0.70–1.06; P = .15). EHR intervention 1 was temporally associated with a trend reduction in HO-CDI (annual change in IR from baseline, 0.85; 95% CI, 0.75–0.96; P < .01) and EHR intervention 2 was temporally associated with both level (IRR, 1.29; 95% CI, 1.06–1.56; P = .01) and slope increases in CDI rate (annual change in IR from baseline 1.20; 95% CI, 1.08–1.33; P < .01; Fig. 4B). At hospital C, the HO-CDI rate was not significantly trending upward in the baseline period (annual change in IR, 1.13; 0.81–1.59; P = .47). The gatekeeper intervention temporally correlated with both level (IRR, 0.42; 95% CI, 0.22–0.81; P < .01) and trend reductions in HO-CDI rate relative to the baseline period (annual change in IR, 0.68; 95% CI, 0.50–0.92; P = .01; Fig. 4C).
Rates of Hospital-Onset Clostridioides difficile Infection (HO-CDI) per 10,000 Patient Days by Phase of Intervention: Duke Health January 1, 2017, through December 31, 2019
a HO-CDI cases per 10,000 patient days.
Interrupted time-series analysis of monthly rates of hospital-onset C. difficile by electronic health record intervention period at Duke Health, January 2017–December 2019. (A) Initial model without interaction terms between hospital and EHR intervention. (B) Subanalysis of the effect of interventions at hospitals A and B. (C) Subanalysis of the effect of interventions at Hospital C. Intervention 1: Laxative alert. Intervention 2: Autocancellation of uncollected samples after 24 hours. Intervention 3: Contextual rule-driven order questions linked to laboratory test cancellation. Gatekeeper intervention: (hospital C only) chief medical officer or infectious diseases consultant approval required for all test orders after hospital day 3. Hospital A: 957-bed academic hospital. Hospital B: 354-bed academic-affiliated community hospital. Hospital C: 175-bed academic-affiliated community hospital.
Discussion
Diagnostic test stewardship is an important quality-improvement and high-value care challenge for clinicians. Following a series of EHR-based interventions and the implementation of a test gatekeeper at hospital C, we observed a 41% reduction in C. difficile testing rate and a 28% decrease in our observed HO-CDI rate across our 3-hospital health system compared to our preintervention rates. Importantly, however, the rate of CDT was declining across the 3-hospital system even before the first EHR intervention was launched and continued to decline thereafter; thus, it is not clear that the EHR interventions contributed to the overall decline in testing rate. In contrast, hospital C independently implemented a test gatekeeper. This intervention was temporally associated with immediate large and sustained reductions in C. difficile testing and HO-CDI rates. Taken together, our findings highlight important learnings for future quality-improvement work aimed at reducing unnecessary diagnostic testing: (1) EHR-based clinical-decision support interventions alone in the absence of education and buy-in from clinician stakeholders are weak tools for changing behavior; (2) EHR-based interventions coupled with administrative restriction are more likely to lead to behavior changes; and, (3) local culture is an important determinant of readiness to change and user acceptance of any intervention.
Our findings are similar to those in other published attempts at improving CDT appropriateness through EHR interventions. Reference Mizusawa, Small and Hsu17 Multiple studies have found that a laxative alert alone may not dramatically reduce overall inappropriate C. difficile testing or improve CDI rates. Reference Friedland, Brown, Glick, Lusby, Lemkin and Leekha10,Reference White, Hamilton, Pegues, Hanish and Umscheid14 Providers at our hospitals overrode the laxative alert >70% of the time. Nevertheless, we observed a decrease in the number of laxative alerts as well as a decrease in the proportion of tests ordered on patients who had recently received laxatives over time, suggesting that clinicians learned to defer testing. Notably, planning for the laxative alert and discussions regarding inappropriate testing of patients on laxatives began a year prior to the implementation of the EHR alert and continued throughout the study. Because testing rates declined throughout the baseline and study period but did not decrease with the implementation of the laxative alert, we believe that ongoing educational efforts to gain buy-in from clinicians were important to changing behavior.
Understanding the key drivers of inappropriate test ordering behavior is critical for targeting ongoing education and EHR decision support. The second and third changes to the EHR decision support were designed to address inappropriate C. difficile testing as part of evaluations for fever or leukocytosis in the absence of clinical signs or symptoms of CDI because these were commonly observed reasons for inappropriate CDT ordering at our hospitals. Key features of our EHR interventions included prompts containing pertinent patient information (ie, flow-sheet documentation of diarrhea, recent WBC results, and expert guidance on test appropriateness criteria) to the ordering clinician at the time of order entry. Additionally, our decision support provided real-time feedback to the ordering clinician when a test order would be cancelled based on administrative criteria, increasing transparency between the performing laboratory and the ordering clinician. Finally, the EHR intervention tagged specimens for cancellation by the microbiology laboratory staff, minimizing the burden of review and workflow impact to those team members. Overall, the implementation of the contextual rule-based order questions coupled with laboratory restriction was an innovative strategy that was associated with an immediate decline in testing rate and was well received by providers.
Our study had the advantage of including multiple hospitals providing care to patient populations with varying acuity and staffed by community providers, academic providers, and trainees. We observed the greatest decrease in test rates at hospital C, a primarily nonteaching community hospital that adopted a physician “gatekeeper” function in addition to the EHR changes. This finding is consistent with others who have found true administrative stops to be more effective than EHR alerts or passive decision support. Reference Quan, Yim and Merrill18,Reference Rock, Mizusawa and Small19 Additionally, greater declines in test ordering were observed at hospital B, a community teaching hospital, compared to hospital A, a tertiary-care referral hospital. Although we cannot make any definitive conclusions, our data suggest that interventions to reduce unnecessary testing may be more successful at smaller hospitals because fewer resources are required to provide targeted education. Additionally, providers caring for lower-acuity patients may be more receptive to test avoidance and more accepting of diagnostic and therapeutic guidelines targeting general patient populations. Finally, hospital culture, including the strength of relationships between medical staff and hospital leaders, may also impact the success of interventions to reduce unnecessary testing, particularly when administrative restrictions are implemented.
Our study has several limitations. First, our findings may not be generalizable to other settings with different local cultures or patient populations; even within the same organization, differences in local culture likely played a significant role in the acceptance of interventions. Second, this work was conducted as part of a larger quality-improvement initiative at our health system to reduce rates of HO-CDI and determining the independent impact of our EHR decision support interventions was challenging. We used interrupted time-series analysis to address this limitation. However, the interpretation of our findings leaves some questions. For example, the laxative alert intervention was temporally associated with a decline in HO-CDI rates at hospitals A and B; however, because it was not also temporally associated with a decline in CDT rates, we cannot confidently conclude the decline in HO-CDI at hospitals A and B was a result of the laxative alert itself or other confounding factors. Third, our study was not designed to assess the impact of test stewardship on quality of patient care. We used the C. difficile testing rate as a surrogate for test appropriateness and did not adjudicate test appropriateness through chart review. Furthermore, we did not prospectively review patients’ charts for whom tests were canceled or deferred to detect potential safety concerns that may have gone undetected by our safety reporting system. Finally, we did not assess for changes in antimicrobial prescribing practices over the course of our study, such as increased use of oral vancomycin prophylaxis and/or empiric treatment in the absence of testing. Future studies of test stewardship should assess impact on care quality in addition to resource utilization.
In summary, optimizing C. difficile diagnosis is an important priority for hospital systems to more accurately estimate CDI incidence, to improve hospital performance on HO-CDI reportable metrics, and to improve value-based care. EHR interventions using alerts and contextual rule-driven questions addressing common reasons for inappropriate CDT ordering are attractive interventions because they require relatively fewer resources to reach a broad audience. However, EHR interventions are most likely to be effective when coupled with laboratory- or clinician-based test restriction. Furthermore, all stewardship strategies must be coupled with targeted education and tailored to the specific care environment. Diagnostic stewardship is the epitome of high-value care, and innovative approaches to optimize test utilization whiles improving patient care quality are needed.
Acknowledgments
Financial support
No financial support was provided relevant to this article.
Conflicts of interest
All authors report no conflicts of interest relevant to this article.
