Clostridioides difficile infection (CDI), with an estimated annual incidence of 149 cases per 100,000 population leading to 29,000 deaths per year in the United States, continues to be a major public health concern.1–Reference McDonald, Gerding and Johnson3Clostridioides difficile is the most common healthcare-associated pathogen, and approximately two-thirds of CDI cases are healthcare associated.Reference Lessa, Mu and Bamberg2,Reference Magill, O’Leary and Janelle4 However, since an estimated 8%–15% of hospitalized patients are asymptomatic carriers of C. difficile, positive test results must be interpreted in the context of clinical symptoms.Reference Zacharioudakis, Zervou, Pliakos, Ziakas and Mylonakis5,Reference Alasmari, Seiler, Hink, Burnham and Dubberke6 Inappropriate testing can lead to overdiagnosis, unnecessary treatment, and financial penalties because hospital-onset CDI (HO-CDI) is a nationally reported hospital quality metric.Reference McDonald, Gerding and Johnson3
The 2018 Infectious Diseases Society of America (IDSA) and Society for Healthcare Epidemiology of America (SHEA) CDI clinical practice guidelines recommend not testing for CDI if a patient’s diarrhea is clearly attributable to another cause, and they advise against testing patients who have recently received a laxative.Reference McDonald, Gerding and Johnson3 However, laxatives are frequently prescribed in hospitalized patients, and several studies have indicated that 19%–44% of all C. difficile tests may be inappropriately ordered on patients receiving laxatives.Reference White, Hamilton, Pegues, Hanish and Umscheid7–Reference Dubberke, Han and Bobo10 This scenario provides an opportunity for intervention.
“Nudging,” a behavioral science concept popularized by Richard Thaler, is a means to reliably modify people’s choices by restructuring their environment without eliminating options or imposing financial penalties.Reference Thaler and Sunstein Cass11,Reference Vaughn and Linder12 This concept is effective in many areas of health care including smoking cessation, appropriate antibiotic prescribing, and hand hygiene compliance; it can also be leveraged when designing electronic medical record (EMR) interventions.Reference Sunstein13–Reference Caris, Labuschagne, Dekker, Kramer, van Agtmael and Vandenbroucke-Grauls15 In this study, 4 hospitals within 1 academic healthcare network implemented a simple EMR nudge to encourage providers to cancel C. difficile orders for patients who received a laxative or stool softener in the prior 24 hours. Using a quasi-experimental design with an interrupted time series analysis, we sought to examine the impact of this EMR notification on reducing rates of HO-CDI diagnostic testing.
Methods
Study setting, population, and design
We conducted a retrospective study at 4 hospitals (designated A, B, C, and D) in the same academic healthcare network, comprising ~1,600 inpatient beds. Hospital A is a 511-bed hybrid academic–community, tertiary-care hospital. Hospital B is a 587-bed academic tertiary-care hospital that performs solid organ and hematopoietic stem cell transplantation. Hospitals C and D are 410-bed and 110-bed community hospitals, respectively.
The study population included all hospitalized patients who had a C. difficile order placed after hospital day 3 and were therefore eligible to be included in the Centers for Disease Control and Prevention (CDC) HO-CDI metric. At the patient level, we extracted age, race, hospital location, comorbidities included in the Charlson comorbidity index, dates of hospital admission and discharge, dates and times of all laxatives or stool softeners administered, and dates and times of all C. difficile orders placed from the EMR. Additional hospital-level variables included the monthly number of CDC-defined HO-CDI laboratory-identified (LabID) events and the number of total inpatient days per month. We analyzed a 2-year period before the intervention (February 1, 2015, through January 31, 2017) and after the intervention (February 1, 2017, through January 31, 2019).
Clostridiodes difficile testing and definitions
Clotridioides difficile testing was performed using polymerase chain reaction (PCR; Xpert C. difficile PCR assay, Cepheid, Sunnyvale, CA) at all 4 hospitals throughout the study period. The microbiology laboratory routinely rejects formed stools and repeat specimens collected within 7 days of a previous test. HO-CDI orders were defined as C. difficile orders placed on patients after hospital day 3. A HO-CDI LabID event was defined, according to CDC criteria, as a positive C. difficile test in a patient hospitalized for >3 days, without any prior positive C. difficile tests from the same location in the previous 14 days.
For this study, a C. difficile order was defined as “inappropriate” if it had been placed within 24 hours after a patient received a laxative or stool softener based on the bar-coded administration time stamp in the EMR. The laxatives and stool softeners included in the EMR notification and our analysis were alvimopan, bisacodyl, docusate, sodium bisphophate-sodium phosphate, glycerin, lactulose, magnesium citrate, methylnaltrexone, mineral oil, polyethylene glycol 3350, polyethylene glycol electrolyte solution, senna, and sorbitol.
Primary and secondary outcome measures
The primary outcomes were the changes in the rates of total and inappropriate HO-CDI orders per 1,000 patient days, comparing the pre- and postintervention periods. The secondary outcome assessed the change in rate of HO-CDI LabID events per 1,000 patient days before and after the intervention. A subgroup analysis also evaluated the primary and secondary outcomes stratified by individual hospital.
Intervention
On February 1, 2017, all 4 hospitals implemented a pop-up notification in the EMR (PowerChart Millenium, Cerner, North Kansas City, MO) triggered when providers entered a C. difficile order for a patient who had received a laxative or stool softener within the previous 24 hours. The default option of clicking “okay” without any further steps closes the pop-op and cancels the order, nudging the provider to cancel potentially inappropriate orders. However, providers can also choose to override the notification by checking a box that states, “continue with the order,” before clicking “okay.” No additional standardized, systemwide interventions to decrease HO-CDI testing were implemented during the study period. However, approximately 1 month prior to the EMR intervention (January 2017), hospital B started an audit of all HO-CDI orders in which the hospital epidemiologist could stop the lab from processing inappropriate C. difficile tests. Orders halted through this process were cancelled after they were entered into the EMR; therefore, this intervention did not affect tracking of our primary outcome. Additionally, in May and August 2016, hospitals C and D, respectively, adopted a paper-based, nursing-driven C. difficile testing algorithm to encourage early testing during the first 3 hospital days on any patient with loose stools, followed by a more conservative approach on or after hospital day 4. A similar algorithm was implemented in select units in hospital B in May 2017. Compliance with these initiatives could not be confirmed or quantified, and they were not included in our model due to sample-size limitations.
Statistical analysis
We performed χ2 tests for categorical variables, and means with standard deviation (SD), medians with interquartile ranges (IQRs), Wilcoxon rank-sum tests, or the Student t test as appropriate for continuous variables, to compare patient and C. difficile order characteristics before and after the intervention. We compared the median monthly rate of total and inappropriate HO-CDI orders pre- and postintervention using the Wilcoxon rank-sum test.
Using an interrupted time series design, we analyzed monthly rates of total and inappropriate HO-CDI orders as well as HO-CDI LabID events before and after the intervention, with changes expressed as rate ratios (RR). We used a segmented regression model with a negative binomial distribution since the data was overdispersed. In the models, the outcomes were the number of C. difficile orders (total or inappropriate) or HO-CDI LabID events per month, and the main exposure was a binomial variable indicating if the order, or LabID event, was before or after the intervention. The model also included an offset for the monthly total number of inpatient person days, a time variable to account for time throughout the entire study period, and a “time after” term to account for time after the intervention. The binomial exposure variable allows for detection of an immediate “level change” in the primary outcome following the intervention. The time variable allows for control of unmeasured trends in ordering practices over the entire study period (baseline trend) and the time after variable allows for assessment of a slope or “trend change” in the order rate after the intervention.Reference Penfold and Zhang16 Autocorrelation was assessed using the Durbin-Watson test in SAS for the primary and secondary outcomes. All analyses were completed using SAS version 9.4 software (SAS Institute, Cary, NC) and R Studio software (R Core Team, 2018).
Human subjects
The Emory University Institutional Review Board approved this study with a waiver of the patients’ informed consent.
Results
Patient and C. difficile order characteristics
Among 36,195 C. difficile orders, 17,694 (49%) assessed for HO-CDI in 9,846 patients and 11,143 admissions (Table 1). Approximately half the patients with HO-CDI orders were female (51%), and 54% were white, with a mean age of 60.5 years (SD, 16.0). Comparing the pre- and postintervention periods, patients with orders testing for HO-CDI in the postintervention period more frequently had a Charlson comorbidity index >4 (46% vs 51%; P < .001) and a longer median length of stay (15.1 days [IQR, 9.3–23.9] vs 16.0 days [IQR, 10.4–25.1]; P < .001).
Characteristics of Patients With a Hospital-Onset Clostridioides difficile Ordera

Note. SD, standard deviation; IQR, interquartile range.
a If a patient had more than one hospital-onset C. difficile order placed, only characteristics associated with the first order were included.
b Comparison of pre- vs postintervention: χ2 tests were performed for categorical variables and Wilcoxon rank-sum or Student t tests were used for continuous variables.
Overall, 10,686 HO-CDI orders were placed during the preintervention period and 7,008 orders were placed during the postintervention period. The median number of HO-CDI orders placed per an individual’s hospital admission was 1 (IQR, 1–2), with a median time between admission and HO-CDI order of 10 days (IQR, 6–16). The most common laxative or stool softener administered in the 24 hours prior to an inappropriate HO-CDI order was docusate (387, 30%) followed by bisacodyl (185, 14%) and lactulose (161, 13%). The intervention was associated with a significant decrease in the proportion of inappropriate HO-CDI orders (8% during the preintervention period vs 6% during the postintervention period; P < .001).
Total and inappropriate hospital-onset C. difficile order rates
The median monthly HO-CDI order rate per 1,000 patient days pooled across all 4 hospitals decreased from 10.9 (IQR, 10.5–11.6) to 7.0 (IQR, 6.4–7.6; P < .001) after implementation of the intervention (Table 2). Using an interrupted time series analysis to control for unmeasured changes in ordering practices throughout the study period and change in slope after the intervention, the EMR notification significantly decreased the HO-CDI order rate (level-change RR, 0.79; 95% CI, 0.73–0.86). There was also a modest continuing decline in order rate after the intervention (trend change after intervention RR, 0.99; 95% CI, 0.98–1.0) (Table 3 and Fig. 1).
Characteristics of Hospital-Onset Clostridioides difficile Orders Before and After the Intervention

Note. IQR, interquartile range.
a Comparison of pre- vs postintervention: A χ2 test was used to compare the proportion of inappropriate orders. Wilcoxon rank-sum tests were used to compare median monthly order rates before and after the intervention.
b An order was defined as inappropriate if it had been placed within 24 hours after administration of laxative or stool softener.
c Per 1,000 patient days.
Interrupted Time Series Analysis of the Monthly Rate of Total and Inappropriate Hospital-Onset Clostridioides difficile Orders as well as Hospital-Onset C. difficile Laboratory-Identified Events

Note. CI, confidence interval; HO-CDI, hospital-onset C. difficile infection, EMR, electronic medical record; LabID, laboratory-identified.
Trend in hospital-onset C. difficile order rate per 1,000 patient days before and after the intervention (dashed line). The data points (circles) represent the observed rates and solid line represents the predicted rates from the regression model, with a 95% confidence interval shaded in grey.

For inappropriate orders, the median monthly order rate per 1,000 patient days was 0.8 (IQR, 0.8–1.0) during the preintervention period and 0.4 (IQR, 0.3–0.6) during the postintervention period (P < .001) (Table 2). In the interrupted time series analysis, the intervention was not significantly associated with an immediate decrease in the inappropriate HO-CDI order rate (level change RR, 0.80; 95% CI, 0.61–1.05). However, after the intervention, the order rate significantly decreased with time (trend change after intervention RR, 0.95; 95% CI, 0.93–0.97) (Table 3 and Fig. 2).
Trend in the inappropriate hospital-onset C. difficile order rate per 1,000 patient days before and after the intervention (dashed line). The data points (circles) represent the observed rates and solid line represents the predicted rates from the regression model, with a 95% confidence interval shaded in grey.

We also assessed HO-CDI order rates stratified by hospital, given differences in patient population, baseline C. difficile incidence, and hospital-specific C. difficile testing initiatives. The 2 largest hospitals paralleled the overall trend with a decrease in the HO-CDI order rate associated with the EMR intervention (hospital A level change RR, 0.67; 95% CI, 0.56–0.80; hospital B level change RR, 0.85; 95% CI, 0.77–0.94). Hospital D showed a similar pattern of decline in order rate but did not reach statistical significance (level change RR, 0.73; 95% CI, 0.52–1.03). Hospital C demonstrated a significant decrease in HO-CDI order rate throughout the entire study period (RR, 0.95; 95% CI, 0.93–0.96) but was not influenced by the EMR intervention (level change RR, 0.75; 95% CI, 0.53–1.06) (Supplementary Fig. 1 online). Similarly, for inappropriate HO-CDI orders, the trend in order rates was comparable to the composite primary outcome in hospitals A, B and D with a significant decrease in the trend change after the EMR intervention (Supplementary Fig. 2 online).
Hospital-onset C. difficile laboratory-identified events
In a secondary analysis, the composite rate of HO-CDI LabID events per 1,000 patient days decreased after the implementation of the EMR intervention (level change RR, 0.74; 95% CI, 0.60–0.91) (Table 3 and Fig. 3). However, in the per hospital assessment, only hospital B, which also instituted a HO-CDI audit process 1 month prior to the EMR intervention, had a significant decrease in LabID events associated with the EMR intervention (level change RR, 0.65; 95% CI, 0.49–0.87).
Trend in hospital-onset C. difficile laboratory-identified events per 1,000 patient days before and after the intervention (dashed line). The data points (circles) represent the observed rates and solid line represents the predicted rates from the regression model, with a 95% confidence interval shaded in grey.

Discussion
A simple EMR notification to nudge providers away from testing for C. difficile in patients who had received a laxative or stool softener within the previous 24 hours decreased HO-CDI orders by 21%, and this effect was sustained for 2 years after implementation. In addition, inappropriate HO-CDI orders significantly decreased during the postintervention period. Our results are consistent with previous literature demonstrating that educational campaigns and EMR order sets can decrease the proportion of inappropriate C. difficile orders placed on patients who have recently received laxatives.Reference White, Hamilton, Pegues, Hanish and Umscheid7,Reference Buckel, Avdic, Carroll, Gunaseelan, Hadhazy and Cosgrove8 Additionally, a large academic hospital recently demonstrated decreased rates of HO-CDI using real-time electronic tracking of bowel movements and laxative administration and instructing laboratory technicians to cancel inappropriate orders directly.Reference Truong, Gombar and Wilson17 However, having to fill out cumbersome clinical decision support order sets can be onerous for healthcare providers, and real-time electronic tracking is often unrealistic given the significant time, effort, and personnel required for implementation and maintenance. We posit that EMR nudges may represent a simpler, more palatable strategy to reduce HO-CDI orders, especially in healthcare systems with concurrent or competing quality improvement initiatives.
Although our EMR nudge was associated with an immediate decrease in the total HO-CDI order rate, the effect on the inappropriate HO-CDI order rate was more gradual. Providers who were initially skeptical of the new EMR notification may have acclimated to the nudge with repeated exposure and awareness. This finding may in part explain why there was a continued significant downward trend in the inappropriate order rate postintervention. The relatively low inappropriate order rate also likely contributed to the immediate level-change effect not reaching statistical significance. The EMR notification may have also influenced providers to alter their overall C. difficile ordering practices, even on patients not receiving laxatives or stool softeners, which could have contributed to the decrease seen in total HO-CDI orders. Lastly, awareness of diagnostic stewardship and appropriate C. difficile testing has been increasing at our institution and broadly in the field of academic medicine in the past 2 years, which may have also contributed to the continued downward trend in total and inappropriate HO-CDI order rate after the intervention.
In a per-hospital analysis, the EMR intervention was effective at decreasing total and inappropriate HO-CDI orders in 3 of the 4 study hospitals, which included both community and academic practices. Hospital C was a notable outlier. We believe this is because they initiated a nursing-driven testing algorithm prior to the intervention to encourage testing for C. difficile during the first 3 hospital days followed by a more conservative approach on or after the fourth hospital day. This resulted in a substantially lower baseline total and inappropriate HO-CDI order rates at the time of the EMR nudge, which may have limited any potential impact of this intervention, but we were unable to model this at the hospital level due to sample size.
In a secondary analysis, the EMR nudge was also associated with a significant reduction in composite HO-CDI LabID events; however, this result was driven by the largest hospital in this study (hospital B), where more than half of all orders testing for HO-CDI were placed. The intervention did not have a significant effect in reducing HO-CDI LabID events at the other 3 hospitals. Hospital B’s decline in HO-CDI events may also be attributable to the audit of orders performed by the hospital epidemiologist that was initiated at a similar time as the EMR nudge, making it difficult to differentiate between the effect of these 2 interventions. Therefore, EMR nudges may help improve diagnostic efficiency and reduce excess spending (the estimated material cost of a C. difficile PCR assay is $33, excluding labor and processing costs),Reference Sharp, Ruden, Pohl, Hatcher, Jayne and Ivie18 but alone they may not be enough to improve HO-CDI rates.
This study has several limitations. First, the study was retrospective and did not have a control group. A segmented regression analysis allows for theoretical control of unmeasured trends in ordering practices over time, including unique hospital initiatives, but we were unable to account for all possible confounding factors. We were not able to specifically control for the nursing-driven testing algorithm instituted in hospitals C and D before the EMR nudge, highlighting real-world challenges in analyzing quasi-experimental data, including difficulty in accounting for the impact of process improvement initiatives, which can rely on paper-based audits. Although we included 4 hospitals, our study was limited to a single healthcare network and may not be generalizable to other healthcare systems or hospitals, particularly those with different or older EMRs. We were also not able to track provider-level “accept” or “decline” rates for the pop-up notification, which could be a more direct measure of efficacy. Lastly, while our EMR notification alerts providers when a patient has received a laxative in the previous 24 hours, the IDSA guidelines now suggest that orders placed up to 48 hours after laxative administration can be inappropriate.Reference McDonald, Gerding and Johnson3 By restricting the time frame to 24 hours, we may have underestimated the effect on the inappropriate order rate.
A simple EMR nudge reduced C. difficile orders that may otherwise have been classified as HO-CDI by 21%. After the intervention, we also observed a significant decrease in the number and proportion of orders that were considered inappropriate based on recent laxative or stool-softener use. Although the intervention was not associated with an immediate decrease in the inappropriate HO-CDI order rate, we observed a significant decreasing trend in the inappropriate order rate each month following the intervention. Implementing an EMR nudge is a relatively simple method that can help healthcare systems improve diagnostic efficiency and limit the number of C. difficile tests ordered. However, depending on individual hospital practices, this intervention alone may not be sufficient to reduce HO-CDI rates.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/ice.2020.12
Acknowledgments
We are grateful to the infection prevention teams and pharmacists at all study hospitals who work to improve diagnostic stewardship for C. difficile every day and helped develop this study initiative. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Financial support
J.H.A. is supported in part by the National Center for Advancing Translational Sciences of the National Institutes of Health (grant nos. UL1TR002378 and TL1TR002382). J.T.J. is supported in part by the Prevention Epicenters Program of the Centers for Disease Control and Prevention (grant no. U54CK000164).
Conflicts of interest
All authors report no conflicts of interest relevant to this article.







