Clostridoides difficile infection (CDI) rates have increased markedly over the past 2 decades,Reference Freeman, Bauer and Baines1, Reference Reveles, Lee, Boyd and Frei2 and CDI is now the leading type of healthcare-associated infection (HAI) in the United States.Reference Miller, Chen, Sexton and Anderson3 Nearly 500,000 patients develop CDI each year,Reference Lessa, Mu and Bamberg4 resulting in significant morbidity and mortality.Reference Martin, Monaghan and Wilcox5, Reference Postma, Kiers and Pickkers6 Almost 30,000 patients die annually from CDI,Reference Lessa, Mu and Bamberg4 and survivors experience prolonged hospitalizations and disease recurrence.Reference Dubberke, Reske, Olsen, McDonald and Fraser7 With associated costs exceeding $1.5 billion annually,Reference Zimlichman, Henderson and Tamir8 CDI has become a focus of pay-for-performance mechanisms such as the Centers for Medicare and Medicaid Services hospital-acquired condition measure penalizing health systems for high rates of hospital-onset CDI (HO-CDI).9 it is also the focus of other hospital infection prevention and antimicrobial stewardship programs nationwide.
Both diagnosis and reporting of CDI thus are under increased scrutiny; traditional surveillance and feedback of CDI rates remain a primary component of infection prevention. Quality improvement efforts have focused on more accurate CDI diagnosis, with many hospitals transitioning over the last 10 years from inexpensive, low-accuracy enzyme immunoassay (EIA) methods to highly sensitive nucleic acid amplification tests (NAATs).Reference Humphries, Uslan and Rubin10–Reference Selvaraju, Gripka, Estes, Nguyen, Jackson and Selvarangan14 Similarly, the laboratory-identified surveillance method (LabID) enacted by the National Healthcare Safety Network (NHSN) in 2013 leveraged the electronic integration of laboratory and admission data to more accurately distinguish between hospital- and community-onset CDI.
Several theories have been proposed to explain the emergence of CDI as a leading HAI, including spread of the BI/NAP1/027 clone, increased antibiotic use, and more elderly patients.Reference Freeman, Bauer and Baines1, Reference Loo, Bourgault and Poirier15–Reference Warny, Pepin and Fang17 Our earlier preliminary data suggested that simply changing CDI diagnostic testing methods from EIA to NAAT may increase a hospital’s reported rate of HO-CDI by as much as 50%.Reference Moehring, Lofgren and Anderson18 Other reports have noted that patients positive according to NAAT but negative according to EIA (ie, the NAAT+/toxin− phenotype) had similar outcomes to patients with NAAT−/toxin− test results,Reference Polage, Gyorke and Kennedy19 suggesting that patients with NAAT+ stools but no infection are included in reported CDI rates. In fact, the most recent guidelines for CDI clinical practice from the Infectious Diseases Society of America (IDSA) and Society for Healthcare Epidemiology of America (SHEA), published after the present study period, recommend against use of NAAT alone unless prior institutional criteria have been agreed upon.Reference McDonald, Gerding and Johnson20
The main objective of this study was to conduct a larger replication of our previous investigation of the impact on reported CDI rates of switching from EIA to NAAT testing.Reference Moehring, Lofgren and Anderson18 We used a larger dataset encompassing 47 hospitals over 9 years (an average of 3.2 years before and 4.2 years after the switch), and we analyzed monthly rather than weekly rates to reduce measurement noise. A secondary objective was to investigate the utility of change-point detection methodsReference Hinkley21–Reference Picard24 to statistically determine the timing and magnitude of changes in reported CDI rates.
Methods
Data and setting
Monthly surveillance data were collected prospectively between January 2009 and December 2017 from 47 community hospitals in the southeastern United States as part of routine reporting to the Duke Infection Control Outreach Network (DICON).Reference Anderson, Miller and Chen25 Data acquired from each hospital included monthly patient days, intensive care unit patient days (ICU-PD), dates of known CDI clusters based upon documentation from past outbreak investigations, and (if applicable) the date the hospital switched from nonmolecular (EIA) to molecular (NAAT) testing. A CDI event was defined by either NHSN traditional surveillance definitions or the LabID CDI module.Reference Durkin, Baker and Dicks26 For each event, the specimen collection date and traditional and/or LabID surveillance category (infection source and recurrence type) were retrieved from the DICON database. CDI events marked as duplicate, recurrent, or continuation were excluded from the analysis.
Hospitals were divided retrospectively into 3 groups: (1) the test group, comprising all hospitals with sufficient data (minimum 6 months before and after) to accurately assess pre- and postswitch CDI rates and trends; (2) control group 1, including all hospitals that either used EIA tests throughout or switched late in the study period such that postchange rates were not evaluable; and (3) control group 2, including all hospitals that either used NAAT the entire study period or switched during the first 6 months of reporting. The 2 control groups were used to ensure that regression estimates were generalizable (as described in the following section). Data from EIA and NAAT use periods <6 months and <12 months of data from 1 hospital that switched to 2-stage CDI testing were discarded. Analysis of all data was approved by the Duke University Health System and Northeastern University institutional review boards. All statistical analyses were conducted using MATLAB software (MathWorks, Natick, MA).
Poisson regression analysis
Individual hospital estimates of the relative change in incidence rates of reported CDI caused by the switch from EIA to NAAT testing were determined through interrupted time series (ITS) analysis using linear Poisson regression.Reference Campbell, Stanley and Gage27–Reference Cook and Campbell29 Fitted models included the monthly count of reported CDI cases (grouped by specimen date) as the dependent variable, the corresponding patient-days denominator as an offset variable, and the following 6 predictors: monthly binary indicators for use of NAAT testing, the presence of a hospital-level CDI cluster, the use of LabID surveillance categories, the percentage of ICU-PD relative to total patient days (as proxy for case severity), and the number of months until the test switch date (set to 0 for all postswitch time points) and after the test switch date (set to 0 for all preswitch time points).
We pooled data from the test (centered on the switch date), control group 1 (aligned at the final time point), or control group 2 (aligned at the initial time point) to obtain network-level estimates in a similar manner. Repeating the analysis using 1 or both control groups enabled us (1) to contrast direct estimates (before and after within the test group) with indirect estimates (control group 2 vs control group 1) of the NAAT-over-EIA incidence rate ratio (IRR), (2) to compare longitudinal trends and covariate effects between groups, (3) to confirm the absence of sampling biases, and (4) to ensure generalizability of resulting regression estimates. To account for differences between hospitals, network-level analyses were repeated using a mixed-effects Poisson regression model that additionally included hospital identifier as a random effect with full covariance structure for all predictors except NAAT use. Regression estimates were computed as IRRs with 95% confidence intervals (95% CIs).
Change point detection
As a complementary approach, we performed change-point detection analysis on the 24 hospitals in the test group. In contrast to ITS analysis that prespecifies known change date(s), change-point methods search for any dates with statistically significant before-and-after differences in model parameters. We analyzed unadjusted monthly CDI rates (per 10,000 patient days) separately for each hospital, searching for a specified maximum number of statistical changes in mean or slope.Reference Quandt30, Reference Quandt31 Although other types of change-point methods exist, such as searching for changes in only the mean,Reference Page23, Reference Page32 the variance,Reference Hsu33, Reference Wichern, Miller and Hsu34 or either,Reference Horváth35, Reference Horváth36 this type of analysis was used because we expected multiple longitudinal trend and step changes in reported CDI rates due to switching to NAAT testing, LabID classification, or CDI cluster periods.
Analyses were conducted with a maximum of 1, 3, and 5 change points for each hospital to allow for the test switch and up to 4 other changes in CDI rates or trends (eg, start or end dates of known or previously undetected infection clusters, LabID surveillance, or improvement efforts). In all cases, we imposed a minimum delay of 3 months between successive change points to avoid overfitting high variability periods. For each identified change point, we recorded the type of change (mean, slope, or both), its magnitude(s), and offset from the test switch date. Detected change points were attributed to the switch in CDI testing method if they occurred within 6 months after the reported switch date.
Results
Data summary
Analyzed data consisted of 22,861 CDI cases from 13,530,367 patient days at 47 hospitals, of which 1,303,059 (9.6%) were ICU days. At various times during the 9-year study period, 37 of the 47 hospitals switched to molecular testing. Two hospitals switched within the previous 6 months of the study period and thus were added to the control group 1 with the 10 hospitals that used EIA throughout. Eleven hospitals had <6 months of data prior to the test change date and thus formed the control group 2. The remaining 24 hospitals had adequate data both before and after their switch to NAAT (average of 38 and 51 months, respectively) and comprised the test group.
Average unadjusted CDI rates increased after the test switch from 10.9 to 23.9 CDI per 10,000 patient days at hospitals that transitioned to NAAT testing (Table 1). A similar difference was found between the 2 sets of control hospitals, with control group 1 (using EIA) averaging 11.5 CDIs per 10,000 patient days and control group 2 (using NAAT) averaging 22.6 CDIs per 10,000 patient days (neither adjusting for time, clusters, LabID use, nor proportion of ICU-PD). Notably, the distribution of reported CDI rates had inflated rates of zeros in hospitals using EIA testing and significantly more right skewness (control group 1 and test hospitals before switch), both of which suggest underreporting (Fig. 1).
Summary of Study Dataset, Including Number of Hospitals Per Group (Test, control group 1, control group 2), Timespan of Data, Monthly Patient Days (hospital-wide and intensive care unit (ICU)), Use of LabID Surveillance Categories, and Nonrecurrent Clostridium difficile Infection (CDI) Incidence and Cluster Durations
Note. SD, standard deviation.
Empirical distributions of reported monthly rates of new (nonrecurrent) Clostridium difficile infections (CDIs). Comparison of CDI incidence per 10,000 patient days before and after the switch from nonmolecular (EIA; left panels) to molecular (NAAT; right panels) diagnostic testing. Histograms (top plots) and boxplots (bottom plots) illustrate the test-induced differences in CDI rates, with data pooled across all 47 study hospitals.
Previously known infection clusters occurred in 12 hospitals (9 in the test group, 1 in control group 1 and 4 in the control group 2) and had an average duration of 7.6 months. Clusters were roughly 5 times more frequent in hospitals using NAAT testing (average, 0.05 per hospital per year) than in those using EIA (average, 0.01). Hospitals that switched to NAAT testing before or during the analysis period (the test group and control group 2) had higher patient volumes (average, 4,868 vs 3,166 patient days) but nearly equal ICU utilization rates (average, 11.5% vs 11.9% of patient days) compared to hospitals using non-molecular methods throughout. LabID surveillance usage overlapped substantially, but not fully (average, 85%), with the period during which hospitals used NAAT testing.
Impact of test switch
Hospital-specific test switch IRRs ranged from 0.75 (95% CI, 0.37–1.54) to 5.47 (95% CI, 2.66–11.20) across the 24 hospitals that transitioned to molecular testing (Fig. 2). Of these hospitals, 20 exhibited increases in reported CDI incidence after their test switch, which were statistically significant (P < .01) in 12 of these, and roughly half exhibited significant longitudinal trends (results not shown). We detected a significant NAAT-over-EIA IRR of 1.75 (95% CI, 1.62–1.89; P < .001) at the network level (Fig. 3), but no significant effect of time either before (IRR, 1.004; 95% CI, 0.998–1.010; P = .21) or after the test switch (IRR, 1.001; 95% CI, 0.997–1.005; P = .67).
Hospital-specific estimates of relative changes in CDI incidence rates (IRR) due to switching from non-molecular to molecular diagnostic testing. Error bars denote 95% confidence intervals. An IRR of 1 (dashed line) indicates no change. Asterisk denotes statistical significance (P < .01). Solid line indicates network-wide IRR.
Network-wide model of the effect on reported Clostridium difficile infection (CDI) incidence rates of switching from nonmolecular to molecular diagnostic testing. Poisson regression fit (continuous line) assumes no infection clusters, use of LabID surveillance categories, and an average ratio of 11.7% intensive care unit patient days per total patient days. Dashed lines denote 95% confidence intervals (CIs). Dots indicate individual monthly observations from the 47 hospitals included in the study.
All other covariates had statistically significant impacts on reported CDI rates (Table 2). We detected a significant network-wide cluster effect (IRR, 1.45; 95% CI, 1.28–1.64; P < .001). Individual hospital IRRs were between 0.90 (95% CI, 0.79–1.03) and 3.92 (95% CI, 2.37–6.51), indicating that reported CDI rates increased on average 1.5-fold during cluster periods, irrespective of the diagnosis test used. Switching to LabID surveillance categories increased reported CDI rates by an average of 38% (network-wide IRR, 1.38; 95% CI, 1.10–1.72; P < .01); individual hospital IRRs increased between 0.0005 (95% CI, 0.000–0.002) and 7.13 (95% CI, 3.58–14.20). The proportion of ICU-PD had a small significant effect on CDI rates (2% more nonrecurrent CDIs per additional percent of ICU-PD; network-wide IRR, 1.02; 95% CI, 1.00–1.03; P < .05). Similar network-wide effects were observed when analyzing only the 24 hospitals that switched tests, combining the test set with only 1 of the control groups (Table 2), and not adjusting for variation across hospitals (results not shown).
Network-wide Results of Interrupted Time Series Mixed Effects Model of Clostridium difficile Infection (CDI) Incidence Rates as a Function of Molecular (NAAT) Versus Nonmolecular (ELISA) Testing, Time, Known Clusters, Use of Laboratory-Identified (LabID) Surveillance Categories, and Proportion of Intensive Care Unit (ICU) Patient Days
Note. IRR, incidence rate ratio.
a Statistical significance at P < .01.
Change point analysis
For the 24 test switch hospitals, statistically significant shifts in the mean or slope of CDI rates occurred an average of 1.4 ± 1.0 months after the reported test switch date when fitting a single change point, 1.4 ± 1.2 months if allowing for 3 change points, and 1.8 ± 1.2 months if allowing up to 5 change points. The analysis did not detect any change points within 6 months after the test switch in 12 hospitals if searching for a single change point, 14 hospitals when searching for 3 change points, and 13 hospitals when searching for 5 change points. In all other cases, it resulted in a partitioning of the analyzed time series into 2–6 segments with noticeably different levels or trends in CDI rates (see Fig. 4for examples). Increasing the maximum number of change points did not result in finding smaller shifts closer to the test switch dates.
Time series change point analysis of Clostridium difficile infection (CDI) rates per 10,000 patient days for 2 example hospitals, allowing up to 3 (upper panels) or 5 (lower panels) change points. In each case, detected changes in mean or slope are indicated by green vertical lines, test switch dates are indicated by blue dashed vertical lines, and change points most closely following test switch dates are marked with arrows. For hospital 11, a significant change was detected 3 months after the test switch date in both analyses. For hospital 19, a change in mean and slope was detected 8 months after the test switch for the 3-point analysis and 5 months after the test switch for the 5-point analysis. Both hospitals exhibit different time segments with increases or decrease in reported CDI rates.
In 37% of analyses, the identified change points were associated with either a significant (P < .05) shift in reported CDI rates from one segment to the next (positive for 18% of cases and negative for 6%; data not shown) or a significant change in slope (increase in 10% of cases and decrease in 10%; data not shown). For each hospital, the first change point after the reported test switch date was associated most often with an increase in CDI rates: 85% of cases when allowing for a single change point, 74% for up to 3 change points, and 63% for up to 5 change points (Table 3). These increases averaged 6.2 CDIs per 10,000 patient days, or roughly 56% more than the EIA baseline; these findings are similar to and support the ITS results.
Results of Change-Point Detection Methods Applied to Monthly Unadjusted Rates of Reported Clostridium difficile Infections (CDIs) per 10,000 Patient Days (PD)a
a Average values and counts (number of hospitals detected) of significant increases or decreases in mean and slope are given for the nearest change point within 6 months after the test switch date.
Discussion
Accurate detection of CDIs is important for effective surveillance, quality improvement, and timely interventions. To our knowledge, our multicenter study is the first large-scale analysis of the impact of testing methods on CDI surveillance outcomes, including 9 years of data from 47 hospitals representing over 13.5 million patient days. Our results estimate that transitioning from an EIA-based test to NAAT leads to an average increase of 75% in reported CDI cases after accounting for CDI clusters, longitudinal trends, and transition to LabID. Interestingly, the increase in reported CDI rates resulting from switching to NAAT exceeded that of a typical CDI cluster in our cohort of hospitals (IRR, 1.45).
These estimated increases generally are consistent with, though larger than, prior assessments. Our earlier evaluation of 2.5 years of data from 32 hospitals estimated an average postswitch increase of 56%.Reference Moehring, Lofgren and Anderson18 Similarly, Longtin et alReference Longtin, Trottier and Brochu37 performed both NAAT and EIA tests on stool samples collected at a single academic hospital and reported an increase of 52% in healthcare-facility–associated CDI when using NAAT versus EIA. The observed rate increase also is larger than the NHSN reporting adjustment for NAAT versus EIA testing (of ∼25%).Reference Dudeck, Weiner, Malpiedi, Edwards, Peterson and Sievert38 The current study, however, analyzed all nonrecurrent CDI rather than only HO-CDI and did not consider the same covariates as the NHSN model.
Compared with prior research, our current results are based on a larger cohort, longer study period, more hospitals that transitioned to NAAT, and both negative and positive controls (ie, hospitals that used either EIA or NAAT the entire time). Although previous work also accounted for known CDI clusters and longitudinal trends during the study period, we additionally controlled for the transition from traditional CDI surveillance to LabID, which had a large impact on nonrecurrent CDI rates (IRR, 1.38) and could be a significant confounder.
Although better detection of true CDIs is important, a potential consequence of switching to NAAT could be an increase in false-positive results due to increased sensitivity of detecting C. difficile colonization but decreased specificity of detecting true clinical infection.Reference Humphries, Uslan and Rubin10–Reference Carroll12 Our results thus support recent recommendations from IDSA and SHEA that hospitals should not use NAAT alone unless an institution-level policy exists to guide CDI specimen collection and receipt.Reference McDonald, Gerding and Johnson20 In fact, results suggest hospitals should be cautious in using NAAT even when institution-level protocols for appropriateness exist. Our findings and those of Goldenberg et al,Reference Goldenberg, Price, Tucker, Wade and French39 who noted a 57% increase in reported CDI rates at a single UK hospital after transitioning to a 2-step NAAT, imply that following IDSA/SHEA guidelines, while theoretically very accurate,Reference Swindells, Brenwald, Reading and Oppenheim11, Reference Selvaraju, Gripka, Estes, Nguyen, Jackson and Selvarangan14 nonetheless may not eliminate increases in false-positive cases of C. difficile.
Results also illustrate change point analysis as a novel methodology to evaluate and confirm the timing and scale of increases in CDI rates. Infection prevention surveillance often is hampered by small numbers and arbitrary points of evaluation (eg, annually). Even though ITS analyses can be used to evaluate the impact of process changes with known timing, change-point analysis can help identify and estimate the timing of an unknown change or the time lag after which a known change started to have impact. These 2 methods may be viewed best as complementary rather than competing. In the present study, change points allowed us to better evaluate the time lag of the impact of switching to NAAT on reported CDI rates. In particular, our results indicate that the NAAT-induced increase often was not immediate, averaging a 1-month lag but with some variability. To our knowledge, change point analysis has not been used to identify the existence and timing of changes in rates of empirical CDI surveillance data, although Texier et alReference Texier, Farouh and Pellegrin40 recently illustrated its utility for HAI outbreak surveillance using simulated data.
As with our earlier analysis,Reference Moehring, Lofgren and Anderson18 the current study is limited by its nonrandomized, retrospective nature and the lack of detailed clinical data to help differentiate CDI colonization from true infection, although the inclusion of multiple types of controls may mitigate these limitations. Although we were able to account for many covariate factors, other potential confounders still may exist: patient population risk profiles, hospital size, and structural characteristics, antimicrobial utilization patterns, presence of CDI clusters not detected through traditional surveillance, changes in infection prevention practices (eg, cleaning and disinfection), and unreported process improvement efforts. These factors may impact NAAT-over-EIA IRR estimates at either hospital or network levels. Finally, since our study only included data from community hospitals in the southeastern United States, generalizability to other practice settings might be limited.
In conclusion, C. difficile is the leading cause of HAIs in US acute-care hospitals, resulting in significant costs, extended stays, and patient suffering. Hospitals experience increased scrutiny and potential penalties based on CDI surveillance data at payer and accreditation levels. Our results indicate that reported CDI rates increased significantly in the studied hospitals and time frame with the use of NAAT due to its high sensitivity and potential low specificity. This finding also may support the IDSA/SHEA recommendation to use 2-step testing for diagnosis of CDI.Reference McDonald, Gerding and Johnson20
Acknowledgments
None.
Financial support
This work was supported by the Agency for Healthcare Research and Quality (AHRQ grant no. R01-HS023821-02).
Conflicts of interest
All authors report no conflicts of interest relevant to this article.
