Central-line–associated bloodstream infections (CLABSIs) cause considerable morbidity and mortality.Reference Siempos, Kopterides, Tsangaris, Dimopoulou and Armaganidis 1 In Belgium, surveillance data for infections acquired in intensive care units (ICUs) indicated a CLABSI rate of 2.5 per 1,000 central-line days in 2012.Reference Mertens 2 , Reference Mertens, Morales and Catry 3 Several studies have demonstrated that CLABSIs are largely preventable.Reference Pronovost, Needham and Berenholtz 4 – Reference Palomar, Alvarez-Lerma and Riera 9
Surveillance is a necessary tool for monitoring CLABSI rates as well as the progress toward preventing CLABSIs at the hospital and national levels. A challenge for this specific surveillance is accounting for exposure to invasive devices because patients without a central line are, by definition, not at risk for CLABSI. Thus, the outcome indicator recommended as a “gold standard” in the US and European surveillance protocols is the CLABSI rate per 1,000 device days (ie, central-line days).
However, if not automated, as in the majority of Belgian hospitals, the daily counting of central-line days (CLDs) remains labor intensive. Operational research on alternative methods to exhaustive counting of CLDs is scarce; however, 4 studies have explored the validity of estimating CLDs from weekly samples. Estimates were calculated by multiplying the device utilization ratio (DUR, the ratio of CLDs to patient days) with the aggregated count of the number of patients (patient days). These studies concluded that sampling on weekdays versus exhaustive counting provided a reasonable estimate of CLDs in ICUs and non-ICUs, and a weekly sampling strategy was introduced in the device-associated module from the Centre for Disease Prevention and Control (CDC) in 2015.Reference Shelly, Concannon and Dumyati 10 – Reference Thompson, Edwards and Bamberg 13 This research was conducted exclusively in US settings for inter-hospital comparison and larger sampling intervals (eg, once per month or bimonthly) were not evaluated.
Building on these findings in the present study, we aimed to compare the accuracy of different sampling strategies in the direct estimation of CLDs in ICUs in Belgium. A secondary objective was to provide recommendations regarding sampling strategies for specific settings by identifying factors and strategies associated with better performance.
METHODS
Data Source and Exclusion Criteria
We used data collected within the framework of patient-based surveillance of ICU-acquired infections between 2004 and 2012. The surveillance system has been described elsewhere, and case definitions and methodology used in this study followed those of the European Centre for Disease Prevention and Control (ECDC).Reference Mertens, Morales and Catry 3 No ethical approval was acquired for this study. All involved hospitals were anonymous, and analysis was based on aggregated data, which do not allow for patient identification.
According to the ICU protocol, the minimal registration period for participation in the surveillance is 3 months per year. Data are collected by the dedicated staff of the hospital hygiene teams in participating hospitals. Data collection involved daily recording of exposure to invasive devices, including central lines for all patients admitted to the ICU for at least 2 days (ie, patients at risk for ICU-acquired infections). Admitted patients who were not discharged at the end of the trimester were followed for 1 more month or until discharge. Only 1 CLD was counted per patient per day, irrespective of the actual number of central lines. We obtained data on admission and daily exposure to central lines for each eligible patient admitted during 1 of 378 ICU trimesters, ie, data reported from single participating ICUs for a period of 3 months, with an additional month to follow-up patients who were not discharged by the end of the period. From those periods, we excluded 14 ICU trimesters in which less than 20 eligible admissions were recorded (ie, ≤5th percentile).
Data Analysis
Sampling strategies and extrapolation rules for estimating CLDs
Samples of CLDs were drawn from the original dataset according to 2 different strategies (Table 2): (1) on a fixed day of the month or week (eg, every Monday or the first Monday of the month) or (2) weekly on random week days (eg, on Monday of week 1, Thursday of week 2, etc.). Because results of the random sampling schemes are not unique, 100 samples were generated per random scheme and ICU trimester.
Central-line days were estimated per ICU trimester via an extrapolation rule by dividing the number of CLDs in the sample by the number of sampling days per sampling period and multiplying it by the total number of days per sampling period:
$$\left( {{{\mathop{\sum}\!{CLD_{i} } } \over {n\,specific\,sampling\,days}}{\times}(total\,number\,of\,days\,in\,sampling\,period)} \right),$$
in which i is the number of central lines observed on day i, for i=1,…,n. For example, if there are 13 Mondays in a given ICU trimester, then the estimated CLD is computed as follows:
$$\left( {{{number\,of\,CLD\,counted\,on\,Mondays\,in\,the\,sampling\,period} \over {13}}{\times}120} \right).$$
Accuracy of estimated CLD
The accuracy of the estimates was assessed by calculating the CLD percentage error (CLD % error) per ICU trimester, ie, the relative percentage difference between the observed exhaustive counts and estimated CLDs:
$$\left( {{{observed{\minus}estimated\,CLD} \over {observed\,CLD}}{\times}100} \right).$$
Linear mixed regression models were used to compare CLD % errors between sampling schemes because these models take into account the structure of the dataset: ICUs were nested within hospitals and CLD % errors for different sampling strategies were clustered by ICU trimester.
Rules for choosing a sampling strategy
We established a simple set of rules for choosing a sampling strategy by first identifying the best predictor of CLD % error among 6 ICU-trimester characteristics (ie, the predictor with the smallest Akaike information criterion). These characteristics were the observed exhaustive counts of CLDs, patient days, and admissions, in addition to DUR, average length of stay (LOS), and trimester of the year.Reference Siempos, Kopterides, Tsangaris, Dimopoulou and Armaganidis 1 – Reference Pronovost, Needham and Berenholtz 4 For the single best predictor, we estimated the cutoffs beyond which the model predicted, with at least 90% confidence, that an ICU trimester would have an absolute value of CLD % error of less than 5% or 10% (C5 and C10).
This set of rules was validated via parametric bootstrap and simulation. 14 All models were fit using the linear and nonlinear mixed effects model (nlme) package in R statistical software, version 3.0.1.
RESULTS
The dataset included information from 56 ICUs in 39 hospitals during 33 trimesters. Table 1 provides an overview of the characteristics of the 364 ICU trimesters.
Characteristics of 364 ICU Trimesters: Belgian Hospital Surveillance Data, 2004–2012
NOTE. Patients stayed at least 2 days in ICU.
ICU, intenive care unit; IQR, interquartile range (25th–75th percentile).
Assessing the Accuracy of Different Sampling Strategies
In Table 2, we present the distributions of CLD percentage error (CLD % error) for each of the sampling strategies tested. The higher the number of days sampled per trimester and per week, the more ICU trimesters had a CLD % error ≤10% and the smaller the CLD % error distribution (25th–75th percentile range). Tuesdays and Saturdays yielded the highest proportion of ICU trimesters (≥85%), with CLD % error within 10% when sampling once per week on a fixed day. This proportion was ≥90% when sampling at least twice per week on random days and was <65% when sampling once or twice per month.
Summary of Absolute Percentage Error for All Sampling Schemes for All 364 ICU Trimesters: Belgian Hospital Surveillance Data, 2004–2012
NOTE. Statistics from random sampling schemes are based on 100 random samples from the original data set. Patients stayed at least 2 days in the ICU.
ICU, intensive care unit; IQR, interquartile range.
A linear mixed model confirmed that for fixed sampling once per week, Tuesdays provided the best estimate of observed CLDs: The average % error from Tuesday sampling was closest to 0 of the 7 weekdays and was found to be significantly different from the averages of the remaining 6 days. A comparison was made of weekly sampling from the 7 fixed days versus a randomly selected day; the random scheme resulted in the mean % error closest to 0, and this mean % error was significantly different from that of Tuesday sampling, as well as those of the remaining 6 days. Sampling randomly once per week tended to yield estimates that were too large (average % error, 0.12; 95% confidence interval [CI], 0.03–0.22). However, as expected, increasing the frequency of random sampling per week brought the mean % errors closer to 0 and reduced the variability of the estimates.
Factors Influencing the Accuracy of Estimated CLDs
Through fitting linear mixed models, the observed number of CLDs was found to be the best predictor of a smaller percentage error, irrespective of the applied sampling scheme. Based on this result, we devised a set of rules for the identification of the optimal sampling strategy according to the (minimum) observed number of CLDs in the ICU trimester. The optimal sampling strategy is the strategy requiring the smallest number of sampling days needed to provide estimates of the total CLDs in the ICU trimester within 10% (C10) of the observed CLDs with at least 90% confidence (Table 3). The results clearly showed that with higher CLDs per ICU trimester, a wider interval between sampling was allowed.
Set of Rules for Sampling and Validation via Simulation: Belgian Hospital Surveillance Data, 2004–2012
NOTE. ICU, intensive care units; CLD, central-line days; C10, minimum CLDs per ICU trimester to predict error within 10% with 90% confidence; CI, confidence interval; NA, no minimum CLD (C10) was found using our dataset. We tested up to 1,800 CLDs per ICU trimester, and we conclude that ICUs with <1,800 CLDs per trimester should avoid using these methods.
Based on this criterion, we found that ICUs with >650 CLDs per trimester should sample 3 times per month on days 5, 15, and 25. ICUs with more than 480 CLD/trimester should sample every Tuesday, and ICUs with >250 CLDs per trimester should choose 2 random days of the week on which to sample. We validated this set of rules using a simulated data set (see Methods section). Estimations of the total number of CLDs obtained from applying these rules were within 10% of the observed CLDs for >95% of ICU trimesters in this simulated data set (Table 3). Results are not shown here for the parametric bootstrap; estimates of proportions were similar, but the confidence intervals were wider in cases where a limited number of ICUs were included in the dataset.
DISCUSSION
Approximately 80% of the ICU trimesters had an estimation of the actual CLDs within 10% of the observed CLDs when using estimations derived from sampling CLDs once per week (fixed day or random day) or 3 times per month. When increasing the numbers of day of sampling per week, this proportion increased to >90%. For bimonthly and monthly sampling, <60% had an estimation of the actual CLDs within 10% of the observed CLD. Sampling CLDs on Tuesdays provided better estimations than did sampling on other days of the week. Higher CLDs per ICU trimester were significantly associated with better CLD estimates (ie, % error <10%). From these findings, we established the following sampling rules: (1) Sampling should be conducted 3 times per month for ICUs with >650 CLDs per trimester, and (2) sampling should be conducted each Tuesday for ICUs with >480 CLDs per trimester (Table 3). By following these guidelines, we expect that an ICU can achieve, when sampling by trimester of admittance, a percent error ≤10% with at least 90% confidence. We rather arbitrarily consider 90% accuracy to be an acceptable compromise between (reduced) time spent on data collection and precision of the CLD estimate. Indeed, CLD data collection can only be done manually in most Belgian hospitals, and consequently, these data are collected in few hospitals. To our knowledge, no real assessment of the resource burden has been done in Belgium, but studies in the United States, where the electronic maturity is higher, showed that infection prevention specialists spend ~50% of their time on activities related to surveillance.Reference Stone, Pogorzelska-Maziarz and Herzig 15
The results of the present study are consistent with those of Klevens et alReference Klevens, Tokars, Edwards and Horan 11 and Thompson et al,Reference Thompson, Edwards and Bambert 12 which showed that weekly sampling yielded estimates of CLDs in ICUs that were not meaningfully different from actual figures based on daily data collection. Thompson et al also suggested that there exists a threshold for CLDs below which sampling generates less accurate estimates. In 2015, the CDC protocol for the device-associated module for bloodstream infections recommended sampling on the same (random) day once per week in the ICU and other ward locations with an average of ≥75 CLDs per month. 16 This protocol was validated in a published studyReference Thompson, Edwards and Bamberg 13 that included 89 wards (ICU and non-ICU) in 66 acute-care hospitals, wherein the number of CLDs per ward per month clearly influenced the accuracy of the estimated CLDs generated by the sampling data.
Our cutoffs for CLDs differed from those estimated by Thompson et al, which were included in the CDC protocol. This difference might be due to the fact that Thompson et al estimated total CLDs over a year, based on 52 weekly sampling estimates per ICU, whereas we estimated total CLDs over a quarter, based on 12–13 sampling estimates per ICU. This difference might also explain why our results indicated the need for higher CLD cutoffs per ICU trimester, which are better suited for ICU settings using quarterly surveillance (as recommended by the European protocol 17 ).
We also used a different methodology to calculate the estimated CLDs. In previous publications, estimation of CLDs was obtained by first calculating the DUR from the sample, then multiplying this number by the observed number of patient days within the same period. Here, we sampled CLDs directly and used the sample average (eg, for once weekly sampling on Monday we divided the sum of CLDs counted on Mondays by the number of Mondays in the trimester) and multiplied it by the total number of days in the specific trimester. This method seems to be easier to use in the field. We did not compare differences in percentage errors obtained using these 2 methods.
Our study has several limitations. First, we only took into account the “best” predictor (here, the CLDs) to create a set of rules for sampling schemes; we did not use other ICU characteristics (eg, DUR and LOS). However, the sampling rules were validated by simulations based on our choice of ICU trimester characteristics (ie, median observed LOS, median observed DUR, and CLD range [20–1,800 days]). Second, we only studied ICUs. It seems clear that sampling yields less accurate estimates for smaller CLDs and DURs, as can be expected in a non-ICU setting. However, Climo et alReference Climo, Diekema and Warren 18 found that there were 2.4 times as many patients with central lines outside the ICU than within the ICU setting. A small-scale study based on data from 6 hospitals concluded that a weekly measure of device utilization ratio provided a valid estimate of CLDs outside the ICU assessed using differences in percentile ranking.Reference Shelly, Concannon and Dumyati 10 Further research is needed to assess factors influencing the precision of estimates in the setting of non-ICU services.
We believe that the gold standard for measuring CLABSI rates using exposure (CLDs) as the denominator has its drawbacks. First, successful attempts at decreasing CLABSIs might result, contraintuitively, in an increase in CLABSI rate. This will be the case if prevention efforts result in decreased exposure (eg, by removing unnecessary lines), and if the numerator (CLABSIs) decreases proportionally less than the denominator (CLDs). This is likely to happen when less severely ill patients are the ones taken off central lines. This trend has been observed in at least 1 hospital in Belgium. A similar impact applies to rates of catheter-associated UTIs computed per catheter days.Reference Wright, Kharasch, Beaumont, Peterson and Robicsek 19 In addition, measuring CLABSI rates based on patient days is sufficient if the objective (as in Belgium) is monitoring trends at the hospital level rather than (as in the United States) inter-hospital comparison with benchmarking and ranking.Reference Pittet and Wenzel 20 , Reference Eggimann, Harbarth, Constantin, Touveneau, Chevrolet and Pittet 21 For this reason, the Belgian government introduced CLABSI rates based on patient days as a national quality indicator to evaluate prevention efforts between hospitals because these data are readily available. Finally, when device-utilization ratios (DURs) are relatively stable, trends of CLABSIs per CLD seem to be strongly correlated with CLABSIs per patient day.Reference Horstman, Li, Almenoff, Freyberg and Trautner 22 Therefore, counting CLDs in Belgium remains important to monitoring the level of exposure (DUR) in a given unit because these data are key to prevention efforts and to the interpretation of the CLABSI rate indicator.
Our findings add further evidence in support of sampling CLDs as a valid alternative to daily counting of CLDs. We established sampling rules for the ICU setting based on the number of CLDs per trimester; the higher the CLD per trimester, the smaller the number of sampling days in the trimester needed for obtaining accurate CLD estimates. By following these guidelines, we expect that an ICU can achieve, when sampling by trimester of admittance, a % error ≤10% with at least 90% confidence. These results represent an important step toward reducing the burden of data collection at the hospital level in European settings.
ACKNOWLEDGMENTS
The authors wish to thank the participating hospitals and their dedicated healthcare workers.
Financial support: The surveillance of infection data in ICUs in Belgium is funded by the Federal Public Service Health, Food Chain Safety and Environment. This manuscript was developed and completed in the absence of an outside funding source. The Department of Healthcare-Associated Infections and Antimicrobial Resistance at the Scientific Institute of Public Health financed a consultancy for statistical support from the University of Ghent (Crescendo Stat-Gent).
Potential conflicts of interest: All authors declare that there are no potential conflicts of interest related to this article.
