Frequently ordered diagnostic tests among hospitalized patients include urinalysis and urine culture. These tests may be ordered due to signs and symptoms specific to urinary tract infection (UTI) but also due to nonspecific indications such as altered mental status, fever, or leukocytosis. Testing for broad indications results in substantial use of laboratory resources and associated costs. Reference Saint1 Antibiotics are also frequently prescribed if urine cultures are positive, even in the absence of UTI symptoms. An estimated 20%–50% of antibiotics prescribed in hospitalized patients are inappropriate or unnecessary. Reference Gandhi, Flanders, Markovitz, Saint and Kaul2,3
To address this problem, a variety of successful methods to reduce unnecessary testing have been implemented by hospitals. Reference Pavese, Saurel and Labarère4,Reference Trautner, Grigoryan and Petersen5 One method is diagnostic stewardship, which is the optimization of the process of ordering, performing, and reporting diagnostic tests. Reference Morgan, Malani and Diekema6 In one such test, urinalysis with reflex to culture (UARC), by policy, a urine sample goes to culture only if the urinalysis is consistent with the presence of UTI by prespecified reflex criteria. Institutions implementing UARC protocols Reference Lynch, Appleby-Sigler and Bork7,Reference Dougherty, Rickwa and Guy8 have demonstrated decreases in rates of urine cultures sent after their introduction. Reference Munigala, Rojek and Wood9 This intervention also has the potential to decrease inappropriate antibiotic use Reference Ourani, Honda, MacDonald and Roberts10 and laboratory costs by reducing the number of inappropriate cultures performed. Reference Howard-Anderson, Ashraf, Overton, Reif, Murphy and Jacob11 However, standardized reflex criteria have not been established. A previous study found that a combination of ≥5×109/L white blood cells (WBC) and positive nitrites on urinalysis yielded the highest positive predictive value for a positive urine culture result, Reference Ourani, Honda, MacDonald and Roberts10 but a positive urine culture does not necessarily indicate the presence of infection and must still be interpreted with patient clinical characteristics and symptoms in mind. A stricter UARC criterion of ≥10×109/L WBC has also demonstrated efficacy of the intervention without associated adverse events. Reference Claeys, Zhan and Pineles12
In this study, we evaluated the impact of changing the UARC reflex criteria to a stricter criterion related to urine culture rates and diagnostic efficacy. We also assessed potential harm after the intervention implementation by evaluating antimicrobial prescription suggestive of missed UTI and subsequent sepsis. Furthermore, we evaluated the impact on clinical outcomes, including antimicrobial prescription for suspected UTI and catheter associated urinary tract infection (CAUTI) rates. These results may support the use of a stricter UARC reflex criterion.
Methods
Study setting
This retrospective study included patients admitted to Tufts Medical Center (TMC), a 415-bed academic medical center who had urine testing ordered during their admission between December 1, 2018, and October 31, 2020. For patients with multiple tests ordered, only the initial test from their admission was included in the study. Specimens from patients with and without urinary catheters were included. Patients aged <18 years and those who had testing performed in an outpatient setting were excluded from the study. The data selection process is represented in Supplementary Figure 1. This study was approved by Tufts Health Sciences Institutional Review Board and granted exempt status.
Sample size
We calculated the sample size for the primary outcome of urine culture rates. Sample size and power were calculated for an ANCOVA analysis as an estimate for the interrupted time-series analysis (ITSA). To be conservative, a difference of 2.5 cultures per 1,000 patients between study periods was utilized, corresponding to an effect size of 0.1 (SD, 25). Assuming a 2-sided α of 0.05 and power of 80% to detect a difference using an ANCOVA, the required sample size was 787 cultures overall. To determine the sample size required for the adverse events analysis chart review, a 95% confidence interval (CI) with a 5% margin of error and 2% event rate was assumed; thus, at least 120 chart reviews were needed.
Data collection
Patient and laboratory data were abstracted from the hospital laboratory ordering database. Data included patient demographic characteristics (ie, age and sex) and laboratory test results (ie, urinalysis and urine culture). Antimicrobial data was abstracted from the pharmacy database and was matched to data obtained from the laboratory database by patient medical record number and time of antimicrobial prescription. CAUTI data, including urinary catheter days and number of infections were independently collected by the TMC infection prevention department. Potential harm events were assessed by chart review (performed by J.P.) from patients who had tests ordered in the post-intervention period that did not reflex but would have in the preintervention period (ie, patients with ≥5×109/L WBC but <15×109/L WBC or only >trace leukocyte esterase or positive nitrites), evaluating for the following outcomes: (1) antimicrobial prescription for all causes, (2) timing of antimicrobial therapy relative to urine testing, (3) antimicrobial prescription for UTI, (4) documented symptoms consistent with a UTI, and (5) sepsis.
Definitions
Urine cultures with growth of an organism ≥100,000 colony-forming units per milliliter (CFU/mL) were considered positive culture results for the purpose of this analysis. Urine cultures that were negative for significant growth (<100,000 CFU/mL) or that grew mixed flora consistent with urogenital contamination were considered negative. Antimicrobials prescribed for suspected UTI were identified from pharmacy data using the selected provider indication of “urinary tract infection.” Catheter-associated urinary tract infections (CAUTIs) were defined according to the National Healthcare Safety Network (NHSN) definition 13 as a UTI in which an indwelling urinary catheter was in place for >2 consecutive days in an inpatient location on the date of event, with day of device placement being day 1. In the analysis of potential harms, sepsis was defined as a patient meeting quick Sepsis-Related Organ Failure Assessment (qSOFA) sepsis criteria, Reference Evans, Rhodes and Alhazzani14 with sepsis secondary to a missed UTI defined as a positive urine culture from a pathogenic organism in a patient meeting those criteria with documented symptoms of a UTI within 7 days of initial testing. Symptoms considered consistent with UTI were fever, suprapubic tenderness, costovertebral angle pain or tenderness, urinary urgency, urinary frequency, or dysuria. 13
Intervention
The intervention started on October 22, 2019. The preintervention period for this study was December 1, 2018, to October 31, 2019, and the postintervention period was November 1, 2019, to October 31, 2020. At the time of the intervention, the UARC reflex criteria changed from ≥5×109/L WBC or trace or more leukocyte esterase or positive nitrites on urinalysis to only ≥15×109/L WBC. A diagrammatic figure demonstrating this intervention is included in Supplementary Figure 2, along with a snapshot of the provider orderable in Figure 3. This intervention was guided by a prior institutional analysis evaluating the future impact of changing the reflex criterion to either ≥10×109/L WBC or ≥15×109/L WBC, Reference Durgham15 which projected a decrease in culture rates of 64% without increasing rates of false negative results if the criterion was ≥15×109/L WBC. A summary of this analysis is included in the Supplementary Figure 2 and the summary of preintervention analysis. Previous studies Reference Claeys, Zhan and Pineles12,Reference Jolkkonen, Paattiniemi, Kärpänoja and Sarkkinen16 have demonstrated the effectiveness of stricter reflex criteria. One prior multicenter retrospective study demonstrated the efficacy of a similar intervention using ≥10×109/L WBC as the criterion without significant differences in gram-negative bloodstream infections. Reference Claeys, Zhan and Pineles12
Outcomes
The primary outcome was the change in culture rates of tests ordered as UARC per 1,000 patient days before the intervention (December 1, 2018–October 31, 2019) and after the intervention (November 1, 2019–October 31, 2020). Additional primary outcomes were measures of testing utilization, including (1) the change in culture positivity, which is the proportion of cultures reflexed from UARC with bacterial growth from cultures performed and (2) UARC performance, which is the proportion of cultures reflexed from UARC from UARC ordered. Secondary outcomes included (1) antimicrobials prescribed for suspected UTI per 1,000 patient days and (2) CAUTIs per 1,000 urinary-catheter days.
Statistical analysis
Patient characteristics in the overall study period were reported, and comparisons of the intervention periods were made using the Student t test and the χ2 test. Primary and secondary outcomes, at the monthly level, were summarized by median with interquartile range, and comparisons of the intervention periods were made using the Wilcoxon rank-sum test. For the primary analysis, an ITSA with negative binomial regression was performed to estimate the change in culture performance and clinical outcomes, as well as to assess pre- and postintervention trends. Negative binomial regression was utilized rather than Poisson regression to address overdispersion. This analysis was represented graphically using regression lines with Newey-West standard errors with lag(0). P < .05 was considered statistically significant. Because the study period included the onset of the COVID-19 pandemic, a sensitivity analysis excluding March–May 2020 (the period with greatest volume of COVID-19 patients from the initial surge) was performed. All statistical analysis was performed using R studio version 4.1.1717 software (R Core Team, Vienna, Austria). Graphical representation of the ITS analysis were created utilizing Stata version 16.1 software (StataCorp, College Station, TX).
Results
Patient characteristics
During the study period, 11,322 UARCs were ordered, of which 5,521 tests were ordered before the intervention and 5,801 tests were ordered after the intervention. Average age of the study population was 63.6 years (IQR, 49–75) and 52.3% of the population was male. There were no significant differences between intervention periods in age (P = .58) or patient sex (P = 1).
Urine culture characteristics
Table 1 includes culture data from the overall study cohort as well as pre- and postintervention periods analyzed at the month level. Comparing medians, we detected a significant decrease in cultures rates from tests ordered as UARC as well as performance of cultures. Culture positivity also increased significantly: 32.5% before the intervention versus 49.2% after the intervention.
Culture Characteristics During the Pre- and Postintervention Periods
Note. IQR, interquartile range; UARC, urinalysis with reflex to culture; UTI, urinary tract infection; CAUTI, catheter-associated urinary tract infection. Data analyzed on month level. Median (IQR) reported for all outcomes.
a Compared via Wilcoxon rank-sum testing.
b Cultures reflexed from UARC per 1,000 patient days.
c Culture performance measured as cultures reflexed from no. of UARCs divided by no. of UARCs ordered.
d Culture positivity measured as culture positive for organisms or culture reflexed from UARC.
e Antibiotic prescribed for suspected UTI per 1,000 patient days.
f CAUTI rate per 1,000 urinary-catheter days.
In the ITS analysis, which is represented graphically in Figure 1 (complete results in supplemental material, Table 1), we detected a significant decrease in the rate of cultures performed (32.5 cultures per 1,000 patient days before the intervention versus 8.6 cultures per 1,000 patient days after the intervention; P < .001), which corresponded to a 73.3% reduction in culture rates from the beginning of the preintervention period to the beginning of the postintervention period. We detected a significant preintervention monthly decrease (4% per month; P = .001) and subsequent postintervention increase (2.9% per month; P = .01) in culture rates. Performance of cultures from UARC also significantly decreased (42.9% before the intervention vs 19.6% after the intervention; P < .001) without observed pre- or postintervention trends. Culture positivity significantly increased after the intervention: 34.8% before the intervention versus 62.1% after the intervention (P = .01).
Interrupted time-series analysis (ITSA) of urinalysis with reflex culture characteristics. Top left: UARC culture rate per 1,000 patient days. Top right: UARC performance. Bottom left: Culture positivity. Dotted line represents time of intervention with study start date of December 1, 2018, and end date of October 31, 2020. Note. UARC, urinalysis with reflex to culture.
Antimicrobial prescription for UTI
Comparing median prescription rates, we detected a significant decrease in antibiotic prescription for suspected UTI between the pre- and postintervention periods (Table 1). The results from the ITS analysis for this outcome are shown in Figure 2. We detected a significant change in antibiotic prescription rates: 20.5 antibiotic prescriptions per 1,000 patient days before the intervention versus 14.2 antibiotic prescriptions per 1,000 patient days after the intervention (P = .047), with a significant preintervention monthly decrease of 3.6% per month (P = .0006).
Interrupted time-series analysis (ITSA) of antibiotic and CAUTI rates. Left: Antibiotic rate per 1,000 patient days. Right: CAUTI rate per 1,000 urinary catheter days. Dotted line represents time of intervention with study start date of December 1, 2018, and end date of October 31, 2020. Note. CAUTI, catheter-associated urinary tract infection.
Catheter-associated urinary tract infections (CAUTIs)
Comparing median CAUTI rates (Table 1), we did not detect a significant change between the intervention periods. The results from the ITS analysis are demonstrated graphically in Figure 2 and show no significant change in CAUTI rates after the intervention: 2.7 CAUTIs per 1,000 urinary catheter days before the intervention versus 0.56 CAUTIs per 1,000 urinary catheter days after the intervention (P = .29).
Analysis of potential harm events
To evaluate potential harm associated with the intervention, we analyzed a subset of patients from the postintervention period. This subset included 646 patients, of whom 130 were randomly selected for chart review. Of these reviewed charts, 82 had received antibiotics, 15 (11.5%) of which were for a suspected UTI, which could represent possible missed UTI diagnoses after the intervention. Only 6 of those 130 patients who received antibiotics for a suspected UTI had documented symptoms, consistent with an infection rate of 4.6%. Only 1 patient (0.8%; 95% CI, 0.04%–4.8%) was identified with potential sepsis secondary to a UTI, for which an alternative explanation (hemorrhage) existed. Table 2 demonstrates documented causes of sepsis, as well as timing of initiation of antimicrobial therapy. The timing of all antimicrobial prescription is reported in Supplementary Table 2. Among those with an unidentified cause of sepsis, 2 of 3 cases had urine testing performed at time of antimicrobial initiation, and a third case received antimicrobials prior to testing.
Documented Cause of Sepsis and Timing of Antimicrobial Therapy Relative to Urine Testing
Note. CAP, community acquired pneumonia; HAP, hospital-acquired pneumonia; VAP, ventilator-associated pneumonia; qSOFA, quick Sepsis-Related Organ Failure Assessment.
a Sepsis defined as patient meeting qSOFA criteria for sepsis. Causes were abstracted from electronic medical record documentation.
b Unidentified cause defined as patient meeting qSOFA criteria, but without source identified on testing (sepsis, unspecified cause documented in electronic medical record).
c Timing of initiation of antimicrobial therapy was determined by assessment of testing date and antimicrobial start date as provided by pharmacy database.
Sensitivity analysis
A sensitivity analysis was performed by removing data from March 1–May 31, 2020, the period of greatest pandemic impact on the study center. With this period excluded, no meaningful change was detected in the study outcomes (Supplementary Table 3).
Discussion
In this study, implementing a stricter reflex criterion for urine testing ordered as UARC resulted in the anticipated decrease in culture rates with subsequent increase in culture positivity. Previous single center Reference Lynch, Appleby-Sigler and Bork7–Reference Munigala, Rojek and Wood9 and multicenter studies Reference Howard-Anderson, Ashraf, Overton, Reif, Murphy and Jacob11,Reference Claeys, Zhan and Pineles12 have demonstrated similar findings after the implementation of UARC as a method of diagnostic stewardship. More restrictive interventions have demonstrated a >30% decrease in culture rates by cancelling urine cultures deemed low risk without negative clinical consequences. Reference Hertz, Lescallette, Barrett, Ward and Self17 In our study, a significant decrease in culture rates of tests ordered as a UARC occurred after the intervention took place, and previous studies have demonstrated cost savings associated with this intervention. Reference Munigala, Rojek and Wood9 Although some cost savings accrued to the hospital associated with this reduction in inpatient urine cultures, this savings was offset by a decrease in revenue-generating cultures performed on outpatients.
Criteria for reflex to urine culture are typically based on the presence of pyuria. Although the absence of pyuria is known to have a high predictive value for negative urine cultures, Reference Simerville, Maxted and Pahira18,Reference Pallin, Ronan, Montazeri, Wai, Gold, Parmar and Schuur19 there is no consensus for the optimal laboratory cutoff value. A previous multicenter study conducted at the Veterans’ Health Association Reference Claeys, Zhan and Pineles12 evaluated the change in culture rates after UARC implementation in sites utilizing a permissive versus restrictive criterion and demonstrated a decrease in urine culture rates only at sites utilizing the stricter criteria. Reference Claeys, Zhan and Pineles12 Our study yielded similar findings, providing more evidence for the use of stricter UARC criteria in testing algorithms. Importantly, we did not detect harms associated with this intervention. Of the charts reviewed, 15 patients whose UARC did not reflex to culture received antibiotics for suspected UTI (11.5%), representing potentially missed UTI diagnoses. Among these cases, only 6 patients (4.6%) had documented symptoms consistent with a UTI. However, 1 patient who had an alternative explanation for their condition met criteria for sepsis secondary to a UTI. Prior studies utilizing stricter reflex criteria Reference Claeys, Zhan and Pineles12 did not report any harms associated with the intervention.
In our study, a decrease in antibiotic prescription for suspected UTI occurred after the UARC criteria intervention. However, this decrease occurred primarily during the preintervention period, and it is unlikely that the intervention itself was the cause. Previous studies have shown the impact of UARC interventions on antimicrobial prescription. A prior single-center study Reference Ourani, Honda, MacDonald and Roberts10 reported a 30% reduction in inappropriate antibiotic prescription after implementing UARC. A larger multicenter study successfully used diagnostic stewardship measures in urine testing to reduce antimicrobial prescription rates. Reference Lee, Leung, Lee and Lai20 In another study, patients in whom testing reflexed to culture had 4.92 times the odds of antibiotic prescription, Reference Klein, Elman, Townes, Lewis and McGregor21 even after adjusting for the presence of urinary symptoms. In our analysis, we detected a significant preintervention trend that appeared to contribute the most to the difference in antimicrobial prescription rates between the intervention periods. During the preintervention period, there was a strong institutional focus on provider education for the use of appropriate diagnostic testing as well as antimicrobial stewardship, which may have contributed to the observed preintervention trend, although these interventions did also continue in the postintervention period. A more detailed analysis of patient characteristics associated with antimicrobial prescription would contribute to our understanding of which patients are more likely to receive antimicrobials and potential areas of future interventions.
CAUTI rates did not significantly change after this criterion intervention. The impact of UARC implementation on CAUTI rates has been studied previously, with no significant change in CAUTI rates after this diagnostic stewardship intervention. Reference Munigala, Rojek and Wood9 In our study, we observed a preintervention decrease in institutional CAUTI rates, but it was not significant. During the preintervention period there was a strong institutional focus on appropriate diagnostic testing, including several unit-led CAUTI initiatives. These initiatives were affected by the COVID-19 pandemic, but no change in the outcome was seen in the sensitivity analyses. The rate of CAUTI at our institution was low at baseline, so it is unlikely that a statistically, or clinically, significant decrease would be observed with any intervention.
Our study had several limitations. This single-center study included minimal information on patient characteristics, which would have better informed our analysis of reasons for antimicrobial prescription, and we lacked data on the method of specimen collection. An absence of data on the number of diagnosed UTIs is also a potential limitation; these diagnoses are not routinely monitored and are often poorly documented in medical records. In this population, CAUTIs represent a small proportion of total UTIs, and CAUTI data were not available due to active hospital surveillance protocols. Another limitation was the use of indication selection for antibiotic prescription data; this is not always reliable but was the only means of linking prescription to testing, which likely underestimated the prescription rate. The absence of a control group is another potential limitation, making it difficult to ascertain the impact of the laboratory intervention versus other coexisting interventions intended to change provider ordering behavior.
Our study also had several strengths. Our data set included many tests performed at an academic medical center with an accessible electronic ordering system, providing complete data. This study also utilized an ITSA, which is a robust study design allowing the analysis of pre- and postintervention trends. Reference Kontopantelis, Doran, Springate, Buchan and Reeves22 The ability to link ordered tests with microbiology results and antimicrobial prescriptions also allowed for the analysis of important clinical outcomes.
In conclusion, implementing a stricter UARC criterion was associated with significantly decreased urine culture rates and increased subsequent culture positivity. A chart review revealed only 1 potential case of sepsis secondary to a missed UTI. These findings support the use of stricter reflex criteria for inpatients at institutions where this testing algorithm is used. Continued refinement of these diagnostic stewardship interventions, as well as provider education are needed to supplement these efforts and to improve clinical decision making.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/ice.2022.178
Acknowledgments
We thank Dr Anna Modest for her assistance in generating the figures demonstrating the ITSA results. This study was supported by the Tufts University Clinical and Translational Science Institute. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the National Institutes of Health (NIH).
Financial support
This study was supported by the National Center for Advancing Translational Sciences (NCATS) and the NIH (grant no. UL1TR002544 to J.P.). This study was also supported by the Francis P. Tally, MD Fellowship in the Division of Geographic Medicine and Infectious Disease at Tufts Medical Center and by the Natalie V. Zucker Research Center for Women Scholars Award.
Conflicts of interest
All authors report no conflicts of interest relevant to this article.
