Infectious diarrhea is prevalent in the United States, with estimates of 178.8 million illnesses, 473,832 hospitalizations, and 5,072 deaths per year in the United States.Reference Elaine, Patricia, Frederick, Robert and Robert 1 Most cases of gastroenteritis are self-limited and do not require microbiologic diagnosis unless patients are immunocompromised or present with dysentery or an outbreak investigation is underway.Reference Shane, Mody and Crump 2 , Reference Smieja and Goldfarb 3 Polymerase chain reaction (PCR)–based, culture-independent identification methods allow for efficient specimen transport and provide rapid identification of more organisms.Reference Cybulski, Bateman and Bourassa 4 These methods have shifted the paradigm of evaluation of diarrheal illnesses away from labor-intensive and insensitive direct stool examination and culture methods. Cost-effectiveness is unknown because multiplex PCR panels are more expensive than traditional methods but have the potential to be more cost-effective than multiple individual tests.Reference Ramanan, Bryson, Binnicker, Pritt and Patel 5 , Reference Beal, Tremblay, Toffel, Velez and Rand 6 A multiplex-panel cost–benefit analysis demonstrated that despite an additional $29,392 implementation cost, the hospital could save $88,066 by reducing inpatient isolation time from 2,202 to 1,447 days.Reference Goldenberg, Bacelar, Brazier, Bisnauthsing and Edgeworth 7
The FilmArray Gastrointestinal Panel (GIPP) (BioFire, Salt Lake City, UT) is both more rapid and more sensitive than traditional stool culture and detects 22 common pathogensReference Smieja and Goldfarb 3 (Supplementary Material online), with sensitivity of 94.5%–100% and specificity >97%.Reference Buss, Leber and Chapin 8 Multiplex panels like the GIPP could potentially impact the reduction of empiric antimicrobial therapy for acute gastroenteritis and avoidance of subsequent unnecessary workup.Reference Beal, Tremblay, Toffel, Velez and Rand 6 In one academic institution, the use of the FilmArray test led to a significant reduction of empiric antibiotic therapy (23.5% vs 40%; P = .0148).Reference Cybulski, Bateman and Bourassa 4 Despite the usefulness of GIPP, it is expensive to perform. Efficiency and rapid turnaround may lead to unnecessary use, particularly in the inpatient setting where pathogens other than C. difficile and norovirus (both of which have separate tests available) are uncommon. Inappropriate stool testing has been reported even with stool cultures, and clinical decision support (CDS) interventions can be helpful in reducing inappropriate testing.Reference Nikolic, Richter, Asamoto, Wyllie, Tuttle and Procop 9 , Reference Tewell, Talbot and Nelson 10
In 2015, our institution implemented the GIPP for patients with diarrheal illnesses in lieu of the traditional stool culture. Our previous experience described shortened turnaround time from 55.5 hours to 3.9 hours.Reference Murphy, Fowler, Iwen and Fey 11 Clostridioides difficile (which is present on the panel but not reported to clinicians) was the most commonly identified organism followed by norovirus (8.9%), with 7.8% of stools having multiple pathogens.Reference Murphy, Fowler, Iwen and Fey 11 Prior to implementation of the GIPP, we offered a variety of stool diagnostic tests, including traditional culture, special culture upon request (eg, Yersinia and Vibrio), standalone testing for norovirus, adenovirus, and rotavirus, routine parasite detection with enzyme-linked immunosorbent assay (EIA) test for Cryptosporidium and Giardia, and traditional stool ova and parasite exams upon special request. To promote appropriate use and guide interpretation, the antimicrobial stewardship program (ASP) provided guidance at the time of GIPP implementation.Reference Van Schooneveld, Fey and Rupp 12 The guidance suggested that inpatients have the test performed only once per admission and not after hospital day 5, with an ASP plan to review initial use after implementation and adjust guidance as needed. We sought to evaluate the compliance with ASP recommendations and then to measure the impact of a diagnostic stewardship intervention on inpatient use of the GIPP.
Methods
Setting and testing
This study was performed at an 830-bed Midwestern tertiary-care medical center serving a large population of complex and immunocompromised patients. The GIPP was performed daily using the FilmArray system upon receipt of stool samples, without restrictions on the quality of stool. Traditional stool cultures were no longer offered. Clostridioides difficile test results from the GIPP were not reported and in accordance with published guidelines, we used algorithmic testing for C. difficile using combination glutamate dehydrogenase (GDH)/toxin enzyme immunoassay, with GDH+/toxin specimens reflexed to PCR testing.Reference McDonald, Gerding and Johnson 13
At the time of GIPP implementation, the ASP in concert with the clinical microbiology laboratory published guidance on test utilization, suggesting that GIPP testing not be repeated or performed after hospital day 5. These guidelines (see Supplementary Material online) were initially distributed via institutional newsletter and laboratory alert, and were available on the institution’s antimicrobial stewardship website.Reference Van Schooneveld, Fey and Rupp 12 Inpatient GIPP use was reviewed over the first 12 months (July 2015 through June 2016) after implementation, which revealed that 14.8% of inpatient testing was inappropriate. A secondary evaluation of the GIPP using a more stringent cutoff of testing >72 hours after hospitalization or repeated during a hospital stay revealed that 19.9% of tests were inappropriate, and no meaningful pathogens were detected in the inappropriate tests.
Intervention
Based on the findings of the GIPP review, the ASP, clinical microbiology laboratory, and the institutional clinical effectiveness (CE) program initiated a partnership focused on updating existing ASP guidelines and improving adherence to appropriate use of the GIPP. The team met on 3 occasions over a 1-month period in early 2017. They first reviewed baseline data on the adherence to current guidelines, set a goal to reduce GIPP utilization by 20% based on updating guidelines to recommend no duplicates or testing >72 hours following hospitalization, and then collaborated with the informatics team to create decision support tools to enforce this recommendation.
In April 2017, updated guidelines were published,Reference Van Schooneveld, Fey and Rupp 12 and CDS tools were implemented within the electronic health record (EHR; EPIC), providing guidance on appropriate use and alternative testing and eliminating the ability to order duplicate testing and tests in patients hospitalized for >72 hours. CDS information was included with the test, and an order validation alert occurred if the testing was repeated or after 72 hours hospitalized. An override of this process could be accomplished by contacting the microbiology laboratory director. The GIPP test ordering workflow is illustrated in Fig. 1.
GIPP test-ordering workflow. Upon initial order entry, the user encounters an informational clinical decision support screen advising when the GIPP will be prevented from being ordered and offering alternative stand-alone testing if necessary. If the user attempts to order the GIPP inappropriately, an alert appears, preventing them from signing and directing them to call the microbiology laboratory director if an override is desired. The microbiology laboratory director makes determination whether override testing is appropriate. Note. GIPP, gastrointestinal pathogen panel.

A silent best-practice advisory (BPA) was also triggered when users opened the GIPP order panel. This allowed for measurement of the number of times the order panel was opened but not completed, which was used as a surrogate marker for the effectiveness of the CDS and hard stop ordering restriction. Prior to the silent BPA, incomplete order information was not discretely captured in the EHR. The silent BPA data analysis measured the number of encounters rather than discrete orders to avoid overinflation of the effectiveness of the CDS and cost savings.
Cost savings
Cost savings were derived based on the institutional direct cost for the GIPP test and number of tests completed, compared to the baseline average monthly direct cost in the 15 months prior to intervention. The silent BPA data were used to estimate the “soft” cost savings of additional GIPP encounters prevented above the monthly average direct cost savings, based on the number of encounters GI pathogen orders that were opened and avoided (accounting for alternative testing costs where applicable).
Statistical analysis
The rate of GIPP panel testing was used to compare the preintervention period (period 1: January 2016 through March 2017) and the postintervention period (period 2: April 2017 through March 2018). Data are presented as rate per 1,000 patient days. We tested the null hypothesis that the rate of GIPP ordering (also appropriate and C. difficile testing) during period 2 did not differ from the rate of ordering during period 1. The mean number of monthly GIPP orders was compared between the periods using the Student t test. The overall rate of GIPP orders for the pre- and postintervention periods were compared using negative binomial regression. No interventions were undertaken with regard to C. difficile testing, so the rate of C. difficile testing was used as a counterbalance measure to ensure that any reduction of test ordering was not a result of overall reduced testing for any gastrointestinal illnesses. We followed the approach outlined by Bernal et alReference Bernal, Cummins and Gasparrini 14 and hypothesized that the introduction of ASP guidelines in April 2017 was associated with both a level and slope change in the rate of GIPP ordering between the 2 time periods. P < .05 was considered statistically significant. Analyses were conducted using SAS version 9.4 software (SAS Institute, Cary, NC).
Human subjects protection
This project, implemented with the ASP and CE teams for quality improvement purposes, was reviewed and was not considered human subjects research by the University of Nebraska Medical Center Institutional Review Board.
Results
During period 1, 1,587 total GIPP tests were ordered over 212,212 patient days; 341 (21.5%) were deemed inappropriate. During period 2, 1,165 total GIPP tests were ordered over 222,343 patient days, and 57 (4.9%) were deemed inappropriate (P < .001). The ratio of GIPP tests per month to average daily census was 0.23 during the preintervention period and decreased to 0.16 during the postintervention period (P < .001). During period 1, 67 of 341 inappropriately ordered GIPP tests (19.6%) yielded 1 or more pathogens. The microbiology laboratory director approved only 57 test overrides during period 2 (average, 4 per month), and only 5 of these were positive (8.8%) (Table 1). A review of the results of all inappropriate or overridden tests in each period was performed. During period 2, the only pathogens detected on override were Cryptosporidium and norovirus, neither of which resulted in treatment.
Summary of Positive Gastrointestinal Panel (GIPP) Test Results Before and After the Intervention

The Poisson model estimated a 30% reduction in GIPP ordering rates between the 2 periods (relative risk, 0.70; 95% CI,: 0.63–0.78; P < .001) (Fig. 2). For period 1 (January 2016 through April 2017) the GIPP rate was per 1,000 patient days was 7.48; for period 2 (April 2017 through June 2018), the GIPP rate was 5.24. We captured the C. difficile testing rate as a counterbalance measure; rates during period 1 and period 2 were 2.66 and 2.38, respectively (relative risk, 0.89; 95% CI, 0.78–1.02; P = .094) (Fig. 2).
Monthly GIPP ordering rate per 1000 patient days and the estimated slope for the 2 periods with monthly C. difficile ordering rates as a counterbalance measure. The figure demonstrates that GIPP ordering rates were increasing in the preintervention period, and were decreased in the postintervention period. There was no difference in rates of C. difficile ordering in both periods. Note. GIPP, gastrointestinal pathogen panel.

Table 2 describes results from the interrupted time-series analysis. The rate of GIPP ordering was increasing (2.45% per month) in the 15 months prior to the intervention (P < .001). The rate of GIPP ordering following the intervention was no longer significantly increasing (P = .58). The intervention was associated with a statistically significant change in the rate of GIPP ordering by −2.78% per month (95% CI, −4.35% to −1.22%; P ≤ .001) as measured by the ITS model.
Interrupted Time-Series Analysis Exploring Shift and Slope Change Between the 2 Periods

Note. GIPP; gastrointestinal pathogen panel; IRR, incidence rate ratio.
The total savings calculated, which factored only GIPP orders that triggered the hard stop, was ∼$67,000. When including encounters in which the GIPP test order was initiated but not completed (using silent BPA data), GIPP testing was reduced by 46%, for a potential savings of $168,000 (Fig. 3). These savings also accounted for the cost of alternative testing.
Monthly GIPP ordering and GIPP savings from encounters prevented since Go-Live compared to expected ordering trend. The figure demonstrates that continuous monthly savings calculated from encounters where inappropriate GIPP test ordering was prevented. Note. GIPP, gastrointestinal pathogen panel.

Discussion
We demonstrated that restriction of GIPP testing is feasible and safe, and resulted in significant cost savings. Consistent with published literature, we found that stool testing in patients hospitalized for >72 hours or those who have previously been tested is unlikely to be clinically relevant,Reference Hitchcock, Gomez and Banaei 15 , Reference Park, Hitchcock, Gomez and Banaei 16 which suggests that restricting repeat GIPP testing in the inpatient setting is reasonable. Reduction of unnecessary (either inappropriate or repetitive) laboratory testing is an attractive goal for quality initiatives, clinical microbiology laboratories, and ASPs. Benefits of diagnostic stewardship include improved patient satisfaction, reduction of irrelevant or false-positive test results and their nonbeneficial treatment, and significant cost savings. Interventions that combine clinician education, audit/feedback, and real-time EHR ordering restrictions are more likely to successfully reduce unnecessary laboratory testing.Reference Eaton, Levy and Soong 17
Clinical utilization review efforts have improved care and decreased healthcare costs by reducing low-volume, high-cost laboratory test frequency using a variety of methods.Reference Krasowski, Chudzik and Dolezal 18 EHR ordering restrictions and alerts have successfully reduced unnecessary and duplicate stool tests with significant cost savings.Reference Luo, Spradley and Banaei 19 , Reference Otto, Shuptar and Milord 20 The design of such restrictions and alerting is important because the utility of EHR alerts can be limited by the type of alerting (passive vs active or interruptive), where alerts appear in the clinical workflow, alert messaging and the ability to directly act upon the alert, and the development of “alert fatigue.” Clinicians are unlikely to act on alerts that do not appear in their workflow. Alerts that lack specific recommendations, or even if specific recommendations are included, lack the ability to link immediately to the desired action are much less likely to be successful. Although active or interruptive alerts are more likely to be noted, repeated and excessive alerts result in decreased clinician acceptance, especially in more complex clinical situations.Reference Ancker, Edwards and Nosal 21 Therefore, our clinical effectiveness program only approves active, actionable EHR alerts, which must be clearly justified and pass through rigorous committee review. Strong operational support of an override workflow by the microbiology laboratory directors was critical to the successful implementation of a hard-stop alert in the EHR.
We chose a “hard stop” alert that prevents the clinician from ordering the test if appropriateness criteria are not fulfilled. A CDS of this type is very effective at changing practice, but it must be carefully deployed to prevent patient harm and clinician dissatisfaction. A single-center report described implementation of a hard stop in the EHR, rejecting any orders for repeat C. difficile testing within 7 days of an initial test.Reference Luo, Spradley and Banaei 19 This resulted in a 91% reduction in repeat C. difficile PCR testing, from 14.5% to 1.3% (P < .001), with an estimated savings of almost $55,000 based on the cost of the test alone.Reference Luo, Spradley and Banaei 19 Another health system implemented an EHR fix to eliminate duplicate laboratory testing by comparing a hard stop preventing clinicians from ordering a duplicate test to a smart alert suggesting that the test not be duplicated with bypass allowed.Reference Procop, Keating and Stagno 22 Both options presented the ordering clinician with the rationale for avoiding duplicate testing and reduced duplicate ordering, but the hard stop was more effective (92% duplicate orders avoided vs 43% with smart alert alone; P < .0001).Reference Procop, Keating and Stagno 22 The estimated cost savings for the hard stop nearly doubled that of the smart alert ($94,225 vs $45,681).Reference Procop, Keating and Stagno 22
We included a process for ordering GIPP that did not meet appropriateness criteria for safety reasons; it required a phone call and approval by the laboratory director. These requests allowed clinicians a way to request testing outside of our parameters and were rare, usually approved unless after consultation an alternative test was more appropriate per clinician need, and infrequently resulted in positive results. The bypass process allowed for clinician autonomy in certain patient populations where it may be appropriate to obtain stool testing after >72 hours hospitalized, such as persons living with HIV, those older than 65 years of age, and neutropenic patients.Reference Bauer, Lalvani and Fehrenbach 23 Notably, of the 57 overridden tests in our cohort, only 5 were positive, and clinical review of these cases revealed no change in management based on the results. Thus, the restriction as implemented is likely safe and further restriction of even these approved results may be appropriate.
Stool-testing patterns may be influenced by other factors such as hospital census and patient factors, so we utilized a rate of GIPP testing as our primary outcome. Despite a consistently increasing patient census and a steady trend of increasing GIPP testing in the preintervention period, in the post-intervention period the rates of GIPP testing immediately dropped and slowly declined thereafter. We utilized C. difficile testing as an independent control for inpatient stool testing and noted that C. difficile testing did not change significantly. This finding suggests that the changes noted in GIPP testing were due to the intervention alone.
This study was a single-center, retrospective study that employed 1 of 3 available multiplex panels, which may limit its applicability in institutions that do not have these panels available. However, restriction of inappropriate testing is not dependent on the type of panel used; therefore, our results are likely generalizable to other institutions. Additionally, we utilized our EHR to develop the alerting, and not all centers have EHRs with robust CDS and alerting functionalities. Our bypass process required evaluation by the director of clinical microbiology, and other facilities may not have personnel that are available or that have the knowledge and experience to determine whether a test is appropriate. We did not evaluate the actual time the alert fired, but implementation of the restriction did not significantly influence laboratory workflow. Furthermore, we did not evaluate specific clinical outcomes such as length of stay or antibiotic use. Others have demonstrated that the implementation of the multiplex panels significantly decreased inappropriate antibiotic use for diarrheal illnesses from 43% to 26% (P = .023) in a center where empiric therapy for diarrhea was common.Reference Keske, Zabun, Aksoy, Can, Palaoğlu and Ergönül 24
Existing data support multiplex GIPP panels as useful additions to clinical care. Although these multiplex panels can provide rapid and useful results, at times they may contain information that is difficult for clinicians to interpret or to determine the best course of actions, such as being positive for multiple pathogens or positive for pathogens of debatable significance only (Enteropathogenic E. coli or Enteroaggregative E. coli). Given this information, along with the cost associated with the GIPP and the high incidence of inappropriate use, institutions should consider ways to ensure appropriate testing by engaging with their microbiology laboratory and antimicrobial stewardship teams. Incorporating diagnostic stewardship into the ASP can ensure that the costs do not outweigh the benefits of the test. Considering the degree of savings that a simple intervention can create, there is also a compelling argument for the value of an ASP, as $168,000 could potentially cover a large majority of a stewardship full-time equivalent (FTE).
In conclusion, GIPP testing in patients hospitalized for >72 hours or who have already had GIPP testing is almost always inappropriate. Implementing a hard stop in the EHR can significantly reduce inappropriate GIPP testing. Responsible test ordering of low-yield tests such as the GIPP leads to significant cost savings without affecting high-quality patient care. This study and our results highlight the value of diagnostic stewardship in ASP and collaboration with the microbiology laboratory.
Author ORCIDs
Jasmine R. Marcelin, 0000-0003-0504-753X
Acknowledgments
We thank the microbiology laboratory and clinical informatics team for their assistance in creating and maintaining the EHR alert for this project.
Financial support
No financial support was provided relevant to this article.
Conflicts of interest
All authors report no conflicts of interest relevant to this article.





