Introduction
The Surgical Care Improvement Project (SCIP) was a Joint Commission national initiative that included public reporting of evidenced-based antimicrobial guideline compliance metrics in high-risk surgeries. 1 During the period of active policy support, the program led to high rates of adoption of evidence-based practices (>95%). Reference Branch-Elliman, Elwy and Lamkin2–Reference Stulberg, Delaney, Neuhauser, Aron, Fu and Koroukian4 SCIP targeted major cardiac, orthopedic, vascular, colorectal, and gynecologic procedures. The effective project led to substantial improvements in peri-operative antimicrobial stewardship. Reference Stulberg, Delaney, Neuhauser, Aron, Fu and Koroukian4 SCIP antimicrobial use reporting was discontinued in 2015, as the manual measurement and reporting process was costly and was expected to yield limited additional benefits.
During the period since the SCIP reporting program was discontinued, the Centers for Disease Control (CDC) Surgical Site Infection (SSI) Prevention guidelines and the Society for Healthcare Epidemiology in America SSI Prevention Guidelines Reference Calderwood, Anderson and Bratzler5,Reference Berríos-Torres, Umscheid and Bratzler6 have been updated to include a recommendation to discontinue antimicrobials immediately after skin closure, rather than supporting up to 24 hours as was permitted under SCIP INF-3 (48 h for cardiac surgery). In the absence of a national surveillance program and external pressure to support practice change, little is known about how institutions plan to incorporate these guideline updates.
The SCIP program focused on public reporting for a limited number of surgeries, a form of audit-and-feedback to facilities, and a force of external pressure on facilities and providers to adopt guideline-concordant practices (Table 1).
Brief Summary of Metric and Guideline Recommendations
To support policy recommendations, facility-level implementation strategies, defined as tools and methods to support the uptake of evidence-based interventions, treatments or practice Reference Powell, Waltz and Chinman7 such as developing local policies and procedures, education on antimicrobial guidance, creation of order sets and checklists, and nursing-driven management protocols were locally developed and implemented to support compliance by sites. In theory, local (or facility-specific) maintenance of these implementation strategies should continue to support compliance with SCIP best practice measures in the absence of the public reporting requirement (Figure 1). Additionally, the creation and maintenance of these internal strategies may support the diffusion of antimicrobial practice guidelines to surgical procedures and specialties not included in the original SCIP program.
Internal and external pressures supporting uptake of evidence-based perioperative antimicrobial use recommendations.
This study aimed to examine antimicrobial steward’s perceptions about whether best practices have been maintained and to identify measures perceived to support ongoing compliance or lack thereof. A secondary aim was to determine which policies and procedures may have supported the spread of implementation strategies to procedures and surgical specialties not included under the umbrella of the original SCIP program. This survey, conducted across a variety of US healthcare settings, was conducted to inform a mixed methods analysis within the VA healthcare system designed to evaluate the long-term sustainability of best perioperative antimicrobial use practices following discontinuation of public reporting. Findings will be used to inform additional qualitative and quantitative assessments of SCIP program impact in the VA national healthcare care system.
Methods
Overview
Utilizing the Society for Healthcare Epidemiology of America (SHEA) Research Network (RN), we conducted a brief (approximately 15 min) survey to assess key stakeholders’ perceptions regarding maintenance of the SCIP program and supporting policies and procedures. The online survey was approved and administered by the SHEA RN committee over the course of 8 weeks from June through August of 2023.
Respondents
The SHEA RN is a voluntary group comprised of 114 healthcare institutions worldwide. We used the SHEA RN platform to request participation every two weeks, for a total of three requests. Clinical staff with expertise in perioperative infection prevention and antimicrobial stewardship working in facilities that perform major surgeries (eg, CABG, total joint replacement, etc.) were invited to participate. Participation in the survey was anonymous and confidential.
Survey
The survey consisted of forty questions, including questions about facility demographics and characteristics, SCIP compliance, the spread of SCIP best practices, and finally, a space to upload facility-specific materials (eg, order sets, check lists, protocols, etc.). The compliance questions focused specifically on SCIP INF-1 (antibiotic administered within 60 min prior to incision) and SCIP INF-3 (appropriate discontinuation post-operation). The survey also queried which implementation strategies continued to be employed within their facility and perceptions about which strategies were most effective and important for sustaining compliance with SCIP best practices in the absence of external pressure from public reporting programs. In addition to questions about maintenance of SCIP best antimicrobial stewardship practices, the survey also collected information about perceptions about spread to uncovered surgeries within a surgical specialty (eg, within orthopedics, from total joint replacement procedures to arthroscopic procedures) and to surgical specialties not included in the original program (eg, spread of INF-3 compliance to otolaryngology). A final section collected data regarding knowledge of the recent guideline update to discontinue prophylaxis after skin closure and about plans and strategies for implementing the change. The surveys consisted of both structured questions and open-ended responses. Respondents had the option of responding to all questions or partial survey responses. For some questions, due to branching logic (eg, if you have this policy, then…), not all participants were eligible to respond.
Analysis
Simple tabulations were used for analysis of the structured questions response rates. For the overall response rate, the study participation rate was calculated as the total number of SHEA research network facilities providing a full or partial response to the survey. For individual questions are presented as proportion of positive responses out of the population responding. Open-ended responses were categorized and reported qualitatively.
Ethics
This study was approved by the VA Boston Research and Development Committee as exempt research prior to data collection and analysis.
Results
Demographics and characteristics of respondents and facilities
The survey was distributed to 112 SHEA RN members; there was a 41% overall response rate. The respondents’ roles varied; 29% identified as infection control practitioners, 27% as antimicrobial stewardship program (ASP) physicians, 8% as ASP pharmacists, and 37% as having another role. Over 60%, (30/49, 61.2%) of respondents had personal experience implementing the SCIP program. Most respondents represented in the sample (60%) worked in facilities with more than 500 beds (30/50, 60%). Facility level results comparing key survey questions by bed size is presented in Table 2.
Demographics and key responses
*ASP – Antimicrobial stewardship program
SCIP compliance
Over 75% (28/37, 75.6%) of respondents reported that compliance with SCIP INF-1 has been maintained or mostly maintained since the period when public reporting was discontinued. Furthermore, over 85% (31/36, 86.1%) of respondents reported that SCIP INF-3 has been maintained or maintained mostly since the period when public reporting was discontinued. Almost half of respondents, 46% (17/37, 45.9%) for SCIP INF-1 and half, 50% (18/36, 50%) for SCIP INF-3, reported that there has been a similar impact across different surgical specialties since public reporting was discontinued.
Respondents reported that SCIP INF-1 compliance was supported by ongoing policies and procedures such as electronic order sets (29/36, 80.6%), facility policies (26/36, 72.2 %), and provider education (24/36, 66.7%). Of these policies and procedures, respondents reported that the most effective in supporting compliance with SCIP INF-1 were electronic order sets, timeouts, and facility policies. For SCIP INF-3, respondents reported that compliance was supported with electronic order sets (27/36, 75.0%), facility policies (19/36, 52.8%), provider education (16/26, 44.4%), and antimicrobial stewardship review with audit and feedback (16/26, 44.4%). For SCIP INF-3, respondents reported that electronic order sets, facility policies, and automatic stop orders were the most important in maintaining compliance.
SCIP antimicrobial use was monitored locally at some level for (19/36, 52%) of facilities. Of the facilities that continued to monitor SCIP metrics, (11/35, 31.4%) of respondents reported that compliance had stayed the same since the program was discontinued in 2015; (1/35, 2.9%) reported that compliance has increased; and (5/3514.3%) reported that compliance has decreased. Of the facilities that continued to monitor SCIP metrics, (17/34, 50%) of respondents reported that they provide feedback to providers. Finally, since the active discontinuation of SCIP reporting, over (29/36, 80%) of respondents reported strong ongoing facility leadership support for peri-operative antimicrobial stewardship.
Diffusion of SCIP best practices
Over (28/31, 90%) of respondents reported that SCIP best practices spread within the same specialty, this spread was facilitated by policies and procedures being applied to surgeries not covered in the program (19/31, 61.3%), providers adopting SCIP practices to less invasive surgeries (16/31, 51.6%), and change facilitated by electronic health record (EHR) order sets (15/31, 48.4%). Respondents reported spread of SCIP best practices to surgical specialties not initially covered in the program at most facilities: universal improvements in practice (11/31, 35.5%), slow but steady improvements in practice (11/31, 35.5%), very slow improvements in practice (6/31, 19.4%), and a small amount reported no improvements in practice (3/31, 9.7%). Of the sites where best practices spread to other surgical specialties, the top three facilitators of that change were EHR order sets (15/31, 48.4%), adoption of anesthesia standard operating procedures (15/31, 48.4%), and prolific implementation of surgical timeouts (11/31, 35.5%). The majority (27/31, 87.1%) indicated that the spread of SCIP best practices to other surgical specialties was positive.
SSI prevention guideline update
Most respondents were aware of the SHEA SSI prevention guideline update to discontinue antimicrobials immediately after skin closure (19/29, 65.5%). When asked if they would implement the new recommendation for shorter post-operative duration, (19/29, 65.5%) said yes, while (7/29, 24.1%) said it was not applicable because they were already practicing immediate discontinuation following skin closure. Those who said yes to implementing the new recommendation, (15/28, 53.6%) reported they would do provider education, (13/28, 46.4%) reported leadership engagement, and (11/28, 39.3%) reported updating peri-operative order sets. The leading reasons for those who indicated they would not implement the recommendation were being too busy or having competing priorities (3/24, 12.5%) and key stakeholders in surgery opposing the change (2/24, 8.3%). Finally, when asked if there was a plan to track changes in post-operative antimicrobial use following the guideline update, (20/29, 69%) of respondents reported no. The top three reasons for not planning to track changes were limited staff to support peri-operative stewardship activities (11/26, 42.3%), competing priorities (8/26, 30.8%), and limited technical support for updating current EHR order sets (5/26, 19.2%).
Discussion
Overall, facility antimicrobial stewardship experts reported perceptions that compliance with SCIP INF-1 and 3 metrics had been fully or mostly maintained in the absence of the external pressure applied by public reporting requirements. Half of facilities reported ongoing support for surveillance systems to measure part or all of SCIP INF-1 and 3.There was a strong perception that local policies and procedures, such as electronic order sets, supported the spread of SCIP best practices to other procedures within a surgical specialty (eg, best practices adopted for total joint arthroplasties spread to lower-risk arthroscopies), but opinions were varied concerning whether SCIP practices spread to surgical specialties not included in the program, such as otolaryngology and urology.
The 2022 SHEA SSI Prevention guidelines included a major update to recommend discontinuation of prophylactic antimicrobials immediately after skin closure, rather than permitting use up to 24 hours after procedure. Reference Calderwood, Anderson and Bratzler5 The rationale for this update was evidence of harms associated with extended durations without evidence of benefit. Reference Calderwood, Anderson and Bratzler5,Reference De Jonge, Boldingh and Koch8–Reference Branch-Elliman, O’Brien, Strymish, Itani, Wyatt and Gupta13 Although mandatory reporting of the SCIP INF metrics have been discontinued, this guideline update directly impacts the measurement and ongoing interpretation of metrics that measure compliance using INF-3 definitions. The majority of responding SHEA RN-participating facilities reported awareness of this change, but more studies are needed to evaluate how these updated guidelines are implemented, particularly in light of limited surveillance tools and external policy pressure to adopt the updated, and more restrictive antimicrobial use guidance. This change highlights the need for a dynamic approach to measurement of compliance and the challenge of evaluating long term policy sustainment.
The local policies and procedures that were the most implemented were also those perceived to be the most effective: electronic order sets, facility policies, provider education, and surgical timeouts. Ongoing surveillance with audit-and-feedback was perceived to be less critical for supporting SCIP compliance. Notably, facilities with ongoing surveillance were more likely to report compliance had decreased since SCIP was discontinued. Overall, respondents supported the concept that the internal practices developed and deployed to support compliance with SCIP during the period of active reporting continue to sustain the practices.
Spread of best practices within a specialty was perceived to be supported and adoption of these strategies to support the spread led to overall practice change. The application of policies and procedures for significant surgeries and minor surgeries was also perceived to be necessary. In contrast, facility-wide interventions, such as standardized anesthesia protocols, surgical time outs, and standardized order sets, facilitated spread to other surgical specialties not included in SCIP. In other words, spread within a specialty was driven by a combination of surgeon practice change and facility support, whereas spread between specialties was perceived to be driven almost entirely by surgery-wide policies and procedures applied to other services such as anesthesiology that may work across surgical specialties and apply lessons learned from one surgical setting to another.
We conducted post-hoc analysis comparing facility responses by bed size, (under 500 beds and over 500 beds), due to small group sample sizes <500 beds (n = 15), and >500 beds (n = 24) statistical analysis was not appropriate. However, while data suggests that respondents from larger facilities had more experience with SCIP, it appears both large and small facilities maintained SCIP about the same.
Limitations
This study has several limitations. First, this was a survey completed within the SHEA Research Network. Respondents in this network are more likely to be in academic medical centers with relatively strong support for stewardship activities. For the purposes of SCIP reporting, this has both benefits and downsides. SCIP was applied to major surgeries, including cardiac surgery, and these procedures necessitate that any hospital performing them have advanced services. This would tend to make the SHEA research network more representative of these facilities than would be expected if SCIP targeted more minor or less invasive surgeries that are performed in a wider range of healthcare settings. The major downside is that SCIP may have downstream positive impacts in the facilities where it was applied and the overall program impact might be lower in less complex facilities; prior work by Branch-Elliman and Mull suggest differential impacts in high versus low complexity facilities due to interfacility spread of practice improvements. Reference Branch-Elliman, Pizer and Dasinger14 Thus, reports about the policies and procedures that are available may not be generalizable to facilities without this same support and is out of the scope of this paper. This survey will inform a larger investigation into the sustainment and spread of SCIP practices also supplemented by qualitative interviews about SCIP practices at low complexity sites Reference Branch-Elliman, Elwy and Lamkin2
Second, one of the most reported strategies for supporting ongoing compliance was use of electronic order sets; these tools are not available in healthcare settings without EHRs. Hospitals without an EHR may rely on other, potentially limited strategies for ensuring compliance. Third, this is a survey of antimicrobial stewards and infection prevention experts. It is possible that surgeons, anesthesiologists, and other interested parties in surgical management may have different opinions about which practices are most important and effective for maintaining SCIP, as well as the importance of the SCIP program in general. In addition, perceptions of the respondents may not be reflective of actual practice and behaviors. Finally, our survey did not assess whether spread beyond the original SCIP-covered surgeries was universally an improvement; it is possible that uptake of policies and protocols that promoted peri-operative antimicrobial use may have led to inappropriate increases for surgeries for which prophylaxis is not indicated.
Conclusions
Experts in infection prevention and antimicrobial stewardship reported high rates of internal strategies to support ongoing compliance with SCIP INF-1 and 3 despite discontinuation of mandated public reporting. Internal policies and processes were perceived to be largely effective for maintaining compliance and supporting spread within and between surgical specialties. To help spread of SCIP best practices to surgeries not covered initially under the umbrella of the program, engagement of leadership and specialties such as anesthesia that work with different types of surgical specialists, may be necessary for facilitating practice improvement. Findings may inform additional analyses of long-term sustainability and spread of best practices and to create a list of strategies that can be used to advance peri-operative antimicrobial stewardship activities.
Acknowledgements
We would like to thank the SHEA research network for hosting our survey and completing the analysis. We would also like to thank the respondents who took the time to complete this survey and a special thank you to Nkechi Emetuche for her support administering the survey and providing feedback throughout the process.
Financial support
This study was supported by VA HSR&D IIR 20-101 (MPI, Mull and Branch-Elliman) and VA HSR Research Scientist Award, RCS 23-081 (co-I Dr. Elwy). This study was conducted with support from the Society for Healthcare Epidemiology in America Research Network.
Disclosures
All authors have nothing to disclose.
