Hostname: page-component-6898cfcb55-th4rj Total loading time: 0 Render date: 2026-11-12T03:22:44.382Z Has data issue: false hasContentIssue false
  • English
  • Français

Performance of surgical site infection risk prediction models in colorectal surgery: external validity assessment from three European national surveillance networks

Published online by Cambridge University Press:  20 June 2019

Rebecca Grant,
Martine Aupee,
Nicolas C. Buchs,
Kristine Cooper,
Marie-Christine Eisenring,
Theresa Lamagni,
Frédéric Ris,
Juliette Tanguy,
Nicolas Troillet and
Stephan Harbarth
...Show all authors
Rebecca Grant
Affiliation:
Infection Control Programme, University of Geneva Hospitals and Faculty of Medicine, Geneva, SwitzerlandCentre for Global Health, Institut Pasteur, Paris, France
Martine Aupee
Affiliation:
Coordination Center for Prevention and Control of Nosocomial Infections (CClin) Ouest, Rennes, France
Nicolas C. Buchs
Affiliation:
Service of Visceral Surgery, Department of Surgery, University of Geneva Hospitals and Faculty of Medicine, Geneva, Switzerland
Kristine Cooper
Affiliation:
National Infection Service, Public Health England, London, United Kingdom
Marie-Christine Eisenring
Affiliation:
Swissnoso, National Center for Infection Prevention, Bern, SwitzerlandService of Infectious Diseases, Central Institute of the Valais Hospital, Sion, Switzerland
Theresa Lamagni
Affiliation:
National Infection Service, Public Health England, London, United Kingdom
Frédéric Ris
Affiliation:
Service of Visceral Surgery, Department of Surgery, University of Geneva Hospitals and Faculty of Medicine, Geneva, Switzerland
Juliette Tanguy
Affiliation:
Coordination Center for Prevention and Control of Nosocomial Infections (CClin) Ouest, Rennes, France
Nicolas Troillet
Affiliation:
Swissnoso, National Center for Infection Prevention, Bern, SwitzerlandService of Infectious Diseases, Central Institute of the Valais Hospital, Sion, Switzerland
Stephan Harbarth
Affiliation:
Infection Control Programme, University of Geneva Hospitals and Faculty of Medicine, Geneva, SwitzerlandSwissnoso, National Center for Infection Prevention, Bern, Switzerland
Mohamed Abbas*
Affiliation:
Infection Control Programme, University of Geneva Hospitals and Faculty of Medicine, Geneva, Switzerland
*
Author for correspondence: Mohamed Abbas, Email:mohamed.abbas@hcuge.ch
Rights & Permissions [Opens in a new window]

Abstract

Objective:

To assess the validity of multivariable models for predicting risk of surgical site infection (SSI) after colorectal surgery based on routinely collected data in national surveillance networks.

Design:

Retrospective analysis performed on 3 validation cohorts.

Patients:

Colorectal surgery patients in Switzerland, France, and England, 2007–2017.

Methods:

We determined calibration and discrimination (ie, area under the curve, AUC) of the COLA (contamination class, obesity, laparoscopy, American Society of Anesthesiologists [ASA]) multivariable risk model and the National Healthcare Safety Network (NHSN) multivariable risk model in each cohort. A new score was constructed based on multivariable analysis of the Swiss cohort following colorectal surgery, then based on colon and rectal surgery separately.

Results:

We included 40,813 patients who had undergone elective or emergency colorectal surgery to validate the COLA score, 45,216 patients to validate the NHSN colon and rectal surgery risk models, and 46,320 patients in the construction of a new predictive model. The COLA score’s predictive ability was poor, with AUC values of 0.64 (95% confidence interval [CI], 0.63–0.65), 0.62 (95% CI, 0.58–0.67), 0.60 (95% CI, 0.58–0.61) in the Swiss, French, and English cohorts, respectively. The NHSN colon-specific model (AUC, 0.61; 95% CI, 0.61–0.62) and the rectal surgery–specific model (AUC, 0.57; 95% CI, 0.53–0.61) showed limited predictive ability. The new predictive score showed poor predictive accuracy for colorectal surgery overall (AUC, 0.65; 95% CI, 0.64–0.66), for colon surgery (AUC, 0.65; 95% CI, 0.65–0.66), and for rectal surgery (AUC, 0.63; 95% CI, 0.60–0.66).

Conclusion:

Models based on routinely collected data in SSI surveillance networks poorly predict individual risk of SSI following colorectal surgery. Further models that include other more predictive variables could be developed and validated.

Information

Type
Original Article
Information
Infection Control & Hospital Epidemiology , Volume 40 , Issue 9 , September 2019, pp. 983 - 990
Copyright
© 2019 by The Society for Healthcare Epidemiology of America. All rights reserved. 

Surgical site infection (SSI) is a leading cause of healthcare-associated infections.Reference Cassini, Plachouras and Eckmanns1 Among surgical procedures, colon and/or rectal surgery is associated with the highest incidence of SSI, despite widespread implementation of evidence-based practices to reduce them. The incidence of SSI is reported to be 14.4% in Switzerland,Reference Troillet, Aghayev, Eisenring and Widmer2 9.2% in England,3 5.9% in France,4 and 11.1% in the United States.Reference Morris, Graham, Chu, Cannon and Hawn5

Surveillance is essential for preventing SSI, and a core component of effective infection prevention and control programs.Reference Anderson, Podgorny and Berríos-Torres6–Reference Haley, Culver and White9 SSI surveillance networks track surgical outcomes and improve the quality of patient care through benchmarking and peer comparisons. Risk-adjustment and risk-prediction models based on data collected by SSI surveillance networks, such as the US National Nosocomial Infections Surveillance (NNIS) risk index, are useful to improve prediction of the risk of SSI for better infection control.Reference Culver, Horan and Gaynes10,Reference Gaynes, Culver, Horan, Edwards, Richards and Tolson11 However, this risk index score is not specific to any surgery and has been abandoned by the National Healthcare Safety Network (NHSN) in favor of procedure-specific multivariable models to determine a standardized infection ratio (SIR).Reference Mu, Edwards, Horan, Berrios-Torres and Fridkin12

In this study, we assessed the performance of different scoring systems applied to data from national SSI surveillance networks to predict the risk of SSI following colorectal surgery through the following objectives: (1) assessment of the external temporal and geographic validity of the COLA score (contamination class, obesity, laparoscopy, American Society of Anesthesiologists [ASA])Reference Gervaz, Bandiera-Clerc and Buchs13 in 3 validation cohorts from Switzerland, France, and England; (2) assessment of the temporal and geographic external validity of the NHSN risk models for colon surgery and rectal surgeryReference Mu, Edwards, Horan, Berrios-Torres and Fridkin12 in the Swiss cohort; (3) construction of a new predictive model using routinely collected surveillance data and multivariable regression analysis in the Swiss cohort.

Methods

We analyzed prospectively collected SSI surveillance data from 3 national networks: Swissnoso in Switzerland (134 hospitals from June 1, 2009, through October 31, 2017), ISO-Raisin (infection du site opératoire – Réseau Alerte Investigation Surveillance des Infections) in France (158 hospitals from January 1, 2014, through December 31, 2016), and Public Health England (PHE) Surgical Site Infection Surveillance Service (SSISS) in England (60 hospitals between January 1, 2007, and December 31, 2016).

The COLA score was originally developed using 534 patients across 24 hospitals in Switzerland who underwent colorectal surgery between October 2008 and November 2010.Reference Gervaz, Bandiera-Clerc and Buchs13 The score assigned 1 point for each risk factor present: contamination class contaminated or dirty, body mass index (BMI) ≥30 kg/m2, laparotomy (versus laparoscopy), and ASA class III–V. Possible COLA scores thus ranged from 0 to 4. Temporal validation was investigated in the Swissnoso cohort, whereas temporal and geographic validation was performed on the ISO-Raisin (France) and PHE SSISS (England) patient cohorts.

The NHSN risk models for colon and for rectal surgery were developed using data reported to the NHSN for operations performed between January 2006 and December 2008. The NHSN risk model for colon surgery includes age, ASA class, duration of surgery, surgical approach, contamination class, medical school affiliation of the healthcare facility, and bed capacity of the healthcare facility. The NHSN risk model for rectal surgery includes sex, duration of surgery, surgical approach, and contamination class.Reference Mu, Edwards, Horan, Berrios-Torres and Fridkin12 The external validation of the NHSN models for colon surgery and rectal surgery, and the construction of a new multivariable risk models for colorectal surgery, colon surgery, and rectal surgery were performed on the Swiss cohort only, due to availability of data.

From each cohort, any patient (aged ≥18 years) who had undergone elective or emergency colon or rectal surgery, and had no missing data in items of the COLA score or NHSN scoring system, was included. We excluded children because they differ from adult patients in terms of procedures, underlying conditions, and baseline SSI rate.Reference Feng, Sidhwa, Cameron, Glass and Rangel14 In all analyses, patients with missing outcome data were excluded.

The primary outcome was occurrence of SSI within 30 postoperative days, as defined by each network (Supplementary Appendix online).

Statistical analysis

We calculated the COLA score for each patient in all validation cohorts. The discriminative ability of the COLA score was assessed using ROC curves. The calibration of the COLA score was investigated by calculating the observed risk of SSI by level of the score and by the standardized infection ratio (SIR) comparing the observed risk of SSI and the risk predicted by the score. A SIR <1 means that the COLA score overestimated the risk of SSI. In addition, multivariable logistic regression models were computed in each validation cohort to assess the associations between SSI and the items of the COLA score. Adjusted odds ratios (aORs) are reported with their 95% confidence intervals.

To construct the new multivariable risk model, we performed univariable analysis to identify pre- and intraoperative characteristics routinely reported in the Swiss network (using the same database and timeframe as for validation of COLA) that were associated with SSI following colorectal surgery, then colon and rectal surgery separately. Multivariable models included variables with P < .20 in univariable analysis and final models for colorectal surgery, colon surgery, and rectal surgery, constructed using the backward, stepwise elimination method, including all variables with a P < .10. Variables used to construct the multivariable risk model were dichotomous (ie, sex, presence of colorectal cancer, multiple surgeries performed, elective or emergency surgery, presence of an implant, renal insufficiency as measured by creatinine clearance below 50 mL/min, surgical approach, administration of antibiotics, timing of antibiotic administration within 1 hour of incision, administration of a second dose of the antibiotic during the operation), categorical (ie, ASA class BMI, contamination class, NNIS risk index score), or continuous (ie, age, duration of procedure). Predictive score models for colorectal surgery, colon surgery, and rectal surgery were constructed based on the regression coefficients of each variable retained in the final multivariable model. These models were not validated.

Performance of each model was evaluated using a receiver operating characteristic (ROC) curve and by calculating the area under the ROC curve (AUC).

Subgroup analyses were performed restricting the outcome to deep or organ-space SSI only because these types of SSI are more clinically relevant and because they are more likely to be detected at readmission in settings where only passive postdischarge surveillance is performed. To assess the impact of missing data, we performed sensitivity analyses using all patients in each of the cohorts who had any of the 4 variables of the COLA score reported. Due to the large amount of missing data for some of the COLA variables, multiple imputation was not deemed adequate. For each patient, missing data were attributed 0 points in a ‘best case scenario’ and 1 point in a ‘worst case scenario.’

Two-tailed P values <.05 were considered statistically significant. All statistical analyses were performed using the epiDisplay: Epidemiological Data Display Package in R, version 3.2.2.0 software (https://CRAN.R-project.org/package=epiDisplay) and the pROC: Display and Analyze ROC Curves package (https://cran.r-project.org/web/packages/pROC/index.html).

Ethics statement

The data from each cohort were anonymized and deidentified prior to transfer for analysis. In accordance with Swiss law, approval from an institutional review board was not required. Data sharing agreements were signed between all study partner institutes to safeguard and maintain the confidentiality of the data, in accordance with the statutory and corporate requirements. We used secure data transfer methods via a server, which included storage in a data center that meets the requirements of European General Data Protection Regulation (GDPR), data encryption, and password-protected access to the server. The study is reported in accordance with the TRIPOD statement.Reference Moons, Altman and Reitsma15

Results

External validation of the COLA score

We included 40,813 patients in the external validation of the COLA score: 25,535 from the Swissnoso cohort, 13,171 from the PHE SSISS cohort, and 2,107 from the ISO-Raisin cohort. Due to missing data, we excluded 21,784 (46.0%) patients from the Swissnoso cohort, 20,949 (61.4%) from the PHE SSISS cohort, and 9,234 (81.4%) from the ISO-Raisin cohort. Of these records with missing data, 675 were missing outcome data and most (54.6%) had incomplete height and weight data, which are used to calculate the obesity parameter of the COLA score, and 675 records lacked follow-up.

The mean ages and proportions of female patients were similar across validation cohorts. Table 1 summarizes the demographic and clinical characteristics of patients in each cohort. The incidence rates of SSI across the study period for each cohort were 14.8% for the Swissnoso cohort, 10.2% for the PHE SSISS cohort, and 6.2% for the ISO-Raisin cohort. Figure 1 indicates the time trend in annual SSI incidence across the study period for each cohort.

Fig. 1.

Annual surgical site infection incidence in each national colorectal surgery surveillance network.

Table 1.

Comparison of Characteristics of Patients in Original Derivation and Cross-Validation Cohort and 3 External Validation Cohortsa

Note. PHE-SSISS, Public Health England Surgical Site Infection Surveillance Service; SD, standard deviation; ASA, American Society of Anesthesiologists; SSI, surgical site infection.

a Numbers are n and (%), except for age.

b Across the 3 networks.

Multivariable logistic regression

Table 2 shows the adjusted odds ratios (aOR) and the 95% confidence intervals (CI) for each of the 4 variables of the COLA score in each of the validation cohorts compared to the original derivation cohort.13 All 4 variables of the COLA score were independent risk factors for SSI in each of the validation cohorts, except for contamination class and ASA class in the ISO-Raisin cohort, which did not reach statistical significance.

Table 2.

Multivariable Logistic Regression Model for Prediction of Risk of Surgical Site Infection (SSI) of COLA Variables in Original Derivation Cohort and 3 External Validation Cohorts

Note. COLA, contamination class, obesity, laparoscopy; PHE-SSISS, Public Health England Surgical Site Infection Surveillance Service; ASA, American Society of Anesthesiologists; CI, confidence interval; OR, odds ratio.

a Italic P value indicates statistical significance.

Calibration and discrimination of COLA score

Table 3 indicates the distribution of patients with and without SSI by level of COLA score for each cohort. It shows poor agreement between the numbers of predicted and observed SSIs. That is, the COLA score overestimated the number of SSI for each level, and in each of the validation cohorts. Supplementary Table 1 (online) also shows the ORs for each level of the COLA score, from 0 to 4. Each 1-point increase in the score was associated with a statistically significant increase in the risk of SSI. Supplementary Figure 1 shows the ROC curve for each of the validation cohorts. The AUC was 0.64 (95% CI, 0.63–0.65) for the Swissnoso cohort, 0.62 (95% CI, 0.58–0.67) for the ISO-Raisin cohort, and 0.60 (95% CI, 0.58–0.61) for the PHE SSISS cohort.

Table 3.

Calibration Table of the COLA Score Showing Observed and Predicted Risk of Surgical Site Infection (SSI) for Each of the Validation Cohorts

Note. COLA, contamination class, obesity, laparoscopy; PHE-SSISS, Public Health England Surgical Site Infection Surveillance Service; AUC, area under the curve; CI, confidence interval; SIR, standardized infection ratio.

Sensitivity analyses

Restricting analysis to consider only deep or organ-space SSI did not improve the predictive ability of the COLA score, with an AUC of 0.57 (95% CI, 0.53–0.59) for the Swissnoso cohort, 0.54 (95% CI, 0.43–0.66) for the ISO-Raisin cohort, and 0.55 (95% CI, 0.52–0.60) for the PHE SSISS cohort (Supplementary Table 2 online).

Missing data

Sensitivity analyses performed to assess the impact of the missing data did not improve the predictive ability of the COLA score (Supplementary Tables 3 and 4 online).

Table 4.

NHSN multivariable logistic regression model for predicting risk of SSI after colon surgery in NHSN derivation cohort (N = 62,777) and Swissnoso validation cohort (N = 42,551)

Note. ASA, American Society of Anesthesiologists; CI, confidence interval; OR, odds ratio.

a Italic P value indicates statistical significance.

External validation of the NHSN risk models

We included 42,511 patients from the Swissnoso cohort who underwent colon surgery in the external validation of the NHSN colon-specific risk models for (1) all SSIs and (2) deep or organ-space SSIs only. Table 4 shows the aORs for each of the components of the NHSN risk model for all SSIs (data for deep or organ-space SSIs not shown) for both the derived cohort and the validation cohort. All variables were found to be independent risk factors for SSI, and deep or organ/space SSI specifically, except for hospital size (number of beds) and patient age, which showed an inverse relationship with SSI risk. Absence of medical school affiliation was an independent risk factor for both NHSN colon-specific risk models in the derivation cohort, whereas in the validation cohort (Swissnoso), absence of medical school affiliation was significantly protective.

Supplementary Figure 2 (online) shows the ROC curves for the NHSN colon-specific risk model in the validation cohort. The AUCs were 0.61 (95% CI, 0.58–0.61) for all SSIs and 0.62 (95% CI, 0.61–0.63) for deep or organ-space SSIs (data not shown).

We included 2,665 patients who underwent rectal surgery in the external validation of the NHSN rectal-specific risk models. Table 5 shows the aORs for each of the components of the NHSN rectal-specific risk models for all SSI and deep or organ-space SSIs only. All variables were independent risk factors for SSI and deep or organ-space SSI, except for hospital size in the deep or organ-space SSI risk model. In the derived NHSN rectal-specific risk model for all SSIs, laparoscopy was an independent risk factor for SSI, whereas in the validation cohort (Swissnoso), laparoscopy was significantly protective.

Table 5.

NHSN Multivariable Logistic Regression Models for Predicting Risk of SSI After Rectal Surgery in NHSN derivation cohort (N = 1215) and Swissnoso validation cohorts (N = 2665) – Model A: all SSI, Model B: deep or organ-space SSI only

CI, confidence interval; OR, odds ratio.

Supplementary Figure 3 (online) shows the ROC curves for each NHSN rectal-specific risk model in the validation cohort. The AUCs were 0.57 (95% CI, 0.53–0.61) for all SSIs and 0.63 (95% CI, 0.59–0.67) for deep or organ-space SSIs.

Construction of a new multivariable risk model

We included 46,320 patients from the Swissnoso cohort who underwent elective or emergency colon or rectal surgery in the construction of a new multivariable risk model. Overall, 49% of patients were male, and the mean age was 66.6 years (SD, 14.1 years).

Univariable analysis

Based on the univariable analysis, the following variables were associated with higher incidence of SSI following colorectal surgery: age (P < .001), male sex (P < .001), BMI (P < .001), ASA class III–V (P < .001), multiple surgical procedures being conducted (P < .001), emergency surgery (P < .001), creatinine clearance <50 mL/min (P = 0.006), laparotomy (P < .001), contamination class III–IV (P < .001), NNIS risk index (P < .001), duration of procedure (P < .001), number of antibiotics administered (P < .001), timing of antibiotic administration >1 hour before incision or after incision (P < .001), and a second dose of the first antibiotic during the operation (P = .01).

Multivariable logistic regression

Table 6A shows the aORs (and 95% CI) of each component of the final multivariable risk model for colorectal surgery. Age, male sex, ASA class III–IV, emergency surgery, laparotomy, contamination class III–IV and duration of procedure were all independent risk factors of SSI.

Table 6.

New Multivariable Logistic Regression Model for Predicting Risk of Surgical Site Infection (SSI) Using the Swissnoso Cohort for Colon or Rectal Surgery, Colon Surgery Only, and Rectal Surgery Only

Note. CI, confidence interval; OR, odds ratio.

a Italic P value indicates statistical significance.

A second multivariable model was constructed for colon surgery only (Table 6B). The final multivariable risk model for colon surgery retained the same variables as the multivariable model for colorectal surgery: age, male sex, ASA class III–IV, emergency surgery, laparotomy, contamination class III–IV and duration of procedure.

A third multivariable model was constructed for rectal surgery only (Table 6C). Laparotomy, duration of surgery, and no antibiotic administration were independent risk factors of SSI.

Supplementary Figure 4 (online) shows the ROC curves for each model: (a) colorectal surgery (AUC, 0.65; 95% CI, 0.64–0.66), (b) colon surgery only (AUC, 0.65; 95% CI, 0.64–0.65), and (c) rectal surgery only (AUC, 0.63; 95% CI, 0.60–0.66).

Discussion

We compared different multivariable risk models developed using national SSI surveillance networks and found that the ability of these models remains limited to predict individual SSI risk, particularly when tested for validity in patients who are distinct from the cohort of patients in which the multivariable risk models were constructed.

The external validation of the COLA score confirmed that although all 4 variables remain independent risk factors of SSI, a combined score does not accurately identify patients at increased risk of SSI following colorectal surgery. In the original derivation study, which included internal validation, the COLA score showed moderate discriminative ability, as indicated by the AUC of 0.729, which after accounting for overfitting using a cross-validation procedure, was 0.703. In our study, however, there was a further loss in the calibration and discriminative ability of the COLA score in each of the 3 validation cohorts. The score performed best in temporal validation (AUC, 0.65). Geographic validation showed that its performance was lower compared to both the original derivation and temporal validation cohorts. The results of this study are consistent with, to the best of our knowledge, the only other study having validated the COLA score, a US-based single-center study (AUC, 0.61).Reference Bergquist, Thiels, Etzioni, Habermann and Cima16

The NHSN risk models included variables such as university affiliation and bed size of the hospital; these may be considered a proxy for skill and experience of the surgical teams or more complicated cases referred to tertiary-care centers. It is puzzling that medical school affiliation had differing effects on risk of SSI in the United States and in Switzerland; this may be due to different healthcare systems. In Switzerland, most healthcare is delivered by public hospitals, and teaching hospitals are larger and considered to be referral excellence centers. Despite identifying independent risk factors for SSI, both NHSN risk models lacked accuracy in predicting the risk of SSI following colon and rectal surgery, respectively. For colon surgery, this result was consistent with the predictive ability (AUC, 0.59) detailed in the original derivation study by Mu et al.Reference Mu, Edwards, Horan, Berrios-Torres and Fridkin12 For rectal surgery, the predictive ability dropped substantially compared to the original derivation study.Reference Mu, Edwards, Horan, Berrios-Torres and Fridkin12

Finally, we constructed a new model for predicting SSI after colorectal surgery as well as for colon surgery and rectal surgery specifically. However, the predictive ability of these new models was not sufficiently high to be useful for identifying patients at increased risk of SSI in a clinical setting.

Although annual incidence of SSI following colorectal surgery has slowly decreased over time, further reducing SSI could be achieved by identifying individuals at increased risk of SSI prior to, or shortly after, the surgical procedure. Predictive models based on independent risk factors of SSI can quantify this risk and could be used to optimize preventive measures. The main aims of patient data routinely collected in national SSI surveillance networks is to monitor and improve patient outcomes. These surveillance networks, however, may also provide the opportunity to validate and construct SSI risk prediction models to further these goals.

Other variables, possibly those with greater predictive potential, could be considered for inclusion in future models for predicting excess risk of SSI following colorectal surgery. Other factors that may also be investigated for their predictive potential include diabetes mellitus, tobacco use, perioperative blood transfusion, preoperative bowel preparation (including digestive decontamination with oral antibiotics), preoperative albumin levels, dose and timing of prophylactic antibiotics, carriage of multidrug-resistant microorganisms, and experience of the colorectal surgeon.Reference Bergquist, Thiels, Etzioni, Habermann and Cima16–Reference Nthumba, Stepita-Poenaru and Poenaru18 Despite inclusion of these predictive factors, some uncertainty will persist because of the heterogeneity of human factors (eg, surgical skill), adherence to prevention guidelines, use of specific techniques or devices, among others.

The results of our study are strengthened by the size and multicenter nature of each of the cohorts, as well as the fact that the external validation of the COLA score was performed in cohorts that were both temporally and geographically distinct from the original derivation cohort. Possible limitations of the results are the level of missing data in the SSI surveillance databases and the likelihood of nonuniformity in reporting across healthcare systems, particularly in the postoperative surveillance methods for detecting SSIs. Indeed, in England, only detection of SSI upon readmission is mandatoryReference Meijerink, Lamagni, Eriksen, Elgohari, Harrington and Kacelnik19; in France, postdischarge surveillance is mandatory but incompletely performed20; in Switzerland, postdischarge surveillance is mandatory and performed in >90% of cases.Reference Troillet, Aghayev, Eisenring and Widmer2 Auditing practices between the networks certainly differ. For example, in Switzerland, there is a time- and resource-intensive quality assurance and auditing program in place that includes on-site visits and random chart reviews.Reference Kuster, Eisenring, Sax and Troillet21 This has shown that the surveillance system has positive and negative predictive values of 97% and 99%, respectively. This factor may also contribute to higher SSI incidence in Switzerland. In England, a percentile analysis is regularly performed to identify outliers in terms of SSI rates (lowest and highest 10%), as well as a review of surveillance methodology generally (eg, through case-note review).Reference Meijerink, Lamagni, Eriksen, Elgohari, Harrington and Kacelnik19

In conclusion, our study shows that current risk models using routinely collected data from SSI surveillance databases are insufficient for developing an accurate predictive model for SSI following colorectal surgery. Further models for predicting excess risk of SSI following colorectal surgery that include other, possibly more predictive, variables could be developed and validated.

Supplementary material

To view supplementary material for this article, please visit https://doi.org/10.1017/ice.2019.163

Author ORCIDs

Mohamed Abbas, https://orcid.org/0000-0002-7265-1887

Acknowledgments

We thank Christophe Combescure, PhD, and Professor Thomas Perneger (Clinical Trials Centre and Division of Clinical Epidemiology, Department of Health and Community Medicine, University of Geneva) for their assistance in methodological and statistical aspects of this study. We also thank the numerous, dedicated, professionals involved in the collection of SSI surveillance data in Switzerland, France, and England.

Financial support

No financial support was provided relevant to this article.

Conflicts of interest

All authors report no conflicts of interest relevant to this article.

Footnotes

PREVIOUS PRESENTATION: Preliminary results of this study were presented (oral presentation no. O1177) at the 29th European Conference on Clinical Microbiology and Infectious Diseases on April 16, 2019, in Amsterdam, The Netherlands.

References

Cassini, A, Plachouras, D, Eckmanns, T, et al. Burden of six healthcare-associated infections on European population health: estimating incidence-based disability-adjusted life years through a population prevalence-based modelling study. PLoS Med 2016;13:e1002150.CrossRefGoogle ScholarPubMed
Troillet, N, Aghayev, E, Eisenring, MC, Widmer, A Swissnoso, First results of the Swiss National Surgical Site Infection Surveillance Program: who seeks shall find. Infect Control Hosp Epidemiol 2017:697–704.CrossRefGoogle Scholar
Public Health England. Surveillance of surgical site infections in NHS hospitals in England, 2016 to 2017. Public Health England, December 2017. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/666465/SSI_annual_report_NHS_hospitals_2016-17.pdf Accessed June 13 2019.Google Scholar
Le Réseau d’alerte d’investigation et de surveillance des infections nosocomiales (Raisin). [Surgical site infection Surveillance, France 2016]. Santé publique France, 2018.http://www.cpias-ile-de-france.fr/surveillance/iso/RappRAISIN2016.pdf Accessed June 13 2019.Google Scholar
Morris, MS, Graham, LA, Chu, DI, Cannon, JA, Hawn, MT., Oral antibiotic bowel preparation significantly reduces surgical site infection rates and readmission rates in elective colorectal surgery. Ann Surg 2015;261:1034–1040.CrossRefGoogle ScholarPubMed
Anderson, DJ, Podgorny, K, Berríos-Torres, SI, et al. Strategies to prevent surgical site infections in acute care hospitals: 2014 update. Infect Control Hosp Epidemiol 2014;35:605–627.CrossRefGoogle ScholarPubMed
Zingg, W, Holmes, A, Dettenkofer, M, et al. Hospital organisation, management, and structure for prevention of health-care-associated infection: a systematic review and expert consensus. Lancet Infect Dis 2015;15:212–224.CrossRefGoogle ScholarPubMed
Storr, J, Twyman, A, Zingg, W, et al. Core components for effective infection prevention and control programmes: new WHO evidence-based recommendations. Antimicrob Resist Infect Control 2017;6:6.CrossRefGoogle ScholarPubMed
Haley, RW, Culver, DH, White, JW, et al. The efficacy of infection surveillance and control programs in preventing nosocomial infections in US hospitals. Am J Epidemiol 1985;121:182–205.CrossRefGoogle ScholarPubMed
Culver, DH, Horan, TC, Gaynes, RP, et al. Surgical wound infection rates by wound class, operative procedure, and patient risk index. National Nosocomial Infections Surveillance System. Am J Med 1991;91(3B):152S–157S.CrossRefGoogle ScholarPubMed
Gaynes, RP, Culver, DH, Horan, TC, Edwards, JR, Richards, C, Tolson, JS., Surgical site infection (SSI) rates in the United States, 1992–1998: the National Nosocomial Infections Surveillance System basic SSI risk index. Clin Infect Dis 2001;33(suppl 2):S69–S77.CrossRefGoogle ScholarPubMed
Mu, Y, Edwards, JR, Horan, TC, Berrios-Torres, SI, Fridkin, SK., Improving risk-adjusted measures of surgical site infection for the national healthcare safety network. Infect Control Hosp Epidemiol 2011;32:970–986.CrossRefGoogle ScholarPubMed
Gervaz, P, Bandiera-Clerc, C, Buchs, NC, et al. Scoring system to predict the risk of surgical-site infection after colorectal resection. Br J Surg 2012;99:589–595.CrossRefGoogle ScholarPubMed
Feng, C, Sidhwa, F, Cameron, DB, Glass, C, Rangel, SJ., Rates and burden of surgical site infections associated with pediatric colorectal surgery: insight from the National Surgery Quality Improvement Program. J Pediatr Surg 2016;51:970–974.Google ScholarPubMed
Moons, KG, Altman, DG, Reitsma, JB, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 2015;162:W1–W73.CrossRefGoogle ScholarPubMed
Bergquist, JR, Thiels, CA, Etzioni, DA, Habermann, EB, Cima, RR., Failure of colorectal surgical site infection predictive models applied to an independent dataset: do they add value or just confusion? J Am Coll Surg 2016;222:431–438.CrossRefGoogle ScholarPubMed
Dubinsky-Pertzov, B, Temkin, E, Harbarth, S, et al. Carriage of extended-spectrum beta-lactamase-producing Enterobacteriaceae and the risk of surgical site infection after colorectal surgery: a prospective cohort study. Clin Infect Dis 2018;68:1699–1704. doi: 10.1093/cid/ciy768.CrossRefGoogle Scholar
Nthumba, PM, Stepita-Poenaru, E, Poenaru, D, et al. Cluster-randomized, crossover trial of the efficacy of plain soap and water versus alcohol-based rub for surgical hand preparation in a rural hospital in Kenya. Br J Surg 2010;97:1621–1628.CrossRefGoogle Scholar
Meijerink, H, Lamagni, T, Eriksen, HM, Elgohari, S, Harrington, P, Kacelnik, O., Is it valid to compare surgical site infections rates between countries? Insights from a study of English and Norwegian surveillance systems. Infect Control Hosp Epidemiol 2017;38:162–171.CrossRefGoogle ScholarPubMed
Santé Publique France. Surveillance des infections du site opératoire dans les établissements de santé. Réseau ISORaisin, France. Résultats 2017. February 2019.Google Scholar
Kuster, SP, Eisenring, MC, Sax, H, Troillet, N, Swissnoso. Structure, process, and outcome quality of surgical site infection surveillance in Switzerland. Infect Control Hosp Epidemiol 2017;38:1172–1181.CrossRefGoogle ScholarPubMed
Figure 0

Fig. 1.Annual surgical site infection incidence in each national colorectal surgery surveillance network.

Figure 1

Table 1.Comparison of Characteristics of Patients in Original Derivation and Cross-Validation Cohort and 3 External Validation Cohortsa

Figure 2

Table 2.Multivariable Logistic Regression Model for Prediction of Risk of Surgical Site Infection (SSI) of COLA Variables in Original Derivation Cohort and 3 External Validation Cohorts

Figure 3

Table 3.Calibration Table of the COLA Score Showing Observed and Predicted Risk of Surgical Site Infection (SSI) for Each of the Validation Cohorts

Figure 4

Table 4.NHSN multivariable logistic regression model for predicting risk of SSI after colon surgery in NHSN derivation cohort (N = 62,777) and Swissnoso validation cohort (N = 42,551)

Figure 5

Table 5.NHSN Multivariable Logistic Regression Models for Predicting Risk of SSI After Rectal Surgery in NHSN derivation cohort (N = 1215) and Swissnoso validation cohorts (N = 2665) – Model A: all SSI, Model B: deep or organ-space SSI only

Figure 6

Table 6.New Multivariable Logistic Regression Model for Predicting Risk of Surgical Site Infection (SSI) Using the Swissnoso Cohort for Colon or Rectal Surgery, Colon Surgery Only, and Rectal Surgery Only

Supplementary material: File

Grant et al. supplementary material

Appendix

Download Grant et al. supplementary material(File)
File 214.9 KB