Hostname: page-component-6898cfcb55-kwdml Total loading time: 0 Render date: 2026-11-12T03:48:32.595Z Has data issue: false hasContentIssue false
  • English
  • Français

Changes in the accuracy of administrative data for the detection of surgical site infections

Published online by Cambridge University Press:  17 December 2020

Brian T. BucherOpen the ORCID record for Brian T. Bucher[Opens in a new window],
Meng Yang,
Julie Arndorfer,
Cherie Frame,
Jan Orton,
Matthew H. Samore and
Kristin K. Dascomb
Brian T. Bucher*
Affiliation:
Department of Surgery, University of Utah Schools of Medicine, Salt Lake City, Utah
Meng Yang
Affiliation:
Department of Surgery, University of Utah Schools of Medicine, Salt Lake City, Utah
Julie Arndorfer
Affiliation:
Division of Infectious Diseases, Office of Patient Experience, Intermountain Healthcare, Salt Lake City, Utah
Cherie Frame
Affiliation:
Division of Infectious Diseases, Office of Patient Experience, Intermountain Healthcare, Salt Lake City, Utah
Jan Orton
Affiliation:
Clinical Data Management, Intermountain Healthcare, Salt Lake City, Utah
Matthew H. Samore
Affiliation:
Department of Medicine, University of Utah Schools of Medicine, Salt Lake City, UtahVA Salt Lake City Healthcare System, IDEAS Center 2.0, Salt Lake City, Utah
Kristin K. Dascomb
Affiliation:
Division of Infectious Diseases, Office of Patient Experience, Intermountain Healthcare, Salt Lake City, Utah
*
Author for correspondence: Brian T. Bucher, E-mail: brian.bucher@utah.edu
Rights & Permissions [Opens in a new window]

Abstract

We performed a retrospective analysis of the changes in accuracy of International Classification of Diseases, Clinical Modification (ICD-CM) diagnosis codes for colectomy and hysterectomy surgical site infection surveillance. After the transition from ICD-CM ninth edition to tenth edition codes, there was no significant change in the accuracy of these codes for SSI surveillance.

Information

Type
Concise Communication
Information
Infection Control & Hospital Epidemiology , Volume 42 , Issue 9 , September 2021, pp. 1128 - 1130
Check for updates
Copyright
© The Author(s), 2020. Published by Cambridge University Press on behalf of The Society for Healthcare Epidemiology of America

Surgical site infections (SSIs) continue to remain a significant cause of patient morbidity and increased healthcare costs after operative procedures. Reference Weber, Zwahlen and Reck1,Reference Kanerva, Ollgren, Virtanen and Lyytikäinen2 To assist in SSI prevention several surveillance systems, including the National Healthcare Safety Network (NHSN), were developed to assist hospitals in the identification and reporting of these complications. Reference West and Eng3 However, these surveillance systems rely on labor-intensive manual chart review, which limits the generalizability and feasibility of this approach.

The use of administrative billing codes such as the International Classification of Diseases Clinical Modification (ICD-CM) diagnosis codes has been proposed as a tool to help assist in the surveillance of surgical site infections. Reference van Mourik, van Duijn, Moons, Bonten and Lee4 Several studies have demonstrated a wide range for sensitivity (10%–100%) and positive predictive value (PPV) (11%–95%) of these codes, depending upon the procedures evaluated. Reference van Mourik, van Duijn, Moons, Bonten and Lee4 Given the wide variability in the performance of ICD-CM codes between hospitals, their use for surveillance of SSIs has been limited. However, on September 30, 2015, the Centers for Medicare and Medicaid Services (CMS) transitioned from the ICD-CM ninth edition (IDC-9-CM) to the ICD-CM tenth edition (ICD-10-CM) codes. Reference Khera, Dorsey and Krumholz5 This change resulted in an increase from 13,000 ICD-9 codes to 68,000 ICD-10 codes. Reference Khera, Dorsey and Krumholz5 Several studies have examined the impact of this vocabulary expansion on several condition-specific diagnoses in administrative discharge data; however the effect on the accuracy of ICD-CM codes for the detection of SSIs is unknown. Reference Mainor, Morden, Smith, Tomlin and Skinner6 Therefore, the goal of the present study is to evaluate the change in accuracy of the ICD-10 codes compared to the ICD-9 codes for the detection of SSIs in a single healthcare system.

Methods

The data in the present study were obtained from Intermountain Healthcare, a not-for-profit healthcare system located in the intermountain west. Patients were included in the present study if they underwent a NHSN-defined colectomy or abdominal hysterectomy procedures between January 1, 2012, and April 1, 2019 based on CMS-defined denominator procedure codes. Reference West and Eng3 The reference standard was an NHSN-defined SSI, including superficial, deep, and organ-space infections, as identified by infection preventionists as part of mandatory hospital SSI reporting practices. Reference Berrios-Torres, Umscheid and Bratzler7 For patients included in the study, we obtained all ICD diagnosis codes within 30 days of the operative procedure. We included diagnosis codes from both the index admission and any readmission that occurred within 30 days of the operative procedure. If a patient underwent a surgical procedure on or before September 30, 2015, ICD-9 diagnosis codes were utilized, and if the procedure occurred on or after October 1, 2015, ICD-10 codes were utilized. ICD codes were grouped into a narrow-range SSI category (using only the ICD code for postoperative infection) and an expanded-range SSI category as previously described by the Pennsylvania Health Care Cost Containment Council (PHC4) and defined in the Supplementary Appendix (online). Reference Julian, Brumbach and Chicora8

The performance of the narrow-range and expanded-range ICD-9 and ICD-10 codes were compared to the reference standard and evaluated for sensitivity, specificity, positive predictive value, negative predictive value. We utilized bootstrapping with 2,000 iterations to generate confidence intervals for each point estimate. The analysis was performed using the R version 4.0 of the statistical software package (R Foundation for Statistical Computing, Vienna, Austria).

Results

In total, 16,714 procedures were included in the present study: 10,315 (62%) procedures utilizing ICD-9 codes and 6,399 (38%) procedures utilizing ICD-10 codes. Colon procedures comprised 52% of the cohort and hysterectomy procedures accounted for the remaining 48%. The prevalence of SSI was significantly higher in the ICD-9 cohort than the ICD-10 cohort (4.7% vs 3.6%; OR, 0.75; 95% CI, 0.64–0.88; P < .001) and in the colectomy procedures compared to the hysterectomy procedures (6.2% vs 2.1%; OR, 2.95; 95% CI, 2.48–3.51; P < .001).

The classification performance for the ICD-9 and ICD-10 codes is shown in Table 1. Overall there was a small decrease in the sensitivity of ICD-10 codes compared to ICD-9 codes using either a narrow code range (difference, −0.03; 95% CI, −0.12 to 0.04) or an expanded code range (difference, −0.08; 95% CI, −0.16 to 0.00). In addition, there were small increases in specificity of ICD-10 codes compared to ICD-9 codes using either a narrow code range (difference, 0.007; 95% CI, 0.002–0.01) or an expanded code range (difference, 0.005; 95% CI, −0.002 to 0.01).

Table 1.

Classification Performance of International Classification of Diseases Clinical Modification 9 th and 10 th Edition Codes for Surveillance of Surgical Site Infections

Note. CI, confidence interval; ICD, International Classification of Diseases.

On subgroup analysis, for colectomy procedures using the ICD-10 expanded code range, we observed a small decrease in the sensitivity for SSI detection compared to the ICD-9 expanded code range (difference, −0.11; 95% CI, −0.2 to −0.02). For hysterectomy procedures, we observed a small increase sensitivity for SSI detection with ICD-10 codes compared to ICD-9 codes using an expanded code range (difference, 0.07; 95% CI, −0.11 to 0.26).

The predictive value of information for the ICD codes is shown in Table 2. We detected small decreases in the PPV between the narrow-range codes (difference, −0.01; 95% CI, −0.09 to 0.08) and the expanded-range codes (difference, −0.07; 95% CI, −0.12 to −0.02).

Table 2.

Positive and Negative Predictive Values of International Classification of Diseases Clinical Modification 9th and 10th Edition Codes for Surveillance of Surgical Site Infections

Note. PPV, positive predictive value; NPV, negative predictive value; CI, confidence interval; ICD, International Classification of Diseases.

Discussion

In our study of colectomy and hysterectomy procedures in a single healthcare system, we identified several small but clinically questionable changes in the performance of ICD-10-CM codes compared to ICD-9-CM codes. There were small decreases in sensitivity for SSI after the ICD-CM transition and there was a small decrease in the PPV for expanded ICD-CM codes.

Several previous studies have defined the performance of ICD-9-CM and ICD-10-CM codes or the surveillance of SSIs. Stevenson utilized the PHC4 expanded ICD-9-CM codes and identified a sensitivity of 60% for SSI surveillance. Reference Stevenson, Khan and Dickman9 Other groups outside the United States have utilized ICD-10-CM codes for SSI Surveillance. Kanerva et al Reference Kanerva, Ollgren, Virtanen and Lyytikäinen2 utilized data from Finnish acute-care hospitals and demonstrated a sensitivity of 45% for expanded ICD-10-CM codes. Our work builds upon previous work by identifying the performance of ICD-9-CM and ICD-10-CM codes in a single healthcare system using active surveillance of colectomy and abdominal hysterectomy procedures. Like others, we found limited performance for SSI surveillance using ICD-CM codes. The poor performance did not improve after the transition to the ICD-10-CM edition. Interestingly, the direction of change was different for the procedures studies, in that sensitivity increase with hysterectomy procedure and decreased with colectomy procedures. These data suggest that the impact of ICD vocabulary expansion on SSI surveillance is not uniform across procedures and has implications for ICD-based surveillance of other procedures, including cardiovascular, neurosurgical, or orthopedic procedures.

Our study has several limitations. First, we limited our study to mandatory NHSN-reported colon and hysterectomy procedures and did not include other surgical procedures such as orthopedic or neurosurgical procedures for which the SSI risk extends beyond 30 days. Second, NHSN reporting may have missed SSIs that the ICD codes identified; thus, some false positive cases may have been true-positive cases. Lastly, the decrease in the prevalence of SSIs during the study period may partially explain the decrease in PPV of both the narrow-range and expanded-range code sets.

In conclusion, we observed no significant changes in the performance of ICD-CM codes for the detection of SSI in colectomy and abdominal hysterectomy procedures after transitioning from the ICD-9-CM to the ICD-10-CM codes. ICD-CM codes for SSI detection continue to have low sensitivity and PPV. We cannot recommend the utilization of ICD-10-CM as a screening tool for SSI surveillance. Predictive models utilizing ICD-CM codes in combination with clinical data are needed to assist in SSI surveillance.

Supplementary material

To view supplementary material for this article, please visit https://doi.org/10.1017/ice.2020.1346

Acknowledgments

The Agency for Healthcare Research and Quality (AHRQ) had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication. The views expressed are those of the authors and do not necessarily represent the views or opinions of the US government or the US Department of Veterans’ Affairs.

Financial support

B.T.B. received grant support from the AHRQ (grant no. 1K08HS025776).

Conflicts of interest

All authors report no conflicts of interest relevant to this article.

References

Weber, WP, Zwahlen, M, Reck, S, et al. Economic burden of surgical site infections at a European university hospital. Infect Control Hosp Epidemiol 2008;29:623–629.CrossRefGoogle Scholar
Kanerva, M, Ollgren, J, Virtanen, MJ, Lyytikäinen, O. Estimating the annual burden of healthcare-associated infections in Finnish adult acute-care hospitals. Am J Infect Control 2009;37:227–230.CrossRefGoogle ScholarPubMed
West, N, Eng, T. Monitoring and reporting hospital-acquired conditions: a federalist approach. Medicare Medicaid Res Rev 2014;4:E1–E16.CrossRefGoogle ScholarPubMed
van Mourik, MSM, van Duijn, PJ, Moons, KGM, Bonten, MJM, Lee, GM. Accuracy of administrative data for surveillance of healthcare-associated infections: a systematic review. BMJ Open 2015;5:e008424.CrossRefGoogle ScholarPubMed
Khera, R, Dorsey, KB, Krumholz, HM. Transition to the ICD-10 in the United States: an emerging data chasm. JAMA 2018;320:133–134.CrossRefGoogle Scholar
Mainor, AJ, Morden, NE, Smith, J, Tomlin, S, Skinner, J. ICD-10 coding will challenge researchers: caution and collaboration may reduce measurement error and improve comparability over time. Med Care 2019;57:e42–e46.CrossRefGoogle ScholarPubMed
Berrios-Torres, SI, Umscheid, CA, Bratzler, DW, et al. Centers for Disease Control and Prevention guideline for the prevention of surgical site infection, 2017. JAMA Surg 2017;152:784–791.CrossRefGoogle Scholar
Julian, KG, Brumbach, AM, Chicora, MK, et al. First year of mandatory reporting of healthcare-associated infections, Pennsylvania: an infection control-chart abstractor collaboration. Infect Control Hosp Epidemiol 2006;27:926–930.CrossRefGoogle ScholarPubMed
Stevenson, KB, Khan, Y, Dickman, J, et al. Administrative coding data, compared with CDC/NHSN criteria, are poor indicators of healthcare-associated infections. Am J Infect Control 2008;36:155–164.CrossRefGoogle Scholar
Figure 0

Table 1.Classification Performance of International Classification of Diseases Clinical Modification 9th and 10thEdition Codes for Surveillance of Surgical Site Infections

Figure 1

Table 2.Positive and Negative Predictive Values of International Classification of Diseases Clinical Modification 9th and 10th Edition Codes for Surveillance of Surgical Site Infections

Supplementary material: File

Bucher et al. supplementary material

Table S1

Download Bucher et al. supplementary material(File)
File 12.5 KB