Health claims data are useful for retrospective surveillance of postoperative complications and for tracking complications across institutions and the spectrum of care. The accuracy of identification depends on the validity of the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes chosen for surveillance. Results have been mixed regarding the accuracy of ICD-9-CM diagnosis codes for surgical site infection (SSI) after various procedures. Few studies have validated ICD-9-CM diagnosis codes for SSI after breast procedures,Reference Yokoe, Noskin and Cunningham 1 – Reference Miner, Sands and Yokoe 3 and to our knowledge, no group has validated the coding of noninfectious wound complications (eg, hematoma, fat necrosis) after mastectomy.
We previously reported on the incidence of SSIReference Olsen, Nickel and Fox 4 and noninfectious wound complicationsReference Nickel, Fox, Margenthaler, Wallace, Fraser and Olsen 5 after mastectomy using data from private insurer claims. We used this population to determine the positive predictive value (PPV) of ICD-9-CM diagnosis codes to identify infectious and noninfectious wound complications and ICD-9-CM procedure and CPT-4 codes to identify breast surgical procedures in claims data compared to medical record review.
METHODS
From the HealthCore Integrated Research Database, a private insurer claims database, we established a retrospective cohort of women 18–64 years of age that had an ICD-9-CM procedure code or a CPT-4 code for mastectomy between January 1, 2004, and August 31, 2009, as described previously (see Supplemental Table 1).Reference Olsen, Nickel and Fox 4 Newly coded SSIs, cellulitis, hematoma, seroma, dehiscence, fat necrosis, and tissue necrosis were identified by ICD-9-CM diagnosis codes in inpatient and outpatient facility and provider claims from 2 to 180 days after mastectomy (day 0 for hematoma), as described previously.Reference Olsen, Nickel and Fox 4 , Reference Nickel, Fox, Margenthaler, Wallace, Fraser and Olsen 5 The complication algorithm included diagnosis codes specific to breast complications (eg, 611.3 for fat necrosis), and general postoperative complications (eg, 998.59 for SSI) (Supplemental Tables 2 and 3).
We randomly selected a subset of women with infectious or noninfectious complications for review of medical records. We prioritized medical records from hospitals for patients with an inpatient claim coded for SSI and/or noninfectious wound complication. If there were no hospitalizations coded with a complication, we prioritized records from outpatient facilities, and finally, we considered medical records from providers. The medical records were obtained by a third-party vendor and were redacted of identifying information before review.
We received records that contained clinical information spanning the time period of the complication of interest for 132 women coded for SSI and/or cellulitis and 188 women coded in the claims data for ≥1 noninfectious wound complication. Signs and symptoms of postoperative complications documented by clinicians were abstracted from the medical records by a reviewer blinded to the codes in the claims data (K.E.B.). The Centers for Disease Control and Prevention/National Healthcare Safety Network (NHSN) definition was used to define SSIs in the medical record; the pre-2010 (including cellulitis as an SSI) and 2010 definitions (excluding cellulitis-only) were considered separately. 6 , Reference Mangram, Horan, Pearson, Silver and Jarvis 7
PPV was calculated as the number of cases confirmed by medical record review divided by the number identified by the claims algorithm. The 95% confidence intervals were calculated using a finite population correction factor. Data management and analyses were performed using SAS v9.3 (SAS Institute, Cary, NC) and SPSS version 21.0 (IBM SPSS Statistics, Armonk, NY). This study was approved with a waiver of informed consent by the Washington University Human Research Protection Office and Quorum Review.
RESULTS
We obtained medical records of 275 women coded in the claims data for mastectomy; 260 records (94.5%) included information regarding whether mastectomy was performed. The records showed that 2 women had undergone breast-conserving surgery rather than mastectomy, yielding a PPV for mastectomy codes of 99.2%. The PPV for laterality of the mastectomy was 94.6%; 244 of 258 were coded correctly in the claims data as unilateral or bilateral according to medical record review.
In total, 259 records contained sufficient information to determine whether breast reconstruction was performed and the type of reconstruction. Of 122 women coded in the claims for breast expander or implant reconstruction, 119 were confirmed by medical record review (PPV 97.5%). The records of 48 women included codes for autologous flap reconstruction, and 47 of these cases were confirmed by medical record review (PPV, 97.9%).
The PPVs for individual complications are shown in Table 1. The PPV was 57.5% for SSI using the current NHSN definition of SSI (excluding cellulitis-only), but the PPV increased to 68.9% when the pre-2010 NHSN definition including cellulitis-only as SSI was used. The PPVs of individual noninfectious wound complications ranged from 47.8% for fat necrosis to 96.6% for hematoma.
Positive Predictive Value of Coding in Administrative Claims Data Compared to Medical Record Review to Identify Surgical Site Infection (SSI) and Noninfectious Wound ComplicationsFootnote a After Mastectomy
a Women could have multiple different complications per the claims data. Of the 188 women coded for noninfectious wound complications, 157 were coded in the claims data for 1 noninfectious wound complication, 27 were coded for 2 different noninfectious wound complications, and 4 were coded for 3 different noninfectious wound complications (223 individual noninfectious complications). In total, 45 women were coded in the claims data for both SSI and a noninfectious wound complication.
b Number of women coded in the claims data for complication for which the medical record was received with clinical information spanning the time period of the complication.
c Pre-2010 NHSN definition included cellulitis as SSI. The PPV for this definition, excluding 998.51, was 75.0 (95% CI, 67.6–82.4).
The complications documented in the medical record at the time of false-positive coding for a wound complication are shown in Table 2. Of 100 discrepancies between the claims data and medical records among 90 women, 4 (4.0%) had the correctly coded wound complication at another anatomic site (eg, port infection, cellulitis), and 81 (81.0%) had a different breast wound complication recorded in the medical record. The most common complication noted in the medical records for women with false-positive coding for SSI was cellulitis, followed by seroma and noninfectious wound complications. Of the 13 false-positive SSIs with medical record documentation of seroma, 11 cases had been coded with the ICD-9-CM diagnosis code 998.51 (infected postoperative seroma). The most common error in the coding of noninfectious wound complications involved incorrect use of a code for a different wound complication. Of these discrepancies, 24 involved codes in the claims data for fat or tissue necrosis or dehiscence with medical record documentation of another of these noninfectious complications (Table 2). In the case of fat necrosis, 11 of the 12 complications miscoded as fat necrosis had medical record documentation of tissue necrosis.
Summary of Complications That Were Not Confirmed and Complications Documented in the Medical Records
NOTE. SSI, surgical site infection.
a An individual record could have multiple complications recorded.
DISCUSSION
The PPVs of our ICD-9-CM diagnosis code algorithms to identify infectious and noninfectious complications after mastectomy were variable but generally showed moderate to good results. The PPVs of coding for SSI ranged from 58% to 69% depending on whether cellulitis-alone was considered to meet the SSI definition. Cellulitis-alone was excluded from the NHSN definition of SSI in 2010, 6 , Reference Mangram, Horan, Pearson, Silver and Jarvis 7 and this change has had a considerable impact on reported infection rates because cellulitis is the most common postoperative breast infection.Reference Degnim, Throckmorton and Boostrom 8 To address this impact, we calculated the PPV for SSI using the definition of SSI at the time of the study (2004–2009), as well as the current NHSN definition excluding cellulitis-only. Of the 51 miscoded SSIs (using the strict definition of SSI), 14 cases were recorded as cellulitis in the medical record, and 3 of the 8 cases miscoded with cellulitis were recorded in the medical record as an SSI without clinician documentation of cellulitis.
Most nonconfirmed individual complications coded in the claims were documented in the records as other noninfectious breast wound complications. Some of the miscodings could have been due to misinterpretation via term searches. For example, a common error involving false coding for SSI was due to the use of the diagnosis code for infected seroma (998.51), in women with a documented seroma (without evidence for infection). Similarly, difficulty appeared to exist in discriminating fat necrosis versus tissue necrosis.
These results suggest that care should be used with respect to the inclusion of certain codes in algorithms to identify specific complications (ie, avoid 998.51 for identifying SSIs). Because the majority of errors in coding noninfectious wound complications involved misinterpretation of the specific complication, it may be better to focus on a composite group consisting of fat necrosis, tissue necrosis, or dehiscence, which had a higher PPV. In contrast, the PPVs for seroma and hematoma were very high; therefore, these can be reliably identified as individual complications.
In the literature, validation of claims data for SSI surveillance has yielded mixed results. The sensitivities of algorithms using a comprehensive list of codes to identify SSI in several studies were high at 72%–99%,Reference Sands, Yokoe and Hooper 9 – Reference Calderwood, Kleinman, Murphy, Platt and Huang 11 while the PPVs were lower, ranging from 15% to 51%.Reference Miner, Sands and Yokoe 3 , Reference Rhee, Huang and Berrios-Torres 10 – Reference Calderwood, Kleinman and Bratzler 16 In contrast, in studies that used a small set of SSI-specific diagnosis codes,Reference Yokoe, Noskin and Cunningham 1 , Reference Olsen and Fraser 2 , Reference Calderwood, Ma and Khan 17 – Reference Knepper, Young, Jenkins and Price 22 sensitivities varied greatly depending on the surgery and the data source, from 20% for 998.59 from a single academic medical center after general and vascular surgeryReference Hollenbeak, Boltz, Nikkel, Schaefer, Ortenzi and Dillon 18 to 100% after joint arthroplasty (using 998.5X and 996.66) or vascular surgery (using 998.5X and 996.62) using Medicare claims.Reference Calderwood, Ma and Khan 17 The PPVs in these studies using SSI-specific codes were generally higher, with 4 of 6 studies reporting PPVs >50% and up to 88%.Reference Yokoe, Noskin and Cunningham 1 , Reference Olsen and Fraser 2 , Reference Hollenbeak, Boltz, Nikkel, Schaefer, Ortenzi and Dillon 18 – Reference Yokoe, Khan and Olsen 21
Only 3 studies have assessed the accuracy of coding for SSI codes after breast surgery. We previously reported high sensitivity (88%), specificity (99%), and PPV (88%) for SSI using ICD-9-CM diagnosis codes 611.0, 682.2, 682.3, 996.69, and 998.5X from primarily inpatient billing data after breast surgery at an academic medical center.Reference Olsen and Fraser 2 In a multicenter 2-phase study, Yokoe et alReference Yokoe, Noskin and Cunningham 1 reported the sensitivity of SSI diagnosis codes 998.5X from inpatient billing data as 50% in phase 1 and 70% in phase 2, with PPVs of 58% and 79%, respectively. Miner et alReference Miner, Sands and Yokoe 3 used claims data from a large health-care system to evaluate a complex algorithm for SSI using a variety of ICD-9-CM diagnosis codes and procedure codes for wound care and culture. The PPV of the SSI algorithm using medical claims from inpatient, outpatient, or emergency department encounters within 60 days after breast surgery was only 18%.Reference Miner, Sands and Yokoe 3
The choice of codes used to identify SSIs and corresponding sensitivity and PPV depends on the intended goal of surveillance. Our intent was to develop an algorithm to compare complication rates among facilities and over time; thus, our focus was on maximizing the PPV to identify SSI and noninfectious wound complications. Interestingly, Calderwood et alReference Calderwood, Kleinman and Bratzler 12 , Reference Calderwood, Kleinman and Bratzler 16 reported low PPVs of a comprehensive list of diagnosis and procedure codes used to identify SSIs in Medicare claims after hip arthroplasty and vascular surgery, but they found similar PPVs between hospitals in the best- and worst-performing deciles in terms of risk-adjusted infection rates. These findings highlight the potential benefit of administrative data for comparing SSI rates between hospitals, even when PPVs are low.
The PPVs for the procedure codes to identify mastectomy and breast implant and autologous flap reconstruction were all >97%, consistent with previous reports.Reference Olsen and Fraser 2 , Reference Fisher, Whaley and Krushat 23 , Reference Talia, Agarwal, Momoh, Wilkins and Kozlow 24 As described previously,Reference Olsen, Nickel and Fox 4 before randomly selecting procedures for medical record review, we performed extensive filtering to exclude procedures that were unlikely mastectomy (eg, mastectomy coded only by an assistant, brachytherapy catheters at time of procedure). Thus, the true PPVs of the procedures codes for mastectomy and reconstruction procedures are likely lower than we report. Provided that a comprehensive algorithm is used to identify the appropriate patient population, the high PPVs of procedure codes suggest that accurate denominators can be calculated from claims or billing data to compare SSI rates across different surgical procedures or institutions.
We are aware of some limitations in our study. First, we only reviewed the record from 1 source (facility or provider) per patient, so it is possible that documentation of a complication may have been available in an alternative record. We could only determine the PPV of the ICD-9-CM diagnosis code algorithms and not sensitivity, specificity, or negative predictive values because we did not obtain medical records for women who were not coded for complications. A larger study would be helpful to assess whether our findings for individual noninfectious complications can be replicated.
Using health claims data, we captured complications coded at the time of an inpatient or outpatient hospital visit at the same or different institution than the index mastectomy, as well as complications treated in outpatient clinics. We found moderate PPVs for ICD-9-CM diagnosis codes for fat necrosis, tissue necrosis, and dehiscence, and we found high PPVs for coding of cellulitis, hematoma, and seroma. The PPV was modest for SSI without cellulitis, but it improved when cellulitis-only was considered an infection and when the code for infected seroma was excluded from the algorithm. Given the relatively high PPVs for noninfectious complications, including noninfectious wound complications along with SSI, may provide a more robust measure of quality of care by which to compare complication rates between facilities. New surveillance algorithms to identify infectious and noninfectious wound complications must be developed using ICD-10 codes in future, but this study can be used as the basis to develop ICD-10 algorithms. Finally, the high PPVs for identifying breast procedures are encouraging because they indicate high accuracy of the denominators used to calculate wound complication rates from claims data. Although imperfect, claims data can be used to screen for possible complications, with subsequent confirmation by medical record review, to improve the efficiency of routine SSI surveillance.
ACKNOWLEDGMENTS
We thank Cherie Hill for database and computer management support.
Financial support: Funding for this work was provided by the National Institutes of Health (NIH) (grant no. 5R01CA149614 to M.A.O.). Additional support was provided by the Centers for Disease Control and Prevention (CDC) Epicenters Program (grant no. U54CK000162 to V.J.F.).
Potential conflicts of interest: M.A.O. reports consultant work with Pfizer, Merck, and Sanofi Pasteur and grant funding through Pfizer, Sanofi Pasteur, and Cubist Pharmaceuticals for work outside the submitted manuscript. A.E.W. is an employee of HealthCore, a wholly owned subsidiary of Anthem, Inc., a health insurance company. She has received Anthem stock options and participates in an Anthem employee stock purchase plan. V.J.F. reports that her spouse is Senior Vice President and Chief Medical Officer at Express Scripts. K.E.B. and K.B.N. report no conflicts of interest.
SUPPLEMENTARY MATERIAL
To view supplementary material for this article, please visit https://doi.org/10.1017/ice.2016.271
