Prosthetic hip and knee joint infections (PJIs) are increasing in concert with the rising number of hip and knee arthroplasties. Reference Kurtz, Ong, Lau, Mowat and Halpern1 Forming the large cohorts necessary to adequately study these infections is a challenge because PJIs complicate a minority of primary arthroplasties. Reference Kurtz, Lau, Son, Chang, Zimmerli and Parvizi2,Reference Bozic, Grosso and Lin3 Surgical registries have been used to generate these cohorts, yet they are not available in all jurisdictions, they are costly to maintain, they may not be representative, and they often have incomplete outcome data. Reference Lacny, Bohm, Hawker, Powell and Marshall4 Health administrative databases capturing hospital diagnosis and procedure codes are a potential option because they are available, comprehensive, and standardized. Reference Rusk, Bush and Brandt5
However, using administrative databases for PJI research can be problematic. First, individual hip and knee PJI diagnosis codes have variable and sensitivity, ranging between 60% and 90% across previous studies. Reference Bolon, Hooper and Stevenson6–Reference Curtis, Graves and Birrell8 Second, these diagnosis codes are based upon International Classification of Diseases Ninth or Tenth Revision (ICD-9 or ICD-10) and have usually been tested against surveillance performed by infection prevention and control departments, potentially underestimating the burden of PJIs because both surveillance strategies are imperfect. Reference Rusk, Bush and Brandt5,Reference Bolon, Hooper and Stevenson6 Combinations of diagnosis and procedure codes have been employed in studies of PJI using administrative databases, Reference Marang-van de Mheen and Bragan Turner9 but their performance characteristics have not been assessed.
In this study, we validated ICD-10 diagnosis and procedure codes at 4 hospitals in Toronto, Ontario, Canada, to test the hypothesis that combinations of diagnosis and procedure codes could improve PJI detection from administrative databases.
Methods
We followed the guidelines for reporting of validation studies of health administrative databases. Reference Benchimol, Manuel, To, Griffiths, Rabeneck and Guttmann10
Study population
The eligible study population included all individuals who underwent a primary arthroplasty at 4 hospitals (Sinai Health System, University Health Network, Sunnybrook Health Sciences Centre and St. Michael’s Hospital) in Toronto, Ontario, Canada, from January 1, 2010, until December 31, 2016. Primary arthroplasties were identified using Canadian Classification of Health Interventions (CCI) codes (1.VA.53 for hip and 1.VG.53 for knee) with the appropriate qualifier (‘0’ prior to April 1, 2012 or ‘P’ thereafter). The CCI is based upon ICD-10 procedure codes. A prior validation study has demonstrated 99% sensitivity and 100% specificity for using these procedure codes to identify primary arthroplasties, all of which occurred in a hospital setting in Ontario. Reference Juurlink, Preyra and Croxford11
Flow chart for creation of study population.
Study cohort
All inpatient rehospitalizations following a primary hip or knee arthroplasty from January 1, 2010, until December 31, 2016, were included. Each individual was followed from the time of primary arthroplasty until the end of the study period (December 31, 2016) by searching the discharge abstract database (DAD) at each hospital. This database is the repository for the mandatory information collected upon hospital discharge and includes clinical diagnoses and procedures performed. These codes follow the ICD-10 for diagnoses and the CCI for procedures. 12 A second, independent, cohort was also constructed using operative listings collected by the infection prevention and control (IPAC) departments of each hospital to identify all revision operations. This was used to identify revision arthroplasties that may have been missed by searching the DAD.
Prosthetic joint infection detection
The medical record for each readmission was reviewed by an infectious diseases physician (C.K.) blinded to the administrative database codes to ascertain whether a PJI was present. A PJI diagnosis, considered a deep surgical site infection (SSI), was based on the Musculoskeletal Infection Society (MSIS) criteria, which involves clinical, biochemical, radiographic, histological, and intraoperative characteristics. Reference Parvizi and Gehrke13 A readmission in a patient with a confirmed PJI that was due to a treatment complication (eg, adverse event due to an antibiotic or joint spacer dislocation) was categorized as a PJI admission despite not meeting the MSIS criteria because these complications were the result of PJI treatment. Discordant PJI pairs occurred when a PJI diagnosis code of T84.53 or T84.54 in any of 20 available diagnosis fields was applied in the absence of a PJI identified by chart review or a PJI identified by chart review in the absence of a PJI code. These cases were rereviewed by C.K. and A.M. to ensure correct application of the MSIS criteria. The surgical procedures performed and microbiology were also abstracted by chart review.
Prosthetic joint infection detection algorithms
Diagnosis and procedure codes were tested individually and in combination against the criterion standard, which was whether a PJI was present during a hospital admission based on chart review. These algorithms were prespecified and included PJI diagnosis codes, postoperative infection diagnosis codes, procedure codes for orthopedic surgery operations, and ancillary procedure codes such as the insertion of a peripherally inserted central catheter. The main comparison algorithm was the combination of an arthroplasty code along with a procedure code for the insertion of a peripherally inserted central catheter, which permits outpatient administration of parenteral antibiotics. Other algorithms included an orthopedic procedure code or peripherally inserted central catheter insertion code alone, combination of a PJI code with an orthopedic procedure code, and a PJI code with a specific orthopedic procedure code likely associated with a PJI (joint spacer insertion or excision arthroplasty).
Recursive partitioning
Binary recursive partitioning was applied to the cohort to identify potentially novel detection algorithms using the ‘rpart’ package version 4.1-13 in R software (R Foundation for Statistical Computing, Vienna, Austria). All diagnosis and procedure codes were input as predictor variables with the outcome being a confirmed PJI for each admission. This produced a classification tree using the Gini criterion to split into increasingly homogenous groups until further partitioning was not feasible. The initial classification tree was pruned by adjusting the cross-validated error rate. Reference Breiman, Friedman, Olshen and Stone14 This final pruned tree yielded an additional PJI algorithm to test.
Statistical analysis
For each algorithm, the performance characteristics (sensitivity, specificity, positive predictive value, and negative predictive value) were calculated along with 95% confidence intervals following every joint replacement (eg, 1 individual could contribute up to 4 joints). The reference standard was a PJI identified by chart review. The primary analysis assessed whether a PJI was detected in at least 1 admission following a primary arthroplasty and the denominator was the total number of arthroplasties performed. This denominator was selected as the absence of a hospital admission influences the probability a PJI has occurred. Differences by joint and hospital were assessed, as were secular trends by testing for variation of sensitivity and positive predictive value by year. Secondary analyses assessed performance characteristics for each admission and for PJI detection using readmissions for a potential PJI (ie, excluding readmissions for subsequent primary arthroplasties and those in persons with a PJI already diagnosed). Counts of the CCI procedure codes by PJI operative intervention were tabulated. Statistical analyses were performed using R version 3.4.4.
Results
During the 7-year period, 27,802 primary arthroplasties were performed on 23,419 individuals. There were 8,844 readmissions among 6,518 individuals, with a median time to re-admission of 669 days (interquartile range [IQR], 256–1,249 days). Overall, 4,067 of 8,840 readmissions (46%) were due to another primary arthroplasty. There were 586 readmissions for 354 PJIs (overall, 354 of 27,802, 1.27%) with 1 additional PJI identified by IPAC (Fig. 1). The median time to PJI readmission following primary arthroplasty was 178 days (IQR, 52–534). In total, 558 operations were performed for the treatment of a PJI, including 188 debridement with prosthesis retention (34%), 171 first-stage procedures (30%), 137 second-stage procedures (25%), 42 single-stage procedures (8%), 15 excision arthroplasties (3%), 2 fusions (0.4%), and 3 above-knee amputations (0.5%). Staphylococcus aureus was found in 93 of 354 (26%), followed by coagulase negative staphylococci in 79 of 354 (22%), gram-negative bacilli in 49 of 354 (14%), β-hemolytic streptococci in 20 of 354 (6%) and Enterococcus spp in 21 of 354 (6%), while 99 of 354 (28%) were culture negative.
The PJI diagnosis codes exhibited a sensitivity of 0.88 (95% confidence interval [CI], 0.85–0.92) and a positive predictive value (PPV) of 0.78 (95% CI, 0.74–0.82) for identifying at least 1 admission for a PJI during the follow-up period (Table 1). Combining a PJI code or a procedure code for an arthroplasty along with the insertion of a peripherally inserted central catheter improved sensitivity (0.92; 95% CI, 0.88–0.94) without change in PPV (0.78, 95% CI, 0.74–0.82). Expanding diagnosis codes to include postoperative infections or skin and soft-tissue infection diagnosis codes increased sensitivity, while lowering specificity (data not shown). The algorithm derived using recursive partitioning identified the combination of both the PJI diagnosis code and joint spacer procedure code as the most important, yielding a sensitivity of 0.88 (95% CI, 0.85–0.92) and PPV of 0.78 (95% CI, 0.73–0.82). All algorithms had high specificity and negative predictive values (Table 1). When applied to each individual admission the PJI diagnosis code exhibited sensitivity of 0.85 (95% CI, 0.81–0.87), specificity of 0.99 (95% CI, 0.99–1.00), a negative predictive value of 1.00 (95% CI, 1.00–1.00), and a PPV of 0.77 (95% CI, 0.74–0.80). When the denominator was restricted to readmissions without a primary arthroplasty and only the first admission for a PJI, the sensitivity decreases to 0.83 (95% CI, 0.78–0.87). Using this approach, the relative performance of different algorithms did not change, but all performance characteristics were reduced. The median reductions were as follows: sensitivity, 0.05 (IQR, 0.05–0.06); specificity, 0.04 (IQR, 0.03–0.06); NPV, 0.01 (IQR, 0.01–0.02); and PPV, 0.11 (IQR, 0.08–0.11) (Supplementary Table 1 online).
Performance Characteristics of PJI Identification Algorithms Comprised of Diagnosis and Procedure Codes From the Discharge Abstract Database Tested Individually and in Combinationa
Note. PJI, prosthetic joint infection; PICC, peripherally inserted central catheter.
a These were assessed in all readmissions following primary arthroplasty.
We detected variation by hospital when testing individual PJI diagnosis codes: sensitivity range, 0.81–0.98 (P = .002) and PPV range, 0.73–0.86 (P = .11). We detected variation in the combination of a PJI diagnosis code or an arthroplasty code with the insertion of a peripherally inserted central catheter: sensitivity range, 0.86–1.00 (P = .01) and PPV range, 0.74–0.87 (P = .07). We detected no variation by joint: sensitivity for hip, 0.86 and sensitivity for knee, 0.90 (P = .42) and PPV for hip, 0.79 and PPV for knee, 0.78 (P = .96). And we did not detect any secular trends: sensitivity range, 0.88–0.91 (P = .99) and PPV range, 0.76–0.81 (P = .97).
When a procedure was performed for the treatment of a PJI, a variety of orthopedic procedure codes were applied; an arthroplasty procedure code was the most frequent (474 of 558, 85%) (Table 2). For the first of a 2-stage operation, a spacer insertion procedure code was recorded in 107 of 188 cases (57%), which represented 107 of 120 total spacer codes applied (89%). We were unable to identify procedure codes that could adequately distinguish between different surgical approaches used for the management of PJI.
Canadian Classification of Health Intervention (CCI) Procedure Codes Applied to Surgical Procedures Performed for Management of a Prosthetic Hip or Knee Joint Infection
a Identified by chart review.
Discussion
We have demonstrated that detection of PJIs from administrative databases can be improved by combining diagnosis and procedure codes. Although validation in a larger group of hospitals in different geographic areas may be required, this approach allows for the creation of large observational cohorts, which is vital to improve the management of PJIs and to understanding the epidemiology of these infections. This detection algorithm is intuitive, simple, and can be applied wherever ICD-10 codes are utilized. Its use does require either a single-payer system or a willingness to collaborate among healthcare institutions. This is particularly true for jurisdictions in which joint arthroplasties are performed in outpatient surgical centers.
PJI diagnosis codes
PJI diagnosis codes exhibit a sensitivity of 0.88 and a positive predictive value of 0.78. Comparable performance characteristics using ICD-9 and ICD-10 revisions diagnosis codes have been observed in other jurisdictions with different reference standards, which suggests that our cohort is broadly representative of previous studies. Reference Bolon, Hooper and Stevenson6–Reference Curtis, Graves and Birrell8,Reference Calderwood, Ma and Khan15 PJIs can be challenging to diagnose definitively, so imperfect PJI detection is anticipated. Reference Fillingham, Della Valle and Suleiman16 Other studies have used PJIs identified through surveillance performed by infection prevention and control departments or random sampling of readmissions as a comparison standard, which may be suboptimal because such surveillance has a sensitivity of 80%. Reference Rusk, Bush and Brandt5,Reference Curtis, Graves and Birrell8 Our reference standard included review of all hospital readmissions identified by administrative databases and operative listings to minimize potentially missing PJIs. In addition, using the MSIS criteria for defining a PJI ensures accurate classification because varying definitions impact PJI detection substantially. Reference Tan, Goswami, Fillingham, Shohat, Rondon and Parvizi17 The choice of denominator impacts overall performance characteristics, but it does not alter the improvement observed by combining procedure and diagnosis codes. Overall, database codes have exhibited high specificity with somewhat lower sensitivity, and in our study, which uses ICD-10 codes, this pattern was similarly observed. Reference Rennert-May, Manns and Smith7,Reference Calderwood, Ma and Khan15 The performance characteristics can be used to account for outcome misclassification bias when epidemiologists study PJIs using administrative databases.
PJI detection algorithm
Combining information from multiple sources enhances PJI detection from administrative databases. Reference Gundtoft, Pedersen, Schønheyder and Overgaard18 In our sample, the most robust algorithms involved a combination of diagnosis and procedure codes, including PJI diagnosis codes along with the combination of an arthroplasty code and the insertion of a peripherally inserted central catheter or a joint spacer procedure code. Because most PJIs involve a surgical procedure it seems intuitive that inclusion of a procedure code for arthroplasty increases positive predictive value. The insertion of a peripherally inserted central catheter, which facilitates outpatient parenteral antibiotics, during the same hospitalization as a revision operation, improves PJI detection because these intravenous catheters are unlikely to be inserted for reasons other than PJI treatment. Applying recursive partitioning also identified a combination of diagnosis and procedure codes as the most effective in identifying a PJI, but the results (using a PJI diagnosis code and joint spacer insertion procedure code) were not an improvement over a PJI code alone in our sample.
PJI surveillance
The large numbers of arthroplasties performed makes postoperative infection surveillance by individual infection prevention and control departments challenging. Our PJI diagnosis algorithm will detect >90% of deep postorthopedic infections; reviewing the charts of 423 of 27,802 (~2%) could be used to remove false-positive records and reduce misclassification. The most sensitive algorithm could be used to reduce the number of chart reviews needed to be reviewed when performing hospital SSI surveillance and the uncertainty of a PJI diagnosis can be incorporated into real-time SSI detection systems. This system could also be used to detect clusters of PJIs, although these presumably occur uncommonly, enhanced detection might offer opportunities to intervene both in early diagnosis and prevention of further infections. Reference Baker, Haridy and Salem19 If hospitals collaborate in SSI detection this detection strategy can be similarly used across hospitals to increase the capture of PJIs which are managed at a hospital other than that where the primary procedure was performed. Two recent studies, in Alberta and California, identified that 43% and 17% of individuals with a postoperative infection presented to a different hospital, respectively, from where the primary arthroplasty was performed. Reference Rusk, Bush and Brandt5,Reference Yokoe, Avery, Platt and Huang20 In the short term, such collaborative surveillance systems could be used for feedback to surgical teams for the reduction of early infections. In the longer term, such systems might identify previously unidentified, potentially modifiable, risk factors for late infections.
Detecting PJI operations
A major advantage of prosthetic joint registries is the ability to classify operations performed for a PJI. Obtaining such descriptive operative information from administrative databases would be very helpful in the study of PJIs. However, in our data, CCI procedure codes (based upon the ICD-10 revision classification structure) alone were unable to distinguish between the varying operative strategies performed for the treatment of a PJI. In particular, it was difficult to separate operations involving joint retention from joint removal. Only the joint spacer procedure code could reliably detect a first stage procedure, yet it missed a substantial proportion of these operations. To improve PJI surgery characterization, it will be necessary to combine additional information, such as physician billing codes or specific joint components.
Our study has several limitations. First, PJIs were only detected at the same institution where the primary arthroplasty was performed. Thus, the ~30% of individuals suffering from an SSI present to a hospital other than where the primary procedure was performed will be missed. Reference Rusk, Bush and Brandt5,Reference Yokoe, Avery, Platt and Huang20 If coding for such patients is systematically different than that of patients admitted to the primary hospital, then our estimates of performance characteristics may be incorrect. Second, the proposed detection strategy will miss patients managed with antibiotics alone and those who are not hospitalized. However, most primary arthroplasties are performed electively for osteoarthritis, and chronic PJIs are associated with substantial disability; thus, this proportion of patients is expected to be small. Reference Ravi, Pincus and Wasserstein21 Third, while this validation included 4 hospitals over a 7-year period, it was carried out in a single geographic area in Canada and included only operations performed in a hospital. The performance characteristics between hospitals varied, suggesting that further validation in other jurisdictions would be useful. Fourth, the MSIS criteria available at the time the chart reviews were performed has since been updated. Reference Parvizi, Tan and Goswami22 The additional criteria are not routinely performed in the evaluation of a potential PJI in Toronto and are unlikely to have altered the number of PJIs detected.
In conclusion, an algorithm combining diagnosis and procedure codes can improve the identification of PJIs from administrative databases. The most robust algorithm involves the combination of a PJI diagnosis code or a combination of an arthroplasty procedure code along with the insertion of a peripherally inserted central catheter. Because this information is widely available, infection prevention and control departments can use administrative databases to assist with PJI surveillance, either by using the algorithm directly or by limiting the number of medical records to review. This PJI detection algorithm, with incorporation of the performance characteristics, can also be used to construct large observational cohorts to improve the management of these devastating infections.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/ice.2020.449
Acknowledgments
We are grateful for the assistance of the medical records departments of each participating hospital.
Financial support
C.K. received support from the Eliot Phillipson Clinician-Scientist Training Program from the University of Toronto.
Conflicts of interest
No potential conflicts of interest exist for any author.
