Discussion

Recently, many studies and reviews have focused on the impact of SSI on medical costs. ^{Reference Kashimura, Kusachi and Konishi75} Accurate information on the cost of SSI is required to support decisions concerning the extent of resources hospitals devote to infection control. ^{Reference Kilgore, Ghosh, Beavers, Wong, Hymel and Brossette4} Published cost estimates may not be transferable, and thus, the decision based on them to implement a program may result in undesirable consequences. ^{Reference Fukuda and Imanaka13,Reference Sculpher, Pang, Manca and Drummond14} Also, these estimates are not necessarily comparable due to the variation in the costing scopes, cost accounting systems, unit costs, clinical practice, costing methods, statistical methods used to determine attributable costs, and duration of follow-up. ^{Reference Kilgore, Ghosh, Beavers, Wong, Hymel and Brossette4,Reference Olsen, Chu-Ongsakul, Brandt, Dietz, Mayfield and Fraser35} Thus, it is crucial to assess the quality of the costing methods and the transferability of cost estimates before using them. In this review, we aimed to critically appraise published articles after the reviews made by Fukuda et al ^{Reference Fukuda, Lee and Imanaka6,Reference Fukuda, Lee and Imanaka16} to explore whether researchers addressed their recommendations. However, most researchers did not address these recommendations that make their cost estimates unlikely to be transferable.

Matching and regression analysis represented 80% in our review. Both methods are easy to implement given the hospital information system’s availability. Matching on covariates, unlike regression analysis, does not require complicated statistical analysis. However, with many covariates, the proportion of unsuccessful matches increases. ^{Reference Graves, Weinhold, Tong and Birrell12} Although it is easier to base matching on a propensity score (PS), one scalar that summarizes all covariates is still associated with unsuccessful matches. Thus, reporting the proportion of successful matches to measure the potential bias is recommended. ^{Reference Fukuda, Lee and Imanaka6} Of the 31 studies that used matching on PS or covariates, only 10 (32.3%) had reported the proportion of successful matches. With a lack of controls matching to severely infected cases, the latter is likely to be excluded with subsequent underestimation of SSI cost (ie, selection bias). ^{Reference Olsen, Butler, Willers, Gross, Hamilton and Fraser34}

The whole study sample could only be preserved using regression analysis and PS-based weighting approaches as inverse probability weighting (IPW). ^{Reference Desai and Franklin10,Reference Graves, Weinhold, Tong and Birrell12,Reference Allan, Ramagopalan and Mardekian81} This might explain why more studies used regression analysis than was reported by Fukuda et al (39% vs 26%). Also, regression analysis was used after matching to control for any remaining imbalances in 2 studies.

The impact of different analytical methods on estimated costs is unclear. In the current review, 2 studies compared matching with regression analyses. One study found no difference between them ^{Reference Olsen, Chu-Ongsakul, Brandt, Dietz, Mayfield and Fraser35} , and the other revealed higher estimates with regression. ^{Reference Olsen, Butler, Willers, Gross, Hamilton and Fraser34} A similar unclear pattern was observed by Fukuda et al. ^{Reference Fukuda, Lee and Imanaka6}

As reported by Fukuda et al, ^{Reference Fukuda, Lee and Imanaka6} case review was the least used method to estimate the cost as it is labor intensive. Additionally, if the criteria for inclusion of SSI-related resources were unclear as in the AEP method, ^{Reference Wakefield, Pfaller, Hammons and Massanari82} expert reviewers as physicians or nurses might be required to review records. ^{Reference Wakefield, Pfaller, Ludke and Wenzel83}

SSI increases total costs by extending the LOS. Concurrently, extended time to start of SSI (LOS before SSI) is by itself a cause of SSI. Thus, LOS is an endogenous variable because it simultaneously affects and is affected by SSI. Without adjustment for time to start of SSI, the cost of SSI is usually overestimated as the time to start of SSI is incorrectly assigned to the cost of SSI. ^{Reference Kilgore, Ghosh, Beavers, Wong, Hymel and Brossette4,Reference Graves, Weinhold, Tong and Birrell12,Reference Lloyd-Smith84} Contrarily, adjustment for the “whole LOS” underestimated the cost of SSI. ^{Reference Kilgore, Ghosh, Beavers, Wong, Hymel and Brossette4,Reference Kilgore and Brossette85} In the current review, only a few studies (13%) adjusted for time to start of SSI. None of them adjusted for the whole LOS. Even when the time to start of an SSI was unknown, researchers adjusted for only a part of this time, eg, adjusting for preoperative LOS.

Half of the studies did not report the type of perspective. In a recent systematic review, Sculpture et al ^{Reference Sculpher, Pang, Manca and Drummond14} found that the perspective was defined in only 42% of the studies. When researchers neglect to report the perspective, decision-makers could not interpret if the relevant costs and selected costing methods are consistent with the objective of costing. This will affect the generalizability and transferability of the reported results.

Most of the studies which reported perspectives assigned the monetary values of resources according to the chosen perspective. Since hospital perspective was the most common, studies mainly included inpatient costs and excluded post-discharge costs, mainly borne by patients, caregivers, and the community health service. Excluding post-discharge costs might considerably underestimate the burden of SSI. Perencevich et al ^{Reference Perencevich, Sands, Cosgrove, Guadagnoli, Meara and Platt88} found that the inclusion of post-discharge costs increased SSI cost by a factor of 3. Thus, the choice of hospital perspective should be considered carefully.

Although ICPs are implemented by and in hospitals, considering their perspective provides little incentive to implement ICP because hospitals might not bear the consequences of SSI. We recommend estimating the costs from the perspective of whoever bears the consequences, either the patients or the society. If it proves to be substantial, then legislation should be set to encourage hospitals to keep SSI rates low, even if the hospitals will not gain monetary benefit from reducing SSI.

Overall, researchers chose a perspective that aligns with the main source of health financing in their countries. In the United States, where private insurance dominates, ⁸⁹ payer perspective was mainly adopted. In Europe, where universal health coverage predominates, ⁸⁹ provider perspective was the most common.

Also, researchers were aware of how the chosen perspective affects the assigned costing method. Inconsistent methods of costing were observed in only 25% of the studies. Moreover, 80% of studies that took payer or patient perspective used unmodified charges. Unlike Fukuda et al, we believe that unmodified charges are not always less accurate than actual costs. In countries where out-of-pocket payments constitute the main sources of financing healthcare services, as in low-income countries, charges might be more realistic to evaluate the burden of illnesses or the impact of interventions.

As reported by Fukuda et al, ^{Reference Fukuda, Lee and Imanaka16} few studies (10%) had a high transferability level, and D was the most common scope of costing. No increase in the percentage of studies was reported at the B level, the recommended level that achieves the balance between clarity (which is essential for transferability) and practicality when estimating the cost. ^{Reference Fukuda and Imanaka13}

Half of the studies we reviewed were multicenter studies. In contrast to single-setting studies, multicenter studies ^{Reference Sculpher, Pang, Manca and Drummond14,Reference O’Brien90} generate comparable standardized and transferable cost estimates. Most of the studies were conducted in the United States and Europe, and few were conducted in developing countries. The underrepresentation of low-income countries hinders the transferability to similar countries.

All of the studies that converted currency reported the type of exchange rate used. Unfortunately, all but 1 study ^{Reference González-Vélez, Romero-Martin, Villanueva-Orbaiz, Diaz-Agero-Pérez, Robustillo-Rodela and Monge-Jodra65} used the market exchange rate. The purchasing power parity exchange rate is superior to the market exchange rate because the former is determined by the relative cost of living and inflation rates in different countries. Thus, it allows for transferability and comparability of cost estimates.

In the current review, most studies reported the mean or total costs, and only 13% reported the median cost. Although cost data are known to be skewed, the use of median underestimates the burden of SSI. For decision making, the cost is best presented with the mean. ^{Reference Mani, Lundkvist, Holmberg and Wanhainen91} Additionally, such studies should be accompanied by sensitivity analysis to evaluate the robustness of the estimated costs. ^{Reference Sculpher, Basu, Kuntz and Meltzer92,Reference Briggs93} In this review, only 5 studies conducted sensitivity analyses.

Lack of reporting does not only hinder the transferability of the cost estimates, it also might hinder their usage in clinical support decisions in the setting where they were estimated. For example, without reporting the time horizon (observed in 10% of the studies), readers are left unable to check whether the cost of SSI was entirely captured and whether discounting should have been applied. Neglecting the price year, which was reported in half of the studies, hinders the comparability and transferability of cost estimates between different settings and times.

Unlike Fukuda et al, ^{Reference Fukuda, Lee and Imanaka16} we found that researchers became more inclined to use transferable studies. We observed significantly higher numbers of citations of studies with high transferability as well as studies with a clear scope of costing (A/B) and studies that used actual costing/RCC/RVU.

Here, we updated the reviews by Fukuda et al. ^{Reference Fukuda, Lee and Imanaka6,Reference Fukuda, Lee and Imanaka16} Although we confined our review to only 1 type of HAI, we broadened the criteria for critically appraising the included studies to enhance cost studies’ quality and transferability as economic perspective, types of cost (inpatient, outpatient, out of pocket, productivity losses, capital/overhead, opportunity), cost description measure, time horizon and discounting, price year and inflation adjustment, currency and currency conversion, and sensitivity analysis.

This study has several potential limitations. We limited our search to MEDLINE only via PubMed, included only published studies, and excluded studies in which evaluating intervention or ICPs was the primary aim, although they could have estimated the cost of SSI. The impact of these limitations might be minimal for the following reasons. MEDLINE was capable of identifying from 75% to 92% of relevant studies. ^{Reference Hartling, Featherstone, Nuspl, Shave, Dryden and Vandermeer94,Reference Halladay, Trikalinos, Schmid, Schmid and Dahabreh95} Unpublished studies are less likely to be sources of SSI cost to support decision for implementing ICP. Exclusion of the studies evaluating the economic impact of ICP kept the included studies more homogenous in methodology and reporting. Another limitation is that we did not address other aspects of the costing methodology, for example, which confounding variables are important to include, and how to refer to the time to start of the infection. These factors might be topics for further research.

In conclusion, over the past 14 years, matching and regression analysis were more likely used to control for confounding factors to minimize bias. Although LOS adjustment has decreased, researchers started to correctly use “time to start of infection” instead of “whole LOS” for adjustment. Researchers mainly adopted a hospital perspective and neglected patient and societal perspectives in the postdischarge setting. After Fukuda et al article, we observed no significant improvement in the transferability of published studies; however, unlike Fukuda et al, transferable studies became more likely to be cited. Researchers became selective in using cost estimates derived from other settings, indicating increased awareness about fundamentals in costing methodologies.

Researchers were less likely to consider reporting recommendations made by Drummond et al ^{Reference Drummond, Sculpher, Claxton, Stoddart and Torrance17} and the CHEERS checklist ^{Reference Husereau, Drummond and Petrou18} when estimating the cost of SSI (Table 5). We believe that these recommendations, while not new, are unfortunately not well considered by researchers and should be used when conducting their economic evaluations to estimate the cost.

Table 5.

Recommendations to Improve Costing Methodologies for Surgical Site Infections