In this issue of the journal, authors from the US Centers for Disease Control and Prevention (CDC) introduce a new metric, the cumulative attributable difference (CAD), to efficiently prioritize hospitals with large numbers of preventable healthcare-associated infections (HAIs).Reference Soe, Gould, Pollock and Edwards
1
The metric, which represents the excess number of HAIs for a reporting hospital or ward, is easily constructed by selecting a standardized infection ratio (SIR) goal for HAI reduction. It is also easily interpretable—a CAD value greater than zero indicates excess HAIs. Although these are desirable attributes in a metric, they are perhaps not its most crucial attributes. To fully understand a metric’s impact, we need to anticipate how it can contribute to changing behavior and improving outcomes. Here we examine the roles that perspective, value systems, and implementation may play as CAD is used to guide practice.
The CAD metric is part of a larger strategy advocated by the CDC called Targeted Assessment for Prevention,
2
which provides a framework for linking data to action. Within the framework, decision makers target a SIR (eg, 0.75, representing a 25% reduction below the expected value given the subject's patient mix) and use the resulting CAD to identify hospitals or wards with the highest numbers of excess HAI events. The US Department of Health and Human Services has outlined SIR goals for a number of HAI targets.
3
Decision makers then implement interventions in these hospitals and wards to prevent future HAIs. This strategy has been piloted
4
but since there are not yet any published studies, its effects on behavior and outcomes (both actual and intended) are still unknown.
One persistent question is whether a CAD-driven strategy is the most efficient one. Underlying this strategy are 2 implicit assumptions: that targeting fewer hospitals is more cost-effective because a fixed cost must be incurred when intervening at each targeted hospital,Reference Graves, Halton and Jarvis
5
and that the marginal cost of preventing 1 more HAI is constant, whether within the same hospital or across different hospitals. The first assumption is likely true. An intervention usually needs a champion, educational efforts, and a social campaign at each hospital in order to be successful.Reference Soo, Berta and Baker
6
,
Reference Stetler, Damschroder, Helfrich and Hagedorn
7
Therefore, selecting a smaller number of hospitals in which to intervene would make sense when the fixed costs associated with each intervention are high.
The second assumption is unlikely to be true because of the law of diminishing marginal returns, which states that, at some point, additional units of an input will yield fewer units of the output.Reference Graves, Halton and Jarvis
5
,
Reference Rubin, Jones and Leecaster
8
–
Reference Barnett, Page and Campbell
10
In the case of infection control, this means that as HAIs decrease, increasingly greater effort and resources will be needed to yield further reductions in HAIs. From a healthcare network perspective, it is conceivable that it may be more costly to prevent a single additional infection in a large hospital with a moderate HAI rate than in a small hospital with a high HAI rate. As a consequence, the potential impact of an intervention (the count of HAIs prevented) may not correlate with CAD.
A complicating factor is that poorly performing hospitals may be poorly performing because of local difficulties in the implementation of preventive strategies. Thus, even though the law of diminishing returns can help us predict changes in marginal returns at the same hospital, it remains difficult to predict the difference in cost of preventing 1 more infection in a high-CAD hospital versus a low-CAD hospital. More research is needed on this point.
To maximize impact, it will be necessary to predict both the CAD prevented and the costs of a given strategy. Decision makers would then select the strategy with the highest predicted number of prevented CAD for the resources at their disposal. The strategy also has to be dynamic. Even after initially targeting hospitals for intervention, decision makers need to decide when to continue to pour resources into the same hospitals and when to move on to others. This adds to the complexity of finding the ideal solution.
There is 1 additional facet to be considered: the transmissibility of the pathogens that cause HAIs. It may be that addressing HAIs due to multidrug-resistant organisms at the worst hospitals might stem their spread to other hospitals, thereby benefiting the future state of the entire system.Reference Lee, McGlone and Wong
11
For example, a single hospital with an outbreak of carbapenem-resistant Klebsiella might have a low CAD but may also be the source for many more future CAD in larger hospitals. Thus, an intervention to prevent HAIs at this hospital would have additional benefits on other hospitals over time.
The ideal approach to prioritization may require more than rank prioritization by CAD. The sheer number of ways to allocate resources over time makes it a computationally complex problem. When taking a broader system perspective, research using simulation models could be used to identify potential solutions and explore the interaction between fixed intervention costs at multiple hospitals and marginal costs within hospitals, as well as the ripple effects of interventions on neighboring facilities.
Metrics take different forms, such as counts, risks, and rates, and each form can influence practice differently. The CAD is a count metric. If the number of HAI events were to increase by 1 in either a small hospital or a large hospital of equivalent patient mix, the CAD would increase by 1 in each. However, if we were to compare rates, which is a common practice, the small hospital would have a larger increase in the rate because of differences in denominator size. Different denominators reflect different perspectives and can prompt different responses.
Another important aspect of metrics is whether comparisons are absolute or relative. Comparisons of CAD highlight absolute differences, whereas those of SIR highlight relative differences. As noted by Soe et al,Reference Soe, Gould, Pollock and Edwards
1
prioritization driven by SIR (or other metrics used for relative comparisons), as opposed to CAD, targets a greater number of smaller hospitals. These 2 metrics, which contain largely the same information, encode the priorities of 2 different theories of distributive justice: utilitarianism in the case of CAD and prioritarianism in the case of SIR.Reference Scheunemann and White
12
Several stakeholders are interested in preventing HAIs; the 3 most obvious are society, patients, and hospitals. From a societal perspective, it makes sense to act in ways that bring the greatest good to the most people. In this case, society as a whole is considered better off if there are fewer HAIs (utilitarianism). Patients may take a different perspective, however; at the most poorly performing, low-SIR hospitals, patients may feel that their hospitals deserve the most resources (prioritarianism).
In many cases, utilitarian and prioritarian approaches may target the same hospitals, but prioritization by CAD may lead to intervention in a lesser number of larger hospitals, leaving a sizable number of hospitals without an intervention. Patients who use these hospitals would continue to be exposed to risk for a HAI event that may exceed some threshold for “acceptable” HAI risk, if such a threshold existed. Conversely, had the SIR been used to prioritize hospitals for HAI reduction interventions, a greater number of hospitals would have been identified as targets for intervention. Therefore, either more resources would be required for intervention or less resource-intensive interventions would be necessary. The minimum standard of quality would be raised, but it would come at the cost of either a higher overall number of HAIs or a greater amount of resources expended.
It is rare for either utilitarian or prioritarian approaches to be used in their purest forms, which is likely why the CDC advocates for both the CAD and the SIR metrics. On a societal level, it makes sense to use our resources wisely. This makes the use of CAD within a Targeted Assessment for Prevention framework a reasonable overall approach. However, as dictated by the rule-of-rescue,Reference Scheunemann and White
12
a hospital with particularly poor performance (eg, a very high SIR) would still merit intervention, regardless of its CAD.
The CAD brings a new perspective to the problem of optimizing health systems, but as a metric, it is only a part of the puzzle. We anticipate that decision makers who use the CAD will likely prioritize resources in a utilitarian fashion. The use and availability of CAD data will also foster economic analysis, which often takes a utilitarian, societal perspective. However, until we see published experience using the metric, it is unclear how it will be practically implemented and how its implementation will influence behavior and outcomes. It is conceivable that in the future, new metrics that account for diminishing returns, difficulties in implementation, or future HAI events due to the spread of pathogens may lead the way toward more optimal prioritization of resources than CAD alone. For those that use the Targeted Assessment for Prevention strategy, the SIR will continue to allow stakeholders to identify problem hospitals and work toward common minimum standards of quality. In summary, the Targeted Assessment for Prevention and other potential strategies to optimize HAI outcomes merit further investigation as ways to improve outcomes in a cost-effective manner.
In this issue of the journal, authors from the US Centers for Disease Control and Prevention (CDC) introduce a new metric, the cumulative attributable difference (CAD), to efficiently prioritize hospitals with large numbers of preventable healthcare-associated infections (HAIs).Reference Soe, Gould, Pollock and Edwards 1 The metric, which represents the excess number of HAIs for a reporting hospital or ward, is easily constructed by selecting a standardized infection ratio (SIR) goal for HAI reduction. It is also easily interpretable—a CAD value greater than zero indicates excess HAIs. Although these are desirable attributes in a metric, they are perhaps not its most crucial attributes. To fully understand a metric’s impact, we need to anticipate how it can contribute to changing behavior and improving outcomes. Here we examine the roles that perspective, value systems, and implementation may play as CAD is used to guide practice.
The CAD metric is part of a larger strategy advocated by the CDC called Targeted Assessment for Prevention, 2 which provides a framework for linking data to action. Within the framework, decision makers target a SIR (eg, 0.75, representing a 25% reduction below the expected value given the subject's patient mix) and use the resulting CAD to identify hospitals or wards with the highest numbers of excess HAI events. The US Department of Health and Human Services has outlined SIR goals for a number of HAI targets. 3 Decision makers then implement interventions in these hospitals and wards to prevent future HAIs. This strategy has been piloted 4 but since there are not yet any published studies, its effects on behavior and outcomes (both actual and intended) are still unknown.
One persistent question is whether a CAD-driven strategy is the most efficient one. Underlying this strategy are 2 implicit assumptions: that targeting fewer hospitals is more cost-effective because a fixed cost must be incurred when intervening at each targeted hospital,Reference Graves, Halton and Jarvis 5 and that the marginal cost of preventing 1 more HAI is constant, whether within the same hospital or across different hospitals. The first assumption is likely true. An intervention usually needs a champion, educational efforts, and a social campaign at each hospital in order to be successful.Reference Soo, Berta and Baker 6 , Reference Stetler, Damschroder, Helfrich and Hagedorn 7 Therefore, selecting a smaller number of hospitals in which to intervene would make sense when the fixed costs associated with each intervention are high.
The second assumption is unlikely to be true because of the law of diminishing marginal returns, which states that, at some point, additional units of an input will yield fewer units of the output.Reference Graves, Halton and Jarvis 5 , Reference Rubin, Jones and Leecaster 8 – Reference Barnett, Page and Campbell 10 In the case of infection control, this means that as HAIs decrease, increasingly greater effort and resources will be needed to yield further reductions in HAIs. From a healthcare network perspective, it is conceivable that it may be more costly to prevent a single additional infection in a large hospital with a moderate HAI rate than in a small hospital with a high HAI rate. As a consequence, the potential impact of an intervention (the count of HAIs prevented) may not correlate with CAD.
A complicating factor is that poorly performing hospitals may be poorly performing because of local difficulties in the implementation of preventive strategies. Thus, even though the law of diminishing returns can help us predict changes in marginal returns at the same hospital, it remains difficult to predict the difference in cost of preventing 1 more infection in a high-CAD hospital versus a low-CAD hospital. More research is needed on this point.
To maximize impact, it will be necessary to predict both the CAD prevented and the costs of a given strategy. Decision makers would then select the strategy with the highest predicted number of prevented CAD for the resources at their disposal. The strategy also has to be dynamic. Even after initially targeting hospitals for intervention, decision makers need to decide when to continue to pour resources into the same hospitals and when to move on to others. This adds to the complexity of finding the ideal solution.
There is 1 additional facet to be considered: the transmissibility of the pathogens that cause HAIs. It may be that addressing HAIs due to multidrug-resistant organisms at the worst hospitals might stem their spread to other hospitals, thereby benefiting the future state of the entire system.Reference Lee, McGlone and Wong 11 For example, a single hospital with an outbreak of carbapenem-resistant Klebsiella might have a low CAD but may also be the source for many more future CAD in larger hospitals. Thus, an intervention to prevent HAIs at this hospital would have additional benefits on other hospitals over time.
The ideal approach to prioritization may require more than rank prioritization by CAD. The sheer number of ways to allocate resources over time makes it a computationally complex problem. When taking a broader system perspective, research using simulation models could be used to identify potential solutions and explore the interaction between fixed intervention costs at multiple hospitals and marginal costs within hospitals, as well as the ripple effects of interventions on neighboring facilities.
Metrics take different forms, such as counts, risks, and rates, and each form can influence practice differently. The CAD is a count metric. If the number of HAI events were to increase by 1 in either a small hospital or a large hospital of equivalent patient mix, the CAD would increase by 1 in each. However, if we were to compare rates, which is a common practice, the small hospital would have a larger increase in the rate because of differences in denominator size. Different denominators reflect different perspectives and can prompt different responses.
Another important aspect of metrics is whether comparisons are absolute or relative. Comparisons of CAD highlight absolute differences, whereas those of SIR highlight relative differences. As noted by Soe et al,Reference Soe, Gould, Pollock and Edwards 1 prioritization driven by SIR (or other metrics used for relative comparisons), as opposed to CAD, targets a greater number of smaller hospitals. These 2 metrics, which contain largely the same information, encode the priorities of 2 different theories of distributive justice: utilitarianism in the case of CAD and prioritarianism in the case of SIR.Reference Scheunemann and White 12
Several stakeholders are interested in preventing HAIs; the 3 most obvious are society, patients, and hospitals. From a societal perspective, it makes sense to act in ways that bring the greatest good to the most people. In this case, society as a whole is considered better off if there are fewer HAIs (utilitarianism). Patients may take a different perspective, however; at the most poorly performing, low-SIR hospitals, patients may feel that their hospitals deserve the most resources (prioritarianism).
In many cases, utilitarian and prioritarian approaches may target the same hospitals, but prioritization by CAD may lead to intervention in a lesser number of larger hospitals, leaving a sizable number of hospitals without an intervention. Patients who use these hospitals would continue to be exposed to risk for a HAI event that may exceed some threshold for “acceptable” HAI risk, if such a threshold existed. Conversely, had the SIR been used to prioritize hospitals for HAI reduction interventions, a greater number of hospitals would have been identified as targets for intervention. Therefore, either more resources would be required for intervention or less resource-intensive interventions would be necessary. The minimum standard of quality would be raised, but it would come at the cost of either a higher overall number of HAIs or a greater amount of resources expended.
It is rare for either utilitarian or prioritarian approaches to be used in their purest forms, which is likely why the CDC advocates for both the CAD and the SIR metrics. On a societal level, it makes sense to use our resources wisely. This makes the use of CAD within a Targeted Assessment for Prevention framework a reasonable overall approach. However, as dictated by the rule-of-rescue,Reference Scheunemann and White 12 a hospital with particularly poor performance (eg, a very high SIR) would still merit intervention, regardless of its CAD.
The CAD brings a new perspective to the problem of optimizing health systems, but as a metric, it is only a part of the puzzle. We anticipate that decision makers who use the CAD will likely prioritize resources in a utilitarian fashion. The use and availability of CAD data will also foster economic analysis, which often takes a utilitarian, societal perspective. However, until we see published experience using the metric, it is unclear how it will be practically implemented and how its implementation will influence behavior and outcomes. It is conceivable that in the future, new metrics that account for diminishing returns, difficulties in implementation, or future HAI events due to the spread of pathogens may lead the way toward more optimal prioritization of resources than CAD alone. For those that use the Targeted Assessment for Prevention strategy, the SIR will continue to allow stakeholders to identify problem hospitals and work toward common minimum standards of quality. In summary, the Targeted Assessment for Prevention and other potential strategies to optimize HAI outcomes merit further investigation as ways to improve outcomes in a cost-effective manner.
Acknowledgments
We are indebted to Barbara Jones, MD, for her input into the ideas discussed.
Financial support. Veterans Affairs Salt Lake City Health Care System.
Potential conflicts of interest. M.M.J. reports that he receives funding from the CDC (SHEPHerD 2013-003). R.E.N. reports that he receives support from the CDC (contract 200-2011-42039) M.A.R. reports that he receives support from the CDC (contract 200-2011-42039).
Disclaimer: The contents do not represent the views of the US Department of Veterans Affairs or the US Government.