Adherence engineering (AE), a conceptual framework introduced by the Center for Human Factors and Patient Safety, applies the knowledge of human behavior-shaping factors to examine non-adherence within a specific task and to develop materials or equipment to increase protocol adherence and reduce human error.Reference Sax, Uckay, Richet, Allegranzi and Pittet 1 , Reference Drews 2 AE focuses on affecting human behavior by implementing factors that shape task performance. For example, AE aims at increasing standardization and reducing variability.Reference Berwick 3 , Reference McGlynn, Asch and Adams 4 This reduces task complexity and task-related cognitive demand, which are both associated with performance breakdowns. AE design principles also suggest facilitation of desirable actions by reducing the associated physical effort so that non-adherent actions require more physical effort than adhering to protocol. We applied the AE conceptual framework to catheter care.
Improper central line maintenance (CLM) can lead to central line-associated bloodstream infections (CLABSIs), which substantially increase mortalityReference Klevens, Edwards and Richards 5 , Reference Richards, Edwards, Culver and Gaynes 6 and cost for caring for critically ill patients in the intensive care unit (ICU).Reference Laupland, Lee, Gregson and Manns 7 CLM is performed once a week in our facility by a registered nurse. This process involves cleaning the insertion site(s) as well as replacing the dressing and cleaning and protecting the needleless injection site(s) by protecting the catheter hubs with caps after thoroughly cleaning them. Properly performed CLM can lead to lower rates of CLABSI.Reference Mermel 8 , Reference O’Grady, Alexander and Dellinger 9 Unfortunately, CLM is often performed in the presence of stress,Reference DeLucia, Ott and Palmieri 10 – Reference Drews, Musters and Samore 12 fatigue,Reference Krueger 13 and multi-tasking or task switching,Reference Grundgeiger and Sanderson 14 , Reference Grundgeiger, Sanderson, MacDougall and Venkatesh 15 all of which are common in the ICU and can lead to significant impairments of task performance.
CLM at our facility involves 27 necessary items, such as sterile gloves, syringes, and catheter and dressing materials. Typically, nurses must gather each of these items separately before performing the task. This preparatory step not only takes time but, due to the large number of items, is prone to be incomplete because of the high likelihood of item omission. While several commercially available kits assemble some or even all of the required items in one place, these kits lack any internal structure or guidance regarding which item is to be used when in the CLM process. Applying AE principles, we developed a CLM kit (Medical Central Line Maintenance Kit; MEDClick) and a guide that provides external cues to support nurses to remember the individual steps of the procedure. MEDClick packages the 27 items by grouping the items into categories based on phases of the CLM procedure, namely preparation, sterile field, and needleless injection site change. In addition, within each of the phases the individual items are packaged following the sequence of the CLM task. A pre-post intervention study evaluating this kit found a 100% reduction in CLABSI over 30 months in the medical and surgical ICUs.Reference Drews, Bakdash and Mallin 16 There is an increased cost for packaging the items together in a kit. However, the prevention of rare, high-cost events, such as CLABSIs, may outweigh these packaging costs, leading to overall cost savings. The objective of this study was to perform an economic analysis that weighs the costs and the clinical benefits of this AE intervention.
METHODS
Overview
To examine the cost-effectiveness of the CLM kit, relative to the standard of care (ie, no CLM kit), we constructed a Markov model using TreeAgePro 2013 (TreeAge Software, Inc., Williamston, MA). The model (depicted in Figure 1) simulated a hypothetical cohort of 45-year-old patients (10,000 first-order simulations) in an ICU with a central line placed. Patients entered the model by having a central line placed on a certain day of the week. Following the distribution of line placements at our facility, we assumed that 10% of the patients had their line placed on a weekend (with an equal probability for Saturday or Sunday) and that the remaining 90% of patients had their line placed during the work week (again, with equal probability for each day Monday through Friday). We also assumed that CLM occurred once a week each Tuesday, which is mandated at our facility as part of the central line dressing protocol. The mean number of central line days per patient was 7.14.Reference Galpern, Guerrero, Tu, Fahoum and Wise 17
Markov model. Note: ICU=intensive care unit, CLABSI=central line-associated bloodstream infection.
The model followed patients through 3 different settings: initial ICU stay, then admission to the general ward, followed by discharge. While in the ICU, patients could develop a CLABSI, which would lead to an increased risk of death and prolonged ICU and general ward stay.
Input Parameters
The key cost inputs to our model were those associated with CLM and CLABSI (Table 1). The cost of the CLM kit ($29.45), which included cost for the kit-specific packaging and arranging the various components, was slightly higher than for all of the individual kit components purchased separately ($21.82). The cost of a CLABSI was incorporated in 2 distinct ways: (1) as an estimate of the total cost of the event ($46,485) and (2) as the excess length of stay (LOS) in the ICU (6.9 days) and the general hospital ward (3.5 days). Both the total cost and excess LOS estimates were drawn from a recent meta-analysis.Reference Zimlichman, Henderson and Tamir 18 We created an estimate of the cost of CLABSI by combining the excess LOS estimates with the cost of care per day in the ICU ($6,288) and in a general hospital ward ($3,822), which were obtained for our local facility. Additionally, we included the cost of a 14-day course of oral vancomycin ($154) when estimating the cost of CLABSI through excess LOS. Our analysis was conducted from the healthcare system perspective, and all costs were converted to 2013 US dollars.
Model Input Parameters
NOTE. CLABSI, central line-associated bloodstream infection; CLM, central line maintenance; ICU, intensive care unit; RR, relative risk; SLCVA, Salt Lake City Department of Veterans Affairs Healthcare System.
a Rate per 1,000 line days.
b For all patients with a central line.
Table 1 also includes the probability inputs for our model. The key probability inputs were the rate of CLABSI (3.1/1,000 patient days) and the impact on this rate of infection due to the CLM kit intervention, which were observed during the pre- and post-implementation period of our study, respectively.Reference Drews, Bakdash and Mallin 16 Additionally, our model included a baseline mortality rate for ICU (13.5%)Reference Angus, Linde-Zwirble and Sirio 19 – Reference Young and Birkmeyer 21 and general ward patients (2%)Reference Hall, Levant and DeFrances 22 and the relative risk of mortality for patients with CLABSI (2.27).Reference Stevens, Geiger, Concannon, Nelson, Brown and Dumyati 23
Outcome Parameters
Our model was designed to compare cost and effectiveness outcomes between the two approaches toward CLM. Cost outcomes for each approach were computed following the 2 methods for computing CLABSI costs mentioned above (overall costs and excess LOS multiplied by a cost per day). Effectiveness outcomes were measured in 2 ways: life years (LYs) and quality-adjusted life years (QALYs). QALYs were constructed by weighting the time spent in ICU, general ward, and post-discharge to account for quality of life in those locations. The utility values used for these weights range from 0 to 1, with 0 representing death and 1 representing perfect health. We assigned a utility value of 0.66 for patients while they were in the ICU.Reference Halton, Cook, Whitby, Paterson and Graves 24 Post-ICU discharge utility values were based on patient age.Reference Gold, Franks, McCoy and Fryback 25 To consider differences in mortality between the CLM kit and no CLM kit approaches, we adopted a lifetime horizon and assumed a life expectancy of 78.7 years. Outcomes occurring in future time periods but related to the initial ICU stay were discounted at a rate of 3%. Discounting is the commonly used method of weighting outcomes based on when they occur so that costs and benefits that occur in different time periods can be directly compared. Following recommended practice, our model did not consider future access to healthcare.Reference Brouwer, Rutten and Koopmanschap 26 Therefore, patients who did not die due to CLABSI were assumed to gain 18.7 discounted QALYs and 22.1 discounted LYs. Because all costs associated with the initial ICU stay were assumed to occur in the first year, we did not discount costs. Incremental outcomes were calculated as the difference between the outcomes from each CLM approach.
Sensitivity Analyses
We ran a number of deterministic sensitivity analyses to explore thresholds of input values at which our results might change. We recognized the assumption that the CLM kit led to a 100% reduction in CLABSIs to be strong, although this assumption is based on our previous prospective study. To address this issue, we varied the reduction in CLABSI from 0% to 100% along with 2 other key parameters (the baseline probability of CLABSI in the ICU and the cost of CLABSI) in separate 3-way sensitivity analyses. In addition, we performed probabilistic sensitivity analyses in which all parameter values were allowed to vary simultaneously. This was done using 10,000 second-order Monte Carlo simulations.
RESULTS
Table 2 shows the results from our base-case analysis, in which we assumed that the CLM kit led to a 100% reduction in CLABSIs and a 50% reduction in CLABSIs. Under both assumptions, the CLM kit strategy was less costly using both cost calculation methods and more effective using both effectiveness measures. When using estimates of the total cost of a CLABSI, on average, the CLM kit approach was $860 and $400 less expensive than the no-kit approach when the CLM kit was assumed to reduce CLABSIs by 100% and 50%, respectively. These cost differences were $643 and $306 when using the excess LOS due to a CLABSI to calculate the cost of the event. The CLM kit also yielded 0.13 more QALYs and 0.15 more LYs compared with the no kit strategy under the assumption of 100% reduction in CLABSI and 0.05 more QALYs and 0.06 more LYs compared with the no kit strategy under the assumption of 50% reduction in CLABSI.
Base-Case Results from Cost-Effectiveness Analysis
NOTE. CLM, central line maintenance; CLABSI, central-line associated bloodstream infection; LOS, length of stay; QALYs, quality-adjusted life-year; LYs, life-years. Incremental cost=cost for CLM kit – cost for no CLM kit; incremental effectiveness=QALYs/LYs for CLM kit – QALYs/LYs for no CLM kit; cost calculation method 1=cost of CLABSI calculated as overall cost due to infection; cost calculation method 2=cost of CLABSI calculated as excess LOS due to CLABSI multiplied by cost per day.
The deterministic sensitivity analyses identified threshold values for the input parameter describing effectiveness of the CLM kit in reducing CLABSI risk for which the CLM kit no longer resulted in cost savings for varying levels of baseline CLABSI risk and cost of CLABSI (Figures 2a–i). In these figures, the curves depict the incremental cost of the CLM kit approach compared with the no-CLM-kit approach; the gray curve represents costs calculated using the excess LOS due to a CLABSI and the black curve represents costs calculated using the total cost of a CLABSI. Each graph depicts these incremental costs across a range of the percentage decrease in CLABSI resulting from use of the CLM kit (range: 0–100% reduction). The calculations for each graph also assumed a baseline rate of CLABSI (1, 2, or 3 per 1,000 line days), total cost of CLABSI ($46,685, $25,000, and $10,000), excess LOS in ICU due to CLABSI (6.9, 4, and 2 days), and excess LOS in general ward due to CLABSI (3.5, 2, and 1 day). The threshold of effectiveness of the CLM kit for which CLM kit reduced overall costs fell as both the baseline probability of CLABSI and cost of CLABSI decreased. With the base-case assumptions for the rate of CLABSI and the impact of CLABSI on cost and LOS, the CLM kit reduced overall costs as long as it reduced the risk of CLABSI by at least 2% when using CLABSI cost calculation method 1 and 7% when using CLABSI cost calculation method 2. These CLM kit effectiveness thresholds increased to 42% and 37%, respectively, when the rate of CLABSI was assumed to be 1/1,000 patient days and when CLABSI resulted in costs of $10,000 or excess LOS of 2 days in the ICU days and 1 day in a general hospital ward.
Deterministic sensitivity analyses of (1) effectiveness of CLM kit in reducing CLABSI, (2) baseline rate of CLABSI, and (3) cost of CLABSI. Note: x-axis=% reduction in CLABSI risk due to CLM kit; y-axis-cost of strategy per patient; black line=cost of CLABSI calculated using total cost of infection, gray line=cost of CLABSI calculated using excess LOS. rate of CLABSI (per 1,000 line days), total cost, excess LOS in ICU, excess LOS in general ward=(a) 3,$46,685,6.9,3.5; (b) 3,$25,000,4,2; (c) 3,$10,000, 2,1; (d) 2, $46.685,6.9,3.5; (e)2,$25,000,4,2; (f) 2,$10,000,2,1; (g) 1,$46,685,6.9,3.5; (h) 1,$25,000, 4,2;(i) 1,$10,000,2,1.
Finally, Figures 3a and 3b present scatterplots on the cost-effectiveness plane of the second-order Monte Carlo simulations from the probabilistic sensitivity analyses, with each point representing the result of 1 simulation. In these figures, the horizontal axis depicts the difference in cost between the 2 strategies and the vertical axis depicts the difference in effectiveness (QALYs, in this case) between the 2 strategies. Each point in these figures indicates that the CLM kit resulted in reduced costs (negative incremental cost). Effectiveness was improved (positive incremental effectiveness) in >97% of the 10,000 simulations.
Scatterplots from probabilistic sensitivity analyses (10,000 2nd order Monte Carlo simulations) (a) Cost=calculated using total CLABSI cost Effectiveness-QALYs. (b) Cost=calculated using excess LOS Effectiveness=QALYs. Note: x-axis=difference in effectiveness between CLM kit and no CLM kit strategy; y-axis=difference in cost between CLM kit and no CLM kit strategy.
DISCUSSION
In this paper, we have described the results of a simulation to compare the cost and effectiveness outcomes from using an AE-designed CLM kit (MEDClick) compared with the standard method of CLM (no kit) in patients with a central line in the ICU. The input parameters for our model were obtained from published studies as well as from a pre/post intervention study that we conducted.Reference Drews, Bakdash and Mallin 16 Our base-case analyses indicated that, while the CLM kit was more expensive per use than performing CLM by gathering each component separately, using the CLM kit resulted in a reduction of rare, but expensive, CLABSIs and in an overall cost savings per patient. This situation, in which one intervention is both less costly and more effective than a comparator, is referred to as dominance. With estimates of cost savings from our analysis ranging from $306 to $860 per patient, a hospital with an average of 500 ICU patients per year could see annual savings of $153,000 to $430,000 by implementing this CLM kit.
It is possible that the published estimates of attributable cost and LOS due to CLABSI that we used as parameters in our model are inflated due to time-dependent bias.Reference Graves, Harbarth, Beyersmann, Barnett, Halton and Cooper 27 To examine how our results would change if the cost of CLABSI was much lower, we conducted several deterministic sensitivity analyses. In these sensitivity analyses, we also relaxed the assumption that use of the CLM kit would reduce the risk of CLABSI completely and reduced the underlying rate of CLABSI. The results of these simulations showed that the cost savings from the CLM kit were negatively associated with each of these three variables (the underlying CLABSI rate, the cost of CLABSI, and the effectiveness of the kit in reducing the risk of CLABSI). However, even with an underlying rate of 1/1,000 line days and the lowest assumed cost ($10,000) and excess LOS (ICU: 2, general ward: 1) associated with CLABSI, the CLM kit still reduced overall costs as long as it lowered the risk of CLABSI by at least 43%.
While this is the first study to conduct an economic analysis of an AE-designed CLM kit to reduce CLABSI, other economic analyses have considered other CLABSI-focused interventions and have found dominance in their interventions as well. For example, Cooper et alReference Cooper, Frampton and Harris 28 found dominance when evaluating the cost-effectiveness of a bundle that included education, optimal hand hygiene, chlorhexidine skin antisepsis, barrier precautions, optimal insertion site, and prompt catheter removal interventions to reduce CLABSI in ICUs compared with current clinical practice using inputs specific to the United Kingdom. Halton et alReference Halton, Cook, Whitby, Paterson and Graves 24 found a similar result in a cost-effectiveness analysis of antimicrobial-coated central venous catheters using data from Australian ICU patients. Finally, Ye et alReference Ye, Rupnow, Bastide, Lafuma, Ovington and Jarvis 29 estimated that a hypothetical 400-bed hospital in the United States could save nearly $900,000 per year by adding chlorhexidine gluconate-impregnated sponge dressings to standard care (skin preparation and transparent film dressing) to reduce CLABSIs.
A recent survey of National Healthcare Safety Network hospitals indicated that, while the vast majority of ICUs had written CLABSI prevention policies, adherence to these policies ranged from 37% to 71%.Reference Stone, Pogorzelska-Maziarz and Herzig 30 Additional studies have found that near perfect adherence (>95%) to infection prevention policies is necessary for reducing infection rates.Reference Furuya, Dick, Perencevich, Pogorzelska, Goldmann and Stone 31 , Reference Pogorzelska, Stone and Furuya 32 Other studies have indicated that there is a huge gap between ICU providers’ perceptions of adherence and actual adherence to guidelines. It is clear that despite awareness of the importance of adhering to best practices, many obstacles to successful implementation of these practices remain. By focusing on factors that shape clinician behavior, our results suggest that AE can play a potentially important role in improving the safety of hospitalized patients by removing some of these obstacles.
This study had several limitations. First, the parameter dictating the impact of the CLM kit on both costs and effectiveness (the effectiveness of the kit in reducing CLABSIs) was estimated from a study that used a pre/post design. With no contemporary group of control patients, it is not possible to separate the effect of the kit from the effect of temporal events that may also have led to fewer CLABSIs such as other improvements in infection prevention. However, we conducted deterministic sensitivity analyses to explore the impact that modifications to this effectiveness estimate would make to the overall cost-effectiveness of the kit intervention. Second, the results of this study are based on a single clinical study, limiting the generalizability of the results. Other factors, eg, safety climate, unit morale, etc, can also have an impact on the success of an intervention like the one described. Clearly, there is a need to improve our understanding of the additional factors. In 2001, DrewsReference Drews 33 described a model of human error, which specifies some of these potential influences. Finally, our model focused only on CLABSI in the ICU without considering CLABSI in the acute care setting.
In conclusion, the present work demonstrates that an AE intervention to streamline the CLM process can both improve patient outcomes and can lower costs.
Acknowledgments
The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government.
Financial support: This work was supported by the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Health Services Research and Development Service (grant no. CDA 11–210) and the Department of Veterans Affairs Center for Patient Safety (grant no. XVA91-006).
Potential conflicts of interest: All authors report no conflicts of interest relevant to this article.
