Septic shock remains the most common reason for admission to the intensive care unit (ICU) and the main cause of long-term morbidity and mortality despite improvement in treatments (Reference Adhikari, Fowler, Bhagwanjee and Rubenfeld1). Among survivors, it often leads to persistent physical and mental health problems and poor health-related quality of life that can last years following hospital discharge (Reference Angus and van der Poll2). Septic shock is also one of the most expensive conditions to manage; it accounted for over 6 percent of the total cost of all hospitalizations in the United States in 2013 (Reference Torio and Moore3).
Mesenchymal stem cells (MSCs) are a novel therapeutic tool that can modulate uncontrolled immune activation and reduce multi-organ dysfunction thereby having the potential to improve survival in the event of septic shock. In preclinical settings with animal sepsis models, MSC treatment for sepsis was found to significantly reduce mortality by 73 percent (odds ratio .27, 95 percent confidence interval .18–.40) (Reference Lalu, Sullivan, Mei, Moher, Straus and Fergusson4). Early phase clinical trials have shown that MSCs are safe, and none of these trials report MSC transfusion-associated adverse events (AEs) in acute respiratory distress syndrome (Reference Zheng, Huang, Tong, Shu, Hu and Ge5–Reference Wilson, Liu, Zhuo, Caballero, McMillan and Fang6) and severe sepsis patients (Reference Galstian, Parovichnikova, Makarova, Kuzmina, Troitskaya and Gemdzhian7). A recent study has shown that the use of the maximum feasible dose of MSCs, up to 3.0 × 106 cell/kg, was safe in septic shock patients, and there was no occurrence of MSC transfusion-associated events, deaths within 24 h of cessation of the MSC infusion, or serious and unexpected AEs (Reference McIntyre, Stewart, Mei, Courtman, Watpool and Granton8).
Although stem cell therapy is a promising intervention, cost considerations are increasingly important in a limited resource environment. Evidence to date has shown that several regenerative medicine therapies experienced delayed commercialization and negative recommendations for reimbursement and clinical adoption partly due to high costs and poor cost-effectiveness profiles (Reference McAteer, Cosh, Freeman, Pandit, Wood and Lilford9–13). Significant resources can be sunk into researching and developing advanced interventions that are ultimately unproductive due to overall health system costs that researchers are unaware of and/or have no control over.
Early health economic evaluation is a form of economic evaluation performed ex ante to the technology's full development and implementation. It can help inform the commercial viability of novel therapies and the design of research and development activities. In other words, early economic evaluation can estimate the potential value of the technology in a resource-constrained environment where stakeholders need to consider the benefits of a new treatment according to the opportunity cost of financing it over alternative technologies. The results may lead to the go/no-go decision. A value of information analysis conducted as part of the early evaluation can also inform the design and conduct of future research on novel therapies.
We performed an early economic evaluation to: (i) estimate the therapeutic headroom and maximum reimbursable price (MRP) of MSC therapy in septic shock patients; (ii) determine factors that determine the cost-effectiveness of the MSC therapy for septic shock patients; and (iii) assess whether more research on MSC therapy is required and identify key variables that should be focused in future trials to ensure that MSC therapy is economically competitive in a resource-constrained environment.
Methods
We performed an early economic evaluation from a perspective of Canada's publicly funded healthcare system. The early economic evaluation approach included a therapeutic headroom analysis, MRP estimate, and a value of information analysis. The therapeutic headroom represents the magnitude of improvement on patient outcomes compared to other potential treatments. The headroom provides the maximum incremental cost of a new intervention per patient by estimating the net monetary benefit expected from a new intervention that would be realized by the healthcare system (Equation 1). The headroom analysis, however, assumes that the new intervention is provided at no cost or has external effects to other areas of health care (Reference Girling, Lilford, Cole and Young14). Improved patient outcomes are likely to have cost implications in the form of reduced hospital admissions, reduced risk of costly complications and follow ups, or changes in long-term costs among survivors. When incorporating this cost component, we calculated the MRP of the MSC therapy per patient as the sum of the therapeutic headroom and the expected cost-savings due to MSC therapy. Since we do not know the true cost of providing MSC therapy, the MRP of the MSC therapy sets a ceiling on the per-patient unit cost of MSC therapy that would still be considered cost-effective based on a demand-side willingness to pay threshold (λ) which represents an estimate of what the health system is willing to pay for the health benefit (Reference Vallejo-Torres, García-Lorenzo, Castilla, Valcárcel-Nazco, García-Pérez and Linertová15).
$${\rm Therapeutic\ Headroom\ } = \lambda \ast \Delta {\rm QALY}$$λ denotes the maximum willingness to pay by the health system per QALY gained; QALY denotes quality-adjusted life year.
We used a discrete time Markov model to estimate outcomes over a septic shock patients' lifetime. A base case model represents the current state of care, followed by a series of scenario analyses in which the impact of MSC therapy for patients are asserted to determine how such a treatment would change patient outcomes and health system costs. The treatment effect of MSC therapy is still unknown so we modeled a series of treatment effects and plotted a frontier at which the minimum treatment effect would be cost-effective. The value of information analysis provides a more detailed understanding of the parameters which contribute most to the uncertainty over the cost-effectiveness of MSC therapy; and therefore, have the greatest potential value from further research.
Setting and study population
Our target population included Canadian patients aged 66 years old when diagnosed with septic shock. We used 66-year-olds since this is the median age of patients with sepsis in Canada (Reference Husak, Marcuzzi, Herring, Wen, Capan and Cernat16) and it allowed for a clearer description of the burden for this cohort of patients rather than relying on age-adjusted effects.
Study design
The Markov model consisted of five mutually exclusive health states based on the location of care including hospital-ICU, hospital-inpatient ward, rehabilitation and other outpatient care facilities, home, and death. The model structure was developed in collaboration with an ICU intensivist. The model simulated the cost and health outcomes of a cohort of patients with septic shock aged 66 years old from their index ICU admission over their lifetime (Supplementary Figure 1). Patients started in either the ICU or inpatient ward, with the majority (average 72 percent) in the ICU, consistent with both hospital administrative data (17) and previous literature (Reference Husak, Marcuzzi, Herring, Wen, Capan and Cernat16).
Each of these care locations had associated costs and health utilities that accumulate daily. Each day patients may transit to another state or remain in their current state based on daily transition probabilities. These probabilities were derived from the published literature focusing on septic shock patients (Reference Dick, Hangsheng, Zwanziger, Perencevich, Yoko Furuya and Larson18). Septic shock can impact a patient in a wide variety of ways, many of which are severely debilitating. We captured all the effects of septic shock according to the trajectory of care provided to patients. This had the advantage that literature on sepsis patients is largely based on tracking their needs in the short term (Reference Dick, Hangsheng, Zwanziger, Perencevich, Yoko Furuya and Larson18) so the available data was better suited to this design. Hospital readmissions were excluded from the model since it is a significant uncertainty how readmission rates would change based on the MSC therapy.
Intervention and comparator
The intervention of interest was a single dose of MSC therapy intravenous infusion given at the time of diagnosis of septic shock (i.e. on the first day of the model representing septic shock diagnosis). Our study modeled providing MSC therapy at the index stage as a supplement to usual care. Usual care constituted patients receiving initial resuscitation followed by administration of intravenous antimicrobials. Over subsequent days patients may receive additional antimicrobial treatments and hemodynamic support based on the determined source of sepsis and patient response (Reference Dellinger, Levy, Carlet, Bion, Parker and Jaeschke19).
Input parameters
Daily transition probabilities for patients moving between hospital facilities, discharge, and mortality rates for the ICU and hospital were extracted from The Ottawa Hospital data from 2012 to 2017 (17). The Ottawa Hospital is a tertiary care teaching institution and is the largest referral center in eastern Ontario. A multinomial discrete time hazard model was used to account for competing events of facility transition, mortality, and discharge to estimate daily transition probabilities. Table 1 presents the cumulative transition probability over the stated time interval for reference. Postdischarge outcomes, including discharge from rehabilitation facilities, and long-term mortality were derived from the best available literature on outcomes for sepsis and septic shock patients (Reference Cheung, Tansey, Tomlinson, Diaz-Granados, Matté and Barr20–Reference Lee25). All-cause mortality was derived from general population mortality rates according to Statistics Canada (26). When cause-specific mortality matched that of age-adjusted mortality rate of the cohort, general population mortality was used.
Input parameters
The costs associated with the immediate treatment of patients at the acute moment of septic shock were excluded from the model since these costs are incurred regardless of our modeled intervention. The per-day costs related to inpatient admission to the ICU or other location were included since these may vary based on the duration of stay in the location. The hospital costs were based on The Ottawa Hospital's average daily cost of patients in the ICU or hospital and average emergency department cost among patients who were transferred to admitting. The daily cost of long-term care, rehabilitation facilities, and home care costs were derived from a retrospective cohort study of long-term health outcomes and costs of sepsis patients in Ontario (Reference Farrah, McIntyre, Coyle, Talarico and Thavorn27). Costs are expressed in 2019 Canadian dollars.
Health utility values were based on a 2010 study that surveyed patients who were discharged from the ICU and measured average utility according to location (hospital, rehabilitation, long-term care, or home care) and time since index discharge (Reference Cuthbertson, Roughton, Jenkinson, Maclennan and Vale21). The study used the EuroQOL-5D (EQ-5D)-3L quality of life assessment tool measured at several stages over 5 years. It reported the utilities stratified by age group, for which we used the results for the age group >64 years of age at discharge. Our model assumed that after 5 years, the average utility score plateaus for those living at home and at rehabilitation care facilities, based on the plateauing trend already observed by year five in the study. Our model applies the same assumption as the cohort study that patient quality of life during time in the ICU is zero (no different than death) (Reference Cheung, Tansey, Tomlinson, Diaz-Granados, Matté and Barr20).
Analysis
In our base case, we asserted the effect of MSC therapy on the in-hospital mortality rate. As it is unclear how MSC therapy may affect the rate of recovery from sepsis, we conducted a scenario analysis by considering the effects of MSC therapy on the increased rate of ICU/hospital discharge (proxy for increased rate of improvement). The optimistic plausible ranges of effectiveness were guided by a clinical expert in MSC therapy for septic shock patients (LM). We calculated the aggregate costs and QALYs for patients over the 20-year timeframe and summarized as expected annual cost of care and annual QALY. Costs and QALYs were discounted annually at a rate of 1.5 percent, as recommended by the Canada's Economic Evaluation Guideline (28). The results are shown as a cost-effectiveness frontier, which plots the willingness to pay threshold per QALY gained against the minimum improvement in clinical outcome that needs to be achieved for the intervention to be considered cost-effective.
All outcomes were obtained through probabilistic analysis using a Monte Carlo Simulation method with 10,000 iterations. All uncertain parameters were represented by appropriate probability distributions incorporating their associated uncertainty. Since there are potential interactions of the two-treatment effect on QALY and cost outcomes, we bootstrapped all values in the Monte Carlo to have a random value of risk improvement (rate reduction in the mortality and rate increase in discharge) sampled from 1 to 100 percent, including randomly assigned runs in which one or more treatment effect(s) were deactivated. Using the simulated outcomes data set, we ran a generalized linear mixed regression on all treatment effects to determine the independent effect on the net monetary benefit of each treatment per percent absolute risk improvement while capturing structural variance in the model in the error term.
Finally, the results of the probabilistic sensitivity analysis were used to estimate expected value of information (EVPI) and expect the partial value of information (EVPPI). Value of Information (VOI) is an analytical approach that allows us to determine the cost of uncertainty and to prioritize future research by identifying the areas where new research will lead to reduction in the current level of uncertainty (Reference Briggs, Claxton and Sculpher29). EVPPI allows identification of which parameters have the highest information value; that is, contribute most to the uncertainty over which treatment alternative is cost-effective. Calculating the VOI requires a known price for MSC therapy to appropriately quantify the total possible losses from uncertainty. Consequently, we performed VOI where the price of MSC therapy was set at different points in order to describe the nature of model uncertainty as a function of MSC price. For the EVPPI, we identified those parameters contributing most to decision uncertainty.
Each of the treatment effects was also varied to determine whether uncertainty around the actual treatment effect would be a driver of whether MSC therapy will be cost-effective or if even small positive impacts would prove cost-effective. Due to the paucity of evidence, the treatment effects were varied arbitrarily using a uniform distribution between 1 and 10 percent absolute change.
In order to validate our base case model, we compared our results to a Canadian economic evaluation that provides summary results of their observational study of severe sepsis patients (Reference Linder, Guh, Boyd, Walley, Anis and Russell30). The study did not provide sufficient data for us to use in our model. However, it does offer aggregate information on their control cohort that we will use to determine if our base case reasonably matches their findings. Long-term outcomes of our model cannot be directly validated as there have been no reliable long-term observational or trial studies in Canada.
Results
Our base case model estimates the average hospital length of stay for a septic shock patient to be 29 days (95 percent CI 24–61 days), with the average health system cost being $51,877 (95 percent CI $43,990–$121,100) for the index admission. We estimate the average life expectancy following a septic shock incident for a 66-year-old patient is 12 years, with 36 percent of patients dying prior to discharge from the hospital (see Supplementary Figure 2). This is closely comparable to the previous economic evaluation that reported 39.1 percent before discharge mortality for the 60–79 years old cohort (Reference Linder, Guh, Boyd, Walley, Anis and Russell30). Our cost per patient calculation is higher than the previous study (adjusted to 2019 Canadian dollars) but within their 95 percent confidence interval. The average cost per patient (including nonsurvivors) for the first year following the septic shock incident is $52,903. The costs after 5, 10, 20 years, and lifetime is $57,172, $62,256, and $74,596, respectively. The initial hospitalization makes up nearly the entire average cost in the first year, and represents over 71 percent of total expected costs incurred by the average 66-year-old septic shock patient.
Table 2 shows the therapeutic headroom of improving treatment outcomes using MSC therapy according to what percent absolute change in mortality or discharge rates. Headroom was estimated using a willingness to pay per QALY gained threshold of $50,000, $100,000, and $150,000. A 5 percent permanent reduction in absolute cause-specific in-hospital mortality is estimated to increase average patient life expectancy by roughly .6 years (7 months), which amounts to .42 QALY gained. Depending on the cost-effectiveness threshold we consider, these QALY gains translate to MSC therapy providing value to the health system of $20,941, $41,882, or $62,823, respectively. The therapeutic headroom for discharge outcomes is expectedly far less dramatic since any improvements would be due to postdischarge death rates being lower than that for patients still in hospital. Our model assumption would be that discharge from hospital is indicative of improved patient health overall.
Therapeutic headroom and maximum reimbursable price of MSC therapy
Table 2 also presents the MRP of MSC therapy, which incorporates any added costs or cost savings resulting from the therapy to the headroom calculation. We find that improved mortality outcomes are associated with moderately higher average patient costs, since a higher proportion of patients surviving longer mean a larger proportion of patients staying in hospital and being discharged to incur additional rehabilitation and/or home care costs. The increase in average patient care costs decreases the MRP for MSC therapy mildly but there is still a significant improvement in patient quality of life to make MSC therapy economically feasible. Both treatment effects for ICU and hospital discharge result in cost savings that increase the MRP for MSC therapy. Figure 1 presents the expected MRP of MSC therapy according to treatment effect with 95 percent confidence intervals based on the Monte Carlo probabilistic analysis. Improvements in mortality rate have the highest benefit per marginal percent improvement, though there is a large amount of uncertainty around this analysis as a consequence of wide range in baseline mortality risk (this will be discussed further in the value of information results). Reductions in the duration of ICU or hospital stay—cost proxies for improved morbidity outcomes—are roughly equivalent in expected benefit per marginal improvement, with relatively little uncertainty around these estimates due to fairly small standard errors around our cost parameters. Combining all three effects indicates the highest MRP but it is notable that, due to the interactions between states, the expected MRP is not a compounded sum of the three discrete effects.
Maximum reimbursable price, by percent improvement in patient outcomes (95 percent CI).
Table 3 displays the MRP of MSC therapy wherein we incorporate more than one treatment effect. For simplicity, we combine the effects of MSC therapy on ICU and hospital discharge so that they are always the same, and combine these effects with that of mortality rates. We find that the combined net benefits of interacting treatment effects are not perfectly additive; there is some competing risk between mortality and discharge playing out in the model that means the net benefit of combined treatment effects is less than the sum of each solo treatment effect, however it is not a large effect. Average age of the sepsis patient has a significant effect on the expected therapeutic headroom of MSC therapy. We altered the average age of the cohort being run through the Markov model, with the only adjustment being a proportional change in expected cause-specific mortality that matches that of previous studies (Reference Linder, Guh, Boyd, Walley, Anis and Russell30). We found that, compared to the base case headroom of $20,941 for the average 66-year-old patient, the age groups 70, 60, and 50 had an expected therapeutic headroom of $17,002, $25,057, and $32,481, respectively.
Maximum reimbursable price of combined treatment effects
The results of the value of information analysis verify there is a significant amount of uncertainty that is sufficient to merit additional research into septic shock and the relative effectiveness of MSC therapy. The expected value of perfect information (Supplementary Figure 3) finds that when we assign no cost to MSC therapy, the average patient EVPI is $115 and decreased when the willingness to pay threshold was greater than $11,000 per QALY gained. Based on a willingness to pay threshold of $50,000 per QALY and the annual prevalence of sepsis including septic shock (.064 percent) in hospitalized Canadian patients (31), the value for having perfect information on all factors that influence the treatment choice for patients with septic shock was estimated to be $24,721,059 over 10 years.
When the cost of MSC therapy was assumed to be $10,000 or $50,000, the EVPI increased to $955 and $4386, respectively, suggesting that the value of information increases with the higher price of MSC therapy. The EVPPI analysis found that the relative reduction in in-hospital mortality as a result of MSC therapy, baseline septic shock mortality, and the utility score associated with hospital admission introduced the highest degree of uncertainty (see Supplementary Figure 4). The EVPPI results demonstrate the importance of the wide confidence intervals we also observe in the sampled net benefit results from the probabilistic analysis (see Figure 1).
Discussion
An early health economic evaluation aims to provide guidance to researchers and product developers whether there is sufficient economic evidence to invest in interventions. If the outcome of the model showed an unfeasibly small therapeutic headroom and MRP, then the take away is that further investment is unadvised. Our study highlights that MSC therapy can be priced between $16,563 and $101,450 and still be cost-effective from the Canadian public payer perspective. If the price of the MSC therapy is within the range, our results suggest that the product development is worth proceeding. Furthermore, our findings identify the relative effectiveness of MSC therapy on mortality reduction compared to usual care as an important source of uncertainty; this variable should be prioritized in future research.
The primary treatment effect of interest, in-hospital mortality, had the initially surprising result of having the narrowest therapeutic headroom of the outcomes measured. Although it remains unknown whether MSC therapy improves the rate of recovery from septic shock, our early economic model suggests that changes in ICU and hospital discharge rates resulting from MSC therapy have a lower MRP because they drive both a reduction in cost and an increase in average QALYs. In comparison, while the in-hospital mortality rate improvement has the largest impact on QALYs gained, the average cost of care per patient actually goes up due to higher survivor rates. This consequence from increased survivor rates drives the cost-frontier up.
While we calculate the MRP and present a cost-effectiveness frontier based on expected changes in QALY and costs, this study does not intend to estimate a traditional incremental cost-effectiveness ratio since the actual treatment effect of MSC therapy is unknown. The cost-effectiveness frontier measures the likelihood that a treatment effect is going to be cost-effective based on a given willingness to pay per QALY. The probability is based on both the attributable mean QALYs gained and costs incurred (or saved) per unit of improvement (in this case, percent change in absolute risk) adjusted for the structural uncertainty within the model itself. For an early health economic evaluation, a cost-effectiveness frontier is best paired with the value of information analysis since the EVPI takes a closer look at the structural uncertainty of the model.
The mathematics behind the bootstrapped treatment effect calculation approach is not dissimilar from that of the expected value of information analysis. The primary difference is that while we hold constant the value of one parameter in an EVPI, we bootstrapped the treatment effect parameters to have a randomly assigned value that we run a linear regression on so that any variance we see in the QALY and cost outcomes we can assign to the treatment parameter outcome. If the treatment effect value is poorly associated with the variance in the outcomes, then the treatment effect will be nonsignificant and we can expect the error term to have a higher value. In our case, all three treatment effects were significant though we also only tested three treatment effects that we expected to be major drivers of cost-effectiveness.
This early health economic evaluation is based on secondary data sources that were combined or imputed and required several modeler assumptions. While we can test the robustness of these imputations and assumptions as part of the sensitivity analysis and EVPI, it is still important to identify limitations in interpretation and scope of the model. Our model parameters are derived from the best available data, which often does not differentiate between septic shock and sepsis patients. The transition probabilities may be underestimated as septic shock patients would have worse health outcomes and need more health services than sepsis. This means that any treatment that improves immediate and/or long-term outcomes will have an even greater effect on septic shock patients given their baseline average QALYs will be even lower. However, it also introduces added value to partially perfect information not captured in our value of information analysis. More comprehensive data on septic shock patients' outcomes and health services consumed will add significant value to this model and decision-makers' understanding of the impact of sepsis. Due to the lack of evidence on the effect of MSC therapy on hospital re-admissions among septic shock patients, it remains unknown how reduction in hospital remissions may affect the therapeutic headroom and MRPs. It is conceivable that the economic viability of MSC therapy for septic shock patients may be improved if hospital readmissions were considered. Our hypothesis is supported by a systematic review of MSC therapy in patients with systolic heart failure which reported that MSC therapy was significantly associated with a 34 percent reduction in the incidence of readmission (RR [CI] = .66 [.51–.85], p = .001) (Reference Fan, Huang, Chen, Xia, Chen and Lu32).
By specifying 66-year-olds as our cohort of interest, we greatly decrease the amount of data necessary to parameterize the model. It also lets us avoid age-adjustment effect calculations throughout the model which can be computationally challenging and demand more assumptions in the model given the limited detail in secondary data sources. The disadvantage of this approach is that it reduces the generalizability of the model to any septic shock patients. However, the start age used in our study is comparable to the average age of sepsis and septic shock patients reported in the published studies (Reference Shankar-Hari, Harrison, Rubenfeld and Rowan33–Reference Cox, Carson, Lindquist, Olsen, Govert and Chelluri35).
Conclusion
The MRPs for MSC therapy for patients with septic shock has been optimistically estimated to range between $16,563 and $101,450, depending on the extent by which the MSC therapy can reduce in-hospital mortality and/or increase hospital discharge rates. Our results indicate the potential for commercial viability of MSC therapy and room for a premium price. It should be noted that this study intends to demonstrate the usefulness of a health economic evaluation at an early stage of a product development. A robust health economic model should be capable of simulating the expected effect of an individual or an interacting set of interventions on the base case disease trajectory; effectively creating a cost-effectiveness frontier for the level of treatment effect an intervention has to reach before it is graded as cost-effective. A decision maker should be able to set research priorities based on types of interventions that have the highest chances of being cost-effective. Additionally, a researcher or product developer should be able to determine the clinical effectiveness “threshold” they must overcome to be reasonably confident that the intervention will ultimately be graded cost-effective, all before their intervention is even designed.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462320000781.
Financial support
This work was supported by the Ontario Institute for Regenerative Medicine and Stem Cell Network.
Conflict of interest
The authors have no conflicts of interest to report.
