Medical imaging is confronting seemingly contradictory challenges. On one hand, innovation in diagnostics is set to accelerate with personalized medicine (Reference Nunn1;Reference Hillman and Goldsmith2). On the other, value of imaging is being questioned in many clinical settings (3–Reference Winter and Ray7), payment reductions and usage controls are effectively limiting use of imaging (Reference Levin, Rao, Parker, Frangos and Sunshine8), and reservations about appropriateness of investing in new research and development activities are emerging (Reference Hillman and Goldsmith2). Within these challenges lies an opportunity.
Economic modeling studies conducted early in a product life cycle can be used to guide the development of diagnostics which fulfill a clearly defined clinical need and provide value in a resource-constrained healthcare environment. Integrating clinical and economic information to project the impact of a new diagnostic test through modeling can inform how well-positioned a technology may be for implementation, and the potential desirability of investing in a given technology. Herein, we present the results of an early cycle economic evaluation focused on using diagnostic imaging to improve treatment decision making for men with biochemical recurrence after radical prostatectomy (RP) for prostate cancer (PCa).
Approximately 30 percent of the 90,000 men undergoing RP in the United States each year will experience a cancer recurrence during his lifetime (Reference Caire, Sun and Ode9). Asymptomatic men with low but detectable prostate specific antigen (PSA) elevation following RP present a diagnostic and therapeutic dilemma to the clinician, because site of recurrence (local versus metastatic) cannot be determined well by currently-available diagnostic modalities. Often, these men receive costly, burdensome, and potentially damaging salvage radiation therapy (RT) to the pelvis, but only approximately 50 percent have a durable response, suggesting small volume metastatic disease may have been responsible for PSA elevation (Reference Stephenson, Shariat and Zelefsky10). Therefore, up to half of men treated with salvage RT after RP are subjected to cost and harms of therapy without benefit, and potentially delaying treatment of their metastatic disease. Consequently, a new PCa specific imaging technology capable of identifying residual localized disease versus small volume metastatic disease, as a means of more accurately guiding therapy decisions, could be of enormous clinical benefit. As a guide to diagnostic development efforts, the objective of this study was to assess potential cost-effectiveness of using a novel diagnostic test in this way.
MATERIALS AND METHODS
In previous work, we have described our stakeholder-driven approach to early cycle economic evaluation (Reference Bensink, Wyatt and Musa11). Unlike other approaches (Reference Sculpher, Drummond and Buxton12–Reference Vallejo-Torres, Steuten and Buxton14), which generally start with a given technology, our approach starts with a current disease management paradigm and then identifies clinically and economically important needs within that paradigm from multiple stakeholder perspectives. These needs are then prioritized by the expert group and provided to the industry partner involved in the process, in this case GE healthcare, for alignment of needs with products in development. This allows developers to match clinical needs to pipeline products that are potential diagnostic solutions to appropriately target investment.
For this analysis, we convened a workshop with six experts, three with PCa clinical expertise and three with payer expertise. The focus of the workshop was identification of areas of high clinical need in PCa. Discussions with the six attending advisors centered on the humanistic and economic burden of PCa from screening and diagnosis through treatment, surveillance, and end-of-life care. Ten areas of high need in PCa were identified and prioritized by the advisors and provided to our industry partner for matching. Internal review of the stakeholder's priority list identified a PCa specific functional imaging technology from a portfolio of products in early development as a possible solution to some of the pressing clinical problems in PCa. From among several clinical scenarios in which a novel imaging technology may be of use, the advisors proposed to model the specific scenario of distinguishing between local and metastatic recurrence after surgery, because it represents a common clinical scenario in which decisions are made with imperfect information, with substantial downstream impact on costs, quality of life, and cancer outcome.
The imaging technology is an investigational positron emission tomography (PET) radiotracer being studied in the staging and re-staging of patients with PCa. Anti-1-amino-3-[18F]fluorocyclobutane-1-carboxylic acid (anti-3-[18F]FACBC) is a synthetic amino acid analog with little renal excretion, which avoids the problem encountered when using the renally excreted radiotracer of traditional PET scans for malignancies that lie within the urinary tract. Small scale studies of the technology in development have shown the potential for superior performance to existing imaging alternatives (Reference Schuster, Nieh and Jani15). For extra-prostatic disease, sensitivity has been reported from 55 percent to 100 percent, and specificity from 91 percent to 100 percent. This option was presented back to members of the original stakeholder group, and was received favorably, so economic modeling was undertaken.
Economic Modeling
We used the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) guideline to ensure our methodology was thorough (Reference Husereau, Drummond and Petrou16). We developed a health economic model using Microsoft Excel (2010) to estimate incremental impact on costs and quality adjusted life-years (QALYs) of two alternative strategies: (i) using the new diagnostic test to guide therapy versus (ii) usual care, a combination of CT, MRI, and bone scan used to guide therapy.
The analysis focuses on U.S. practice and uses the perspective of the U.S. healthcare system. We selected the population of men aged 55–74 years because approximately 2/3 of PCa diagnoses affect men in this age group, and salvage RT is not common among men older than 74 (http://seer.cancer.gov/statfacts/html/prost.html). The cycle length for the underlying Markov model is 1 year and the time horizon of the model is the lifetime of recurrent PCa patients; the model cycles for approximately 35 years until the entire cohort resides in the death state.
Model Structure
The model structure has two components: a simple decision tree that is based on clinical pathways asymptomatic PCa patients traverse when their PSA is elevated after primary surgical therapy (Figure 1); and a Markov model, based on the natural history of recurrent PCa (Figure 2).
Decision tree for diagnostic work-up of post-prostatectomy patients who are asymptomatic and have a rising PSA along with the outcomes modeled.
Recurrent prostate cancer Markov diagram.
In our model, a positive test result is considered indicative of metastatic recurrence. False negatives and false positives, for the new diagnostic test and current standard diagnostic work-up, will result in incorrect treatment decisions thus impacting patient outcomes. The aim of the new diagnostic test is to more accurately identify metastatic recurrence to guide appropriate treatment.
In the standard work-up arm of the decision tree, men can receive a variety of tests which may include a bone scan (estimated usage 85 percent, varied from 80–90 percent in sensitivity analysis), a CT scan (50 percent, varied from 33–66 percent), and/or an MRI (50 percent, varied from 33–66 percent), to determine site of recurrence. Men are subsequently presumed to have local or metastatic disease depending on outcome of imaging studies. In the comparator arm of the model, men are given the novel diagnostic test which has better performance characteristics than current standard work-up. Each testing strategy (standard work-up or new imaging) results in four groups of patients: (i) men diagnosed with residual localized disease who truly have residual localized disease “local (true)”; (ii) men diagnosed with residual localized disease who actually have metastatic recurrence “local (false)”; (iii) men diagnosed with metastatic recurrence who truly have metastatic recurrence “metastatic (true)”; and (iv) men diagnosed with metastatic recurrence who actually have residual localized disease “metastatic (false)”. The costs, survival, and quality of life consequences for these four groups are outlined in Figure 1.
Once a patient is placed into one of these four groups, a seven state Markov process begins (Figure 2). In the Markov model, men diagnosed with residual localized disease enter a “local treatment” state where they accrue the survival, negative impact on quality of life (disutility) and costs associated with salvage radiotherapy and associated side effects of therapy (bowel problems, impotence and incontinence) (Reference Stewart, Lenert, Bhatnagar and Kaplan17;Reference Trock, Han and Freedland18). Men diagnosed with metastatic recurrence enter a “metastatic treatment” state where they accrue the survival, disutility and costs associated with hormone therapy (Reference Lu-Yao, Moore, Oleynick, DiPaola and Yao19). Men who complete a year of active treatment (one cycle of the Markov model) enter a continuing state—“metastatic continuing” or “local continuing” where they accrue higher survival, higher quality of life, and lower costs associated with continuing care (Reference Stewart, Lenert, Bhatnagar and Kaplan17). Men in “local treatment” or “local continuing” states can progress to metastatic disease or they can die from other causes. Similarly, men in “metastatic treatment” or “metastatic continuing” states can die from their disease or die from other causes.
To account for different costs and quality of life associated with death from metastatic PCa and other causes of death, patients who transition to the death state first enter a terminal state where they accrue corresponding costs and quality of life for the last year of life (Reference Penson, Moul and Evans20). To account for the situation where men (incorrectly identified as having local recurrence) progress from local disease to metastatic disease and subsequently die from metastatic cancer within the first few years following diagnosis (within the Markov model's first cycles), a small number of men transition directly to “terminal prostate” state (included as dotted lines in Figure 2). These men accrue costs associated with terminal local PCa care (Reference Penson, Moul and Evans20). They do not die from local disease but instead incur different costs than men who transition from metastatic states to terminal states as they transition to the terminal state within the first years post-treatment.
Model Parameter Estimates
Test performance
Performance characteristics of the novel diagnostic test were provided by our industry partner; 75 percent sensitivity, 90 percent specificity, and a cost of USD 2,500 (Reference Schuster, Nieh and Jani15, Reference Schuster, Savir-Baruch and Nieh21–Reference Nanni, Schiavina and Boschi23). The performance characteristics of standard care vary depending on the specific combination of tests used. Based on data showing approximately 50 percent of men presumed to have local disease by negative imaging have disease recurrence following salvage RT, thus suggesting micrometastatic disease was present (Reference Stephenson, Shariat and Zelefsky10), we assumed sensitivity and specificity of standard work-up was 50 percent. Given the variability, and considerable uncertainty with regard to optimal clinical practice in this area (Reference Beresford, Gillatt, Benson and Ajithkumar24), we confirmed appropriateness of this assumption with clinical experts and varied these parameters widely for sensitivity analyses. Specifically, one-way sensitivity analyses were performed for the new technology over ranges 0.50 to 0.90, 0.75 to 0.99, and USD 1,000 to 5,000 for test sensitivity, specificity, and cost, respectively. For standard care, ranges of 0.40 to 0.60 and a Beta distribution were used for sensitivity and specificity in the one-way and all-way (probabilistic) sensitivity analyses. A Beta distribution was used for sensitivity and specificity of the new diagnostic in probabilistic sensitivity analysis.
Clinical Parameters
Clinical parameters included in the model are the proportion of men with true metastatic disease, time to failure for residual localized disease (i.e., progression from residual localized disease to metastatic disease and then to death), and time to failure for metastatic recurrent PCa (i.e., death).
Probability of a true metastatic recurrence for a post-prostatectomy patient, given a rising PSA and no symptoms, was estimated from studies on predictors of metastatic disease in men with biochemical failure and on use of choline-PET/CT to restage patients with biochemical failure (Reference Giovacchini, Picchio and Briganti25;Reference Okotie, Aronson and Wieder26). Both studies found 11 percent of patients with biochemical failure after RP had a positive result (indicative of metastatic disease) from a bone scan or a choline-PET/CT, a value that was also verified by our clinical experts. Combining this probability with the performance characteristics of standard work-up resulted in 44.5 percent of men in the standard work-up arm of the decision tree being considered to have true local disease (1-probability of true metastatic disease [0.11] × specificity [0.5]). Similar calculations were completed for the proportion of men with false local disease, true metastatic disease, and false metastatic disease. Given 75 percent sensitivity and 90 percent specificity for the new diagnostic test, men were similarly categorized. The proportion of men with false local disease decreased from 44.5 percent to 8.9 percent and for men with false metastatic disease from 5.5 percent to 2.75 percent. Correspondingly, the proportion of men categorized with true local disease increased from 44.5 percent to 80.1 percent and true metastatic disease from 5.5 percent to 8.25 percent.
Probability of transitioning from local to metastatic states and then to terminal state, is adjusted to account for men who are misdiagnosed (e.g., men who have metastatic recurrence but receive treatment for residual localized disease and vice versa). This is achieved by including an adjustment to the timing of when men transition to metastatic treatment, metastatic continuing and then terminal states, given a correct or incorrect initial diagnosis (i.e., progression from residual localized disease to metastatic recurrence and then to terminal state will occur for all patients but the time will depend on whether the patient was given a correct initial diagnosis) (Supplementary Table 1, which can be viewed online at http://dx.doi.org/10.1017/S0266462314000476).
Progression from local to metastatic disease and then to death was estimated from a study on PCa-specific survival following salvage radiotherapy in post-prostatectomy men with biochemical recurrence (Reference Trock, Han and Freedland18). This study was selected as the most robust single-institution data available, with 15 years of follow-up information, for this specific population. The Kaplan-Meier survival curve from this publication was retrieved and used as the basis for calculating 16 years of transition probabilities for the model. The proportion of men transitioning from biochemical recurrence to death was combined with the timing adjustment to calculate time spent in metastatic states before the transition to terminal state. For men correctly diagnosed with local disease, time dependent adjustment was 5 years. For example, if the survival curve indicated a transition to death from the time of biochemical recurrence at year 10, these men made the transition from local disease to the metastatic treatment state at year 5, spending 1 year in the metastatic treatment state, 3 years in the metastatic continuing state, and 1 year in the terminal prostate state. For men incorrectly diagnosed, time dependent adjustment was 3 years resulting in less time correctly receiving the benefit of treatment for metastatic disease.
Survival for men with metastatic recurrent PCa was assumed to be the same as the survival of de novo metastatic PCa. This was estimated from a population-based study of oncologic outcomes of hormonal therapy in men with metastatic PCa (Reference Lu-Yao, Moore, Oleynick, DiPaola and Yao19). Again, the Kaplan-Meier curves were retrieved and used to calculate 10 years of probabilities for transition from metastatic recurrence to death. Both survival curves were extrapolated beyond their respective time frame to a lifetime horizon for inclusion in the model (Supplementary Figure 2, which can be viewed online at http://dx.doi.org/10.1017/S0266462314000476). Probability of death from all-cause mortality was estimated from age-specific life tables (Reference Arias27).
Costs
All costs are reported in 2012 USD. Costs not in 2012 USD were adjusted using the consumer price index medical services component (28). Cost parameters included in the model were: cost of the novel diagnostic test, cost of standard work-up diagnostics, cost of PCa treatment, and cost of end-of-life care (Table 1). Cost of the new diagnostic test was provided by GE Healthcare. Costs of specific diagnostic tests were retrieved from the 2012 Medicare fee schedule. A literature search was undertaken to determine the usage of diagnostic tests and treatments in the setting of rising PSA after RP, but there were no available high-quality studies, nor do clinical guidelines from the American Urological Association and National Comprehensive Care Network indicate the extent of usage of testing and treatment in this setting. Therefore, clinical experts indicated what type, and how many tests, a patient would typically receive. This information was used to calculate an aggregate cost for use in the model.
Cost Parameters Included in the Model
Note. A Gamma distribution was used for costs and a Beta distribution for proportions in probabilistic sensitivity analyses.
SE, standard error.
Costs of recurrent PCa treatment and terminal PCa costs were estimated from a study on the economic burden of metastatic and PSA progression in patients with PCa (Reference Penson, Moul and Evans20). This retrospective analysis of healthcare resources associated with American PCa patients in the years after progression (metastatic and “local”), split treatment costs into three phases: initial, continuing and terminal aligning with the design of our Markov model. Initial treatment costs, including salvage treatment for local recurrences and hormonal treatment for metastatic disease, were assigned to initial treatment states. Continuing costs were assigned to PCa patients not undergoing active treatment, but still using healthcare resources for ongoing care. Terminal costs were included to account for healthcare resource usage in the last year of life, when usage is often high (Reference Murphy, McGuire and Costa-Font29). Finally, costs of end-of-life care, not specific to any disease, were retrieved from a study on U.S. hospital palliative care programs (Reference Morrison, Penrod and Cassel30).
Impact on Quality of Life (Disutility)
A review of PCa quality of life research was performed to abstract information on the quality of life impact of various states in the model. Quality of life associated with local, metastatic and terminal PCa, as well as disutility associated with radiotherapy and hormonal therapy (Supplementary Table 2, which can be viewed online at http://dx.doi.org/10.1017/S0266462314000476), was retrieved from a comprehensive study that estimated utilities for PCa health states in men aged 60 years and older (Reference Stewart, Lenert, Bhatnagar and Kaplan17). Quality of life impact of the long-term side effects associated with PCa treatment, including bowel problems, impotence, and urinary incontinence, were weighted by the proportion of patients predicted to experience these effects (Reference Tefilli, Gheiler and Tiguert31–Reference Potosky, Davis and Hoffman34) and then incorporated into the continuing states of the model. To reflect quality of life associated with the terminal year before a patient dies from other causes, an end-of-life utility was calculated using estimates from studies that examined the relationship between quality of life and mortality. End-of-life disutility was estimated for two age groups (65–84 and 85+) as quality of life in the terminal year declines with age (Reference Lentzner, Pamuk, Rhodenhiser, Rothenberg and Powell-Griner35–Reference Myint, Luben and Surtees39).
Sensitivity Analysis
One-way sensitivity analyses were conducted on all parameters to determine their individual impact on results. Parameters were varied within one standard deviation or error from their base case value. If this information was not available, standard error was assumed to be 20 percent of the base case value. A probabilistic sensitivity analysis was performed to explore joint uncertainty of all parameters. Probability distributions were defined for each parameter and 1,000 Monte Carlo simulations run. Results of these simulations were plotted on the cost-effectiveness plane (indicating joint uncertainty in costs and effects). Results were also presented as a cost-effectiveness acceptability curve indicating the probability the new diagnostic test is cost-effective when compared with current standard work-up (y-axis), given different willingness to pay thresholds (x-axis).
RESULTS
The model estimates that a novel diagnostic test capable of accurately identifying site of recurrence in post-prostatectomy patients with low but detectable PSA elevation, compared with standard work-up, would result in improved quality of life for men (1.83 QALYs; 95 percent uncertainty interval [UI]: 1.24–2.64) at an added cost of USD 15,595 (95 percent UI: USD -6,330–44,402). The resulting cost-effectiveness ratio (ICER) is USD 8,516/QALY (95 percent UI: USD -2947–22,372).
The one-way sensitivity analysis showed this result is most sensitive to the discount rate used to adjust for differential timing in QALY outcomes followed by performance characteristics of the test used to identify site of recurrence (Supplementary Figure 1, which can be viewed online at http://dx.doi.org/10.1017/S0266462314000476). The model was sensitive to the discounting rate used in the model (3 percent in the base case and 0 percent to 6 percent in sensitivity analyses), which is an artifact of the long time horizon, 35 years (see the Discussion section). Sensitivity to test performance characteristics is unsurprising considering the importance of minimizing false positives (men identified as having metastatic disease who actually have local recurrence) by means of test specificity, and maximizing true positives (men correctly identified as having metastatic disease) by means of test sensitivity.
The probabilistic sensitivity analysis identified that simulated ICERs were mostly in the north-east quadrant of the cost-effectiveness plane (Figure 3). This result indicates the new test, when compared with current standard work-up, will result in improved outcomes for patients (QALYs on the x-axis) at additional cost (USD on the y-axis). Several simulations can be seen in the south-east quadrant of the cost-effectiveness plane, indicating improved patient outcomes and cost savings. The resulting 95 percent UI for the Monte Carlo simulations is USD -2,947 to 22,372/QALY.
Scatterplot of ICERs (USD/QALY) from probabilistic sensitivity analysis.
These data can be used to present uncertainty in the overall cost-effectiveness ratio using a cost-effectiveness acceptability curve (CEAC). For this analysis, the CEAC indicates the new diagnostic test has a 95 percent probability of being cost-effective at a willingness-to-pay threshold of USD 19,000/QALY and a 98.6 percent probability of being cost-effective at a threshold of USD 25,000/QALY (Supplementary Figure 3, which can be viewed online at http://dx.doi.org/10.1017/S0266462314000476).
DISCUSSION
With currently available imaging technology, 50 percent of patients presumed to have locally recurrent disease after RP fail salvage radiation (Reference Stephenson, Shariat and Zelefsky10), suggesting micrometastatic disease was present. While there are some clinical parameters and even nomograms to help distinguish between localized and micrometastatic disease (Reference Stephenson, Scardino and Kattan22), a novel test that can more accurately identify residual localized disease versus small volume metastatic disease would increase the proportion of men given beneficial RT while at the same time reducing treatment burden, costs, and potentially damaging side effects of RT for men who do not have local disease. Our modeling-based analysis suggests that such a test would improve the overall quality of life of men at a modest cost, the resulting average cost-effectiveness ratio being well below commonly cited thresholds for cost-effectiveness in health and medicine (e.g., USD 50,000 per QALY gained) (Reference Grosse40).
Improvements in patient outcomes identified by the model are primarily due to reductions in the proportion of men experiencing the quality of life burden of non-beneficial RT as well as the survival benefit provided by more accurate use of beneficial RT. Despite higher costs for the new test, reductions in the proportion of men receiving costly initial treatment for residual localized disease provides a substantial cost offset overall, although not enough to completely cover the added cost of using the new diagnostic test in the base case result. Although results were quite sensitive to changes in the discounting rate, this is most likely an artifact of the long time horizon of the model. Discounting of future costs and effects, common practice in economic evaluations, makes current costs and benefits worth more than those occurring in the future. This is done to incorporate “time preferences”—the desire to enjoy benefits in the present while deferring any negative effects of doing so, into long-term evaluations of cost-effectiveness (Reference Torgerson and Raftery41).
Results of our analysis may seem unsurprising to clinicians all too familiar with limitations of current diagnostics in this patient population. However, this result provides preliminary quantitative information on the potential acceptability of a new functional imaging test. Clinicians and patients would embrace a reliable and accurate diagnostic solution that reduces misclassification of the site of recurrence, and minimizes unnecessary use of salvage RT. Payers would need to assess the impact of adoption on constrained budgets given the added costs of the new diagnostic test, but the favorable effectiveness and cost-effectiveness profile identified here would be encouraging for many public and private insurers dealing with the humanistic impact of post-prostatectomy biochemical recurrence and the inaccuracies of current diagnostics.
Our analysis has its limitations. Findings must be interpreted with an understanding that results are dependent on the performance characteristics of a technology that is still in development, and it will be important to confirm and further refine these findings (Reference Vallejo-Torres, Steuten and Buxton14). Similarly, findings must also be interpreted in light of limitations of currently available data. While every effort has been made to use relevant high-quality information, several parameters in the model rely on single studies. Generation of additional data to further inform key parameters used in the model should be incorporated into development and validation efforts where possible, in the form of observational studies or as part of interventional investigations. Specifically, more information is needed on PCa-specific survival following salvage radiotherapy for locally recurrent disease and hormone therapy for metastatic recurrent disease in post-prostatectomy men with biochemical recurrence, along with updated cost of treatment information. Although the level of uncertainty in these parameters was included in the analysis as large standard errors in probabilistic analyses, especially those for cost parameters, further information will help refine estimations of cost-effectiveness.
CONCLUSIONS
Based on the parameters and assumptions incorporated into our model, the results of the analysis presented here suggest that a new PCa specific functional imaging technology, capable of identifying residual localized disease versus small volume metastatic disease would be a cost-effective alternative to current standard work-up. With moderate sensitivity and high specificity, the new diagnostic test reduces the quality-of-life burden of non-beneficial RT at a reasonable cost. This result supports additional investment in developing and validating such a technology.
SUPPLEMENTARY MATERIAL
Supplementary Table 1: http://dx.doi.org/10.1017/S0266462314000476
Supplementary Figure 2: http://dx.doi.org/10.1017/S0266462314000476
Supplementary Table 2: http://dx.doi.org/10.1017/S0266462314000476
Supplementary Figure 1: http://dx.doi.org/10.1017/S0266462314000476
Supplementary Table 3: http://dx.doi.org/10.1017/S0266462314000476
CONTACT INFORMATION
Daniel A. Barocas, Vanderbilt University Medical Center, Nashville, Tennessee
Mark E. Bensink, Kristin Berry, Zahra Musa, Fred Hutchinson Cancer Research Center, Seattle, Washington
Carolyn Bodnar, Robert Dann, GE Healthcare, Pollards Wood, United Kingdom
Scott D. Ramsey (sramsey@fhcrc.org), Fred Hutchinson Cancer Research Center, Seattle, Washington
CONFLICTS OF INTEREST
Funding for this research was provided by a subcontract through Arizona State University (Subcontract No. 11-465) from GE Healthcare (Agreement # 2580220935) to Dr. Ramsey. Dr. Bensink, Dr. Berry, and Ms. Musa are affiliate researchers with the Fred Hutchinson Cancer Research Center and were funded by Subcontract No. 11-465 to complete this work. Ms. Bodnar and Mr. Dann are employees of GE Healthcare. Mr. Dann has shares in GE Healthcare. Dr. Barocas' contribution to the research was supported directly through a consulting agreement with GE Healthcare.
