Innovation is a strong force in healthcare development and has a major economic impact on the healthcare system. In the research-based industry, innovations generate revenues that are part of the companies' profits and impact on further investments in research and development (R&D), giving way to new innovative products. Innovations are subject to public regulation of market access, while coverage and reimbursement by health insurance again impact directly on the manufacturers' attainable revenues (Reference Leidl29). This innovation cycle is illustrated in Figure 1.
The life cycle of innovation.
Economic evaluation is particularly relevant for new technologies and is becoming increasingly important. After having demonstrated quality, safety, and efficacy for market approval, in numerous countries so-called fourth hurdle institutions require new technologies to show evidence of cost-effectiveness before national health services or insurance systems provide coverage (Reference Langley26;Reference Taylor, Drummond, Salkeld and Sullivan41).
For the manufacturer of an innovative medical technology, coverage and adequate reimbursement are the key to a wide application that is essential for economic success. Applied timely in the product development process, economic evaluation provides the manufacturer with useful information on the future economic viability of the new product.
This study forms a part of the EU-funded Inno-HTA research project that aims at developing a methodology for the evaluation of health innovations to broaden the scope of classic HTA. It sets out to explore the potential and actual role of economic evaluation in early phases of product development.
Methodology
We conducted a literature research in February 2007 whose purpose was twofold: (i) to detect methodological contributions regarding early economic assessments, and (ii) to find economic evaluations that actually used data from early phases of product development. Databases researched were PUBMED, The Cochrane Library, CRD (DARE, HTA, and NHS EED), MEDLINE, DAHTA, EconLit, Embase, BIOSYS Previews, the UK Department of Health Database publications library and the Cost Effectiveness Analysis Registry, by text words and MESH terms (phase II, randomized controlled trial, controlled clinical trial, clinical trials, clinical trial phase I, clinical trial phase II, economic evaluation, early pharmacoeconomics, early technology assessment, healthcare evaluation mechanisms, economics, cost, cost analysis). Online available issues of potentially relevant journals were researched (International Journal of Technology Assessment in Health Care, Expert Opinion on Investigational Drugs, Pharmacoeconomics). References of relevant publications were tracked, an additional internet research was conducted by means of Google Scholar, and websites of institutions related to innovations in health care were investigated (acatech, EUROSCAN, NHS National Innovation Centre). In addition, reports of international horizon scanning agencies published in 2004 were investigated for emerging technologies which were researched for available economic evaluations. The year 2004 was chosen to account for the lag in scientific publishing, to enhance chances to find economic evaluations for the identified technologies.
More than 1,000 titles and abstracts were reviewed. Publications in English, German, French, and Spanish were considered when they covered a healthcare delivery context, used early stage data and presented at minimum a cost assessment or comparison. In total, 111 potentially relevant empirical studies have been identified, of which 83 fulfilled the inclusion criteria, while 28 publications were excluded on these grounds. The research also yielded seventy-one methodological contributions.
RESULTS
The presentation of the results is organized in three sections: the first section details major purposes early economic data can serve as revealed by the analysis of the seventy-one methodological papers identified in the literature search. In this respect, strategic R&D decision making comprehending several applications is a central issue, followed by early economic evaluation supporting future pricing and reimbursement. Tools encountered for early economic evaluation make up the second section. The third section presents the use and purposes of early data in the eighty-three actual economic evaluations found in the literature search.
Major Applications of Early Economic Evaluation
Strategic R&D Decision Making
Drug research and development is a long, costly, and risky undertaking. In the early stage, the manufacturer is ignorant of which project is going to be successful, so he has to take decisions under uncertainty. Early economic assessments help to reduce this uncertainty, promoting more economically solid products and avoiding costs for potentially unsuccessful products, enhancing efficiency, productivity and return on investment (ROI) (Reference Langley27;Reference Miller32). This is essential as the incentives to engage in R&D depend on the expected costs and returns of successful innovations, which in turn depend on development expenses as well as on the proportion of drug candidates that fail and at what point of time these failures happen—the later, the more expensive (Reference DiMasi11;Reference Girard17).
Pre-clinical Preliminary Market Assessments
They encompass the investigation of disease state, target population, and epidemiological factors as well as associated costs and current treatments to picture the disease impact and therapeutic benchmarks. Using a distribution of likely values accounts for the inherent uncertainty of the parameters and shows the robustness of the results. Costs and effectiveness of available therapies have to be assessed—the less effective current treatments are, the higher the potential for a new therapy to be cost-effective (Reference Sculpher, Drummond and Buxton36). Available data sources at this stage comprise literature reviews, claims data or national health surveys. The results offer a benchmark for the minimum performance required and a forecast of market potential that can be used in a business opportunity assessment (Reference Álvarez1;Reference Grabowski18;Reference Langley26;Reference Langley27).
This is illustrated in a case study by Poland and Wada (2001) who combine drug-disease and economic models to explore different dosage regimens for an HIV protease inhibitor in development. The drug-disease model predicts efficacy as a function of regimen, patient adherence as well as pharmacokinetic and pharmacodynamic parameters, which the economic model translates into a net present value measure for decision making, based on development and commercial costs, market size, market share, and price. For uncertain input parameters, probability distributions were assessed, yielding a distribution for the resulting net present value (Reference Poland and Wada34).
Go/no-go Decisions, Identification of Potentially Successful Projects
First data available from phase I (small number of healthy volunteers)/phase II (small number of patients) clinical trials can be fed into the business opportunity assessment, and serve as basis for R&D priority setting and “go/no-go decisions,” determining whether drug candidates will be further developed and proceed to phase III trials (large number of patients). As large phase III trials require substantial investments, it is important to evaluate the economic prospects of new products beforehand (Reference Annemans, Genesté and Jolain2;Reference DiMasi, Hansen and Grabowski13;Reference Grabowski18). Empirical findings support these results. DiMasi (2002) finds substantial reductions in costs of up to 8 percent per approved drug if decisions to abandon failures were shifted from phase II to phase I, and even more so when shifted from phase III to phase II or I (Reference DiMasi11).
Pharmaceutical companies often realize a huge part of their profits with a small number of products and depend on these “blockbusters” to cross-subsidize other products, so that it is essential to focus on the development of drugs that can earn long-term, positive returns and to terminate uneconomic projects in time. These portfolio management decisions contribute to allocative efficiency and reduce total R&D spending, whereas falsely terminated projects do not only impact on costs, as already development expenses occurred, but also on revenues in the sense of forgone earnings. It is thus important to identify successful and unsuccessful projects as accurately as possible (Reference Grabowski18;Reference Miller32;Reference Schachter, Ramoni, Baio, Roberts and Finkelstein35).
Empirical evidence shows that the participation of pharmacoeconomic departments in R&D decision making is still rather limited. Although most have at least sometimes been involved, this happens on an occasional rather than regular basis (Reference DiMasi, Caglarcan and Wood-Armany12). The empirical evidence of development projects discontinued for economic reasons is limited (Reference Lee28). DiMasi (2001) investigated reasons for research abandonment in a study on 350 new chemical entities (NCEs) and found that economic factors were the second leading cause for research termination, also occurring rather late in the development process (Reference DiMasi10). Of the roughly forty compounds examined in two studies on discontinued drugs in 2005, one was terminated after a phase II trial as the company preferred to develop other products “that have a higher commercial potential” (Reference Lee28, p.1498), four were stopped for “strategic reasons” (in one case “because other priorities required a shift in resources”) (Reference Kerwin24, p.1491, 28, p.1498), and one drug discontinuation is mentioned “but the reasons for this are commercial in confidence” (Reference Kerwin24, p.1489).
Development of Future Trial Design
With the planning of the clinical trial phases, particularly from phase II onward, economic evaluation impacts on the development of study design and protocols, further improving R&D resource allocation (Reference Miller32).
The choice of the comparator is crucial, as is the choice of outcome parameters—intermediate or final, patient-relevant endpoints or quality of life. It is essential to determine what kind of instrument is required, as its development takes time and efforts (Reference Seidman, Portenoy and Yao37). Instruments and data collection methods can be tested in phase II before entering expensive phase III trials. The selection of outcome parameters depends on where the results are to be presented, as different institutions have varying informational needs and data requirements (Reference Álvarez1;Reference Bombardier and Maetzel3;Reference Langley27).
Economic modeling in early stages can identify parameters to which the estimated cost-effectiveness is particularly sensitive, so that these key items can be prioritized in the data collection. It can help to determine the optimal statistical power (Reference Annemans, Genesté and Jolain2;Reference Chilcott, Brennan, Booth, Karnon and Tappenden6;Reference Sculpher, Drummond and Buxton36), especially when economic data are collected. As particularly cost data usually exhibit a greater variance and are more skewed than efficacy data, a larger sample size is required to come to statistically significant results. In earlier trials, the intended trial design can be tested and first cost data can be collected to estimate their characteristics, so that future trials can be designed accordingly (Reference Mauskopf, Schulman, Bell and Glick31).
Assessment of Future Reimbursement and Pricing Scenarios
With early data, a preliminary evaluation of the cost-effectiveness at different pricing scenarios, patient populations and indications can be carried out. The pricing has to match the clinical value to avoid an unfavorable reimbursement scenario, which means that a new product ends up in a niche market or is restricted, for example, by prior authorization or third-tier drug formulary positioning.
A preliminary reimbursement dossier can be prepared according to the guideline format in the target market. The cost-effectiveness in key market segments can be simulated under different assumptions. Setting up reimbursement data early also helps to identify gaps in the evidence needed (Reference Langley26;Reference Langley27;Reference Miller32).
Price Determination
Pricing of a new product starts early in development. On the one hand, it is central to take its future value to the projected customers and their willingness-to-pay (WTP) into account, understanding the customers' value perceptions and integrating them into R&D decisions. To determine this value, cost effectiveness analysis has emerged as one of the leading methods. Its result, expressed as a ratio of additional costs per additionally gained benefits, can directly be confronted with the payer's WTP. On the other hand, a company needs to ensure that a new product yields a sufficient ROI, so that the price usually ranges between the minimum ROI requirements and the maximally attainable price on the market.
The market placement of the new therapy in terms of target patient group and indication has essential value and pricing implications and should be thoroughly considered. For a global pricing strategy, additional factors have to be considered, including price differentials and parallel imports, public policy issues that impact on pricing as well as public opinion and patients' copayments. They have to be contemplated to determine the commercial potential of a new product (Reference Clemens, Garrison, Jones and Macdonald9;Reference Gregson, Sparrowhawk, Mauskopf and Paul19).
An early economic model can be used to determine which efficacy or clinical profile has to be attained for a given price so that the product is cost-effective, or, for given clinical and economic outcomes, to calculate the cost-effectiveness under different pricing scenarios (Reference Clemens, Garrison, Jones and Macdonald9).
The major problem with early pharmacoeconomic research is the uncertainty of the available data. Outcomes might not yet be fully at hand, future manufacturing costs are difficult to assess and relevant environmental factors, especially public policy decisions, are hardly foreseeable (Reference Clemens, Garrison, Jones and Macdonald9).
With only two publications encountered, empirical evidence on pricing issues for early stage technologies is scarce. Dranitsaris and Leung (2004) explore the use of economic modeling to estimate a product price for a given cost-effectiveness threshold (Reference Dranitsaris and Leung14), and Tanneberger et al. (2002) discuss dosage reductions as the high price of the drug in question limits its broad application (Reference Tanneberger and Lelli40).
Methodological publications show that early health economic evaluation can inform R&D decisions, establish market potential and feasible pricing and ensures that requirements in target markets are met, paving the way for reimbursement (Reference Langley27). Empirical evidence suggests that in recent years the use of early economic assessments was picking up—one study reported that none of the sampled compounds that entered clinical testing from 1990 to 1993 was subject to economic evaluation initiated during phase I, while this was the case for 15 percent of those compounds that entered clinical testing in 1994 (Reference DiMasi10).
Tools Encountered for Early Economic Evaluation
In this section, we aim to discuss several technical tools and concepts which contribute substantially to the use of early economic evaluations and therefore are increasingly applied in those studies. Modeling provides a useful framework to summarize available data but is not without drawbacks, Bayesian techniques as well as value of information analysis are useful when it comes to update information, while clinical trial simulation is particularly apt to enhance trial design and thus R&D efficiency.
Early Health Economic Modeling
This type of modeling serves as a synthesis of available clinical and economic evidence, a framework to analyze various scenarios, and as an interface to external decision makers. It is recommended to deal with the uncertainty inherent in early data, to account for parameters likely to vary and to combine data from different sources (Reference Hill and Freemantle21;Reference Sculpher, Drummond and Buxton36). Early modeling has to cope with data scarcity. Available data stem from literature, expert opinion or early clinical evidence and should be treated with caution, as they impact on cost estimates and economic results (Reference Annemans, Genesté and Jolain2). Data from small, early phase trials entail limitations, for example, intermediate instead of patient relevant endpoints, short follow-up times, study settings that do not reflect routine practice, and small sample sizes with unrepresentative participants that complicate gaining statistically significant results (Reference Mark and Simons30).
The Bayesian Analytical Framework
This framework, which is basically concerned with updating a priori probabilities with new information into a posteriori probabilities, has been suggested for the use in pharmacoeconomics in R&D, as it allows to synthesize pieces of information obtained at different points of time into an updated knowledge valuable to decision makers (Reference Miller32). Bayesian decision theory has also been recommended to optimize phase II trial design to support go/no-go decisions. In recent studies, costs and financial gains have been included to account for the increasing importance of economic evaluation of emerging therapies (Reference Stallard38). The inclusion of a cost function into go/no-go decision making has been further evaluated by Yan and Chen (2004), who also take into account erroneous decision making (Reference Yan and Chen44). Schachter et al. (2007) take the Bayesian framework one step further to predict the clinical success of a NCE based on early stage development data. The Bayesian network model used demonstrated substantial improvements and proved suitable to help eliminate unsuccessful projects early (Reference Schachter, Ramoni, Baio, Roberts and Finkelstein35).
Value of Information (VOI) Analysis
Together with Bayesian decision theory, VOI provides an analytical framework to determine the value of obtaining additional information to support a decision. Founded on statistical decision theory, the underlying principle is the comparison of costs and benefits generated by additionally gained information, assessing the value of investing in further research (Reference Briggs, Sculpher and Claxton5;Reference Claxton and Sculpher8).
The expected value of perfect information (EVPI) is calculated based on prior information which can be combined and updated with the Bayesian methodology. With clinical research, decision problems can be identified where the costs of uncertainty are highest, so that additionally gained information will be most valuable, supporting R&D prioritizing decisions. Given a fixed research budget, it helps to rule out research that is not cost-effective (Reference Briggs, Sculpher and Claxton5;Reference Claxton and Posnett7;Reference Claxton and Sculpher8).
Coverage and reimbursement decisions are closely linked to VOI analysis, as the decision to adopt a new technology suggests the consideration of whether the evidence available is sufficient to support the decision. A recent work informs on two opportunities where VOI analysis has been used in pilot studies in the UK. Even though the VOI analysis provided suitable results, decision makers appeared to be unfamiliar with the methodologies and were reluctant to base their decisions on such evidence (Reference Claxton and Sculpher8).
Clinical trial simulation (CTS)
The computer simulation of clinical trials uses mathematical synthesis to integrate simultaneously models of pharmacokinetic and pharmacodynamic drug action, disease progression, placebo effects and patient variability. The main objective is to increase drug development efficiency by improving trial protocols, maximizing the probability to meet the trial's targets and enhancing the quality of data gained. Key requirements such as dosage or statistical power can easily be established by simulations. Different assumptions about parameters and intended trial design can be tested to detect weaknesses and limitations. The impact of protocol deviation on the outcomes can be explored by conducting “what-if” scenario tests. CTS helps to prevent trial failures and uninformative or unnecessary studies. Costs can be incorporated into the simulation to minimize trial expenditure given a specific study design (Reference Bonate4;Reference Girard17;Reference Holford, Kimko, Monteleone and Peck22;Reference Hughes and Walley23;Reference Miller32).
Early economic evaluations depend on clinical outcomes data which at early stages might still be unavailable or fraught with uncertainty. Efficacy estimates obtained as outputs from CTS are suggested to supply information otherwise unavailable at this stage (Reference Hughes and Walley23;Reference Miller32). In addition, CTS allows population projections by integrating distributions of individual covariates, identifying patient subgroups that particularly benefit from a treatment or that demonstrate a favorable cost-effectiveness profile (Reference Hughes and Walley23).
Empirical examples show that CTS can be used for dosage optimization or trial design adaptation, or to select appropriate test statistics or the optimal sample size. Phase I or II trial data can enter a simulation to evaluate the planned phase III design (Reference Bonate4;Reference Krall and Engleman25).
Empirical Review of Early Economic Evaluation Studies
This part of the empirical review explores the actual use of early data in economic evaluations and the extent to which the suggested conceptual applications can be encountered in practice.
Publications were classified as trial-based when the evaluation is based on concurrently conducted or published phase II trials; as model-based when economic modeling is used, or as HTA reports which were listed separately as they mostly combine reviews including early phase trial data and modeling, are more standardized and supposedly destined for policy information.
The intervention examined in fifty-six of the eighty-three publications is medication, including treatments combining medication with other interventions, whereas twenty-seven studies cover procedures, including surgery, imaging, and novel products or systems, such as the MARS liver support system or drug-eluting stents. Six studies describe diagnostic procedures, all other interventions are curative. The majority of the studies cover malignancies, other indications encountered comprise diseases of the circulatory system, HIV and diabetes. Thirty-two studies were industry-sponsored, while fifty-one publications either did not state any conflict of interest or funding body or were supported with public means. The main characteristics of the included studies are summarized in Table 1. A full reference list and the results of a more detailed analysis of the study characteristics are available from the authors upon request.
Summary of Main Characteristics of the Included Studies
a Fields of indication according to the International Statistical Classification of Diseases and Related Health Problems, 10th Revision, Version for 2007 [42]: I, Certain infectious and parasitic diseases; II, Neoplasms; III, Diseases of the blood and blood-forming organs and certain disorders involving the immune mechanism; IV, Endocrine, nutritional and metabolic diseases; V, Mental and behavioral disorders; IX, Diseases of the circulatory system; XI, Diseases of the digestive system; XIII, Diseases of the musculoskeletal system and connective tissue.
We were not very successful in our efforts to reveal the purposes and the practical relevance of the empirical studies found in our review. In fact, the decision contexts mostly proved to be not clearly identifiable. Papers either offered information or discussed the state of a technology, recommending its use or suggesting further research, while the actual purpose of the study or the potential use of its results was generally not disclosed. Whereas the ten HTA reports can be assumed to have been compiled as policy decision support consistent with their original purpose, this is mainly true for the trial- and model-based studies. Exemptions were one study touching the reimbursement of a surgical procedure with Medicare in the United States (Reference Sternbergh and Money39), a second work offering a preliminary cost-effectiveness estimation in Germany (Reference Hessel20), and another paper explicitly mentioning its purpose as using modeling for a price estimation (Reference Dranitsaris and Leung14).
In summary, we could not clearly assign the empirical works to the proposed uses, but the very number of studies found shows that the idea of starting economic evaluations early in the product life cycle has gained momentum in the past few years.
DISCUSSION
Several methodological contributions were identified, but it proved difficult to capture what significance decisions makers actually attribute to early economic data in practice. Within industry, the reasoning for a decision is hardly accessible—internal strategic decisions are scarcely published, particularly regarding information on project failures, even more so if for economic reasons. Nevertheless, empirical evidence suggests that economic factors do play a dominant role in strategic R&D decisions (Reference DiMasi10).
Apart from this publication bias, it has to be acknowledged that diverse other factors affect decision making, ranging from the political and institutional environment to personal experience, motivation and attitude toward a technology, which of course can be even less transparent (Reference Wirtz, Cribb and Barber43). It is thus difficult to discern what weight economic data have in an individual decision, as it is only one part of all available information and other factors upon which a decision is based.
Certainly the generation and use of early economic evidence in the industry would be fostered if it would play a greater role in health policy decision making, be it in the context of horizon-scanning activities or in early reimbursement communications. While now being a useful but rather supplementary information, this would put a stronger emphasis on early economic data.
The reviewed early economic evaluations were not analyzed according to one of the various lists of quality criteria (e.g., Reference Drummond and Jefferson15;Reference Evers, Goossens, de Vet, van Tulder and Ament16;Reference Philips, Ginnelly and Sculpher33), as the purpose was not the assessment of the studies' quality, but rather the use of early data in accordance with our research subject. We included studies generously even if they would not qualify as proper economic evaluations, as our intention was to explore to what extent economic considerations were actually undertaken with early data. For a few of the trial-based studies, it was difficult to discern whether the described trial was indeed a phase II study. In these cases, we included the study when we believed it to fit in the context of our research as we considered it an early trial examining a new or emerging technology.
CONCLUSION
For the pharmaceutical and medical device industry, there are numerous beneficial applications for early health economic assessments. They support the determination of market potentials, possible price ranges and reimbursement probabilities. Strategic R&D decisions are backed, so that resources can be directed to potentially profitable projects, enhancing resource allocation efficiency and ultimately profitability. They also deliver valuable inputs to optimize the design of further clinical trials.
Even though the idea of starting economic evaluations early in the product life cycle seems to have gained popularity in the past few years, its use holds a great potential for the industry that seems to be not fully exploited yet. This impression could be attributable to a considerable degree of publication bias, as company-internal information is hardly accessible, and the reasons for abandoning a new product are rarely published, especially if economic reasons are involved.
We identified methodological contributions adapting analytical concepts to the particular use in early economic evaluations, for example clinical trial simulation or value of information analysis. This variety of tools readily at hand can be supposed to facilitate and further promote the use of early economic assessments. The economic evaluations found in practice can mostly be characterized as studying a new technology without disclosing a concrete decision context.
Problems with early economic data stem from their preliminary character, the fact that they cover only a short period of time and are likely to differ from real-world practice, so that the conclusions drawn cannot be taken as “hard facts”. This uncertainty has to be accounted for in the decision.
Our report summarizes the uses, benefits and problems of early economic evaluation. Confronted with the current use in practice, there still seems to be considerable potential that decision makers are invited to take advantage of. The way is paved as today techniques are available to mostly overcome the inherent difficulties of conducting economic analyses with early data.
CONTACT INFORMATION
Susanne Hartz, Dipl.-Vw. (susannehartz@web.de), Researcher, Department of Microeconomics, Friedrich-Schiller-University Jena, Carl-Zeiss-Straße 3, 07743 Jena, Germany
Jürgen John, PhD (john@helmholtz-muenchen.de), Deputy Director, Helmholtz Zentrum München—German Research Center for Environmental Health, Institute of Health Economics and Healthcare Management, Ingolstädter Landstr. 1, 85764 Neuherberg, Germany
