Over the past half century, innovation in the medical device realm has been nothing short of remarkable, as evidenced by the continuous stream of new devices introduced into clinical practice. These devices have extended human life; reduced pain, risk, and disability; and often, although not always, proven to be costly. This recognition has triggered growing interest in the extent to which health spending for device-based interventions yields value for money. Now is an opportune time to explore this issue, mainly because the various stakeholders in health care have become increasingly involved in managing the health and economic impact of technological change, and many find this task challenging.
Why is this so? Value is in the eye of the stakeholder; it evolves over time and is context dependent (Reference Porter1). Traditionally, physicians have been the major decision makers in adopting new devices, but over time other stakeholders with different preferences, such as hospital administrators, regulatory agencies, payers, and patients, have become more prominent in this process, which has led to different dimensions of value being weighted differently. With a more patient-centered focus in health care, decision making more prominently incorporates variation in risk aversion, tolerance of treatment burden and the trade-off between functional status and survival. Meanwhile, the continuing emphasis on cost containment and the introduction of new initiatives, such as value-based purchasing, mean that value is increasingly measured in terms of the cost incurred to produce a desired outcome. Given the multiplicity of stakeholders, perspectives on what constitutes value are not always aligned, and the resulting dilemmas must be managed politically, a process that gains legitimacy if informed by rigorous evidence.
The process of generating evidence is shaped by the dynamics of innovation in the medical device sector, which differ substantially from those in pharmaceutical innovation (Reference Porter1–Reference Taylor and Iglesias6). For devices in particular, it is important to remember that the fruits of medical progress do not appear in final form on the physician's or policy-maker's doorstep. Adoption decisions inevitably face considerable uncertainty about the extent of the indications for use, the optimal target population, the range and severity of risks to the patient, and the estimates of cost-effectiveness. Therefore, in addressing medical device valuation, we begin by characterizing the dynamics of device innovation itself, move on to the challenges these dynamics pose for evaluation and, finally, discuss the implications for clinical and policy decision making.
DYNAMICS OF DEVICE INNOVATION
In today's knowledge-based economy, medical device innovations arise within a complex network of public and private sector institutions, including universities and their academic medical centers, national laboratories, and industrial firms. The medical device industry is highly research intensive, investing on average 8 percent of its annual sales in R&D, ranging from 6 percent in the orthopedic sector to 13 percent in the cardiovascular device sector (7). Because the markets for medical devices are often fragmented and relatively small, that industry has historically not invested heavily in basic research. In fact, the medical device industry often exploits scientific and technological capabilities developed in a range of sectors outside of medicine (such as electronics, mathematics, physics, and specialized materials, including high-quality glass for fiber optics or inert materials for prosthetic devices), and integrates these advances with ones in medicine to develop specific devices. The device innovation cycle consists of diverse, partly overlapping, stages, including the development of novel product ideas, device prototypes and manufacturing methods, the evaluation of devices in animals and humans, and the modification of existing products.
The resulting devices constitute a highly heterogeneous group of products, ranging from tongue depressors to lasers, which vary in terms of both complexity and risk (Reference Kirisits and Redekop8). We focus here on the more technologically sophisticated end of the spectrum, including such major device categories as imaging and nonimaging diagnostics, therapeutic devices, and health information technology, which is a rapidly growing sector. In response to market signals about value and technological opportunity, the direction of innovation has shifted. In the past, innovators were mostly concerned about improving clinical efficacy, utility, and safety. In recent years, medical devices have become more patient-centered; for example, less invasive and burdensome, as illustrated by the development of such technologies as a “prep-less” colonoscope. Device innovation has also become more focused on “downstream” healthcare costs, for example, by developing ingestible sensors to help ensure medication adherence and increasing use of remote monitoring to avert hospitalizations.
These demand-side factors and the expanding scientific and technological opportunities affect not only the direction of innovation but also its rate. The device industry produces a steady stream of novel high-risk devices. One metric is regulatory approvals by the Food and Drug Administration (FDA). The FDA, for example, approved 39 devices in 2012 through premarketing approval (PMA) decisions (9). But, this number does not accurately reflect the extent of innovation underway. Devices, as distinct from drugs, are complex technologies, with many component parts, each of which is subject to independent innovation; hence, most device innovation is incremental in nature. Each year, the FDA approves approximately 3,000 510(k) devices (ones that are “substantially equivalent” to devices already marketed before the 1976 Device Amendments to the FDA Act took effect) and between 1,500 and 2,000 supplemental PMAs, which represent incremental improvements in previously approved devices (10).
In Europe, medical devices are required by regulatory agencies to demonstrate safety as a condition for market approval. Payers generally require broader demonstration of both effectiveness and safety to support purchasing decisions. In the United States, the FDA requires that novel high-risk devices and a sub-set of 510(k) devices undergo premarketing evaluation of both effectiveness and safety, determinations that have recently been increasingly based on randomized trials (Reference Hulstaert, Neyt and Vinck11). Such trials eliminate bias in assigning patients to treatment arms, and, thus ensure equal constitution of comparison groups. Their use, however, remains less prominent than in the drug world; only 31 percent of cardiovascular PMAs, approved between 2000 and 2007, involved randomized trials and only 42 percent of PMAs had an active control group (Reference Dhruva, Bero and Redberg12;Reference Chen, Dhruva and Redberg13). The design and conduct of device trials, however, involve unique challenges compared with pharmaceuticals, which we will highlight by focusing on the development and evaluation of left ventricular assist devices (LVADs).
LVADs were originally introduced for patients with advanced “stage D” heart failure, who were decompensating while awaiting cardiac transplantation (so-called bridge-to-transplantation indication). Subsequently, they were evaluated for long-term use in patients ineligible for transplantation, that is, as a “destination therapy” (DT). The REMATCH trial assessed the efficacy and safety of devices for this indication, and based on its results, the FDA approved LVADs for DT in 2002, and the Center for Medicare and Medicaid Services (CMS) approved them for coverage in 2003 (Reference Rose, Gelijns and Moskowitz14;15).
CHALLENGES IN ASSESSING THE VALUE OF NOVEL DEVICES
The first challenge in conducting randomized trials of a novel device is that the comparison arm is often a very different treatment modality, unlike the comparison of two drugs. In REMATCH, for example, the use of long-term LVADs was compared with optimal pharmacological therapy (Reference Rose, Gelijns and Moskowitz14). Such vastly different treatment approaches may engender strong physician and patient preferences, which may make it harder to achieve equipoise or buy-in for randomization (especially in the case of a life-threatening condition). Moreover, in comparison to drugs, blinding is often not feasible, which increases the potential for observational bias, especially in the assessment of more subjective endpoints like quality of life (Reference Parides, Moskowitz, Ascheim, Rose and Gelijns16).
A second challenge is the highly incremental character of change that accompanies medical device innovation. A drug typically does not undergo substantial changes as it moves through the phases of clinical trials (Reference Drummond, Griffin and Tarricone4–Reference Taylor and Iglesias6;17). Devices, however, may see extensive modifications. In the REMATCH trial, for example, we saw numerous improvements in such components as the driveline (the tube carrying the power and electronic controls for the pump), software for the system controller, battery modules, and connections to the blood conduits of the pump, introduced over the 4-year period of trial conduct (Reference Dowling, Park and Pagani18). Such modifications may need to be accounted for by sub-group analyses.
Compared with most drugs, device and procedure trials must typically contend with a more prominent learning curve among users (Reference Drummond, Griffin and Tarricone4–Reference Taylor and Iglesias6). In the REMATCH trial, LVAD implantation (n = 68) was found to double the 1-year survival (from 25 percent to 51 percent), as compared to medical management (n = 61), in this terminally ill population (Reference Rose, Gelijns and Moskowitz14). If we compare patients randomized in the second half of this 4-year trial with those implanted in the first half, we see a significant improvement in survival in the LVAD arm (see Figure 1). This was not the case for the medical treatment arm. The development of transcatheter aortic valve replacement (TAVR), a new percutaneous procedure for replacing the cardiac aortic valve, is another cardiovascular technology in which clinicians and policy makers are contending with learning curves. Such learning has led to technical improvements over the trial period, such as reductions in length of procedure, amount of imaging radiation and exposure to potentially toxic dyes (Reference Alli, Booker and Lennon19). In addition, the introductions of preclosure devices for the catheter insertion site through the skin, as well as improved criteria for selection of patients for specific surgical approaches, have improved complication and mortality rates (Reference Lange, Bleiziffer and Mazzitelli20).
Figure 1.LVAD recipient survival comparison between patients enrolled in the first and second half of the REMATCH trial.
Moreover, with device implantation trials there may be considerable variation in provider skills which may reside at the physician, team or hospital level. Minimally invasive surgical approaches, for example, are technically demanding, and early performance and the trajectory of adoption are greatly affected by variation in provider expertise (Reference Pisano, Bohmer and Edmondson21;Reference Holzhey, Seeburger, Misfeld, Borger and Mohr22). Another case in point is a recent trial of an extracellular matrix, CorMatrix, for pericardial closure to reduce postoperative atrial fibrillation in patients undergoing coronary artery bypass grafting (CABG) surgery. In this study, eight of the fifteen clinical sites participating in the trial had positive outcomes, while the remaining seven institutions had uniformly negative outcomes. Although this variation by center may have occurred by chance, it is more likely that these disparate findings reflect differences in techniques, such as incision approach, bypass methods, heart muscle preservation, heart valve and coronary bypass approaches, and surgical closure techniques. As a result, the effectiveness and costs of these interventions can vary dramatically across treatment venues, much more so than for drugs.
A vexing challenge in trial design is that the target populations for many interventions are often much smaller than for pharmaceuticals (Reference Sorenson, Tarricone, Siebert and Drummond5;15). In the case of LVADs, the U.S. patient populations for the bridge to transplant and destination therapy indications over the first decade following initial device approval for use in humans remained approximately 500 patients annually. For a variety of reasons, the number of patients available may be limited, which makes it difficult to complete trials in a timely manner and argues for novel designs to reduce sample size. Such designs may include using randomized and nonrandomized data to construct a control arm, relaxing the type I error rate, or adaptive trial designs (Reference Parides, Moskowitz, Ascheim, Rose and Gelijns16).
Finally, it is important to recognize that even if these challenges are addressed and well-controlled evidence becomes available, policy makers must still balance trade-offs among benefits, risks and costs of new devices in a context of uncertainty. Uncertainty lingers because premarketing trials are based on a sampling process, typically have limited time frames, and are often underpowered for secondary endpoints. Moreover, they tend to limit patient heterogeneity and are conducted in specialized centers, thus raising questions about generalizability. Diminishing these uncertainties requires widespread clinical use and additional evaluation.
THE IMPORTANCE OF LEARNING BY USING
There remains also a deeper source of uncertainty: devices entail much more postmarketing innovation and so-called “learning by using” than do drugs (Reference Rosenberg23). The adoption of a new device in widespread clinical practice does not signal the end of the development process. Widespread use is typically a prerequisite for garnering insights that provide important feedback to the R&D sector about the need for device improvements or the development of newer generations of devices. REMATCH patients implanted with the first-generation LVADs, which relied on pusher-plate technology and a bulky internally implanted blood reservoir, benefited in terms of survival and overall quality of life, but also were plagued by serious adverse events (Reference Rose, Gelijns and Moskowitz14). Efforts to miniaturize the LVAD, improve device durability, and minimize infections and thromboembolic events drove the evolution from these first generation devices to continuous flow devices, and more recently, to newer generations of micro VADs, which are the size of a AAA battery and are just entering trials.
Moreover, innovation occurs not only in R&D laboratories, but also in clinical practice itself, with changes in patient management, patient selection criteria and sometimes the discovery of new indications for use. Since initial CMS reimbursement approval in 2003, investigators have delineated several patient characteristics, such as poor nutrition, hematological abnormalities and markers of end-organ dysfunction, which stratified patients into risk groups (Reference Lietz, Long and Kfoury24;Reference Cowger, Sundareswaran and Rogers25). At the same time, patient management improved through modifications of the operative procedure, methods to prevent driveline infection and tailoring of anticoagulation regimens (Reference Gregoric, Wadia and Radovancevic26–Reference Holman, Rayburn and McGiffin31). Such learning may fundamentally change both clinical and economic outcomes.
IMPLICATIONS FOR MEASURING VALUE
The learning that followed the introduction of LVADs for DT illustrate the dramatic changes in value. Average survival for LVAD recipients during the 2-year follow-up of the REMATCH trial was 0.99 years as compared to 0.59 years for recipients of optimal medical management (OMM). Therefore, the incremental benefit of device implantation over the follow-up period was 0.40 years (95 percent CI: 0.14, 0.67). The cost associated with device implantation was $315,362 and for OMM it was $72,059, a difference of $243,303. Thus, the incremental cost-effectiveness (CE) ratio for the LVAD based on the REMATCH data was $602,361, far exceeding the threshold of $100,000 per life year saved, above which some economists have argued that a technology is not cost-effective. However, the CE ratio of an emerging technology may not be static. As a result of the above-described learning curve the cost-effectiveness ratio between the first and second half of the REMATCH trial decreased by nearly 45 percent from $898,666 to $505,285.
Moreover, in the 2 years following Medicare reimbursement approval in 2003, continued learning resulted in further reductions in the cost-effectiveness ratio. While there were not significant improvements in LVAD survival during the post-REMATCH period (Reference Lietz, Long and Kfoury24), the cost of the implantation hospitalization decreased by approximately 40 percent (Reference Zietkiewicz, Garlicki and Domagała32). The length of stay for the implant hospitalization, the most costly part of the care process, fell by 25 percent from an average of 44 days in REMATCH (with a mean cost of $210,187) to 33 days (with a mean cost of $148,305) (Reference Zietkiewicz, Garlicki and Domagała32). Furthermore, as a result of incremental improvements in this device, freedom from major device replacement at 1 year increased to 97 percent from 76 percent, representing a decrease in the relative probability of device replacement of approximately 25 percent (Reference Zietkiewicz, Garlicki and Domagała32). In addition to these engineering improvements, new management protocols helped to reduce the incidence of driveline infections and thrombosis associated with LVAD therapy (Reference Zietkiewicz, Garlicki and Domagała32–Reference Long, Healy and Rasmusson35).
During the same period, non-LVAD therapy for advanced heart failure underwent improvements as well. Biventricular pacemakers that synchronized ventricular contraction and improved cardiac output were adopted, as also were implantable cardio defibrillators, which reduced the risk of sudden death from a heart rhythm abnormality. The impact was an improvement in survival with “medical management” and an increase in cost. Specifically, using data from the COMPANION trial, where mean survival in patients managed medically with biventricular pacers and defibrillators increased from 3.37 years to 4.15 years, the survival in our cost-effectiveness model for the OMM group was increased by 25 percent in all time periods after REMATCH (Reference Slaughter, Sobieski, Gallagher, Dia and Silver36). Similarly, using COMPANION and CARE-HF data, OMM patients incurred an additional cost of $30,000 to account for costs associated with biventricular pacemaker and defibrillator implantation (Reference Slaughter, Sobieski, Gallagher, Dia and Silver36;Reference Feldman, de Lissovoy and Bristow37). Factoring these changes into the Markov model resulted in an incremental benefit of device implantation of 0.773 years (estimated survival of 0.737 years in the OMM group and of 1.51 years in the VAD group) with an incremental cost of $187,989 (OMM:$108,617; LVAD: $296,606). See Figure 2 for a diagram of the Markov decision model. Thus, the estimated incremental CE ratio for the LVAD during the immediate post-REMATCH time period decreased to $243,071 due to fewer complications, shorter hospital stay and improvements in the device.
Figure 2.Diagram of the Markov decision model used to estimate an ICER during the post-REMATCH and second generation device eras.
Second generation devices, which are smaller axial flow pumps, were introduced in 2009. These continuous flow devices significantly improved durability compared with 1st generation devices and reduced the device replacement rate (Reference Yao, Freemantle and Calvert38), which was a big-ticket item in that 5 percent of total costs in REMATCH were related to hospital readmissions in which a device replacement occurred. Furthermore, survival curves for LVAD recipients implanted with these continuous flow devices steadily improved, as have adverse event rates (Reference Yao, Freemantle and Calvert38–Reference Boyle, Teuteberg and Ascheim40). The current expected 1-year survival for DT patients is approximately 75 percent (a 40 percent relative improvement over DT patient survival in REMATCH) (Reference Yao, Freemantle and Calvert38;Reference Boyle, Teuteberg and Ascheim40–Reference Slaughter, Rogers and Milano42). Factoring these improvements into our Markov model produced an incremental benefit for device implantation of 1.86 years (estimated survival of 0.74 years in the OMM group and of 2.59 years in the VAD group) at an incremental cost of $199,652 (OMM:$108,617; LVAD: $308,269). This resulted in a further decrease in the incremental CE ratio to $107,569; closer to a more commonly accepted threshold for willingness to pay of $100,000 per life year saved (Figure 3) and (Table 1).
Table 1.Survival, Costs, and Incremental Cost-Effectiveness Ratios by Time Period for the Treatment of Advanced Heart Failure with Left Ventricular Assist Device and Medical Management
Figure 3.Plot of the trend of the incremental cost effectiveness ratios comparing LVAD therapy with medical therapy in patients with advanced heart failure.
CONCLUDING OBSERVATIONS
The innovation processes of drugs are very different from those of devices, the hallmarks of which have been “embeddedness” in skill-based procedures, “incrementalism” in modifications to the device itself, expanding know-how about what patient characteristics define good candidates, changing clinical management strategies, and expanding indications of use. Although, the term incremental is often equated with trivial or unimportant, it is exactly these incremental modifications that over time may lead to major improvements in outcomes and cost. Clearly, such learning occurs in the drug world too, but it is more prominent with devices (Reference Park, Milano and Tatooles43). These differences pose fundamental challenges to the assessment enterprise as well as to policy makers.
Policy makers increasingly seek rigorous evidence about the value of new technologies to guide decisions on their adoption and use, but analytical techniques need better adaptation to the distinctive characteristics of the device world. Rigorous controlled data on a new device intervention will facilitate regulatory and purchasing decisions. However, designing trials to generate unbiased estimates with external validity is more difficult in the device context, not only because trial populations are typically smaller, but also because incremental change and variations in provider skills are important.
Clinical trialists should explore innovative designs for premarketing device trials, such as combining nonrandomized and randomized data, adaptive trial designs, and analytical methods to explore temporal and site variations. Observing a temporal change in outcomes (e.g., survival) within a trial of a novel technology, which is likely to continue after approval and adoption, may be a strong indicator that the cost-effectiveness ratio will evolve. Cost effectiveness analyses typically do not incorporate future technological change and learning, and decision analysts should build these dynamics into sensitivity analyses because doing so can greatly change the interpretation of results. For example, models for change could draw upon adverse events that limit the net benefit of a technology, where variations in clinical center adverse event rates could indicate potential achievable performance. Another option is to survey the R&D community about expected technological improvements that may address current limitations of the device. But device innovations incorporate advances from many different fields (e.g., the material sciences, electronics, immunology) making technological prediction a formidable task.
Because device therapies continuously evolve, a strong argument can be made for assessment throughout the life cycle of the technology. Gathering data, and trying to eliminate all uncertainty, in the premarketing stage will increase the cost of development and delay the release of promising interventions, which itself has a cost. Therefore, policy makers will have to make regulatory and payment decisions based on imperfect information. Because many novel devices see improvements only over time, coverage decisions should not be binary “go/no go” decisions. For these devices, the health outcomes and cost effectiveness ratio, established early in the life cycle of a device, can be expected to change, as LVADs illustrate. The original cost effectiveness ratio led some European countries to approve this device not as a “destination” or long-term therapy, but only as a bridge to transplantation. The cost effectiveness ratio for DT soon improved substantially; however, partly on the basis of evidence from widespread use in the United States, where coverage was approved. Unless “learning” or “ongoing innovation” is incorporated into coverage decisions, valuable technologies may lose a chance to become cost-effective. In other words, healthcare systems may need enough flexibility to accept short-term inefficiencies to garner long-term value.
The dynamics of ongoing device innovation and learning by using argue for policies that are linked to an infrastructure that captures changing outcomes in everyday practice and permits payment decisions to be revisited as technologies evolve. This holds especially for devices that address serious conditions, hold promise for important unmet needs, are costly, and may entail safety concerns. Many European countries have created an infrastructure of detailed clinical registries, and in the United States, promising public-private partnerships are emerging to do the same. In the case of LVADs, for example, the NIH, FDA, and CMS, in collaboration with industry and hospitals, have created the Interagency Registry for Mechanically Assisted Circulatory Support [INTERMACS] registry to capture data on both the long-term safety and effectiveness of these devices over time and the evolution of patient populations. Participation in the registry is a condition that centers must meet to receive reimbursement for device implantation.
In 2006, Medicare formalized this policy direction by introducing coverage with evidence development (CED), by which coverage of new devices is conditionally linked to ongoing data collection efforts such as clinical trials or data registries (Reference Ascheim, Gelijns and Rose44). A recent example of CED is TAVR, a technology that in the premarketing trial demonstrated important survival benefits for patients who otherwise would succumb to their disease (because they were ineligible for traditional open surgery), but the benefits of which came with a prominent stroke risk. Professional societies joined with Medicare, FDA and industry to create the TAVR registry to address questions about evolving outcomes and patient populations.
In the UK, NICE has also introduced similar innovation by creating in 2009 the Medical Technologies Evaluation Programme (MTEP), which evaluates technologies based on the manufacturers’ claimed advantages and cost savings over current management options. Based on the presented evidence and the claims, the Medical Technologies Advisory Committee will either issue a guidance statement that supports adoption of the device, request additional evidence, or route the request to more stringent, long term assessment programs (Reference Tunis and Pearson45-Reference Campbell47). In circumstances where additional evidence is required, MTEP facilitates access to patients for clinical trials by encouraging collaboration between the manufacturer and NHS providers.
A suitable coverage decision-making model (one that neither introduces questionable devices too quickly into practice nor dismisses as cost ineffective ones that improve with time and experience) is bound to be complex, both conceptually and institutionally. The central elements of such an approach are sustained engagement of stakeholders (producers, providers, payers, and regulatory agencies), who agree to an extensive clinical data collection effort that is then used by payers as a source of evidence to revisit payment decisions. To be sure, this model poses sizable problems of design and implementation. It is hard to marshal stakeholders, find the necessary financial resources, and design a rigorous and efficient data collection system that captures changing outcomes. But to the extent that the necessary institution-building and data accumulation harmonize with developments in information technology that the federal government has been promoting and funding in its health reform initiatives, start up costs and managerial perplexities in the new institutional infrastructure for evaluation of devices should be lower. The evolution of Medicare's policies in this arena is instructive. Having launched the coverage with evidence strategy in 2006, the agency grappled with its many difficulties, wondered whether it should pursue CED, and then, late in 2012, reaffirmed it as a cornerstone of its strategy. For policymakers who want society to enjoy the benefits and avert the risks of innovative medical devices, there is no short cut around the challenges of inventing innovative evaluative approaches to guide decisions.
CONTACT INFORMATION
Annetine C. Gelijns, PhD, International Center for Health Outcomes and Innovation Research, Department of Health Evidence and Policy at the Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, New York 10029
Mark J. Russo, MD, Barnabas Health Heart Centers, West Orange, New Jersey
Kimberly N. Hong, MD, MHSA, International Center for Health Outcomes and Innovation Research, Department of Health Evidence and Policy at the Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, New York 10029
Lawrence D. Brown, PhD, International Center for Health Outcomes and Innovation Research, Department of Health Evidence and Policy at the Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York 10029, and Department of Health Policy and Management, Mailman School of Public Health, Columbia University, New York, New York
Deborah D. Ascheim, MD, International Center for Health Outcomes and Innovation Research, Department of Health Evidence and Policy and The Cardiovascular Institute, Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, New York 10029
Alan J. Moskowitz, MD, (alan.moskowitz@mssm.edu) International Center for Health Outcomes and Innovation Research, Department of Health Evidence and Policy Department of Medicine at the Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, New York 10029
CONFLICTS OF INTEREST
All authors report they have no potential conflicts of interest.
You have
Access
Over the past half century, innovation in the medical device realm has been nothing short of remarkable, as evidenced by the continuous stream of new devices introduced into clinical practice. These devices have extended human life; reduced pain, risk, and disability; and often, although not always, proven to be costly. This recognition has triggered growing interest in the extent to which health spending for device-based interventions yields value for money. Now is an opportune time to explore this issue, mainly because the various stakeholders in health care have become increasingly involved in managing the health and economic impact of technological change, and many find this task challenging.
Why is this so? Value is in the eye of the stakeholder; it evolves over time and is context dependent (Reference Porter1). Traditionally, physicians have been the major decision makers in adopting new devices, but over time other stakeholders with different preferences, such as hospital administrators, regulatory agencies, payers, and patients, have become more prominent in this process, which has led to different dimensions of value being weighted differently. With a more patient-centered focus in health care, decision making more prominently incorporates variation in risk aversion, tolerance of treatment burden and the trade-off between functional status and survival. Meanwhile, the continuing emphasis on cost containment and the introduction of new initiatives, such as value-based purchasing, mean that value is increasingly measured in terms of the cost incurred to produce a desired outcome. Given the multiplicity of stakeholders, perspectives on what constitutes value are not always aligned, and the resulting dilemmas must be managed politically, a process that gains legitimacy if informed by rigorous evidence.
The process of generating evidence is shaped by the dynamics of innovation in the medical device sector, which differ substantially from those in pharmaceutical innovation (Reference Porter1–Reference Taylor and Iglesias6). For devices in particular, it is important to remember that the fruits of medical progress do not appear in final form on the physician's or policy-maker's doorstep. Adoption decisions inevitably face considerable uncertainty about the extent of the indications for use, the optimal target population, the range and severity of risks to the patient, and the estimates of cost-effectiveness. Therefore, in addressing medical device valuation, we begin by characterizing the dynamics of device innovation itself, move on to the challenges these dynamics pose for evaluation and, finally, discuss the implications for clinical and policy decision making.
DYNAMICS OF DEVICE INNOVATION
In today's knowledge-based economy, medical device innovations arise within a complex network of public and private sector institutions, including universities and their academic medical centers, national laboratories, and industrial firms. The medical device industry is highly research intensive, investing on average 8 percent of its annual sales in R&D, ranging from 6 percent in the orthopedic sector to 13 percent in the cardiovascular device sector (7). Because the markets for medical devices are often fragmented and relatively small, that industry has historically not invested heavily in basic research. In fact, the medical device industry often exploits scientific and technological capabilities developed in a range of sectors outside of medicine (such as electronics, mathematics, physics, and specialized materials, including high-quality glass for fiber optics or inert materials for prosthetic devices), and integrates these advances with ones in medicine to develop specific devices. The device innovation cycle consists of diverse, partly overlapping, stages, including the development of novel product ideas, device prototypes and manufacturing methods, the evaluation of devices in animals and humans, and the modification of existing products.
The resulting devices constitute a highly heterogeneous group of products, ranging from tongue depressors to lasers, which vary in terms of both complexity and risk (Reference Kirisits and Redekop8). We focus here on the more technologically sophisticated end of the spectrum, including such major device categories as imaging and nonimaging diagnostics, therapeutic devices, and health information technology, which is a rapidly growing sector. In response to market signals about value and technological opportunity, the direction of innovation has shifted. In the past, innovators were mostly concerned about improving clinical efficacy, utility, and safety. In recent years, medical devices have become more patient-centered; for example, less invasive and burdensome, as illustrated by the development of such technologies as a “prep-less” colonoscope. Device innovation has also become more focused on “downstream” healthcare costs, for example, by developing ingestible sensors to help ensure medication adherence and increasing use of remote monitoring to avert hospitalizations.
These demand-side factors and the expanding scientific and technological opportunities affect not only the direction of innovation but also its rate. The device industry produces a steady stream of novel high-risk devices. One metric is regulatory approvals by the Food and Drug Administration (FDA). The FDA, for example, approved 39 devices in 2012 through premarketing approval (PMA) decisions (9). But, this number does not accurately reflect the extent of innovation underway. Devices, as distinct from drugs, are complex technologies, with many component parts, each of which is subject to independent innovation; hence, most device innovation is incremental in nature. Each year, the FDA approves approximately 3,000 510(k) devices (ones that are “substantially equivalent” to devices already marketed before the 1976 Device Amendments to the FDA Act took effect) and between 1,500 and 2,000 supplemental PMAs, which represent incremental improvements in previously approved devices (10).
In Europe, medical devices are required by regulatory agencies to demonstrate safety as a condition for market approval. Payers generally require broader demonstration of both effectiveness and safety to support purchasing decisions. In the United States, the FDA requires that novel high-risk devices and a sub-set of 510(k) devices undergo premarketing evaluation of both effectiveness and safety, determinations that have recently been increasingly based on randomized trials (Reference Hulstaert, Neyt and Vinck11). Such trials eliminate bias in assigning patients to treatment arms, and, thus ensure equal constitution of comparison groups. Their use, however, remains less prominent than in the drug world; only 31 percent of cardiovascular PMAs, approved between 2000 and 2007, involved randomized trials and only 42 percent of PMAs had an active control group (Reference Dhruva, Bero and Redberg12;Reference Chen, Dhruva and Redberg13). The design and conduct of device trials, however, involve unique challenges compared with pharmaceuticals, which we will highlight by focusing on the development and evaluation of left ventricular assist devices (LVADs).
LVADs were originally introduced for patients with advanced “stage D” heart failure, who were decompensating while awaiting cardiac transplantation (so-called bridge-to-transplantation indication). Subsequently, they were evaluated for long-term use in patients ineligible for transplantation, that is, as a “destination therapy” (DT). The REMATCH trial assessed the efficacy and safety of devices for this indication, and based on its results, the FDA approved LVADs for DT in 2002, and the Center for Medicare and Medicaid Services (CMS) approved them for coverage in 2003 (Reference Rose, Gelijns and Moskowitz14;15).
CHALLENGES IN ASSESSING THE VALUE OF NOVEL DEVICES
The first challenge in conducting randomized trials of a novel device is that the comparison arm is often a very different treatment modality, unlike the comparison of two drugs. In REMATCH, for example, the use of long-term LVADs was compared with optimal pharmacological therapy (Reference Rose, Gelijns and Moskowitz14). Such vastly different treatment approaches may engender strong physician and patient preferences, which may make it harder to achieve equipoise or buy-in for randomization (especially in the case of a life-threatening condition). Moreover, in comparison to drugs, blinding is often not feasible, which increases the potential for observational bias, especially in the assessment of more subjective endpoints like quality of life (Reference Parides, Moskowitz, Ascheim, Rose and Gelijns16).
A second challenge is the highly incremental character of change that accompanies medical device innovation. A drug typically does not undergo substantial changes as it moves through the phases of clinical trials (Reference Drummond, Griffin and Tarricone4–Reference Taylor and Iglesias6;17). Devices, however, may see extensive modifications. In the REMATCH trial, for example, we saw numerous improvements in such components as the driveline (the tube carrying the power and electronic controls for the pump), software for the system controller, battery modules, and connections to the blood conduits of the pump, introduced over the 4-year period of trial conduct (Reference Dowling, Park and Pagani18). Such modifications may need to be accounted for by sub-group analyses.
Compared with most drugs, device and procedure trials must typically contend with a more prominent learning curve among users (Reference Drummond, Griffin and Tarricone4–Reference Taylor and Iglesias6). In the REMATCH trial, LVAD implantation (n = 68) was found to double the 1-year survival (from 25 percent to 51 percent), as compared to medical management (n = 61), in this terminally ill population (Reference Rose, Gelijns and Moskowitz14). If we compare patients randomized in the second half of this 4-year trial with those implanted in the first half, we see a significant improvement in survival in the LVAD arm (see Figure 1). This was not the case for the medical treatment arm. The development of transcatheter aortic valve replacement (TAVR), a new percutaneous procedure for replacing the cardiac aortic valve, is another cardiovascular technology in which clinicians and policy makers are contending with learning curves. Such learning has led to technical improvements over the trial period, such as reductions in length of procedure, amount of imaging radiation and exposure to potentially toxic dyes (Reference Alli, Booker and Lennon19). In addition, the introductions of preclosure devices for the catheter insertion site through the skin, as well as improved criteria for selection of patients for specific surgical approaches, have improved complication and mortality rates (Reference Lange, Bleiziffer and Mazzitelli20).
LVAD recipient survival comparison between patients enrolled in the first and second half of the REMATCH trial.
Moreover, with device implantation trials there may be considerable variation in provider skills which may reside at the physician, team or hospital level. Minimally invasive surgical approaches, for example, are technically demanding, and early performance and the trajectory of adoption are greatly affected by variation in provider expertise (Reference Pisano, Bohmer and Edmondson21;Reference Holzhey, Seeburger, Misfeld, Borger and Mohr22). Another case in point is a recent trial of an extracellular matrix, CorMatrix, for pericardial closure to reduce postoperative atrial fibrillation in patients undergoing coronary artery bypass grafting (CABG) surgery. In this study, eight of the fifteen clinical sites participating in the trial had positive outcomes, while the remaining seven institutions had uniformly negative outcomes. Although this variation by center may have occurred by chance, it is more likely that these disparate findings reflect differences in techniques, such as incision approach, bypass methods, heart muscle preservation, heart valve and coronary bypass approaches, and surgical closure techniques. As a result, the effectiveness and costs of these interventions can vary dramatically across treatment venues, much more so than for drugs.
A vexing challenge in trial design is that the target populations for many interventions are often much smaller than for pharmaceuticals (Reference Sorenson, Tarricone, Siebert and Drummond5;15). In the case of LVADs, the U.S. patient populations for the bridge to transplant and destination therapy indications over the first decade following initial device approval for use in humans remained approximately 500 patients annually. For a variety of reasons, the number of patients available may be limited, which makes it difficult to complete trials in a timely manner and argues for novel designs to reduce sample size. Such designs may include using randomized and nonrandomized data to construct a control arm, relaxing the type I error rate, or adaptive trial designs (Reference Parides, Moskowitz, Ascheim, Rose and Gelijns16).
Finally, it is important to recognize that even if these challenges are addressed and well-controlled evidence becomes available, policy makers must still balance trade-offs among benefits, risks and costs of new devices in a context of uncertainty. Uncertainty lingers because premarketing trials are based on a sampling process, typically have limited time frames, and are often underpowered for secondary endpoints. Moreover, they tend to limit patient heterogeneity and are conducted in specialized centers, thus raising questions about generalizability. Diminishing these uncertainties requires widespread clinical use and additional evaluation.
THE IMPORTANCE OF LEARNING BY USING
There remains also a deeper source of uncertainty: devices entail much more postmarketing innovation and so-called “learning by using” than do drugs (Reference Rosenberg23). The adoption of a new device in widespread clinical practice does not signal the end of the development process. Widespread use is typically a prerequisite for garnering insights that provide important feedback to the R&D sector about the need for device improvements or the development of newer generations of devices. REMATCH patients implanted with the first-generation LVADs, which relied on pusher-plate technology and a bulky internally implanted blood reservoir, benefited in terms of survival and overall quality of life, but also were plagued by serious adverse events (Reference Rose, Gelijns and Moskowitz14). Efforts to miniaturize the LVAD, improve device durability, and minimize infections and thromboembolic events drove the evolution from these first generation devices to continuous flow devices, and more recently, to newer generations of micro VADs, which are the size of a AAA battery and are just entering trials.
Moreover, innovation occurs not only in R&D laboratories, but also in clinical practice itself, with changes in patient management, patient selection criteria and sometimes the discovery of new indications for use. Since initial CMS reimbursement approval in 2003, investigators have delineated several patient characteristics, such as poor nutrition, hematological abnormalities and markers of end-organ dysfunction, which stratified patients into risk groups (Reference Lietz, Long and Kfoury24;Reference Cowger, Sundareswaran and Rogers25). At the same time, patient management improved through modifications of the operative procedure, methods to prevent driveline infection and tailoring of anticoagulation regimens (Reference Gregoric, Wadia and Radovancevic26–Reference Holman, Rayburn and McGiffin31). Such learning may fundamentally change both clinical and economic outcomes.
IMPLICATIONS FOR MEASURING VALUE
The learning that followed the introduction of LVADs for DT illustrate the dramatic changes in value. Average survival for LVAD recipients during the 2-year follow-up of the REMATCH trial was 0.99 years as compared to 0.59 years for recipients of optimal medical management (OMM). Therefore, the incremental benefit of device implantation over the follow-up period was 0.40 years (95 percent CI: 0.14, 0.67). The cost associated with device implantation was $315,362 and for OMM it was $72,059, a difference of $243,303. Thus, the incremental cost-effectiveness (CE) ratio for the LVAD based on the REMATCH data was $602,361, far exceeding the threshold of $100,000 per life year saved, above which some economists have argued that a technology is not cost-effective. However, the CE ratio of an emerging technology may not be static. As a result of the above-described learning curve the cost-effectiveness ratio between the first and second half of the REMATCH trial decreased by nearly 45 percent from $898,666 to $505,285.
Moreover, in the 2 years following Medicare reimbursement approval in 2003, continued learning resulted in further reductions in the cost-effectiveness ratio. While there were not significant improvements in LVAD survival during the post-REMATCH period (Reference Lietz, Long and Kfoury24), the cost of the implantation hospitalization decreased by approximately 40 percent (Reference Zietkiewicz, Garlicki and Domagała32). The length of stay for the implant hospitalization, the most costly part of the care process, fell by 25 percent from an average of 44 days in REMATCH (with a mean cost of $210,187) to 33 days (with a mean cost of $148,305) (Reference Zietkiewicz, Garlicki and Domagała32). Furthermore, as a result of incremental improvements in this device, freedom from major device replacement at 1 year increased to 97 percent from 76 percent, representing a decrease in the relative probability of device replacement of approximately 25 percent (Reference Zietkiewicz, Garlicki and Domagała32). In addition to these engineering improvements, new management protocols helped to reduce the incidence of driveline infections and thrombosis associated with LVAD therapy (Reference Zietkiewicz, Garlicki and Domagała32–Reference Long, Healy and Rasmusson35).
During the same period, non-LVAD therapy for advanced heart failure underwent improvements as well. Biventricular pacemakers that synchronized ventricular contraction and improved cardiac output were adopted, as also were implantable cardio defibrillators, which reduced the risk of sudden death from a heart rhythm abnormality. The impact was an improvement in survival with “medical management” and an increase in cost. Specifically, using data from the COMPANION trial, where mean survival in patients managed medically with biventricular pacers and defibrillators increased from 3.37 years to 4.15 years, the survival in our cost-effectiveness model for the OMM group was increased by 25 percent in all time periods after REMATCH (Reference Slaughter, Sobieski, Gallagher, Dia and Silver36). Similarly, using COMPANION and CARE-HF data, OMM patients incurred an additional cost of $30,000 to account for costs associated with biventricular pacemaker and defibrillator implantation (Reference Slaughter, Sobieski, Gallagher, Dia and Silver36;Reference Feldman, de Lissovoy and Bristow37). Factoring these changes into the Markov model resulted in an incremental benefit of device implantation of 0.773 years (estimated survival of 0.737 years in the OMM group and of 1.51 years in the VAD group) with an incremental cost of $187,989 (OMM:$108,617; LVAD: $296,606). See Figure 2 for a diagram of the Markov decision model. Thus, the estimated incremental CE ratio for the LVAD during the immediate post-REMATCH time period decreased to $243,071 due to fewer complications, shorter hospital stay and improvements in the device.
Diagram of the Markov decision model used to estimate an ICER during the post-REMATCH and second generation device eras.
Second generation devices, which are smaller axial flow pumps, were introduced in 2009. These continuous flow devices significantly improved durability compared with 1st generation devices and reduced the device replacement rate (Reference Yao, Freemantle and Calvert38), which was a big-ticket item in that 5 percent of total costs in REMATCH were related to hospital readmissions in which a device replacement occurred. Furthermore, survival curves for LVAD recipients implanted with these continuous flow devices steadily improved, as have adverse event rates (Reference Yao, Freemantle and Calvert38–Reference Boyle, Teuteberg and Ascheim40). The current expected 1-year survival for DT patients is approximately 75 percent (a 40 percent relative improvement over DT patient survival in REMATCH) (Reference Yao, Freemantle and Calvert38;Reference Boyle, Teuteberg and Ascheim40–Reference Slaughter, Rogers and Milano42). Factoring these improvements into our Markov model produced an incremental benefit for device implantation of 1.86 years (estimated survival of 0.74 years in the OMM group and of 2.59 years in the VAD group) at an incremental cost of $199,652 (OMM:$108,617; LVAD: $308,269). This resulted in a further decrease in the incremental CE ratio to $107,569; closer to a more commonly accepted threshold for willingness to pay of $100,000 per life year saved (Figure 3) and (Table 1).
Survival, Costs, and Incremental Cost-Effectiveness Ratios by Time Period for the Treatment of Advanced Heart Failure with Left Ventricular Assist Device and Medical Management
Note. LY, life-year; VAD, ventricular assist device; OMM, optimal medical management.
Plot of the trend of the incremental cost effectiveness ratios comparing LVAD therapy with medical therapy in patients with advanced heart failure.
CONCLUDING OBSERVATIONS
The innovation processes of drugs are very different from those of devices, the hallmarks of which have been “embeddedness” in skill-based procedures, “incrementalism” in modifications to the device itself, expanding know-how about what patient characteristics define good candidates, changing clinical management strategies, and expanding indications of use. Although, the term incremental is often equated with trivial or unimportant, it is exactly these incremental modifications that over time may lead to major improvements in outcomes and cost. Clearly, such learning occurs in the drug world too, but it is more prominent with devices (Reference Park, Milano and Tatooles43). These differences pose fundamental challenges to the assessment enterprise as well as to policy makers.
Policy makers increasingly seek rigorous evidence about the value of new technologies to guide decisions on their adoption and use, but analytical techniques need better adaptation to the distinctive characteristics of the device world. Rigorous controlled data on a new device intervention will facilitate regulatory and purchasing decisions. However, designing trials to generate unbiased estimates with external validity is more difficult in the device context, not only because trial populations are typically smaller, but also because incremental change and variations in provider skills are important.
Clinical trialists should explore innovative designs for premarketing device trials, such as combining nonrandomized and randomized data, adaptive trial designs, and analytical methods to explore temporal and site variations. Observing a temporal change in outcomes (e.g., survival) within a trial of a novel technology, which is likely to continue after approval and adoption, may be a strong indicator that the cost-effectiveness ratio will evolve. Cost effectiveness analyses typically do not incorporate future technological change and learning, and decision analysts should build these dynamics into sensitivity analyses because doing so can greatly change the interpretation of results. For example, models for change could draw upon adverse events that limit the net benefit of a technology, where variations in clinical center adverse event rates could indicate potential achievable performance. Another option is to survey the R&D community about expected technological improvements that may address current limitations of the device. But device innovations incorporate advances from many different fields (e.g., the material sciences, electronics, immunology) making technological prediction a formidable task.
Because device therapies continuously evolve, a strong argument can be made for assessment throughout the life cycle of the technology. Gathering data, and trying to eliminate all uncertainty, in the premarketing stage will increase the cost of development and delay the release of promising interventions, which itself has a cost. Therefore, policy makers will have to make regulatory and payment decisions based on imperfect information. Because many novel devices see improvements only over time, coverage decisions should not be binary “go/no go” decisions. For these devices, the health outcomes and cost effectiveness ratio, established early in the life cycle of a device, can be expected to change, as LVADs illustrate. The original cost effectiveness ratio led some European countries to approve this device not as a “destination” or long-term therapy, but only as a bridge to transplantation. The cost effectiveness ratio for DT soon improved substantially; however, partly on the basis of evidence from widespread use in the United States, where coverage was approved. Unless “learning” or “ongoing innovation” is incorporated into coverage decisions, valuable technologies may lose a chance to become cost-effective. In other words, healthcare systems may need enough flexibility to accept short-term inefficiencies to garner long-term value.
The dynamics of ongoing device innovation and learning by using argue for policies that are linked to an infrastructure that captures changing outcomes in everyday practice and permits payment decisions to be revisited as technologies evolve. This holds especially for devices that address serious conditions, hold promise for important unmet needs, are costly, and may entail safety concerns. Many European countries have created an infrastructure of detailed clinical registries, and in the United States, promising public-private partnerships are emerging to do the same. In the case of LVADs, for example, the NIH, FDA, and CMS, in collaboration with industry and hospitals, have created the Interagency Registry for Mechanically Assisted Circulatory Support [INTERMACS] registry to capture data on both the long-term safety and effectiveness of these devices over time and the evolution of patient populations. Participation in the registry is a condition that centers must meet to receive reimbursement for device implantation.
In 2006, Medicare formalized this policy direction by introducing coverage with evidence development (CED), by which coverage of new devices is conditionally linked to ongoing data collection efforts such as clinical trials or data registries (Reference Ascheim, Gelijns and Rose44). A recent example of CED is TAVR, a technology that in the premarketing trial demonstrated important survival benefits for patients who otherwise would succumb to their disease (because they were ineligible for traditional open surgery), but the benefits of which came with a prominent stroke risk. Professional societies joined with Medicare, FDA and industry to create the TAVR registry to address questions about evolving outcomes and patient populations.
In the UK, NICE has also introduced similar innovation by creating in 2009 the Medical Technologies Evaluation Programme (MTEP), which evaluates technologies based on the manufacturers’ claimed advantages and cost savings over current management options. Based on the presented evidence and the claims, the Medical Technologies Advisory Committee will either issue a guidance statement that supports adoption of the device, request additional evidence, or route the request to more stringent, long term assessment programs (Reference Tunis and Pearson45-Reference Campbell47). In circumstances where additional evidence is required, MTEP facilitates access to patients for clinical trials by encouraging collaboration between the manufacturer and NHS providers.
A suitable coverage decision-making model (one that neither introduces questionable devices too quickly into practice nor dismisses as cost ineffective ones that improve with time and experience) is bound to be complex, both conceptually and institutionally. The central elements of such an approach are sustained engagement of stakeholders (producers, providers, payers, and regulatory agencies), who agree to an extensive clinical data collection effort that is then used by payers as a source of evidence to revisit payment decisions. To be sure, this model poses sizable problems of design and implementation. It is hard to marshal stakeholders, find the necessary financial resources, and design a rigorous and efficient data collection system that captures changing outcomes. But to the extent that the necessary institution-building and data accumulation harmonize with developments in information technology that the federal government has been promoting and funding in its health reform initiatives, start up costs and managerial perplexities in the new institutional infrastructure for evaluation of devices should be lower. The evolution of Medicare's policies in this arena is instructive. Having launched the coverage with evidence strategy in 2006, the agency grappled with its many difficulties, wondered whether it should pursue CED, and then, late in 2012, reaffirmed it as a cornerstone of its strategy. For policymakers who want society to enjoy the benefits and avert the risks of innovative medical devices, there is no short cut around the challenges of inventing innovative evaluative approaches to guide decisions.
CONTACT INFORMATION
Annetine C. Gelijns, PhD, International Center for Health Outcomes and Innovation Research, Department of Health Evidence and Policy at the Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, New York 10029
Mark J. Russo, MD, Barnabas Health Heart Centers, West Orange, New Jersey
Kimberly N. Hong, MD, MHSA, International Center for Health Outcomes and Innovation Research, Department of Health Evidence and Policy at the Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, New York 10029
Lawrence D. Brown, PhD, International Center for Health Outcomes and Innovation Research, Department of Health Evidence and Policy at the Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York 10029, and Department of Health Policy and Management, Mailman School of Public Health, Columbia University, New York, New York
Deborah D. Ascheim, MD, International Center for Health Outcomes and Innovation Research, Department of Health Evidence and Policy and The Cardiovascular Institute, Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, New York 10029
Alan J. Moskowitz, MD, (alan.moskowitz@mssm.edu) International Center for Health Outcomes and Innovation Research, Department of Health Evidence and Policy Department of Medicine at the Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, New York 10029
CONFLICTS OF INTEREST
All authors report they have no potential conflicts of interest.