Technological innovations are known to be a major cost driver in health care (Reference Bodenheimer1). Because healthcare expenditures are rapidly increasing and economic pressure is also expanding, there is growing need for early evaluation of the potential value of such innovations.
The health economic evaluation process alongside the development of new technologies can roughly be divided into three phases (Reference Vallejo-Torres, Steuten, Parkinson, Girling and Buxton2;Reference Girling, Young, Brown and Lilford3). In the very early phase, a technology is still in concept, and not yet under development. Generally, decisions on product development have to be made with no, or only limited evidence on costs and effectiveness being available. Once a positive development decision has been made, a technology enters the development stage. Some clinical evidence on costs and effectiveness becomes available, allowing for early stage evaluation such as iterative decision modelling and one-way sensitivity analyses for determining the key parameters influencing the technology's cost-effectiveness (Reference Vallejo-Torres, Steuten and Buxton4).
Health economic evaluations including cost-effectiveness, cost-utility, and value-of-information analyses are mostly performed at a late stage of development. At this postmarket stage, a technology has already been developed and brought to market, and additional costs and effectiveness data are becoming available from clinical studies. If, at this stage, a technology appears not to be cost-effective, it might not be reimbursed. In this case, considerable investments in research and development have been made that cannot be recovered. To predict these wasted resources, there is interest in making reliable estimates of the cost-effectiveness of innovations at an early stage of development.
As a tool to select those technologies that are most likely to provide value for money from a large pool of ideas, Cosh et al. introduced the headroom method (Reference Cosh, Girling, Lilford, McAteer and Young5). This method examines the potential of a new technology under optimistic assumptions. It aims to quantify the room for improvement in current clinical practice by combining the maximum effect an innovation might have, the maximum willingness-to-pay (WTP) for this effect, and the potential downstream expenses and savings. Thus, the headroom method is an assessment of the maximum potential value of an innovation, informing on the upper ceiling price of a technology, to still be cost-effective (Reference Cosh, Girling, Lilford, McAteer and Young5). If it is realistic to assume that the headroom is large enough to cover the costs of the new technology, further development and economic analyses are worth undertaking. If the headroom is too low, on the other hand, one might rather focus on other innovations.
The headroom method was introduced and is currently used on a quality-adjusted life-year (QALY) basis (Reference Cosh, Girling, Lilford, McAteer and Young5–Reference van de Wetering, Steuten, von Birgelen, Adang and IJzerman7). This suggests that effectiveness is measured in terms of QALYs, which are a combined measure of health-related quality of life and life-years gained from an intervention (8). However, not every innovation is expected to result in QALY gain. This is for example the case in technologies that are expected to increase diagnostic yield or expedite diagnosis in untreatable diseases. As these technologies are not expected to result in QALY gain, performing a headroom analysis on a QALY basis would result in direct rejection, even though such a technology might improve clinical practice in other ways and, therefore, be valuable.
Such a scenario arises in the diagnostic trajectory of complex pediatric neurology. Complex pediatric neurologic disorders are a heterogeneous group of predominantly genetic disorders including epilepsy, movement disorders, neuromuscular disorders, and metabolic disorders. The current diagnostic trajectory for these patients is lengthy, resource-intensive, and has a low diagnostic yield (Reference van Nimwegen, Schieving and Willemsen9). Therefore, it is expected that there is ample room for improvement in this diagnostic trajectory, especially in terms of diagnostic yield and duration. As effective treatments for these disorders are currently often not available, no QALY gain as a result of improving this trajectory may be expected.
This study explores the feasibility and usefulness of the headroom method in the very early health economic evaluation of diagnostic technologies that are not expected to result in a QALY gain. The first part of the study will describe the theory behind the headroom method. In the second part of the study, the case of the diagnostic trajectory in complex pediatric neurology will be used as an illustration on the feasibility and usefulness of the headroom method for technologies that are not expected to result in an immediate QALY gain.
THE HEADROOM METHOD
The headroom method is based on the net-monetary-benefit equation as introduced by Hoch et al. (Reference Hoch, Briggs and Willan10). It involves two main aspects, namely the establishment of what is called the “effectiveness gap”, and the actual calculation of the headroom, which is the maximum additional cost at which the implementation of a new technology could still be considered cost-effective (Reference McAteer11). As we are interested in an upper ceiling price for a new technology, this calculation assumes an optimistic scenario, taking into account the maximum effect a new technology might yield (effectiveness gap), society's WTP for this incremental effect and any costs or savings associated with use of the new technology. This results in equation 1, of which each step will be further explained in the following paragraphs.
$$\begin{equation}
Headroom = \hbox{\it effectiveness}\ gap\ \times WTP - net\ additional\ costs
\end{equation}$$
The Effectiveness Gap
New diagnostic technologies are being developed because they are expected to be more effective than current clinical practice. For an innovation to be more effective, there needs to be room for improvement in current practice. This room for improvement is the maximum increase in effectiveness a new technology could provide compared with the reference standard, also defined as the effectiveness gap (Reference Cosh, Girling, Lilford, McAteer and Young5). This is represented by Equation 2, in which max effectnt is the maximum effect the new technology could provide compared with current clinical practice (effectcp ).
$$\begin{equation}
\hbox {\it Effectiveness}\ gap = max\Delta \hbox{\it Effect} = max\ \hbox{\it effec}{t_{nt}} - \hbox {\it effec}{t_{cp}}
\end{equation}$$
There are several measures to express this effectiveness gap, such as life years gained, number of diagnoses, or unnecessary diagnostic tests prevented. However, the preferred outcome measure is the QALY, which combines health-related quality of life and survival.
For example, McAteer et al. assumed that the use of bowel tissue in substitution cystoplasty, which is the reference standard, after resection for bladder cancer, resulted in a median utility score of 0.95. They compared this technique with tissue-engineered bladder substitute, which has a maximum increment in effect of 1 – 0.95 = 0.05. Assuming that bladder cancer patients live for 10 more years after resection, the effectiveness gap in this case would be (1 – 0.95) × 10 = 0.5 QALY (Reference McAteer, Cosh and Freeman12).
Headroom Calculation
As in net-monetary-benefit calculation, the effectiveness gap should be valued in monetary terms. This is done by multiplying the effectiveness gap with a certain maximum WTP value for an additional unit of effect. For QALYs, such a WTP value is in the United Kingdom well established between £20,000 and £30,000 for every additional QALY gained (13). This means that, according to Equation 1, in the example of McAteer et al., the headroom for tissue-engineered bladder substitute equals 0.5 * 30,000 = 15,000 (Reference McAteer, Cosh and Freeman12).
Additionally, the introduction of a new technology might be accompanied with potential downstream expenses or savings, indirect costs such as productivity losses, or wider infrastructural costs such as staff training, which should also be taken into account in a total headroom calculation (Reference Buchanan, Wordsworth and Schuh14). For example, McAteer et al. considered a potential saving in hospital bed days as a consequence of tissue engineering compared with current clinical practice (Reference McAteer, Cosh and Freeman12). They found a mean saving of four hospital days with an average cost of £317 per day. Hence, in their case, an additional saving of £1,268 could be added to the total headroom.
THE HEADROOM OF THE DIAGNOSTIC TRAJECTORY IN COMPLEX PEDIATRIC NEUROLOGY
In this part of the study, we will illustrate the feasibility and the usefulness of the headroom method in the diagnostic trajectory of complex pediatric neurology. First, we will define the effectiveness gap of the current diagnostic trajectory. Second, we will calculate the headroom, given a certain WTP value and taking into account additional savings and expenses.
Assessing the Effectiveness Gap of the Current Diagnostic Trajectory
Complex pediatric neurologic disorders are a heterogeneous group of predominantly genetic disorders and patients present at the neurologist with nonspecific symptoms. Although these patients clearly have a neurologic disorder, their diagnostic trajectories are generally long-term and a clinical diagnosis is established in only a small minority. Despite complex pediatric neurologic disorders being generally untreatable, a definitive diagnosis might end the diagnostic Odyssey and provide useful information on disease etiology, prognosis, and / or family planning.
Currently, the diagnostic trajectories of these patients take on average 40 months, include extensive imaging, neurophysiologic, and genetic testing, and result in a definitive diagnosis for only 6% of the patients (Reference van Nimwegen, Schieving and Willemsen9). These numbers suggest ample room for improvement in this diagnostic trajectory. In the most optimistic scenario, only two physician visits and one diagnostic test are required to provide a definitive diagnosis for every patient. This could reduce the length of the diagnostic trajectory considerably, while increasing diagnostic yield from 6% to 100%. This results in an effectiveness gap of 94% or 0.94. Note that we are not assuming that in reality a new technology will close all of the effectiveness gap. The effectiveness gap here is the maximum effect that could be achieved hypothetically, informing on an upper ceiling price for a new technology to be still cost-effective. In this way, we assume the most optimistic scenario, with the new technology being a perfect test, providing a diagnosis for all patients without false positive or false negative results.
Monetizing the Effectiveness Gap
To monetize the effectiveness gap, we need to know the monetary value for an additional unit of benefit. Although for QALYs the WTP is well established between £20,000 and £30,000, for intermediate outcomes WTP values are less clear. In 2009, Regier et al. studied families of children with idiopathic developmental disability, in which the exact cause of the disability was unknown (Reference Regier, Friedman, Makela, Ryan and Marra15). Using a discrete choice experiment, they simultaneously obtained monetary values for increasing the diagnostic yield, and for reducing the interval between presentation and diagnosis. They presented test scenarios with varying levels of diagnostic yield (10, 14, 20, and 25 of 100 children tested); time waiting for results (1, 3, 6, and 12 weeks); and cost to the family (CND$750, CND$1100, CND$1750, and CND$2500).
They found that families of patients were willing to pay approximately €85 (CND$131) for one additional child receiving a diagnosis in every 100 patients tested. Converting this 2009 WTP value to the 2015 price level, using price indices, suggests a WTP of €95 per percentage point additional diagnostic yield (16). As mentioned before, in the most optimistic scenario, the diagnostic yield in complex pediatric neurology will increase from 6% to 100%, resulting in an effectiveness gap of 94%. Assuming that the WTP value of €95 found by Regier et al. is a representative value for the societal WTP for one extra diagnosis in every 100 patients in pediatric neurology, the monetized effectiveness gap in current practice would be 94 × €95 = €8,930.
Net Additional Costs
Implementing new technologies into clinical practice might result in additional expenses or savings. As the majority of complex pediatric neurologic disorders have a genetic origin, it is likely that innovations in this diagnostic pathway are in the field of next-generation sequencing (NGS). Applying these technologies in clinical practice may lead to so-called incidental findings, which are clinically relevant mutations that are not related to the disease under investigation (Reference de Ligt, Willemsen and van Bon17). These might lead to follow-on testing and treatment initiation, potentially with complications, and thereby cause downstream expenses. Although these incidental findings are very rare, their downstream costs can be considerable (Reference Buchanan, Wordsworth and Schuh14). In contrast, the application of NGS early in the diagnostic trajectory might partly substitute current diagnostic testing, especially genetic tests.
For this headroom analysis, we assume an optimistic scenario, consistent with its underlying philosophy, which is to determine an upper ceiling price to be still cost-effective in such a scenario. To this end, we assume that there are no downstream costs, as incidental findings are very rare. Moreover, NGS is assumed to substitute all genetics tests of the current diagnostic trajectory, as well as 25% of all physician contacts. These substitutions would result in savings equaling €6,098 per patient (Reference van Nimwegen, Schieving and Willemsen9).
Interpreting the Headroom
When ignoring additional costs or savings, the headroom in the current diagnostic trajectory of complex pediatric neurology equals €8,930. This means that if a new technology would be introduced as an add-on test, the maximal marginal cost would be €8,930.
Taking into account the expected savings of €6,098 by substituting all genetic tests and 25% of the physician visits, the total headroom equals €8,930 + €6,098 = €15,028. Of course, there is uncertainty around the number of current diagnostics that will be replaced by a new technology. To address this uncertainty, additional scenario analyses regarding these substitution savings can be performed to reduce the chance of investing in a technology that at a later stage turns out not to be cost-effective. Figure 1 shows the total headroom in the diagnostic trajectory of complex pediatric neurology in the case of an add-on test and in a scenario in which the new technology partly substitutes current diagnostics.
Total headroom in the diagnostic trajectory of complex pediatric neurology. ΔE, effectiveness gap; WTP, willingness-to-pay; WES, whole exome sequencing.
DISCUSSION
This study shows that headroom analysis is a feasible and useful tool in the early health economic evaluation of diagnostic technologies, also when no immediate QALY gain is expected. An effectiveness gap calculation is informative on the room for improvement in current clinical practice. It indicates whether there is a problem, and allows for quantification of this problem. Additionally, combining a monetary valuation of the effectiveness gap with the foreseeable downstream costs and savings due to a new technology provides an estimation of the potential societal value of this technology. In the case of complex pediatric neurology, headroom analysis indicated a large room for improvement, as in the current trajectory only 6% of all patients receive a diagnosis. In addition, it is realistic to assume that savings will materialize because new technologies can substitute a considerable part of the current diagnostic trajectory. A total headroom of €15,028 was found, indicating that a new technology in this diagnostic trajectory could be cost-effective if its costs do not exceed €15,028.
Although definitely informative, it should be noted that the headroom method has certain limitations. Most notably, the headroom method assumes an optimistic scenario, with the effectiveness gap considering the maximum effect a new technology might theoretically gain. However, in clinical practice, new technologies are unlikely to be perfect and will, therefore, not close the entire effectiveness gap. Therefore, the established ceiling price is likely to be an overestimation, and it cannot be guaranteed that a technology that is brought to the market for less than this ceiling price will actually be cost-effective.
On the other hand, Chapman et al. show that with a specificity of 92%, the headroom method is a very valuable tool in no-go decisions, to avoid investment in technologies that could never be cost-effective, as its expected costs exceed the calculated headroom (Reference Chapman, Taylor and Girling18). Clearly, one could also calculate the effectiveness gap with a range of more realistic estimates of effectiveness in a scenario analysis on the effectiveness. Finally, updating the initial headroom analysis when more evidence becomes available, gives more accurate estimations on the potential value of a technology and thereby reduces chances on investing in technologies that are unlikely to meet societal criteria of cost-effectiveness (Reference Girling, Lilford, Cole and Young19).
The same goes for downstream costs and additional expenses and savings. In early stages, these are highly uncertain. It is hard to decide to what extent and over what time these costs and savings should be taken into account, and to what extent these will vary within and between countries. Here too, scenario analyses can provide insight in the expected value of a new technology.
Finally, the main issue is the fact that intermediate outcomes are difficult to value in monetary terms, complicating headroom analyses of technologies that do not result in QALY gain. Although for QALYs a clear WTP threshold has been established, this is often not the case for intermediate outcomes. Determining the WTP for intermediate outcomes can be done by using, for example, contingent valuation or discrete choice experiments (Reference Lin, Cangelosi, Lee and Neumann20;Reference Johnson, Lancsar and Marshall21). However, these data are seldom readily available, and such experiments are relatively time-consuming and expensive to conduct, requiring large numbers of respondents (depending on, e.g., design, number of choice sets, attributes and levels) (Reference Lancsar and Louviere22). As intermediate outcomes may be valued differently in varying disease areas or over time, such experiments should be performed for every specific context separately (Reference Grosse, Wordsworth and Payne23).
Another issue with the elicitation of WTP values is the choice on whose preferences to incorporate; the patients’ or the general public's. The study of Regier et al., on which the WTP values in this study were based, used the preferences of family members of the patients to elicit WTP. Hence, these WTP values are useful for diagnostics that have to be paid out-of-pocket. However, in many countries, such diagnostics will be covered by health insurance. In these cases, WTP values should be based on the preferences of the general public (24). Taking these factors into account makes the transferability of the WTP values of Regier et al. to our case study questionable.
Despite these limitations, the headroom method seems a useful tool. For the supply side, it does not only inform on the commercial opportunity of a technology by establishing a maximum ceiling price for which it could be cost-effectively brought to market. It is also a quick method for rapid decision making in both selecting the most promising concepts from a larger pool of options, and in investment and development decisions (3). Especially when combined with some additional scenario analyses on both costs and effectiveness parameters, it could be helpful in channeling research and development resources toward those technologies that are most promising.
CONCLUSION
The headroom seems a valuable tool in the very early evaluation of medical technologies, even when no immediate gain in QALYs is expected. It is informative on the room for improvement in a certain disease area and allows for selecting the most promising concepts from a larger pool of options, for decision making regarding investments in research and further development, and for calculating a maximum ceiling price to still be-cost effective. As an optimistic scenario is assumed, it indicates the potential for, but does not guarantee future cost-effectiveness and / or reimbursement. Hence, the headroom method might be most valuable as a rule-out tool to avoid investment in technologies that are very unlikely to be able to meet even the most generous criteria for cost-effectiveness.
CONFLICTS OF INTEREST
All authors declared they have no conflicts of interest to declare.
