Crossref Citations
This article has been cited by the following publications. This list is generated based on data provided by
Crossref.
Grutters, Janneke P.C.
Kluytmans, Anouck
van der Wilt, Gert Jan
and
Tummers, Marcia
2022.
Methods for Early Assessment of the Societal Value of Health Technologies: A Scoping Review and Proposal for Classification.
Value in Health,
Vol. 25,
Issue. 7,
p.
1227.
Hospodková, Petra
Rogalewicz, Vladimír
and
Králíčková, Michaela
2023.
Gynecological Speculums in the Context of the Circular Economy.
Economies,
Vol. 11,
Issue. 2,
p.
70.
Rodriguez Llorian, Elisabet
Waliji, Louloua Ashikhusein
Dragojlovic, Nick
Michaux, Kristina D.
Nagase, Fernanda
and
Lynd, Larry D.
2023.
Frameworks for Health Technology Assessment at an Early Stage of Product Development: A Review and Roadmap to Guide Applications.
Value in Health,
Vol. 26,
Issue. 8,
p.
1258.
Scholte, Mirre
Heidkamp, Jan
Hannink, Gerjon
Merkx, M.A.W. (Thijs)
Grutters, Janneke P.C.
and
Rovers, Maroeska M.
2023.
Care Pathway Analysis to Inform the Earliest Stages of Technology Development: Scoping Oncological Indications in Need of Innovation.
Value in Health,
Vol. 26,
Issue. 12,
p.
1744.
Boudewijns, Esther A.
Otten, Thomas M.
Gobianidze, Mariam
Ramaekers, Bram L.
van Schayck, Onno C. P.
and
Joore, Manuela A.
2023.
Headroom Analysis for Early Economic Evaluation: A Systematic Review.
Applied Health Economics and Health Policy,
Vol. 21,
Issue. 2,
p.
195.
Abu-Zahra, Teebah
Grimm, Sabine E.
Scholte, Mirre
Raymakers, Adam J.N.
Kesselheim, Aaron S.
and
Joore, Manuela
2024.
How health technology assessment can help to address challenges in drug repurposing: a conceptual framework.
Drug Discovery Today,
Vol. 29,
Issue. 6,
p.
104008.
Belowska, Joanna
2025.
Between Patents and Patients: Why Intellectual Property Alone Cannot Sustain Drug Repurposing.
IIC - International Review of Intellectual Property and Competition Law,
Vol. 56,
Issue. 9,
p.
1656.
Owusu Achiaw, Samuel
Hawkins, Neil
Wu, Olivia
and
Mercer, John
2025.
Assessing the Value of Further Investment in R&D Using Mixed Methods: A Case Study of Biosensor-Integrated Arteriovenous Grafts.
Journal of Market Access & Health Policy,
Vol. 13,
Issue. 1,
p.
1.
Abu-Zahra, Teebah
Grimm, Sabine E.
Scholte, Mirre
and
Joore, Manuela
2025.
Can We Make Health Economic Decision Models as Simple as Possible, But Not Simpler? Introducing SMART tool.
PharmacoEconomics,
Vol. 43,
Issue. 10,
p.
1235.
Grutters, Janneke P.C.
Bouttell, Janet
Abrishami, Payam
Ahmed, Sulafa. Y.M.
Cole, Amanda
Dawoud, Dalia
Fernández-Barceló, Carla
Frederix, Geert W.J.
Hawkins, Neil
Karnon, Jonathan
Leadbitter, Sharon
McCabe, Christopher
Mueller, Jani
Achiaw, Samuel Owusu
Partington, Andrew
Sampietro-Colom, Laura
Seyahian, Abril
Sucu, Rabia
Tew, Michelle
van Katwyk, Sasha
Verboven, Yves
and
Wang, Yi
2025.
Defining early health technology assessment: building consensus using Delphi technique.
International Journal of Technology Assessment in Health Care,
Vol. 41,
Issue. 1,
Hainsworth, Rob
Eden, Martin
Ghosh, Jonathan
Hansrani, Vivak
Rogers, Steven
McCollum, Charles
Rogers, Gabriel
and
Payne, Katherine
2025.
Detection and Coil Embolisation of Pelvic Vein Incompetence for Chronic Pain in the Female Pelvis: A Development Phase Economic Analysis.
European Journal of Vascular and Endovascular Surgery,
Vol. 69,
Issue. 3,
p.
474.
Mekniran, Wasu
and
Kowatsch, Tobias
2026.
Early viability assessment of a Business-to-Consumer (B2C) model for digital diabetes screening in Switzerland.
BMC Health Services Research,
Vol. 26,
Issue. 1,
Abu-Zahra, Teebah
Grimm, Sabine
Scholte, Mirre
Casas, Ana I.
Saliba-Gustafsson, Peter
Parini, Paolo
and
Joore, Manuela
2026.
An Early Value Assessment of Biomarker-Guided Precision Medicine for Treatment-Resistant Hypertension: A Headroom and Threshold Analysis.
Applied Health Economics and Health Policy,
Vol. 24,
Issue. 6,
p.
1093.
Abu-Zahra, Teebah
Grimm, Sabine E.
Changoer, Prashant
Gerber, Nicolas U.
Mathes, Stephanie
Netea-Maier, Romana T.
and
Joore, Manuela
2026.
SMART Planning of Early-Stage Health Economic Decision Models for Mechanism-Based Precision Treatments in Rare Cancers: An Application and Tool Update.
PharmacoEconomics,
Vol. 44,
Issue. 10,
p.
1171.
Introduction
Health Technology Assessment (HTA) conducted to inform decision making during the development of health technologies, which we term “development-focused HTA” (DF-HTA), has a number of distinct features when compared with HTA conducted to inform reimbursement and usage decisions (use-focused HTA). We have described these features in a previous article (Reference Bouttell, Briggs and Hawkins1). These features may influence the selection of appropriate methods for DF-HTA.
Firstly, early in the development process, evidence specific to the technology being developed will be limited and rapidly evolving. However, in use-focused HTA, evidence specific to the technology under consideration will be available and the evidence base will be comparatively better established. Secondly, DF-HTA may be used to inform a wide range of decisions concerning whether or not to continue to invest in the development of a technology and the commissioning and design of further studies and analyses to support development and commercialization. In turn, these decisions will require estimates of the acceptable price, expected revenues, and comparative effectiveness for a technology, potentially across a range of candidate clinical contexts reflecting use in different positions in a clinical pathway (which themselves may vary across target markets) and technology design options. By way of contrast, use-focused HTA will be used to inform essentially binary decisions about whether or not a technology should be made available in a specific clinical context.
Thirdly, DF-HTA may need to consider the evaluation of technologies across multiple potential markets that have differing decision-making criteria and evidential requirements. For example, some markets may focus on estimates of cost-effectiveness obtained from decision analytic modeling, whereas others may focus on assessments of comparative clinical value and pricing of comparable products. Use-focused HTA will typically focus on a specific market with well-defined evaluation criteria.
Finally, resources for HTA analysis will often be constrained. Small companies who do not yet have technologies on the market are likely to have limited funds available and larger companies will have multiple candidate technologies competing for available development funds. Proponents of DF-HTA will need to convince potential funders that DF-HTA activities will deliver a positive expected net return on the investment in these activities. There are basically two ways in which investment in DF-HTA activities can deliver a return: (i) by increasing expected revenue for a product by increasing the achieved price and/or by increasing sales volume or (ii) by reducing development costs compared with what would have been achieved had the DF-HTA activities not taken place.
It should be noted that a return in terms of increased revenues can be realized where the development is successful and a technology is commercialized. This is an important issue given the high rates of failure during early development. In principle, DF-HTA could increase expected returns on investment by helping to end development at an earlier stage for technologies that ultimately fail to generate revenue sufficient to cover their development costs. However, given that most technologies fail due to safety issues or a lack of efficacy (Reference Waring, Arrowsmith, Leach, Leeson, Mandrell and Owen2;Reference Hay, Thomas, Craighead, Economides and Rosenthal3), it is unclear how effective DF-HTA will be at terminating development for high-risk candidates.
Developing medical technologies of all kinds is high risk, as investment costs are high and the chances of any individual technology in development reaching commercial success are relatively low (Reference Lehoux, Williams-Jones, Miller, Urbach and Tailliez4). In pharmaceuticals, for example, it is estimated that 40 percent of candidates entering clinical trial phases fail at phase one, 64 percent at phase two, 38 percent at phase three, and a further 10 percent fail before approval (Reference DiMasi, Grabowski and Hansen5). This means that just under 12 percent of pharmaceuticals entering clinical trials are eventually approved (Reference DiMasi, Grabowski and Hansen5). Comparable data are not available for medical devices. In order to produce expected positive returns when used early in development, the costs of DF-HTA need to be commensurate with expected returns on the activity. This means that what might be regarded as best practice for use-focused HTA may not be appropriate in DF-HTA; DF-HTA is not simply HTA conducted earlier in development.
This article aims to set out broad groups of methods available to the DF-HTA analyst (who may be an academic, a consultant, or an employee of the developer's company) to provide examples of how and when the methods have been used and how they are adapted to the features of DF-HTA. We present a toolkit of methods for the DF-HTA analyst to use at different stages of a research and development process. We hope it will be used in conjunction with the framework of features of DF-HTA presented in our earlier article (Reference Bouttell, Briggs and Hawkins1) in order to improve the clarity and appropriateness of methods used in DF-HTA. We also seek to contribute to a dialogue about the nature of, and the best approach to undertaking, DF-HTA.
We identified methods of DF-HTA from a review of published literature. The literature review used a pearl-growing approach (Reference Claxton, Martin, Soares, Rice, Spackman and Hinde6) that aimed to identify examples of methods papers where the intended audience was developers of medical technologies. We used Ijzerman and Steuten's (Reference Ijzerman and Steuten7) 2011 review article as the initial pearl. This approach involves reviewing the references and citations of the pearl for articles of interest and then reviewing the references and citations of the articles of interest until saturation is reached. The search was undertaken in October 2017 and refreshed in February 2019. From the initial review, seven studies listed and described a selected group of methods (Reference Ijzerman and Steuten7–Reference Miller13). The Supplementary Materials include a table showing how the methods listed in previous review papers were categorized for the purposes of this paper (Supplementary Table 1) and provide definitions of methods of DF-HTA (Supplementary Table 2).
Methods of Development-Focused HTA
Table 1 shows three groups of methods useful in DF-HTA: literature review; stakeholder consultation; and decision analytic modeling.
Analytic methods of DF-HTA
DCE, Discrete Choice Experiment; DF-HTA, development-focused HTA; MCDA, multicriteria decision analysis; SEE, Structured Expert Elicitation. See Supplementary Material for source articles and definitions of methods.
The utility of each of the methods shown in Table 1 in DF-HTA depends upon the stage of development of the technology. Figure 1 shows an outline of a generic research and development process for a health technology. It is simplified in that it shows a simple linear process when, in reality, most research and development processes are more complex and involve elements of iteration. The figure also includes the purpose and methods of HTA that may be appropriate at each stage. In the first stage, DF-HTA is helping developers to articulate the value proposition for the technologies, and at this stage, there may be multiple value propositions. As the research and development of the technology progresses, methods become more familiar to analysts undertaking HTA for reimbursement processes. The same methods are relevant at each stage, but the sophistication of the method increases as the development process extends when more resource for assessment may be available.
Purpose and methods of health technology assessment at different stages of a research and development process.
Stage 1: Basic Research
In this stage, DF-HTA activities can help developers to articulate the potential value propositions for the technology in development. By “value proposition,” we mean how the technology fulfills some need or want that potential customers may value. It is key at this stage to understand the current clinical pathways in jurisdictions of interest and how the technology may influence these pathways. Resources are likely to be restricted and methods of literature review, stakeholder consultation, and decision analytic modeling need to be quick and simple.
Literature review is a key source of information in HTA. Published literature can provide information about prevalence and incidence in relevant populations, clinical pathways, current and potential comparators, and estimates of costs and clinical effectiveness to populate decision analytic models. In the earliest stage of development, targeted literature reviews looking at only a small number of authoritative sources or even a single source may be appropriate (Reference Chapman14–Reference Brandes, Sinner, Kääb and Rogowski16). Clinical experts (and/or the developers) may be able to identify key sources such as relevant recent trials, guidelines, or comprehensive HTA reports.
Both qualitative and quantitative methods of consulting clinical experts, patients, or other stakeholders are widely used in DF-HTA. At the earliest stage of technology development, only the most informal methods are likely to be feasible (interviews, for example) and the number of stakeholders and range of settings may be limited. Qualitative methods of stakeholder consultation encourage early interaction with stakeholders which can help position the technology, articulate the value proposition/s, and may lead to changes in direction of the development or design of the technology. A clear set of value propositions may help developers to raise finance and market their technology. Scenarios developed through stakeholder interaction may be useful to indicate where to focus effort in cost-effectiveness analysis. A technology needs to have the potential to add clinical value in terms of better health outcomes unless it is likely to be cheaper than the existing standard of care. Although it may be difficult to generalize results from consulting a small group of stakeholders from limited settings, it should be remembered that at this stage, we are focusing on the potential of the technology to add value. If it can do that in one place in the opinion of one set of stakeholders, then this may be sufficient to justify continuation of the development.
Decision analytic modeling can be used at the earliest stages in development to estimate commercial pricing (headroom) or a threshold for performance for the technology to be considered cost-effective in a given jurisdiction. This is useful for anticipating the maximum achievable price in jurisdictions where cost-effectiveness analysis based on an explicit threshold is a primary driver of reimbursement and usage decisions. Headroom is calculated by multiplying the increase in Quality-Adjusted Life Years (or other relevant effectiveness metric) expected to be generated by the introduction of the new technology by an acceptable cost-effectiveness threshold and then adjusting for cost savings/additional costs included in the calculation. The headroom estimate represents the maximum price achievable for the technology if it is to be considered cost-effective (Reference Chapman14;Reference Vallejo-Torres, Steuten, Buxton, Girling, Lilford and Young17). This price can be multiplied by the expected volume of sales and costs of development deducted to provide an initial estimate of return on investment for the developer. The analysis may focus only on cost savings rather than valuing health outcomes (Reference Markiewicz, Van Til, Steuten and Ijzerman18) and may even be limited to the analysis of a single category of savings (Reference McAteer, Cosh, Freeman, Pandit, Wood and Lilford19).
Example
Kluytmans et al. (Reference Kluytmans, Tummers, Van Der Wilt and Grutters20) undertook an “early assessment of proof of problem” relating to a surgical instrument in development. The assessment comprised a stakeholder consultation and headroom calculation. The value proposition put forward by the developers was that the new flexible instrument could save time and reduce infections in meniscus surgery. The stakeholder consultation took the form of interviews with orthopedic surgeons from two settings in the Netherlands. The surgeons considered that the maximum time saving that could be achieved was 5 minutes from a 25-minute operation, which would probably not be enough to impact theater schedules. They also noted that infection rates were already very low. Using the information from the stakeholder consultation, a headroom estimate was calculated. The estimate took an optimistic approach that assumed that the technology delivered the full 5-minute time saving and reduced infections to zero. A range of time savings and assumptions about price of the instrument were modeled to inform the developers. The conclusion of the analysis was that the instrument may need to be focused on an alternate indication as the estimated headroom was only €55.
Stage 2: Translational Research
In this stage, more resource may be available to extend the methods of DF-HTA beyond the quickest and simplest approaches described in Stage 1. Literature reviews may become more extensive and include a wider range of sources. Reviews are still likely to fall short of systematic reviews, as even the briefest systematic review using automated tools has been estimated to require over 60 man hours (Reference Clark, Glasziou, Del Mar, Bannach-Brown, Stehlik and Scott21) and a typical review takes over 800 man hours' estimate (Reference Pham, Bagheri, Rios, Pourmasoumi, Robson and Hwee22). The range of sources at this stage may include clinical guidelines, previous HTA reviews, online patient forums, regulatory reviews (such as European Public Assessment Reports), summaries of product characteristics and market analyses. Stakeholder consultation from the earliest stage of DF-HTA may have identified important sources of, or requirement for, information. For literature reviews during product development, Abel et al. (Reference Abel, Shinkins, Smith, Sutton, Sagoo and Uchegbu23) recommend a “targeted” approach where searches are conducted on a flexible, iterative basis and are interspersed with expert elicitation and stakeholder consultation.
Stakeholder consultation may also begin to use more formal methods and a wide range of both qualitative and quantitative methods are available. Relevant stakeholders will generally be potential users of the new technology (patients and clinicians) and/or decision makers who may ultimately be involved in the purchase of the technology. Qualitative methods include interviews, focus groups, surveys, patient forums, online discussion groups, and “Delphi” panels. Delphi panels involve multiple rounds of consultations with experts with the aim of reaching consensus (Reference Iglesias, Thompson, Rogowski and Payne24). Quantitative methods include multicriteria decision analysis (MCDA), discrete choice experiments (DCE), and structured expert elicitation (SEE).
MCDA involves identifying a set of criteria for decision making, determining the weight of each of the criteria, then scoring the alternatives against the identified criteria (Reference Ryan, Scott, Reeves, Bate, Van Teijlingen and Russell25). A simple points allocation approach is feasible where resources are limited (Reference Ryan, Scott, Reeves, Bate, Van Teijlingen and Russell25). A key issue in the use of MCDA in DF-HTA is the extent to which the MCDA exercise can formulate the questions and mimic the weights that actual decision makers will use.
In DCE, respondents are asked to choose their preferred option among a set of options with differing attributes, one of which may be price. This reveals preferences and estimates of the relative importance of and willingness to trade between individual attributes associated with healthcare interventions or services. It differs from MCDA in that the “weights” given to the individual attributes are revealed through the choices made rather than being explicitly elicited. Again, this could be used to model future decisions regarding the reimbursement or purchase of a new technology. In such case, the choices made should represent the preferences of the reimbursement agency or individual that will ultimately make the decision.
Structured Expert Elicitation (SEE) uses quantitative methods of elicitation. Estimates may include probability distributions making them particularly suited to populate decision models for cost-effectiveness analysis. Online resources are available for SEE (Reference Oakley and O'Hagan26). Structured methods such as SEE, MCDA, and DCE are likely to be beyond the budget of developers of health technologies in the early stages of development. Bojke et al. (Reference Bojke, Soares, Colson A, Fox and Jackson27) recently undertook an expert elicitation exercise to validate a new framework and this took 5 months of researcher time. Indeed, even in the relatively well-resourced environment at the National Institute for Health and Care Excellence (NICE), expert elicitation is often done informally (Reference Peel, Jenks, Choudhury, Lovett, Rejon-Parrilla and Sims28). Our review identified a number of examples of the application of structured methods of stakeholder consultation, but studies have generally been undertaken in the context of a translational research body such as the Center for Translational Molecular Medicine (CTMM) (Reference Hummel, Boomkamp, Steuten, Verkerke and Ijzerman29;Reference Wissing30). Strengths of quantitative methods of stakeholder consultation include the explicit consideration of a range of relevant criteria that may inform developers in the design or required performance of the technology. The main limitation of these methods is the expertise and resource required to conduct them.
In the translational research stage, decision analytic modeling can be used to explore a range of potential value propositions. For example, a technology may have potential in a number of places in the clinical pathway. A number of models could be produced to explore different levels of performance at different places in the clinical pathway. As there is generally little clinical evidence available and resources may still be constrained, modeling methods need to be simple. There may be a focus on cost–minimization or cost–consequence analysis where this is seen as sufficient to inform decision making by developers. Indeed, the development of a conceptual framework for a future cost-effectiveness analysis might be sufficient to identify evidential needs and to inform decisions regarding the commissioning and design of future studies. Other possible simplifications include shorter time horizons and the use of intermediate outcomes. This may avoid the need for sophisticated modeling, but care needs to be taken to ensure that the consequences of these simplifications are understood.
Grutters et al. (Reference Grutters, Govers, Nijboer, Tummers, Van Der Wilt and Rovers31) set out details of thirty-two assessments of technologies in development undertaken by their academic group in the Netherlands. In all thirty-two cases, the models developed were simple, deterministic models and sensitivity analysis undertaken was one-way rather than probabilistic. However, Annemans (Reference Annemans, Genesté and Jolain32) has argued that models constructed while technologies are in development should be at the same level of complexity as required to support an adoption decision at a later stage. It is argued that this is required so that the full complexity of the clinical and disease pathways is captured and all gaps in evidence identified. However, this approach may not be available to smaller developers where resources are constrained and may also be inefficient given that a large proportion of technologies in development do not reach commercial launch (Reference DiMasi, Grabowski and Hansen5).
It should also be noted that an iterative approach can be taken using a simple, deterministic model at first and then developing a more complex model as required when specific clinical evidence becomes available. Initial low resource models may provide estimates that guide developers in positioning the technology and identify price and performance thresholds.
Example
Gc et al. (Reference Gc, Manca, Casson, Antrobus and Iglesias33) conducted an early HTA of wearable digital health technologies (WDHTs) to monitor chronic kidney disease (CKD) to support research and development decisions. As well as a literature review, the project included qualitative stakeholder consultation with patients and developers about what patients wanted and what this technology would be likely to add to existing provision. Economic evaluation was carried out in two stages: first, a simple initial evaluation identified key drivers of value for money, and then, following a discrete choice experiment that shed light onto patient preferences, a model-based cost-effectiveness analysis was undertaken incorporating headroom analysis, return on investment, and one-way sensitivity and scenario analyses. The analysis showed that the novel WDHT had the potential to be cost-effective in the UK at a price of up to £280.
Stage 3: Clinical Research
As evidence specific to the technology starts to emerge, through the development process, more advanced methods of decision analytic modeling are appropriate. Stakeholder consultation becomes less relevant at this stage as the application of the technology has been narrowed down and the most promising indication and position in pathway identified. As evidence of costs and clinical effectiveness becomes available, probabilistic analysis can be undertaken, which quantifies the level of uncertainty. The value of information analysis, undertaken from the perspective of expected commercial returns for commercial sector developers, can be ascertained to illustrate where more evidence would be useful to reduce uncertainty. Such research may be worthwhile for the developer if reduced uncertainty increases the chances of the technology being adopted and thereby increasing revenues beyond the cost of the evidence-generating activities. The evidence base for health technologies other than pharmaceuticals may be underdeveloped as medical devices can be licenced in many jurisdictions with minimal levels of clinical evidence. There is a disincentive to produce evidence as “fast-followers” may benefit without incurring cost if intellectual property protection is weak. In practice, resource constraint and the underdeveloped evidence base may restrict the use of VOI for DF-HTA. None of the 32 DF-HTA projects described by Grutters et al. (Reference Grutters, Govers, Nijboer, Tummers, Van Der Wilt and Rovers31) involved VOI and Steuten et al. (Reference Steuten, van de Wetering, Groothuis-Oudshoorn and Retèl34) found only three examples of VOI informing premarket research and development. VOI conducted from a commercial perspective may be carried out more frequently, but the extent of this is not clear. Budget impact analysis may be a useful extension of CEA undertaken using a decision analytic model, as budget impact forms a part of reimbursement decision making in some jurisdictions.
Example
Vallejo-Torres et al. presented an iterative economic evaluation of absorbable pins for hallux valgus at three different stages of development (Reference Vallejo-Torres, Steuten, Parkinson, Girling and Buxton35). The authors used retrospective data for this analysis to recreate the dynamic process of DF-HTA occurring in real-time alongside the development process. At the earliest stage, the authors used observational data on standard methods and assumptions to estimate headroom for the absorbable pins. When some data became available specific to the new technology, the authors built a decision model and presented one-way sensitivity analysis. Only when evidence was available from a randomized controlled trial did the authors introduce additional complexity to their model in the form of probabilistic sensitivity analysis and value of information analysis.
Discussion
In this paper, we set out three broad groups of methods of DF-HTA: literature review, stakeholder consultation, and decision analytic modeling. We explained how the use of these methods changes throughout the development of a technology and we provided examples. We also described how the methods were adapted to the particular features of DF-HTA. DF-HTA seeks to inform a broad range of decisions such as whether to continue to invest (go/no go decisions), pricing strategy, optimum positioning in clinical pathways, technology design, and evidence generation strategy. Methods need to be suited to answering this broad range so that DF-HTA has much greater focus on consultation with stakeholders than use-focused HTA. In particular, stakeholders can be useful to help position the technology at the earliest stages of development, to map out care pathways, and to provide estimates of parameter values in the absence of clinical evidence specific to the technology. Evidence from the literature is also of importance in DF-HTA, but resource constraints mean that systematic reviews are seldom feasible and targeted reviews may need to be undertaken or expert guidance provided about the most authoritative sources to use. Informal methods of stakeholder consultation and simpler forms of decision analytic modeling (such as cost–minimization or cost–consequence rather than cost–utility analysis, deterministic models, and one-way sensitivity analysis) may be required when resources are constrained.
These adaptations to methods may sit uncomfortably with experienced HTA analysts accustomed to the rigorous approaches required by publicly accountable reimbursement bodies such as the National Institute for Health and Care Excellence (NICE) in the UK (36). However, it should be remembered that in case of resource constraint, there may not be a choice between formal and adapted methods, a choice just between adapted methods, and no HTA input (Reference Craven, Allsop, Morgan and Martin37). We would argue that developers have much to gain from the consideration of issues such as where the technology should be positioned, the articulation of its value proposition/s, initial cost-effectiveness analysis calculations, and informal contact with stakeholders even at the earliest stages of development. The analyst must, of course, take care to ensure that the limitations of any adapted methods used are made clear to the developers at each stage of the analysis.
Current guidance on appropriate methods in the academic literature is limited. Commercial DF-HTA is often not published as there is no incentive to do so and the contents may be commercially sensitive (Reference Grutters, Govers, Nijboer, Tummers, Van Der Wilt and Rovers31). There are bodies of work from projects such as Multidisciplinary Assessment of Technology Centre for Healthcare (MATCH UK) (Reference Chapman14;Reference Craven, Allsop, Morgan and Martin37–Reference Girling, Young, Brown and Lilford39) and the Center for Translational Molecular Medicine (CTMM) (Reference Ijzerman and Steuten7;Reference Markiewicz, Van Til, Steuten and Ijzerman18;Reference de Graaf, Postmus, Westerink and Buskens40;Reference Markiewicz41) that we have drawn on in this study. However, this published work may not be representative of DF-HTA, as within bodies such as MATCH and CTMM, more resource may be available than is typically available for DF-HTA. Previous review articles have set out and described selected methods considered useful in the assessment of technologies in development (Reference Ijzerman and Steuten7–Reference Miller13). Markiewicz et al. (Reference Markiewicz, van Til and IJzerman11) set out to provide a comprehensive listing of methods of what the authors termed “Early HTA.” Stome et al. set out analytical approaches useful in the early assessment of innovation in a healthcare setting (Reference Støme, Moger, Kidholm and Kvaerner42). This paper builds on these previous articles by grouping similar methods and removing methods that are not methods of assessment (such as bench studies and clinical trials that we would class as methods of research and development). Although we acknowledge that our categorization of methods is subjective, we believe that the simplification is justified as it makes the selection of methods of DF-HTA easier for the analyst. Recently, Grutters et al. (Reference Grutters, Govers, Nijboer, Tummers, Van Der Wilt and Rovers31) provoked a debate about the appropriate methods of health economic modeling. In this paper, we make a further contribution to that debate.
There are several limitations to our work. The most important limitation is that little DF-HTA is published due to commercial sensitivities and a lack of incentive to publish (Reference Love-Koh43). Our literature review may not have retrieved all relevant articles and both our inclusion criteria and categorization were necessarily subjective. However, we hope that this toolkit of methods, together with our framework of features previously described (Reference Bouttell, Briggs and Hawkins1), will provide an introduction to the particular requirements of HTA conducted to inform developers. For the wider academic community, we hope to add to the debate about what are appropriate methods of HTA to inform developers. Further research that would be of use includes studies examining the methods used in DF-HTA in the commercial context.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462321000507.
Acknowledgments
JB was funded by the Glasgow Molecular Pathology Node, an MRC ERSC funded project (MR/N005813/1). The other authors received no funding in relation to this work.
Conflict of Interest
AB reports personal fees outside the submitted work from Takeda, Roche, GSK, Novartis, Merck, Bayer, Daiichi Sankyo, and Astra Zeneca. NH reports personal fees outside the submitted work from Janssen-Cilag, Daiichi Sankyo, Bristol Myers Squibb, Bayer, and Lilly. JB reports no conflict of interest.