The term disinvestment in health care is used with a range of meanings, although most of them refer to the processes of withdrawing (partially or completely) health resources from any existing healthcare practice, procedures, technologies, or pharmaceuticals that are deemed to deliver little or no health gain for their cost, and are thus not efficient health resource allocation (Reference Elsaugh, Hiller and Tunis1). There are three key issues that need to be highlighted in this definition.
Firstly, disinvestment is a process that, in an ideal situation, ends up with the optimization of care and the reallocation of resources to better options. Viewing disinvestment as a process could explain the many alternative terms that are currently in use to refer to this (Reference Leggett, Noseworthy and Zarrabi2;Reference Prasad and Ioannidis3). In fact, a scoping review of Niven et al. identified forty-three different terms used to refer to this process, being “disinvest” (39 percent) the most frequently cited one (Reference Niven, Mrklas and Holodinsky4;Reference Gnjidic and Elshaug5).
Secondly, disinvestment is focused on existing or established health technologies whose use represents no or low added value. Thus, once approved, any technology could be subjected to disinvestment if its use shows to be of no or low added value. This concept highlights the need for the continuing reassessment of technologies.
Thirdly, the continuous reassessment of technologies could identify uses beyond the boundaries of established net benefit (indication creep/leakage), what turns the focus out of the technology or the service itself to its appropriate application. This is one of the areas where disinvestment can make the most impact through refining use of a technology/medicine/service to the patient group/s most likely to benefit, preventing its use from creeping to nonindicated groups.
The traditional linear concept of health technologies life cycle assumes that once decisions on reimbursement were taken, health technologies remained unassessed up to their disuse by health professionals (Reference Gutiérrez-Ibarluzea, Chiumente and Dauben6).
An obsolete technology, by definition, is always susceptible to disinvestment; however, what mainly defines a candidate technology is the added value of its use, more than its life-cycle stage. This conception challenges the Health Technology Assessment (HTA) movement, not only because it extents its frontiers beyond the assessment of new technologies before their acceptance to a continuing assessment of their use once adopted, but also, because it confronts HTA promoters to the need of developing and implementing new strategies and methodologies to face this demand.
The challenge for the HTA agencies and units is where to start given the thousands of separate interventions currently available (Reference Gallego, Haas, Hall and Viney7). Besides, the considerable workload of HTA units in assessing new technologies limits their possibilities in assessing existing items (Reference Elshaug, Moss and Littlejohns8;Reference Elshaug, Hiller and Moss9). Even more, there is considerable variation in processes for selecting and prioritizing potential candidates for disinvestment, which represents an important barrier for HTA organizations interested in implementing these processes. Indeed, a rational and reproducible selection process will facilitate the best approach to reviewing technologies and subsequently support the successful changes in policy and practice. These are unlikely to happen if there is dispute around why a given technology has been selected for reassessment, especially if the decision is to disinvest.
Thus, we undertook an interpretative review on national or international disinvestment initiatives including a systematic review to compare the methods for identifying low or no-value technologies, practices or services. On the basis of this evidence, a framework was proposed to address this issue and provide support to Health Technologies Organizations and Governments working in the field of disinvestment. This research aims also to contribute to a methodological toolkit on disinvestment promoted by the Interest Group on disinvestment and early awareness of the international society HTAi (http://www.htai.org).
Methods
An interpretative review was conducted to develop a framework for identifying potential candidates technologies, practices, and services for disinvestment through the critical analysis of the body of evidence on this topic.
Evidence Search
A systematic literature search was performed to identify secondary or tertiary research related to disinvestment initiatives and/or any type of research that specifically described one or more methods for identifying potential candidates technologies, services, or practices for disinvestment. We searched the following electronic databases from 1 January 1990 to 27 June 2018: PubMed; the Virtual Health Library; the Cochrane Database of Systematic Reviews and Epistemonikos. Search terms included the words “disinvestment” and synonyms or related terms like: “Health Technology Reassessment”; “de-adoption”; “de-implementation”; “medical reversal”; “withdrawal”; “delist”; “decommission”; “defunding”; “rationing”; “low value”; “no value”; “marginal value”; “cost-ineffectiveness”; “obsolescence”; “out-dated”; “out-moded”; “superseded”; “abandoned”; “harmful”. Search terms were combined using the appropriate Boolean logic, and included wildcards to account for plural words and variations in spelling. The search strategy included similar combinations of terms within the other databases. One researcher hand searched references of included studies to identify additional reviews (see Supplementary Table 1: search strategies).
Selection of the Evidence
An article was considered eligible when it was related to one or more disinvestment initiatives, framework, or model or when it specifically described one or more methods for identifying potential candidates for disinvestment. Titles of all retrieved references were assessed by one researcher and classified as eligible according to the established criteria. An article was finally included in the review when it fulfilled at least one of the following criteria: (a) the research was a systematic review, overview, scoping review, integrative review, and/or critical interpretative synthesis of one or more than one international disinvestment initiatives; (b) the research proposed a disinvestment model or framework; (c) the research specifically described one or more methods for identifying potential candidates for disinvestment; (d) the research listed not to do recommendations or low/no added value technologies, practices, or services. Two researchers (MEE; NIR) independently assessed the abstracts of all eligible articles and defined their inclusion or not on the basis of the inclusion criteria. A third researcher (IGI) resolved discrepancies between reviewers. Finally included articles can be found in Table 1.
Description of Included Studies
Notes: (Φ) Although it is not a systematic review, authors decided to include it as it summarizes main issues on disinvestment, including LVNT (Low and No Value Technology) identification methods and criteria.
Extraction of Information of Disinvestment Initiatives and Methods for Identifying Potential Candidates
Articles that fulfilled inclusion criteria were retrieved in full text. One researcher (MEE) extracted information from reviews on disinvestment initiatives, considering the following characteristics: publication date; type of research; names, country and year of the disinvestment initiatives included in the review. On the basis of the information provided by included studies, each disinvestment initiative was described, considering the following characteristics: country, year, financing and/or promoting organization, organization in charge of its implementation, scope, name, type of disinvestment initiative, and aims and description of the methodologies used for identifying candidates for disinvestment.
Development of a Framework
An iterative and critical analysis of the methods described alongside the disinvestment initiatives was performed. Emergent themes that aroused from this analysis were grounded in the Lomas et al. definition of evidence (Reference Lomas, Culyer, McCutcheon, McAuley and Law10). As a result, a new framework for identifying potential technologies, practices, and services candidates for disinvestment was proposed. Methods for identifying candidates for disinvestment, which were identified through the systematic review, were mapped according to the different categories of the framework to test its coherence and potential usefulness.
Results
Seventeen reviews on disinvestment were identified (Reference Leggett, Noseworthy and Zarrabi2;Reference Niven, Mrklas and Holodinsky4;Reference Gallego, Haas, Hall and Viney7;Reference Orso, de Waure and Abraha11–Reference Sutton, Qureshi and Martin24). All identified disinvestment initiatives worldwide, except two that mainly focused on conceptual and terminology issues in relation to disinvestment and medical reversal, respectively (Reference Soril, Niven, Esmail, Noseworthy and Clement23;Reference Sutton, Qureshi and Martin24). One review is specifically focused on methodologies for identifying and prioritizing candidate technologies for disinvestment (Reference Valentín and Blasco13). After summarizing the evidence of reviews on disinvestment, forty-five initiatives were identified (one has an international scope and group different national or regional initiatives that share the same methodology: “Choosing Wisely”). A full description of each initiative is provided in Supplementary Table 2.
Three different but related themes evolved from the critical analysis of the included papers in relation to methods for identifying potential candidates for disinvestment; these were approaches, triggers, and methods (ATM).
“Approach” is defined by the Cambridge Dictionary as “the way of dealing with a situation or a problem” (25). In respect to the identification phase of the disinvestment process, the approach related to the way organizations deal with the multiplicity and diversity of existing technologies whose “value for money” is not well established or uncertain, thus representing an inappropriate and inefficient use of resources. “Triggers” represent a set of criteria that when present, allow the identification of potential candidates for disinvestment. “Methods” refers to the methodological strategy an organization uses for applying triggers or identification criteria.
Besides, we decided to use the broad definition of evidence provided by Lomas et al. (Reference Lomas, Culyer, McCutcheon, McAuley and Law10) to categorize ATM. According to Lomas, there are three basic categories of evidence: “medical effectiveness research” (context-free scientific evidence); “social science-oriented research” (context-sensitive scientific evidence); and “the expertise, views, values, and realities of stakeholders” (colloquial evidence). Mixed evidence could also be considered when decisions are informed by more than one of these categories.
The final product of this interpretative review is a new evidence-informed ATM framework for identifying potential candidates of disinvestment, which proposes seven basic approaches (three predominantly based on the use of one type of evidence and four mixed approaches based on the combination of the different types of evidence); a set of triggers and two types of methodological strategies (ad hoc and embedded methods) (Figure 1).
The ATM evidence-based approach for the identification and prioritization of LNVT.
Basic Approaches for Identifying Opportunities for Disinvestment
Seven basic approaches for identifying opportunities for disinvestment are proposed on the basis of Lomas definition of evidence (Reference Lomas, Culyer, McCutcheon, McAuley and Law10):
• “Context-free scientific evidence-driven approach” (see Figure 1, Approach 1): potential candidates for disinvestment are identified on the basis of concerns with their effectiveness and safety provided by research studies, such as systematic reviews, evidence-based clinical practice guidelines, and HTA reports.
• “Context-sensitive scientific evidence-driven approach” (see Figure 1, Approach 2): identification of no or low added value technologies (NLVT) is performed on the basis of evidence on the implementation, organizational capacity, economics, legal, and ethical issues related to the use of a specific technology in a certain context. Research into clinical practice variation, economic approaches like PBMA or cost-effectiveness analysis are examples of the use of this approach.
• “Colloquial evidence-driven approach” (see Figure 1, Approach 3): NLVT are identified through the use of evidence that comes from the expertise, views, and realities of stakeholders. This type of approach is used in nomination or consultation processes, through which stakeholders identify potential candidates for disinvestment.
• “Combined evidence-driven approaches” (see Figure 1, Approaches 4, 5, 6, and 7): considered separately, each approach has its strengths and weaknesses; thus, the majority of disinvestment initiatives include different types of evidence when identifying potential candidates, resulting in a “Combined-mixed approach”. This last approach includes four different options, depending on the amount and types of evidence that are combined.
Triggers for Identifying Candidate Technologies for Disinvestment
Taking into consideration the three pure evidence-informed approaches previously described (“Context-free scientific evidence driven approach”; “Context-sensitive scientific evidence driven approach”; and “Colloquial evidence driven approach”), we proposed a list of triggers for identifying candidate technologies for disinvestment adapted from the framework proposed by Elshaug et al. (Reference Elshaug, Moss and Littlejohns8) (Figure 1). Categories are defined on the basis of the type of evidence used to value the presence or absence of each criterion.
1. “Triggers based on context-free scientific evidence”: scientific evidence from systematic reviews, guidelines, Health Technology Assessment—HTA—reports, economic evaluations, subsequent trials, post-market surveillance, or an equivalent source showing one or more of the following:
• Evidence on Ineffectiveness/Patient Safety concerns/Inefficiency
– Patient Safety Concerns: the technology is harmful or associated with moderate/serious adverse events.
– Ineffectiveness: the technology is ineffective/ appreciably less effective in comparison to an alternative or showed to be not beneficial for patients (even when safety is not compromised).
– Inadequate Benefit-Risk Balance: risks of applying the technology overcome its benefits.
– Inefficiency: the technology is not cost-effective compared to other treatment approaches.
• Displacement of an old intervention by a new one
When a new intervention is considered a potential replacement for (an) established comparator(s) for that indication, then that comparator for that patient indication is automatically considered and assessed for disinvestment.
In some cases, the absence of scientific evidence could be the criteria for identification:
• Uncertainties related to “Legacy” technologies: long-established technologies whose effectiveness/patient safety/efficiency has never been assessed.
• Uncertainties related to “newer/extended uses” of a technology: when an intervention has evolved to the point that it differs markedly from the initial or prototype intervention that was originally assessed or funded, then the initial intervention should be reviewed.
2. “Triggers based on context-sensitive use or variation evidence”: Scientific evidence provided by context-sensitive research or data analysis showing any of the following:
• Geographic variations in care: geographic variations after adjusting for demographics and location of centers of excellence suggest differences in clinical opinion about the value of the interventions.
• Provider variations in care: local evidence showing inappropriate variability of care. Clinical heterogeneity of procedures where the choice of intervention varies for the same stage of the disease or condition.
• Practice inconsistency with evidence-based standards: medical practice is inconsistent with clinical practice guidelines, clinical college or scientific society position statements, or Cochrane Review recommendations (and where there is no Cochrane Review on that technology).
• Temporal variations in volume: a trend in item volume between time-points (i.e. 2, 3, or 5 years) of a substantial percentage change (e.g. 30 percent, 50 percent, or 80 percent). An increase after adjusting for trends in incidence or prevalence rates may flag “leakage” (usage beyond restrictions/indications) or indication “creep” (see the following criteria). It could also be a decrease, showing that the medicine has fallen into disuse or is no longer available.
• Leakage: technology use (with reimbursement) outside the evidence-based indications.
3. “Triggers based on colloquial evidence”: Colloquial evidence provided by key stakeholders related to a negative experience, perspective, or beliefs related to a certain technology or its use. However, the presence of criteria based on colloquial evidence is generally not sufficient to consider the technology as a potential candidate for disinvestment and needs to be accompanied by the presence of at least one criteria based on any type of scientific evidence. Triggers based on colloquial evidence include:
• Negative experiences or perceptions from community members: expressions (to media, letters to editors, enquiry submissions) and/or nominations provided by patients, consumer advocacy and support groups, and community groups, highlighting negative (or ineffective) experiences following a diagnostic or therapeutic procedure.
• Negative experiences or perceptions from health system workers, administrators, and/or funders: expressions of interests, nominations, or lists of potential candidate technologies for disinvestment provided by stakeholders working at the health system (clinical, nursing, allied health and technical staff, healthcare administrators, and funders (including both public and private health insurance)) and/or medical associations and colleges.
Methods for Identifying Candidate Technologies for Disinvestment
There are different methods for applying triggers and identifying potential candidates for disinvestment, which can be categorized into two types: “Ad hoc methods” and “Embedded methods” (Figure 1).
I. Ad Hoc Methods: methods that are specifically designed and temporally implemented with the objective of identifying candidate technologies for disinvestment and, contrary to “embedded methods”, are not performed on a routine basis.
(1) Horizon or Environmental Scanning: It combines different sources of evidence, such as Internet, websites of HTA agencies or units, the International Information Network on New, Emerging and Obsolete Health Technologies (Euroscan), and other researchers and health professionals’ networks, that notifies the existence of a technology susceptible to be assessed for disinvestment. Many disinvestment initiatives have employed horizon scanning to identify potential candidate for disinvestment (Reference Ibargoyen-Roteta, Gutierrez-Ibarluzea, Asua, Benguria-Arrate and Galnares-Cordero26). Besides, some researchers used this exhaustive methodology to identify and list candidates for disinvestment (Reference Elshaug, Watt, Mundy and Willis27;Reference Malik, Marti, Darzi and Mossialos28)
(2) Identification of opportunities for disinvestment from evidence-based guidelines and/or HTA reports: high-quality evidence-based guidelines and/or HTA reports are very valuable sources for informing these processes. A high number of guidelines and HTA reports contain “against recommendations” on the basis of high-quality evidence of ineffectiveness, minor comparative effectiveness in relation to a new practice or technology, association to serious adverse events and low patient benefits. In a case study, conducted in Argentina, ninety-two “do not recommendations” were identified after reviewing fifty-seven Argentinian guidelines (Reference Esandi and Gutiérrez-Ibarluzea29).
(3) Identification of potential candidates for disinvestment from systematic reviews (SRs): SRs are a useful source for identifying potential candidates for disinvestment. In 2009, NICE piloted a project to use Cochrane Reviews as a basis for identifying low-value healthcare practices for local decision making. However, as Gardner et al. (Reference Garner, Docherty and Somner30) highlighted, there are a number of challenges that should be taken into account when using this source of evidence, like SR of non-prioritized topics; lack of inclusion of clinical significant comparators and a high proportion of SR showing absence of evidence rather than evidence of a lack of efficacy and effectiveness.
(4) Adaptation of existing list of no-value technologies: many governments and health agencies, professional bodies, commissioner agencies, and researchers have produced lists of no-value or low-value technologies. These lists can be used by other organizations to guide the identification of opportunities for disinvestment (Reference Paprica, Culyer, Elshaug, Peffer and Sandoval31). However, as Harris (Reference Harris, Allen and Brooke32) pointed out, users of these lists should confirm their validity and appropriateness before implementation, as there is high heterogeneity in the methods used to generate the lists as well as in the definition of “low-value technologies”.
(5) Comparative Effectiveness Research (CER): CER is the systematic appraisal of the benefits and risks of alternative treatments and other healthcare interventions (e.g. screening). The inclusion of costs in the appraisal is not explicit (Reference Haas, Hall, Viney and Gallego33). As an example, a new coordinating agency in the United States, the Patient-Centered Outcomes Research Institute (PCORI), was established to promote CER on priority topics (34).
(6) Research into clinical variation practices: the systematic investigation of practice variations may identify candidate technologies for disinvestment through the application of “triggers based on context-sensitive use or variation evidence”. Variations may represent widespread inefficiencies in the healthcare system due to the overprovision of treatment or the provision of ineffective or unnecessary care in some regions versus others. This may be due for instance to differences in commercial activities between regions including financial incentives for physicians; alternatively differences in regional policies reducing interactions between physicians and commercial organizations (Reference Spurling, Mansfield and Montgomery35–Reference Fleischman, Agrawal and King37). The Medical Practice Variation Atlas, in Spain, is an example of how the study of geographic variations can guide the identification of technologies of doubtful use (Reference García-Armesto, Angulo-Pueyo and Martínez-Lizaga38). Recently, Badgery-Parker et al. (Reference Badgery-Parker and Pearson39) described the frequency trends and cost implications of low value care episodes in the public Australian hospital setting through contrasting administrative data to do-not recommendations. Other examples of the use of “real-world data” to guide disinvestment decisions are provided by Brett et al. (Reference Brett, Elshaug and Bhatia40).
(7) Program Budgeting and Marginal Analysis (PBMA): PBMA is a process that helps decision makers maximize the impact of healthcare resources on the health needs of a local population or meet other specified goals such as equity considerations. PBMA has been proposed as a method of rational disinvestment (Reference Donaldson, Bate, Mitton, Dionne and Ruta41–Reference Grocott43). However, it is a relatively resource-intensive activity, requires high-quality data (i.e. cost-accounting), and qualified professionals to guide the whole process as well as the commitment and cooperation of clinicians and managers (sometimes from competing programs). Thus, although being considered a valuable method, it still remains quite difficult to achieve in practice (Reference Gallego, Haas, Hall and Viney7;Reference Harris, Allen and Brooke32).
(8) Nomination and consultation methods: in contrast to the previous methods, nomination and consultation methods generate a list of potential candidate technologies for disinvestment on the basis of stakeholders “perspectives”. The Health PACT describes these methods as “bottom-up” approaches, which are mainly characterized by the engagement and empowerment of stakeholders. The new disinvestment process in Brazil includes such approaches (Reference Lemos, Guerra Junior and Santos44). According to Elshaug et al. (Reference Elshaug, Moss and Littlejohns8) there are two methods that capture stakeholders’ opinions and perceptions: nomination and consultation.
II. Embedded methods: methods that are already integrated into organizational infrastructure and/or routinely apply triggers for identifying candidate technologies for disinvestment alongside different organizational processes.
(1) Horizon scanning of existing technologies as a routine task of the HTA process: HTA bodies execute horizon scanning for identifying emerging technologies. Existing technologies could also be scanned to identified potential candidates for disinvestment, as was already proposed by the HTAi Policy Forum (Reference Henshall and Schuller14).
(2) Triggers application alongside purchasing and procurement processes: incorporation of considerations (like prompts, triggers and/or mandatory requirements) for disinvestment into the existing decision-making infrastructure in charge of purchasing and procurement processes.
(3) Systematic identification of “DO NOT” recommendation alongside the guideline development process: prompts, triggers, and/or mandatory requirements to consider disinvestment and “Do Not” recommendations could be introduced into the guideline development and authorization processes: NICE Commissioning Guides are an example of this type of guidelines (45) as well as the production and diffusion of deprescribing guidelines are examples of this methodology (46;47).
(4) System redesign processes related to resource allocation: system redesign describes a range of methods and tools that have been adapted for use in health care. Although some of the reported reasons and motivations for system redesign are consistent with the principles of disinvestment (e.g. better use of existing resources, maximizing value, and eliminating waste, increasing efficiency, and reducing duplication of service), no example could currently be identified in the literature review (Reference Harris, Allen and Brooke32).
(5) Routine use of local data to identify potential candidates of disinvestment: health organizations and systems may have information monitoring systems, such as registries or databases, that could be used in this stage of the disinvestment process. Context-sensitive criteria could be incorporated to trigger the identification of potential candidate technologies for disinvestment.
Discussion
The principal output of this review is the proposal for a new evidence-informed framework for identifying potential candidates for disinvestment, which is the result of a comprehensive systematic review of disinvestment initiatives worldwide combined with a critical and iterative analysis of the multiple methods for the identification of low value technologies currently in use.
Most of the disinvestment initiatives included in the reviews reported the description of the methodologies used for identifying and prioritizing candidate technologies. However, there was a need to clarify the use of terms to describe the strategies most commonly used: the mechanisms described by Niven et al. (Reference Niven, Mrklas and Holodinsky4) are quite similar to the trigger criteria proposed by Elshaug et al. (Reference Elshaug, Moss and Littlejohns8) or the ongoing consultations approaches proposed by the HTAi Policy Forum (Reference Henshall and Schuller14). Mayer and Nachtnebel differentiate “methods” (literature-based methods and expert-opinion methods) from criteria (Reference Mayer and Nachtnebel16). In other words, the review of the published evidence showed that there was overlap among the terms and concepts used for describing the process of identifying potential candidates for disinvestment. Consequently, we make a proposal differentiating basic approaches, triggers, and methods that can be used to enact this. We hope this contributes to reduce confusion among those organizations interested in implementing such processes.
The underlying principle of the proposed frameworks is related to the use of evidence to inform disinvestment decisions. However, our approach highlights the broad definition of evidence that was proposed by Lomas et al. (Reference Lomas, Culyer, McCutcheon, McAuley and Law10) and Paprica et al. (Reference Paprica, Culyer, Elshaug, Peffer and Sandoval31). In fact, each type of evidence (context-free, context-sensitive, and colloquial) could contribute to inform the process of identifying potential candidates for disinvestment. Context-free scientific evidence allows the identification of ineffective and/or harmful technologies on the basis of valid and reliable methods. However, this evidence needs to be contextualized and aligned to the priorities of the health system and the society. Context-sensitive scientific evidence establishes, among other issues, which technology or practice is relevant in a certain area or institution due to its variability, burden, and/or budget impact. Finally, inclusion of key stakeholders and their perceptions (colloquial evidence) alongside the identification process increases its legitimacy, being the last one of the biggest challenges facing health authorities when making disinvestment decisions. The need for contextualization, alignment with local priorities and legitimacy, highlights the crucial importance of considering context and stakeholders preferences when determining the “not to do” practices or recommendations. Consequently, whilst being more time-consuming and complex to achieve, the combined approach is the recommended practice for identifying potential candidate technologies for disinvestment.
We believe that the distinction among approaches, triggers, and methods that are proposed by the evidence-informed framework improves our understanding of the prevailing methods that are currently under use and they define their success. The three categories define the “core components” of the identification process, meaning by these, those components that are fixed and represent an essential function of the strategy. In other words, independently of the context, any organization interested in identifying potential candidates for disinvestment needs to define the whole approach, the triggers, and the methods that it will apply to accomplish its objective.
To select the “whole approach”, the organization needs to define the type of evidence that will be hierarchized in the identification and prioritization of LNVT. In some way, this will also define who will be the stakeholders involved in the process. Sources of information and stakeholders that the organization considers relevant and possible to be involved will influence this definition. Triggers will be chosen according to the selected approach. Finally, there are different available methods to apply the selected triggers and each organization will define which is the most appropriate. However, this selection will be conditioned by the interest and capacity of the organization to apply this method once a time (“ad hoc method”) or as a routine practice (“embedded method”).
In sum, these “fixed” components could incorporate different forms according to local context: the set of criteria could differ among disinvestment initiatives so do the methods for applying these criteria. There are many contextual factors, that is access to different disinvestment information sources, local capability to implement the NLVT identification process, and availability of valid and reliable local data, which influence the type of strategy that a certain organization can choose and implement. In other words, the different types inside each category could vary from organization to organization. Thus, they represent the “peripheral components” of the framework, meaning those that are adaptable depending on the circumstances and situation of each organization.
Variability of these components of the framework not only depends on the context but also on the evolving knowledge in this field. After reviewing the body of evidence, we were able to identify seven basic approaches, eleven triggers, and thirteen methods. However, this number could be reduced or increased in the future with the advancement of knowledge in the field.
It should be noted that this framework was built on the basis of LNVT identification initiatives identified in published systematic reviews, and although a comprehensive search was performed, it is possible that some initiatives could have been omitted. However, as far as we know, this is the most updated review on LNVT identification and we believe it provides a representative picture of what is being done in this field.
It must be acknowledged that the purpose of this research was to design a framework on the basis of almost half of hundred of reported initiatives that have been implemented by HTA agencies since the 1970s. Future applications of this framework by the HTA community will provide invaluable inputs for its validation and feasibility assessment.
In summary, although identification methods have been described in the literature and tested in different disinvestment initiatives, the proliferation of terms and concepts used to describe this process creates considerable confusion among HTA organizations, health providers, and governments. Some existing initiatives and methodological reports, such as the recently published guidance on health technology performance assessment (Reference Guerra-Júnior, Pires de Lemos and Godman48), would benefit from this research and this finding could be incorporated to new proposals and constitute the basis for local, regional, or national initiatives on reassessment and then disinvestment.
Acknowledgments
This research is part of a joint effort performed by HTAi IG on DEA, IG on ethics, EuroScan network, and INAHTA that aims to elaborate a toolkit that could aid organizations and individuals on the steps to be developed when considering disinvestment activities.
Supplementary Material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462319000734
Source of Funding
None to be declared.
