In many developed and developing countries, decision making regarding health technology funding is based on results from health technology assessments (HTAs) (Reference Paris and Belloni1;Reference Barnieh, Manns and Harris2). These assessments often include the same studies conducted for licensing—the randomized clinical trials (RCTs), which are considered to provide “gold standard” evidence (Reference Tugwell and Knottnerus3). However, RCTs, especially those conducted by the manufacturer, have important limitations. Since they are designed to show the best possible result for the new technology, Phase III studies typically are conducted under ideal and highly controlled conditions to seek high internal validity and maximise the chance of demonstrating clinical benefit (Reference Malmstrom, Godman and Diogene4), they frequently select patients more likely to present the best response, and do sometimes use placebo as a control even when standard treatment already exists (Reference Gold, Kappos and Arnold5). As a result, it is expected that technologies do not always perform in real life in the same way as they performed in controlled conditions. As consequence, RCTs can fail to offer evidence of the effectiveness of technologies in routine clinical care especially as patients may well be older and more co-morbid than those contained in RCTs (Reference Joppi, Cinconze and Mezzalira6).
Beyond concerns with RCTs, inherent limitations in several studies used for HTA decisions include the use of surrogate measures and secondary outcomes, using grouped outcome measures, using a single pivotal trial or over-powering trials, all of which may enhance the efficacy and/ or understate the adverse effects of technologies. Concerns with surrogate markers have been highlighted in several situations (Reference Divittorio, Jackson, Chindalore, Welker and Walker7–Reference Henshall, Sansom and Eichler10). It has also been shown that in the United States single pivotal trials gave support to almost a third of new drug approvals from 2005 to 2012. More than two-thirds of new drugs were also approved on the basis of studies lasting 6 months or less (Reference Downing, Aminawung, Shah, Krumholz and Ross11;Reference Avorn and Kesselheim12). Adding to this, despite ethical and statutory mandates, lack of transparency and publication bias are still major concerns. For instance, it has been shown that less than 50 percent of RCTs are published in a 5-year window since registering them onto ClinicalTrials.gov (Reference Anderson, Chiswell, Peterson, Tasneem, Topping and Califf13).
Furthermore, scientific methods used to assess whether or not it is worth paying for new higher priced technologies are typically based on evidence provided by the manufacturer and tend to favor the perception that it is, indeed, worth paying a premium price. The consequence is the potential of ever increasing healthcare expenditure as each newly reimbursed technology is used to establish new cost-effectiveness threshold levels (Reference Gafni and Birch14). This is a concern as several countries are struggling to fund new higher priced medicines, which is not in the best interest of any key stakeholder group (Reference Malmstrom, Godman and Diogene4). In addition, most countries that use economic analyses for reimbursement decisions are still reluctant to set economic threshold levels (Reference Paris and Belloni1). Authors have argued that by defining a threshold, health authorities are setting and disseminating a price ceiling to which manufacturers will seek to reach instead of negotiating lower prices (Reference McCabe, Claxton and Culyer15). This has consequence of pushing up prices and increasing the burden on public health budgets, as seen for instance with new cancer medicines and those for orphan diseases (Reference Howard, Bach, Berndt and Conti16;Reference Simoens, Picavet, Dooms, Cassiman and Morel17). Even in cases where health authorities choose to set a threshold value, there are debates in literature that lower thresholds should be established for new medicines in prevalent conditions (Reference McCabe, Claxton and Culyer15) as the current threshold levels are seen as too high (Reference Raftery18), especially with little empiric basis for establishing these values.
Adding to this, after new technologies are incorporated into reimbursement lists, there is often limited further evaluation of their real value unless they are part of managed entry agreements (Reference Ferrario and Kanavos19). This is a concern both in terms of opportunity costs as well as patient care. For instance, meta-analyses and independent cohort studies have continued to show no difference in effectiveness between isophane insulin, rDNA insulins, and long-acting insulins despite considerable differences in prices (Reference Caires de Souza, de Assis Acurcio, Guerra Junior, Rezende Macedo do Nascimento, Godman and Diniz20). In other case, studies have shown surprisingly differing results in real life in the case of medicines for transplantation (Reference Gomes, Guerra Júnior and Lemos21). On the other hand, despite early concerns, long-term cohort studies have demonstrated the effectiveness and safety of tumor necrosis factor (TNF) alpha inhibitors in patients with immune diseases although the effectiveness of treatments in real life may be different from those seen in randomized clinical trials (Reference Raaschou, Simard, Holmqvist, Askling and Group22). These situations raise the question of whether health systems actually obtain the results they are expecting when they choose to pay for new technologies.
In light of this, we believe new approaches are needed to continually re-assess technologies and evaluate their performance in the real world, given continuing pressure on resources. In this article, we describe the development of the Guideline for Health Technology performance Assessment (HTpA) produced by the SUS Collaborating Centre for Technology Assessment and Excellence in Health (CCATES) in collaboration with many international experts and the support of the Pan-American Health Organization (PAHO) and the Department of Management and Incorporation of Technologies from Brazilian Ministry of Health. The objective of the Guideline was to stablish the monitoring of funded technologies using real-world evidence to assess their performance and update clinical guidelines. The main focus will be on disinvestment as this is a growing area of interest given pressure on resources. We hope this will be of interest not only in Brazil but wider.
HTpA
HTpA refers to the systematic evaluation of properties, effects, and/or impacts of a health intervention or health technology in the real world. It provides information to health systems to support the update of clinical guidelines and/or continued reimbursement by contrasting health and safety results in the whole population with initial RCT information. The HTpA matches real-life results with technology values. In the process of evidence-based decisions, HTpA is undertaken after a positive reimbursement/public provision decision, leading to clinical guideline updates (Figure 1). In Brazil, the inclusion of new technologies in updated clinical guidelines is essential for their reimbursement within the public health system (SUS); otherwise they are not reimbursed (Reference Caires de Souza, de Assis Acurcio, Guerra Junior, Rezende Macedo do Nascimento, Godman and Diniz20;Reference de Oliveira Costa, Almeida-Brasil and Godman23).
Process of evidence-based decisions. HTpA is the evaluation of the results of financed technologies in real-world settings. Flow chart developed with Bizagi Modeler Version 3.1.0.011
HTpA may indicate the need to disinvest in one or more of the four modalities as shown below (adapted from Daniels et al. (Reference Daniels, Williams, Robinson and Spence24): (i) Full delisting: Delisting of the technology. It is the most difficult modality to implement and has great potential for unmanaged substitution; (ii) Restriction: The provision of the technology will be restricted to groups or subgroups of users, with strictly criteria for its use. It may be seen as discriminatory. It can be reversed or enlarged as required; (iii) Retraction: The technology will be supplied in a smaller amount to each individual, for example, one mammogram/year instead of two. Potentially more acceptable than full delisting; (iv) Substitution: The currently offered technology will be replaced by a cost-effective alternative, ensuring equivalent treatment/service.
Price renegotiations may be applied with restrictions to defined patient populations to help target resources to the patient populations of highest value. This may be preferable to not reimbursing patented medicines in a class once generics become available in that class.
DEVELOPMENT OF THE GUIDELINE
A review was conducted to identify necessary aspects to be addressed in the guideline. Scientific literature and HTA Agencies/Government were scanned for eligible reports. Key principles for HTpA were adapted from HTA principles (Reference Frønsdal, Facey, Klemp, Norderhaug, Mørland and Røttingen25), which helped orientate the development of the first draft of the guideline. Similar to HTA, the performance assessment must be current and timely, its scope should be explicit and relevant, and the process must be transparent and actively involve interested stakeholders. When only a few technologies are evaluated, there must be a clear process of selection of these technologies to avoid distortions in investment decisions and to ensure transparency. HTpA should be conducted with appropriate and rigorous methods for assessing costs and benefits to generate confidence among key stakeholders including the general public in its conclusions. Even though HTpA reports must be designed and/or destined to health authority managers with clear and objective recommendations, it is important to develop strategies to disclose the study and its conclusions to the different social actors.
Development of the First Version of the Guideline Using a “Snowball” Technique
Public health researchers, health managers and policymakers were invited to contribute by means of email and in meetings held by CCATES to help develop the guideline. After receiving contributions to the draft, the first version of the guideline was developed. One of the proposals to the draft was the implementation of a program to institutionalize the continuous assessment of incorporated health technologies. The implementation of such a program would serve as a landmark for the possibility of disinvestment for all existing technologies and for all technologies candidates for potential incorporation.
The guideline was divided in eight sections: (i) Introduction; (ii) Steps of disinvestment process (iv–vii); (iii) Permanent program for assessing the performance of technologies; (iv) Identification of potential technologies for disinvestment; (v) Prioritization; (vi) Health technology reassessment; (vii) Implementation of disinvestment decision; and (viii) Final remarks. This version was entitled “Disinvestment Guideline Proposal.”
The proposed activities for the permanent program for assessing the performance of technologies were: (i) the continuous monitoring of the effectiveness of financed technologies, (ii) the establishment of a legal framework and of organizational mechanisms to enable and facilitate technology disinvestment where pertinent, (iii) the active search for candidate technologies for disinvestment, and (iv) monitoring the impact of the disinvestment.
We proposed that the health technology indication for performance assessment could arise from within health systems or from society. To be eligible for HTpA, a reimbursed/provided technology must comply with at least one of the requirements as specified in Table 1.
Criteria for Identification of Financed Health Technologies for Health Technology Performance Assessment
For prioritization, we used “Multicriteria Decision Analysis” incorporating Value Measurement Models, using criteria adapted from those proposed by Elshaug et al. (2009) (Reference Elshaug, Moss, Littlejohns, Karnon, Merlin and Hiller26) and those used by the Canadian Agency for Drugs and Technologies in Health (CADTH) for prioritizing technologies for evaluation in the context of incorporation (Reference Husereau, Boucher and Noorani27).
The HTpA section addresses the performance assessment modalities (full withdrawal/full delisting, restriction, retraction, and substitution), price renegotiation, the HTpA process itself, and the elaboration of recommendations to health managers. It also addresses the need to indicate in the HTpA report a period of time sufficient to implement the decision, that is, the transition period.
The implementation of any disinvestment decision resulting from the performance assessment section should address the organizational implications to be taken into account and overcome by health manages. This includes the necessity to redistribute, hire, and train personnel and the necessity to implement reverse logistics measures to collect any remaining disinvested products, that is, medicines, medical devices, etc. In addition, decisions may address the necessity to update clinical protocols and to produce different versions of the same protocol to reach different stakeholder groups, for example, health professionals, health managers, politicians and patients.
The strategies for the dissemination of any decision should focus on the main social actors involved in the provision of the technology. All media should aim at transparently explaining the decision, and there should be clear explanations of the causes and consequences of any subsequent disinvestment decision to ensure full understanding of those who provide and use the technology, as well as to prevent the occurrence of unmanaged substitution and any judicial demands. Judicial demands are particularly prevalent in Brazil as seen with insulin glargine (Reference Caires de Souza, de Assis Acurcio, Guerra Junior, Rezende Macedo do Nascimento, Godman and Diniz20). Suggested strategies to address this include academic detailing, production and dissemination of videos and bulletins, and the creation of a direct communication channel to address potential doubters.
Development of the Second Version of the Guideline after the Panel of Experts to Discuss Key Points and Clinical Cases to Improve the Document
In November 2015, an International Panel on Disinvestment was held at Federal University of Minas Gerais. This includes representatives from the Brazilian Ministry of Health; PAHO; the University of Strathclyde, United Kingdom; Karolinska Institute, Stockholm, Sweden and Stockholm County Council, Sweden; OSTEBA (Basque HTA), Fondo Nacional de Recursos (Uruguay); National Prescribing Service (Australia); and Universidad Nacional del Litoral (Argentina) to review the first version of the guideline.
As basis for debate, in small group sessions, participants were invited to assess two performance assessment case studies, one focused in specialized care, that is, evidence of the comparative effectiveness of beta interferon 1A 6,000,000 IU (30 µg) for multiple sclerosis, and the other in primary care regarding insulin analogue glargine. Both cases were accompanied by real-life evidence of effectiveness.
The most important decision arrived from the panel was to change the focus of the guideline from disinvestment to technology monitoring and clinical guideline update. The first discussion by the panelists concerned the term “disinvestment,” which was considered to have a negative tone, and to be frequently associated with a “loss of rights” and treatment denial, and not automatically associated with opportunity costs. In addition, as shown previously in the first version, technologies need to be continuously evaluated after financing decisions; this action was summarized under the term Health Technology performance Assessment.
Another important decision was to explicitly indicate to key stakeholder groups that continuous funding of a given technology is conditional on the beneficial results in real life, and this is a valid statement for technologies yet to be financed. Another relevant aspect that was later included in the guideline was the need for horizon-scanning programs to work closely with programs for assessing the performance of technologies in routine clinical care to identify key potential areas for disinvestment and avoid unnecessary evaluations.
In the updated version, two different overlapping flow diagrams were proposed, one for technologies not yet financed, and one for already financed ones (Figure 2). For already financed technologies, prioritization criteria were updated to include the rate of launching of new technologies; technologies from fast evolving areas may be prioritized because there may be a greater chance for substitution (Table 2). It is noteworthy that older technologies can sometimes have worse evidence as seen in the review of older medicines that had been superseded by newer more effective medicines in France (Reference Parkinson, Sermet and Clement28). However, this may not always be the case as seen with tacrolimus versus cyclosporine and insulin glargine versus NPH insulins in Brazil (Reference Caires de Souza, de Assis Acurcio, Guerra Junior, Rezende Macedo do Nascimento, Godman and Diniz20;Reference Gomes, Guerra Júnior and Lemos21). Overall though, older funded technologies may be prioritized for HTpA to assess their continued funding.
Flow diagrams for HTpA implementation. For financed technologies, HTpA may be suggested by any member of society: patient, health professional, medical society, industry, etc. For technologies yet to be financed, appointment for HTpA will be assessed during or right after incorporation. Flow chart developed with Bizagi Modeler Version 3.1.0.011
Criteria for Prioritization of Financed Health Technologies for HTpA
Note. Adapted from: Elshaug et al. (2009) (52) and Husereau et al. (53)
In addition, after the institutionalization of HTpA, not all technologies incorporated would be a priority for assessment. Technologies with high cost per treatment or high unitary cost, recently licensed technologies, and technologies/treatments considered groundbreaking may be prioritized for HTpA. Examples shaping usage and funding including real-world studies with the anti-TNF alphas for rheumatoid arthritis where fears of increased rates of cancer and infection have not been realized in practice (Reference Raaschou, Simard, Holmqvist, Askling and Group22) and the WOSCOPS study in Scotland where 20-year follow-up of statin therapy for 5 years is associated with improved survival and a substantial reduction in cardiovascular outcomes, supporting the wider adoption of primary prevention approaches (Reference Ford, Murray, McCowan and Packard29).
Technologies prioritized to HTpA will be evaluated using real-world evidence and incorporating resource usage data in view of potential concerns. As part of this, it is recommended that a permanent program be established for monitoring the clinical effectiveness and the quality of use of medicines, procedures, and equipment in Brazil in routine clinical care. It is responsibility of the program's staff to coordinate horizon scanning activities, receive external requests, and for prioritization of activities. The design and conduct of any HTpA studies may be undertaken in cooperation with academic units taking into account any conflicts of interest. Society involvement in all stages of HTpA is highly recommended, both because HTpA includes the collection of outcomes from patients in real life and societal participation enhances the acceptability of disinvestment activities.
Presentation and Debate in the National Commission for Health Technology Incorporation (CONITEC/Brazil) and Public Consultation by means of Online Form in CONITEC/Brazil
After presenting the Guideline to the plenary of CONITEC/ Brazil, the proposal was submitted to public consultation in March 2016. A total of eighty-nine comments were received through the online form in the CONITEC/Brazil Web site. Most of the contributors were from pharmaceutical companies. The author's answers for the key points raised by contributors were:
Appropriateness of the guideline for technologies for rare diseases: For rare diseases, the HTpA would be guided by effectiveness and safety results. It is noteworthy that in the absence of therapeutic alternative technologies for rare diseases, they probably would not be candidates for full delisting. However, these technologies should be evaluated for price adjustments where this is a concern as well as the identification of patient sub-groups who may truly benefit from the technology. This follows the experiences in the Netherlands with enzyme replacement therapy for Fabry and Pompe disease where pressure was exerted on the Ministry of Health to fund these technologies up to €15million/quality-adjusted life-year, despite coverage with evidence schemes showing limited benefit in clinical practice (Reference Simoens, Picavet, Dooms, Cassiman and Morel17).
Health professionals and patients participation in HTpA: The participation of patients and health professionals is fundamental to HTpA. We believe that public consultations on the recommendations are very important and have the power to change or improve decisions regarding disinvestment and reinvestment. As indicated in the Guideline, it is recommended that public consultation mechanisms are adopted at all stages of the process. This includes patient participation in HTpA studies.
Medical advocates: The adoption of clinical practices based on best evidence produced without conflicts of interest ensures professionals they are providing their patients with the best possible care within available resources. The HTpA process will present the best evidence for the updating of clinical guidelines, regardless of the recommendation to disinvest or not.
“Passive disinvestment” as discussed in the public consultation process, assumes that all medical professionals have access to updated scientific evidence from reputable sources and act upon it. However, this does not always happen in practice as seen by the continued use of tacrolimus for kidney transplantation despite real-world evidence supporting greater effectiveness with cyclosporine at substantially higher costs (Reference Gomes, Guerra Júnior and Lemos21). Moreover, physician prescribing can be influenced by “success bias” in which prescribers take note of their successful results ignoring cases of nonsuccess, which may be due to patients not returning, giving the impression that the adopted therapeutic approach was optimal when, in fact, it was not.
Companies participation in HTpA: During or soon after incorporation, manufacturers may submit a simplified methodological approach for assessing the performance of the technology in routine clinical practice using outcome parameters adopted by the submitted cost-effectiveness analysis. It is noteworthy that the conduction of HTpA in the country strengthens local and regional company medical and pharmacoeconomic divisions.
Substitution modality could make the process vicious: HTpA of technologies incorporated before the adoption of this guideline, and those not prioritized during the incorporation process, will occur after indication (demand) and prioritization (Table 1). To indicate a technology, or demand a HTpA, the applicant must present at least one study showing compliance with at least one of the criteria shown in Table 2. We believe that the institutionalization of HTpA will increase the competitiveness of companies.
Financing of HTpA studies: There is room in the national health research agenda for HTpA activities financed by the Ministry of Health in Brazil following directives from the Federal Constitution. In addition, new acquisition arrangements can be established in which a part of the price paid for the technology is used to cover the cost of HTpA activities. We recognize this may be different in other countries. However, we believe this is an optimal model ensuring transparency and addressing concerns with conflicts of interest.
Reallocation of resources: The Guideline recommends that, whenever possible, it should be stated where and when the resources raised by disinvestment will be reinvested, which preferably should be for treatments in the same disease area. This should enhance the acceptability of the findings among key stakeholder groups.
Guideline Adjustments and Approval of the Final Version by CONITEC/Brazil and the Ministry of Health
After presentation of the remarks to the Public Consultation and how these were addressed, the final version of the Guideline was accepted in the plenary of the CONITEC and sent for approval of the Minister of Health. The approval by the Brazilian Ministry of Health of the Guideline for Heath Technology performance Assessment was published on December 2, 2016, and is now being taken forward. We will be assessing its impact in the future to determine whether there should be any revisions to the approved Guideline.
CONCLUSION
The development of the guideline has been an interactive process involving key personnel within Brazil and international participants. During its development, we started from the narrow objective of evaluating disinvestment to a wider objective of institutionalizing the continuous monitoring of funded health technologies in routine clinical care. We believe this has been a benefit to all stakeholders concerned. The adoption of the new concept, Health Technology performance Assessment, makes a distinction between the HTA activities for incorporation, which typically only consider controlled studies conducted by the producer for licensing and incorporation, and from Health Technology Reassessment, which does not necessarily incorporate real-world evidence.
Institutionalization of HTpA has the potential to continuously improve clinical guidelines to offer the best care to patients. In doing so, it may fulfil another important objective: the optimal reallocation of health resources, reinvestment, for the benefit of the society. It is clear that because neglecting care must be firmly avoided, reallocation of resources is automatic. Patients previously treated with the disinvested technology will be treated with other more efficient options. In the case of therapeutically equivalent generics and biosimilars, the resources released can be used to finance the increased demand for health care (Reference Parkinson, Sermet and Clement28).
Alongside long-term, real-world setting evaluations, it is urgent that RCTs conducted for licensing and funding assessments be improved. This could be achieved through the choice of more pragmatic and meaningful clinical outcome measures (Reference Henshall, Sansom and Eichler10) and use of a gold-standard treatments as control. Both measures may go against the financial interests of companies, but may be implemented with stronger regulations by agencies worldwide. Technologies incorporated with stronger and credible evidence, including outcomes as opposed to surrogate measures, and with a justifiable cost, may have the lowest possible chance of ever be indicated or prioritized for HTpA.
HTpA conducted right from funding decisions may be concurrent with conditional reimbursement schemes or managed entry arrangements, both of them conducted when there are concerns regarding value for money. Performance assessment adds to these concepts as it is a continuous task, reaching longer periods of evaluation and aiming to regularly update clinical guidelines. Performance assessment can include comparison with different treatment approaches with the rise in electronic health records and other systems to link databases together to assess their effectiveness and safety in routine clinical care.
Two important aspects that can complicate HTpA adoption are the additional costs required and the need for more robust administrative and/or clinical data collection and storage. Costs involved in HTpA can be seen as an investment to achieve better health care; nevertheless, immediate financial resources need to be allocated to improve or create relevant IT systems. In Brazil, the law guarantees investment in research related to the public health system, and we believe that part of this resource can be allocated to HTpA.
Another option valid across countries are arrangements with the companies and the creation of research funds. The publication of this Guideline, along with the creation of a specific program for HTpA, will allow the institutionalization and continuous improvement of the scientific methods to collect real-world evidence toward optimization of available resources. Disinvestment per se is one of the consequences of HTpA. The institutionalization of HTpA will encourage companies to adopt practices to improve quality use of their technologies to ensure their effectiveness and cost-effectiveness, favoring patients and the health system as a whole.
POLICY IMPLICATIONS
Real-world evidence is emerging as an important tool to aid decision making regarding health technology financing. Over the past decade, countries have developed health and/or administrative registries capable of providing quality data for assessing health results in real life. At the same time, budgetary constraints and the pressure to finance emerging high-priced technologies are making it inevitable for health authorities to start assessing whether patients and society are getting the results from financed technologies agreed upon during investment decisions. This study presents the concept of Health Technology performance Assessment and the development of a guideline, commissioned by the Brazilian Ministry of Health, to implement such activity. We believe it is the first guideline to formally incorporate the use of real-world evidence for updating clinical guidelines, price renegotiation, and disinvestment decisions.
CONFLICTS OF INTEREST
All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers' bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge, or beliefs) in the subject matter or materials discussed in this manuscript.
