In recent decades, the countries of Latin America have progressively increased their public spending on health, achieving important health improvements and they made progress toward achieving universal health coverage. However, substantial inequalities, inefficiencies, and sub-optional health outcomes remain (Reference Dmytraczenko, Torres and Aten1–Reference Cotlear, Gómez-Dantés and Knaul3).
In this context, Latin American decision makers face strong pressures from citizens, politicians, and healthcare providers to make better informed decisions for priority setting of healthcare resources. This is accompanied by the growing necessity of accountability regarding the processes, criteria, and the values upon which decisions are based (Reference Terwindt, Rajan, Soucat, Schmets, Rajan and Kadandale4;5).
Health technology assessment (HTA) is a process to determine the value and potential impact of a health technology, for example, to determine whether it should be added to a benefit package (6). This process involves the use of implicit or explicit criteria to define the value and to prioritize the technologies to be financed.
HTA agencies and health systems across the globe have been able to define more explicitly the dimensions to be considered in the evaluation of a technology, the methods and criteria to be used in measuring these dimensions, and how the information obtained is valued in decision making. The recent development of value frameworks is part of this movement to make the decision-making process more inclusive and explicit, both in defining which criteria are considered relevant and the stakeholders who need to participate (Reference Oortwijn, Sampietro-Colom and Habens7–Reference Neumann, Willke and Garrison10).
Value frameworks generally include criteria to measure the degree of health benefit generated by the health technology and some measure of efficiency or cost-effectiveness. They can also incorporate other criteria, such as the impact for relevant stakeholders and ethical, legal, and social considerations. A health technology is assessed according to how it performs on each of the criteria considered relevant by the stakeholders involved, after which a final decision is made (whether explicitly or implicitly) about the value of the technology. The process can be based on a more quantitative approach (e.g., multi-criteria decision making) or, as more frequently seen, a combination of quantitative and deliberative approaches in which the information resulting from the assessment regarding the different dimensions is used to facilitate discussions among the key stakeholders.
In recent years, many value frameworks have proliferated in different health systems, mainly in North America and Europe, as well as in certain areas (e.g., oncology, diagnostics). These phenomena can be explained by the emergence of more complex health technologies, increasing healthcare costs (often associated with these new technologies), the perception that the cost of some health technologies are outweighing their benefits, and the changes in social values and perceptions about the necessity to include more stakeholders, such as clinicians and patients in the HTA process (Reference Oortwijn, Sampietro-Colom and Habens7).
The Global Policy Forum of Health Technology Assessment International (HTAi) was founded 14 years ago with the objective of creating a neutral space for strategic discussions among industry, patients, and other stakeholders about the current state of HTA, its development, and its implications for the health system (11). The first Latin American HTAi Policy Forum was held in Costa Rica in 2016, and good practice principles to guide HTA in the region was the topic discussed. Among the principles identified by countries as highest priority were transparency in the production of HTA, stakeholder involvement (the topic discussed during the second Forum held in Lima, Peru, in 2017) (Reference Pichon-Riviere, Soto, Augustovski and Sampietro-Colom12), clear priority setting mechanisms, a process for appeal, and clear links between the assessment and decision making (Reference Pichon-Riviere, Soto, Augustovski, García Martí and Sampietro-Colom13). These principles informed the selection of the topic of the 2018 Latin American Policy Forum, which was the use of value frameworks as a strategy to apply these principles and to promote transparency in the evaluation of health technologies whilst facilitating the involvement of patients, health professionals, technology developers, and other stakeholders throughout the process.
The objectives of the 2018 Latin American Policy Forum were to explore the current international experiences in the use of value frameworks to inform health technology reimbursement, to discuss the potential application of these frameworks in HTA systems in Latin America, and to identify the criteria that are considered most relevant to use in the region.
The Third Latin American HTAi Policy Forum was held in Montevideo, Uruguay, on 23 and 24 April 2018 and comprised forty-three participants: thirteen representatives of HTA agencies, six representatives of public funders (social and private insurance organizations); fifteen representatives from industry (companies producing pharmaceuticals, medical devices, and diagnostic tests); one representative from the Pan American Health Organization, one representative from a patient association, and seven academics and members of the scientific secretariat and organizers of the Forum. In total, twelve countries in the region were represented (Argentina, Brazil, Chile, Colombia, Costa Rica, Ecuador, El Salvador, Mexico, Panama, Paraguay, Peru, and Uruguay). The acknowledgements section contains a list of participants, their affiliations, and country.
This paper was based on the presentations and discussions held during the Forum. It is not a formal consensus paper from the participants and should not be taken to represent the views of individual participants or of the organizations for which they work.
Methods
The Forum was structured by presentations, working groups (N = 4), and plenary discussions. A background paper (Reference Martí S, Riviere A and Augustovski9) was prepared to inform the meeting and discussions. This document was based on a literature search and a review of organizational websites related to the development of value frameworks, which drew largely from the background paper of the 2017 HTAi Global Policy Forum that addressed the same topic (Reference Oortwijn, Sampietro-Colom and Habens7;Reference Oortwijn14). The paper was also informed by input from the Forum Organizing Committee, and Policy Forum members provided feedback on the drafts and the final version. The purpose of the background paper was to summarize the state of knowledge on this topic to ensure a shared understanding among the Forum participants about the key concepts. To this end, the background paper summarized the main experiences of using value frameworks around the globe, including eight case studies from HTA institutions outside the region (Australia, Canada, Sweden, The Netherlands, United Kingdom, and the United States), five value frameworks in the field of oncology as used in the United States, along with the experiences of four countries in the region (Brazil, Colombia, Peru, and Uruguay).
The meeting began with a presentation by Dr. Wija Oortwijn (Ecorys/Radboud University Medical Centre) on the characteristics and use of value frameworks around the world and their relation to HTA. Thereafter, four countries in the region having more explicit HTA approaches in place (Brazil, Colombia, Mexico, and Uruguay) presented the methods and criteria used in their country along with their experiences of assessing and appraising technologies. Following this, a representative of the pharmaceutical industry and a patient representative provided their perspectives on the topic.
Throughout the 2 days of the event, discussions were held in working groups to: (a) discuss the potential importance of the use of value frameworks in the Latin American countries’ decision-making processes; and (b) to identify the main criteria that should be included in value frameworks to be used in the region, along with the barriers and facilitators to their effective implementation. Regarding the criteria to be used in value frameworks, Forum participants discussed and prioritized a list of twenty-seven potential criteria identified in the literature and used by different HTA agencies and health systems around the globe. These criteria were provided along with a brief description of their meaning (Supplementary Material 1).
During the prioritization exercise, the working groups classified the criteria into three categories: (a) Essential or “core” criteria that should always be used in any value framework; (b) high priority criteria that are desirable but that would not necessarily need to be part of all value frameworks, at least in the first instances in which a country or health system begins to explicitly apply value frameworks; and (c) medium or low priority criteria that are desirable but that could be integrated (or not) into value frameworks. Then, the results of the group work were presented during the plenary session and discussed. The final classification was established by consensus among all the Forum participants.
The Forum was conducted according to the Chatham House Rule (15). All the material was produced in Spanish and English. The Spanish version of this article is available at: https://doi.org/10.1017/S0266462319000126.
Results
Presentations by Policy Forum members about the state of affairs in the region showed that Latin America has made important progress in terms of a more explicit definition of the criteria that should be taken into account during coverage decision making within public health systems. In several countries (e.g., Argentina, Brazil, Chile, and Mexico), such criteria have been included in the legislative body that regulates the process to incorporate technologies in the health system. The following criteria are often mandatory for inclusion in the assessment: effectiveness, safety, cost-effectiveness, and budget impact. However, less progress has been made in the availability of data and reliable information for assessing these criteria and, especially, in defining the link between the assessment and decision making.
Advantages of Using Value Frameworks in Latin America
In the discussion about the potential relevance of value frameworks in Latin America, participants agreed that they would be a valuable tool to improve health system effectiveness, efficiency, sustainability, and equity. In particular, the use of value frameworks was considered as a strategy that could:
• Promote transparency in the decision process to limit political influence.
• Enhance legitimacy of decision making and create a common language that is understandable to all stakeholders.
• Provide a more comprehensive assessment of health technologies based on including a broader range of criteria.
• Facilitate the accountability of decisions and potentially reduce judicialization (i.e., use of the system of courts to obtain access to health technologies not included or covered by the health benefit package).
• Facilitate stakeholder participation.
• Guide prioritization processes for resource allocation.
• Establish clearer “rules of the game” for all stakeholders that would sustain over time.
• Facilitate the institutionalization of HTA.
During the discussions, different aspects related to devices were mentioned. It was agreed that the general framework to assess and define their value is similar to other technologies (e.g., pharmaceuticals) but that the specific characteristics of these technologies must be taken into account, for example, their considerable impact on organizational aspects and the learning curve required. Given that this is an area where the evidence available for the assessment is more complex and often of lower quality, and where the regulatory context is less restrictive, many participants remarked that it would be valuable to introduce a value framework for decisions related to medical devices. Nevertheless, for the reasons mentioned above, the application of value frameworks could initially focus on high-cost or high-risk devices.
Challenges of the Application of Value Frameworks in Latin America
During the working groups and plenary discussions, the following potential disadvantages associated with the application of value frameworks were identified:
• Because the application of comprehensive value frameworks would probably require more information and analysis than most of the evaluation processes currently operating in the region, it could result in delays in the assessment process and subsequent decision making.
• Value frameworks that do not take into account a deliberative aspect can result in a more rigid decision-making process particularly in cases where decisions are based on an algorithmic-type of multi-criteria decision analysis.
• The use of value frameworks involving all relevant stakeholders could expose the decision-making process to imbalanced pressures, for example, dominance of the deliberation by certain interest groups with greater power and influence than others if the rules of the deliberation are not clearly established.
• A value framework is not easy to adapt to all health conditions and technologies (e.g., orphan diseases, high cost drugs, therapeutic versus diagnostic technologies, etc.) There is not a one-size-fits-all value framework.
The potential challenges regarding the application of value frameworks in Latin America are presented in Table 1.
Potential Challenges to the Application of Value Frameworks in Latin America
Priority Criteria to Include in Value Frameworks in Latin America
Of the twenty-seven criteria analyzed, seven were listed as essential or core criteria; five were identified as high priority, that is, desirable but not required to be part of an initial value framework; and three were scored as medium or low priority. For four criteria, there was no clear agreement on their level of priority. The other remaining criteria were considered redundant or as concepts already captured in other criteria, and for this reason, they were not assigned to any of the three prioritization categories.
The burden of illness and severity of the disease, the effectiveness and safety of the technology, the quality of the evidence, the cost-effectiveness, and budget impact were identified as essential criteria that, ideally, should form part of the core of value frameworks used in the region. There was broad consensus about these criteria; however, the cost-effectiveness criteria was initially classified as essential by two discussion groups whereas two others ranked it as high priority. During the plenary discussion, when each of the groups presented the results of their classification process, representatives from several countries mentioned that cost-effectiveness is already a core criterion in their current HTA processes (e.g., Argentina, Brazil, Chile, and Mexico). However, some countries with less developed HTA structures face limitations to apply this criterion in terms of technical staff capacity, the availability of data, and time required to conduct the assessments, although it is also recognized as a criterion of great importance. For these reasons, cost-effectiveness was determined to be included in the final group of core criteria. Table 2 presents the main results of the prioritization exercise, with the criteria grouped according their priority.
Prioritization of Criteria to Include in Value Frameworks in Latin America
*For some participants, the uncertainty regarding clinical benefit (that includes other dimensions beyond the quality of the evidence) could also be considered a core criterion.
** Cost-effectiveness was considered as essential or core by two of the four discussion groups. The other two groups considered it as high priority in light of the challenges that some countries with less developed HTA processes face in taking this forward.
The availability of alternative treatments, organizational requirements, the capacity to reach the entire population, and difficulties with the implementation of the technology in the health system were the criteria that could not clearly be included in any of the three categories. Participants evaluated these in largely heterogeneous ways with some groups considering them as essential or of high priority, whereas others considered them as medium or low priority.
Forum participants also discussed the need to support actions to actively promote more formal use of HTA in decision making and to ensure that HTA includes more explicit value frameworks with a broad range of criteria. Table 3 shows the main actions that were proposed.
Actions Suggested to Promote Decision Making in the Region Based on Value Frameworks Informed by HTA
Concluding Remarks
The development of value frameworks to inform healthcare resource allocation is emerging around the globe. Forum participants agreed with most of the potential benefits of using value frameworks as exemplified in other countries (Reference Oortwijn, Sampietro-Colom and Habens7). The presentations and discussions revealed clearly that value frameworks used in the HTA process could be a valuable tool in Latin America to promote greater transparency in the decision-making process and to facilitate the participation of different stakeholders. However, participants agreed that a particular value framework, as well as the time and the processes needed to implement the value framework, must be adapted to the local context. Furthermore, value frameworks are not to replace deliberative processes related to the incorporation of technologies and the allocation of health budgets. Table 4 summarizes the main messages that emerged during the Forum.
Key Messages from the Forum discussions
The legal and regulatory context was mentioned as a key factor in the correct application of value frameworks in HTA, in using the results, and in formulating recommendations for the decision-making processes. It can also ensure that the decisions resulting from this process are put into practice. The lack of full institutionalization of HTA and the judicialization processes, which are common and particularly unique to the Latin American region, constitute important barriers in many countries in the region to achieve this objective. Furthermore, the lack of data and reliable information for assessing many of the criteria in the region is an important barrier to the correct application of the value frameworks in the decision-making process.
Regarding organizational aspects, the number of dimensions/criteria that can be considered in value frameworks, the depth with which they can be assessed, and the degree of stakeholder participation in defining these is limited by the scarcity of resources (technical, time and funding) available within HTA agencies in the region. The participants believed that it is, therefore, important to advance in a “step-by-step” or gradual approach and to focus efforts on those criteria identified as the highest priority. The results of the prioritization process conducted during the Forum identified a series of core criteria that can be a valuable input for countries when deciding which dimensions are the most important to be assessed when deciding on the incorporation of health technologies.
Another advantage of using value frameworks identified by Forum participants was their potential to allow patients to participate in a more explicit and transparent way in the decision-making process, even with potentially conflicts of interest. However, effective participation cannot be expected without experience and training, and HTA agencies should collaborate in providing training to patients and other stakeholders.
Forum participants agreed that the next steps should be to find appropriate processes and methodologies, adapted to the context of each country, which will permit the application of value frameworks to improve the link between HTA and decision making.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462319000072
Conflicts of interest
The organization and support of the Forum was financed by Health Technology Assessment International (HTAi). The manuscript was prepared by the Institute for Clinical Effectiveness and Health Policy (IECS) with financial support from HTAi. Dr. Wija Oortwijn was employed by Ecorys which received grants and non-financial support from HTAi, during the conduct of the study; grants from Eli Lilly and Company, grants from BMS, non-financial support from Roche, grants from Neprofarm, and grants from FME, outside the submitted work.
Acknowledgment
We thank all the participants of the Forum for their valuable contributions: Adriana Granizo Martínez (Ministerio de Salud Pública - Ecuador), Alejandro Millan (Pfizer – Uruguay), Alicia Ferreira (Fondo Nacional de Recursos – Uruguay), Alicia Granados (Sanofi – Spain), Ana Eduviges Sancho Jiménez (Ministerio de Salud - Costa Rica), Ana Pérez Galán (Ministerio de Salud Pública – Uruguay), Analía López (Ministerio de Salud – Argentina), Andrea Rappagliosi (Edwards Lifesciences – Switzerland), Catherine de la Puente (Ministerio de Salud – Chile), Cristina Nuñez (Edwards Lifesciences – Brazil), David Debrott (Superintendencia de Salud – Chile), Esteban Hernández (Centro Nacional de Excelencia Tecnológica en Salud – Mexico), Fabiane Sztanjbok (MSD – Brazil), Felice Leonardi (Eli Lilly – Colombia), Francisco Caccavo (PAHO – United States), Giovanni Guevara (Ministerio de Salud – El Salvador), Homero Monsanto (Merck & Co. – Puerto Rico), Hudson Santos (Johnson & Johnson – Brazil), Iñaki Gutierrez-Ibarluzea (Vicepresidente HTAi – Spain), Isabel Peirano (Novartis – Argentina), Jaime Calderón Herrera (Instituto de Evaluación de Tecnologías en Salud – Colombia), Jeremias Bernal Gascón (Ministerio de Salud PTY – Panama), Joice Valentim (Roche Pharma – Switzerland), José Luis Estrada (Instituto Mexicano del Seguro Social – Mexico), Kariluz Maestre (Novartis – Colombia), Kátia Wendt (Janssen – Panama), Lizbeth Acuña Merchán (Cuenta de Alto Costo – Colombia), Lucas Najún Dubos (Roche – Argentina), Lucía Bettati (PAMI – Argentina), Manny Papadimitropoulos (Eli Lilly – Canada), Pedro Galván (Instituto de Investigaciones en Ciencias de la Salud – Paraguay), Raúl Sanchez (Sanofi – Mexico), Ricardo Pérez Gómez (Caja Costarricense de Seguro Social – Costa Rica), Silvia Kelles (UNIMED-BH – Brazil), Vania Cristina Canuto Santos (Ministerio de Salud – Brazil), Víctor Suárez (Instituto Nacional de Salud (Peru), Virginia Llera (Geiser Foundation – Argentina), Wija Oortwijn (Ecorys / Radboud university medical centre – the Netherlands).
