Introduction
The value determination, appraisal, and reimbursement of pharmaceutical interventions raise challenges for decision makers at the regulatory and reimbursement level, as well as for healthcare professionals, patients, and developers. In recent years, several initiatives have been undertaken to implement health technology assessment (HTA) (Reference Angelis, Lange and Kanavos1). Value frameworks have been perceived as potential tools to assist decision makers in their decision process as they help in accounting for a range of relevant and contextual factors (Reference Angelis, Lange and Kanavos1;Reference Walker2). With few countries having a clear roadmap for HTA implementation, many are looking into examples from high-income countries whereby different decision-making frameworks have been developed (Reference Angelis, Lange and Kanavos1;Reference Walker2). Furthermore, a number of new initiatives have emerged in the development of multiple criteria decision analysis (MCDA) tools for valuing new medicines. The MCDA approach has been promoted as a means to support a systematic consideration of a broad range of decision criteria when appraising healthcare interventions (Reference Walker2). Although it is important to capture all relevant aspects of value, these may not be fully transferable without taking into account country-specific elements, such as population size, gross domestic product per capita, social values, public health priorities, healthcare financing, as well as the availability of processes, structures, and human resources (Reference Angelis, Lange and Kanavos1;Reference Oortwijn and Klein3). Thus, it is recommended to reflect country-specific factors when using MCDA in the decision process especially in low- and middle-income countries (Reference Oortwijn and Klein3;Reference Marsh, Thokala, Youngkong and Chalkidou4).
Policy makers, particularly in contexts of high unmet needs and limited resources, are facing a number of challenges when making decisions about resource allocation (Reference Angelis, Lange and Kanavos1;Reference Walker2;Reference Marsh, Thokala, Youngkong and Chalkidou4;Reference Angelis and Kanavos5). Over the past decade, academics and HTA bodies have explored the use of decision-making criteria and value frameworks to assist policy makers when valuing and appraising pharmaceutical interventions. Such MCDA frameworks, which combine conflicting multiple criteria that capture all relevant social and scientific values, offer an assessment tool to support policy makers in valuing interventions and making reimbursement decisions in the context of HTA (Reference Angelis, Lange and Kanavos1;Reference Walker2;Reference Marsh, Thokala, Youngkong and Chalkidou4;Reference Angelis and Kanavos5). Such frameworks allow the integration of the values and preferences from different perspectives and in that way support a transparent and collaborative decision-making process (Reference Wagner, Samaha, Cuervo, Patel, Martinez and O'Neil6).
Lebanon is a small country with a population size of approximately 6,850,000 in 2018, of which almost 25 percent are refugees. Health expenditure as a share of Gross Domestic Product (GDP) for Lebanon was 6.4 percent in 2014 and increased to 8.2 percent in 2018 (7–10). Over the past few years, the Lebanese healthcare system has been struggling with resource allocation, which has been overstretched due to the social, political, and economic instabilities and the increasingly high prevalence of diseases such as cardiovascular disease, noncommunicable diseases, and cancer (8–11).
The environment has been rendered more complex given the high number of health coverage operators (public and private). Although the Ministry of Public Health provides coverage to almost half of the population, the National Social Security Fund (NSSF), Lebanon's national social insurance system, provides coverage to almost one-third of the population until the age of 64 years. Some patients are being denied access to treatment due to the ministry failing to meet its financial obligations (8;Reference Ammar, Kdouh, Hammoud, Hamadeh, Harb and Ammar9). Reimbursement decisions are mainly made by following the lowest pricing strategy across different sectors, and in certain situations, other reimbursement tools such as Managed Entry Agreements (MEA) are followed. In the Middle East and North Africa (MENA) region and mainly in Lebanon, it was reported that when a public or private institution decides to run MEAs, financial-based schemes are mostly used (Reference Maskineh and Nasser12).
Embedding MCDA in the decision-making process would assist the Lebanese health system to efficiently allocate limited resources and make reimbursement decisions (Reference Walker2;Reference Marsh, Thokala, Youngkong and Chalkidou4). Health economics and outcomes research (HEOR) has been playing a major role in guiding decision makers in health care around the world (13). Drug spending and pricing, universal health coverage, and access equity are among the top 10 HEOR trends (13). Despite challenges to implement HTA in Lebanon, HEOR has recently emerged as a tool of interest to optimize the management of resources, and pharmacoeconomics has also been recognized by decision makers as an important tool to assist in reimbursement decisions and in the assessment of interventions (Reference Maskineh and Nasser12). In an attempt to engage in a meaningful conversation about value, an exploratory exercise on MCDA was conducted as part of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Lebanon workshop. The objective was to reflect and discuss which considerations or criteria would be relevant for value determination of pharmaceutical products in the Lebanese context.
We conducted the MCDA exercise by leveraging the criteria outlined by two different frameworks deemed to be comprehensive. The Evidence and Value: Impact on Decision Making (EVIDEM) multicriteria framework consists of twenty generic criteria (Reference Goetghebeur, Wagner, Samaha, O'Neil, Badgley and Castro-Jaramillo14). The Advance Value framework consists of five domains of value and eleven decision criteria (Reference Angelis and Kanavos5). These two frameworks are commonly used as they capture not only scientific value such as burden of disease and therapeutics outcomes but also country-specific economic and social elements (Reference Angelis and Kanavos5;Reference Goetghebeur, Wagner, Samaha, O'Neil, Badgley and Castro-Jaramillo14).
To our knowledge, the MCDA method has not been used by Lebanese health authorities. The use of a validated value framework could potentially improve the decision process while accounting for country-specific social, economic, and health-system-related challenges.
Methods
Design
A face-to-face workshop was conducted as part of the ISPOR Lebanon Chapter in February 2019. Invitations were sent to all fifty ISPOR members in Lebanon, including hospital pharmacists, key policy makers in the public sector, members in academic institutions, and members in the pharmaceutical industry. Additionally, an announcement was placed on the Chapter's Web site. Thirty-seven members representing all sectors joined a 4-h session on a voluntary basis to explore the concept of MCDA through a hands-on approach with the intention that this exploratory exercise would trigger discussion on the two frameworks. Following a short presentation of the concept of MCDA and of the two published frameworks at hand, namely EVIDEM and the Advance Value Framework, the interactive exercises included the qualitative and quantitative exploration of the frameworks’ criteria and their importance in decision making (Reference Angelis and Kanavos5;Reference Goetghebeur, Wagner, Samaha, O'Neil, Badgley and Castro-Jaramillo14). The qualitative exercise involved discussing which criteria should be considered for reimbursement decision making. Participants were asked to answer, from their own perspective, the question as to whether the given criteria should be formally considered in reimbursement decision making in Lebanon. More than thirty questions were answered by yes, no, or do not know answers (twenty questions on EVIDEM criteria and eleven on the Advance Value Framework) (Reference Angelis, Lange and Kanavos1;Reference Goetghebeur, Wagner, Samaha, O'Neil, Badgley and Castro-Jaramillo14). The quantitative exploration aimed at ranking the different criteria by assigning weights based on their relative importance in the appraisal process for drug reimbursement. Hierarchical point allocation was used, which involved dividing 100 points across the domains, criteria, and subcriteria of the frameworks (Reference Goetghebeur, Wagner, Samaha, O'Neil, Badgley and Castro-Jaramillo14). Participants were invited to provide their answer in writing. Throughout the exercise, each criterion definition was outlined by the moderators; participants were then invited to fill the questionnaire. The session was conducted under the Chatham House Rule to encourage openness, the sharing of information, and to treat shared information confidentially.
Data Analysis
For data analysis, all responses from the questionnaires and the discussion were collected and entered in an Excel calculator. Normalized weights (/obtained by distributing the weights across all criteria to sum up to 1 for each participant) were analyzed and compared using descriptive statistics (mean ± standard deviation [SD]).
Comparison and Usefulness of Both Frameworks
Although comparing both frameworks is not within the scope of this paper, it is important to note that both frameworks are comprehensive, support decision making by breaking down complex problems into multiple criteria, and evaluate a wide range of aspects of HTA from unmet needs/disease severity to efficacy, safety, and socioeconomic impact of adopting an intervention. A general comparison of the criteria of each framework is presented in the Supplementary Table (Reference Angelis and Kanavos5;Reference Goetghebeur, Wagner, Samaha, O'Neil, Badgley and Castro-Jaramillo14). EVIDEM is a more detailed value framework with seven criteria and twenty sub-criteria (Reference Angelis and Kanavos5;Reference Goetghebeur, Wagner, Samaha, O'Neil, Badgley and Castro-Jaramillo14), whereas the Advance Value Framework has five criteria and eleven sub-criteria (Reference Goetghebeur, Wagner, Khoury, Levitt, Erickson and Rindress15). EVIDEM's additional criteria and subcriteria include the size of the population, type of therapeutic intervention, quality of evidence, clinical guidelines, context-specific normative criteria, and context-specific feasibility criteria, making it a more holistic value framework. In addition, EVIDEM has disease severity and size of the affected population as two separate criteria. In fact, clinical guidelines have a considerable impact on practice and, thus, on decision making, and the type of therapeutic intervention allows to capture the nature of the health benefit of the intervention at both the population and individual levels. Furthermore, the affordability criterion is also an important consideration because most healthcare systems face budgetary restrictions (Reference Goetghebeur, Wagner, Khoury, Levitt, Erickson and Rindress15). In general, the MCDA approach has been perceived as a decision support tool, as shown in a study using Advance Value Framework in four European countries to assess different treatment options (Reference Angelis, Linch, Montibeller, Molina-Lopez, Zawada and Orzel16). This is in line with the rationale for conducting the workshop for Lebanese stakeholders who would perceive such framework as a useful process in their decision making where the evaluation of new options has been based on several factors, such as disease severity, the need of new treatment, clinical guidelines, and affordability. Furthermore, the severity of a disease has been a major factor that has been used over the past few years, and, thus, intervention for a severe disease has been considered to be of more value than an intervention for a mild disease. Similarly, an intervention that benefits a large number of patients has been viewed as having more value than one that benefits a small number of patients. Both EVIDEM and Advance Value Framework are expected to be useful in Lebanese practice as they include these evaluation criteria in a structured and measurable format (Reference Angelis, Linch, Montibeller, Molina-Lopez, Zawada and Orzel16).
Results
Organization and Participants
Thirty-seven participants attended the workshop; 70 percent of them were from the pharmaceutical industry, 19 percent were academics in a university setting, and 11 percent were stakeholders from other settings or affiliations (private insurance companies, hospital pharmacists, public sector, patient advocacy groups, and consultants). All participants were key stakeholders of the Lebanese health ecosystem.
Qualitative Assessment of Criteria
Participants unanimously agreed (100 percent) on the relevance of the inclusion of the following criteria of the Advance Value Framework: (i) the direct meaningful endpoints of treatment, (ii) safety and tolerability of the drug, (iii) contraindications and warning of the drug, (iv) patient convenience, (v) public health, and (vi) direct costs. (i) Unmet need and disease severity, (ii) indirect/surrogate endpoints, (iii) spill-over effect, and (iv) indirect costs were considered to be relevant by more than 75 percent of the participants, whereas 50 percent considered the inclusion of the mode of action (Figure 1). No criteria were deemed to be irrelevant.
Relevance of the inclusion of the advance value framework criteria in reimbursement decision making in Lebanon.
Three participants answered “do not know” for each of the following items: Unmet need and disease severity; indirect/surrogate end points, and spillover effect. Two participants answered “do not know” for the mechanism of action.
When discussing EVIDEM, participants unanimously agreed on the relevance of disease severity, unmet needs, comparative effectiveness, comparable safety/tolerability, patient-perceived health/PRO, impact of other medical costs, and quality of evidence. More than 75 percent of the participants agreed on the relevance of type of therapeutic and preventive benefit, budget impact/cost of intervention, impact on nonmedical costs, rare diseases, other priorities, opportunity costs and affordability, system capacity, and appropriate use of intervention. Around half of the participants highlighted the relevance of size of the population affected by disease, expert consensus/clinical practice guidelines, and common goals and specific interests. The majority of participants did not find relevance in the mandate and scope of the healthcare system, environmental impact, political/ historical/cultural context.
A number of participants did not know if the mandate and scope of the healthcare system (8), environmental impact (Reference Wagner, Samaha, Cuervo, Patel, Martinez and O'Neil6), political/historical/cultural context (Reference Oortwijn and Klein3), common goals and specific interests, expert consensus/clinical practice guidelines, budget impact/cost of intervention, limitations of current interventions (unmet needs), and size of the population affected by disease (one for each) were relevant criteria (Figure 2).
Relevance of the inclusion of the EVIDEM framework criteria in reimbursement decision making in Lebanon.
Quantitative Assessment
When weighing the different subcriteria of the Advance Value Framework, participants assigned the greatest relative weight to unmet need and disease severity (mean normalized weight .27, scale [0–1]) followed by direct and meaningful end points (.15), safety and tolerability (.12), and contraindications and warnings (.08). The direct cost ranked seventh with a mean normalized weight of .05. Spillover effect, mechanism of action, and indirect cost (.05, .04, and .04), respectively, received the lowest weights. Unmet need/disease severity (SD ± .08) and direct/meaningful end points (SD ± .05) showed the largest variations in weight (Figure 3).
Advance value framework mean (SD) normalized weights assigned to each criteria.
When weighting the different criteria of the EVIDEM framework, participants assigned the greatest relative weight to disease severity with a mean normalized weight of .1 (scale: 0–1), followed by the size of affected population (.09), the type of therapeutic benefit (patient level; .09), and the type of preventive benefit (population level; .08). Efficacy/effectiveness, budget impact/cost of intervention, and the quality of evidence received an equal weight of .07.
Unmet need, safety and tolerability, expert consensus/clinical practice guidelines, impact on other medicals costs, PROs, and impact on nonmedical costs received the lowest weights. Disease severity (SD .05) and the type of preventive benefit (patient-level) (SD .05) showed the largest variations in weight (Figure 4).
EVIDEM framework mean (SD) normalized weights assigned to each criteria.
Discussion
The objective of this exploratory exercise was to engage in a meaningful discussion on decision-making criteria to be considered and potential implementations in the Lebanese healthcare system through the application of MCDA. This was done, using the lens of the EVIDEM and Advance Value Framework. To our knowledge, this was the first attempt to engage in a conversation on using an MCDA approach as part of the emerging HTA processes in the Middle East.
In recent years, MCDA has been suggested as a tool to explicitly consider multiple criteria and structure decision processes as a mean to render decision more consistent and transparent, specifically with regard to reimbursement (Reference Marsh, Thokala, Youngkong and Chalkidou4;Reference Baran-Kooiker, Czech and Kooiker17–Reference Lasalvia, Prieto-Pinto, Moreno, Castrillón, Romano and Garzón-Orjuela19).
MCDA could serve as a useful tool, particularly in low- and middle-income countries, where reimbursement decision-making processes frequently lack transparency and consistency, making it difficult for manufacturers, payers, patients, and society as a whole to discuss or justify reimbursement decisions or to compare HTA outcomes between countries and regions (Reference Baran-Kooiker, Czech and Kooiker17).
Disease Severity
In our analysis, disease severity was consistently identified with both frameworks as one of the most important criteria for decision making for drug reimbursement in Lebanon. In fact, disease severity is recognized as a major aspect in describing a health problem and as a prioritizing principle in drug reimbursement in several European countries (Reference Wagner, Samaha, Cuervo, Patel, Martinez and O'Neil6;Reference Gilabert-Perramon, Torrent-Farnell, Catalan, Prat, Fontanet and Puig-Peiró20;Reference Iskrov and Stefanov21). Although no published data are available on the use of MCDA in the MENA region, disease severity and the impact of new treatments to alleviate disease burden have been used as key criteria when assessing options for decision making. In Lebanon, although the relative importance of other decision criteria is not agreed upon, disease severity is considered a priority with special regulations for serious and chronic diseases. The Ministry of Public Health provides outpatients with medicines free of charge for debilitating diseases such as cancer, Human Immunodeficiency Virus (HIV), and multiple sclerosis through its central warehouse or drug dispensing center (7;Reference Ammar, Kdouh, Hammoud, Hamadeh, Harb and Ammar9).
Unmet Needs
Although the criterion on unmet needs was identified as relevant in the qualitative assessment, the weighting exercise allowed to show some discrepancies in the relative weights attributed using each framework due to differences in definitions and the inconsistent application of weighting techniques. In the Advance Value Framework, unmet needs obtained the highest weight; conversely, in the EVIDEM framework, it had a lower rank. This discrepancy may be explained by the differences in definition between the two frameworks. The Advance Value Framework consolidates the unmet need and disease severity elements, arguing that both dimensions are preference-dependent and, thus, must be considered together, whereas the EVIDEM framework presents them separately. This finding appears to be concordant with that of Vreman et al. (Reference Vreman, Heikkinen, Schuurman, Sapede, Garcia and Hedberg22), who report that in current practice, unmet medical need is taken into consideration in discussions on the benefit-risk profile and is important when reaching a positive conclusion on drug approval.
Efficacy, Safety, and Type of Intervention
Efficacy was another key consideration for stakeholders as measured in EVIDEM and the Advance Value Framework, and the variability in results are likely inherent to the weighting approach used as well as the limitations of the exercise (Reference Marsh, Sculpher, Caro and Tervonen23;Reference Németh, Molnár, Bozóki, Wijaya, Inotai and Campbell24). This could imply that a framework adapted to the setting and supported by the appropriate training would be of high value. As for safety relative to others, a low weight was attributed to this criterion. This seems to conform to what is observed in most countries and is particularly true in Lebanon where there is an important lag and delay between regulatory approval and reimbursement.
The type of therapeutic intervention was ranked relatively high in EVIDEM. This could be related to the organization of the healthcare system and coverage in Lebanon. Although the Ministry of Public Health does not cover ambulatory care services, it provides vaccines and medications for chronic diseases, as well as medications for severe diseases (e.g., cancer, HIV, and some psychiatric illnesses), free of charge to all beneficiaries (Reference Ammar, Kdouh, Hammoud, Hamadeh, Harb and Ammar9). An unexpected finding was that stakeholders did not seem to prioritize budget impact in their weighting exercise. As in other resource-constrained settings, drugs with a high budget impact are less likely to be reimbursed or may be subject to access restrictions. This finding underlines one of the key limitations of our activity, as the majority of participants were from the pharmaceutical industry.
Limitations
As part of this exercise, we aimed to engage in discussion on the multiplicity of criteria that could be considered for a more consistent and transparent decision-making process. Our study has several limitations. The methodology used in our exercise does not allow us to draw conclusions based on the weights in any robust way (Reference Keeney25). In the real world, different criteria have different levels of importance based on the values of the evaluator and the disease context. Multiple MCDA approaches have been developed and these are all based on Keeney and Raiffa's foundation (Reference Marsh, Thokala, Youngkong and Chalkidou4;Reference Goetghebeur, Wagner, Samaha, O'Neil, Badgley and Castro-Jaramillo14;Reference Keeney and Raiffa26–Reference Thokala, Devlin, Marsh, Baltussen, Boysen and Kalo28). Although MCDA has been designed to generate transparent reports with a clear description of scoring and weighting of a full range of criteria, decision makers may argue that such analysis may inappropriately weigh certain criteria or fail to integrate important factors of preference (Reference Baltussen, Jansen, Bijlmakers, Grutters, Kluytmans and Reuzel29;Reference Baltussen, Marsh, Thokala, Diaby, Castro and Cleemput30). The application of MCDA frameworks has been criticized for being mechanistic, not paying enough attention to guidelines, and not capturing opportunity costs, thereby leading to suboptimal allocation of resources (Reference Baltussen, Marsh, Thokala, Diaby, Castro and Cleemput30).
Another limitation is the challenge to connect weights with criteria scales, specifically in this study where weights should be carefully interpreted due to the methodology used. This is one of the commonly addressed concerns with MCDA, as it may ignore the overlap between criteria and disregard the ranges of consequences with the simple weighting techniques (Reference Marsh, Sculpher, Caro and Tervonen23).
Implementing a process based on MCDA is not without challenges, especially in low- and middle-income settings. The choice of criteria, the methodology to elicit weights and scores, and importantly how the output of such an exercise would be integrated in the decision-making processes are key. An additional layer of complexity stems from the fact that low- and middle-income countries function in a heavily constrained context where methodological capacity may be limited. Acknowledging that the weight elicitation methods used for this exercise may be challenged from a purist perspective, using two approaches has allowed participants to gain a first-hand experience on MCDA while prompting a discussion on the comprehensive view of value (Reference Marsh, Sculpher, Caro and Tervonen23;Reference Németh, Molnár, Bozóki, Wijaya, Inotai and Campbell24). Finally, it is worthwhile to reiterate the need for a tool that supports and does not replace the existing decision-making process.
Conclusion
In an attempt to achieve value-based health care, MCDA has emerged as a likely alternative to address shortcomings in HTA by offering a holistic perspective to value assessment.
The results reported in our survey are concordant with those from other countries such as France, Italy, and Spain. MCDA methodology could be used as a cornerstone to enhance evidence- and ethical-based discussions among Lebanese stakeholders involved in evaluation and decision-making processes.
This study allows for the initiation of conversation regarding the value of leveraging MCDA as a tool to support decision making. An inclusive conversation with all stakeholders with further field testing will be required to establish a reference decision-making framework that is relevant to the Lebanese context.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462321000398.
Acknowledgments
The authors would like to acknowledge the Lebanese American University, Byblos, Lebanon; Certara Inc, Princeton, NJ, USA; and the Professional Society for Health Economics and Outcome Research, ISPOR Lebanon Chapter.
Funding
This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.
Conflict of interest
HK and DS are employed at Certara Evidence and Access.
