Introduction
The reimbursement of orphan drugs poses persistent methodological and ethical challenges for health systems (Reference Nestler-Parr, Korchagina and Toumi1;Reference Eichler, Kossmeier, Zeitlinger and Schwarzer-Daum2). Orphan drugs target rare conditions affecting fewer than 5 in 10,000 people in the European Union and are often severe, chronic, and life-threatening, or–in exceptional cases–are granted orphan designation due to insufficient expected return on investment. Small sample sizes (Reference Eichler, Kossmeier, Zeitlinger and Schwarzer-Daum2) and heterogeneous patient populations lead to uncertainty about their clinical benefit, further exacerbated by limited knowledge on disease natural history and the frequent occurrence of rare diseases in pediatric populations, which complicate clinical trial design (Reference Nestler-Parr, Korchagina and Toumi1;Reference Hughes, Tunnage and Yeo3–Reference Pizzamiglio, Vernon, Hanna and Pitceathly10). At the same time, the elevated prices of these innovative therapies compared to non-orphan drugs (Reference Chambers, Silver, Berklein, Cohen and Neumann11) challenge payers in determining whether the orphan drug provides adequate value for money, especially given that, for many orphan drugs, supportive care is the only available comparator. This lack of adequate active comparators complicates the assessment of relative effectiveness, and, consequently, the justification of price (Reference Nestler-Parr, Korchagina and Toumi1;Reference Pearson, Rothwell, Olaye and Knight8;Reference Tambuyzer12–Reference Herder14).
Over the years, national health technology assessment (HTA) agencies have adapted their conventional appraisal processes to better account for the complexities of orphan drugs (Reference Ollendorf, Kamal-Bahl, Gleason, Schrandt and McElwee15–Reference Nicod, Annemans, Bucsics, Lee, Upadhyaya and Facey19). In addition to traditional HTA criteria such as efficacy, safety, and costs, many agencies increasingly consider broader elements of value, including patient preferences, unmet medical need, and disease severity (Reference Drummond and Towse20–Reference Felippini, Biglia, Lima and Aguiar26). Whereas traditional criteria primarily support deliberation aimed at improving allocative efficiency, these broader criteria are more closely aligned with societal values related to equality in outcomes and access across both rare and non-rare disease populations.
However, the extent to which these broader criteria influence the final reimbursement decision is often not clear (Reference Stafinski, Glennie, Young and Menon27), and there exist no guidelines on how national agencies trade-off these broader criteria against traditional criteria. As a result, these criteria and their weights may be considered more implicitly by reimbursement committees. Furthermore, HTA reports often fail to emphasize any additional, contextual elements that did not come up during assessment, but which were considered by committees during the appraisal (Reference Blonda, Barcina Lacosta, Toumi and Simoens24;Reference Blonda, Denier, Huys, Kawalec and Simoens28). However, this approach may introduce bias and perverse effects, which may steer the decision in a certain direction. Moreover, as we have shown in a previous study, the limited transparency on their assessment and implicit weighting may lead to inconsistent decision making (Reference Blonda, Denier, Huys, Kawalec and Simoens28).
Multi-criteria decision analysis (MCDA) has therefore been proposed as a framework to structure the assessment and appraisal of orphan drugs (Reference Picavet, Cassiman and Simoens29–Reference Schey-Schulmann39). In MCDA, a multidisciplinary team identifies, prioritizes, and scores key criteria relevant to a decision, which can then be aggregated into an overall performance score and compared against an external benchmark (Reference Belton and Steward40–Reference Thokala, Devlin and Marsh43). Finally, publishing the the results of this process can enhance the legitimacy and accountability of reimbursement decisions (Figure A1 in Supplementary Appendix A).
MCDA may address the limitations of conventional HTA approaches for orphan drugs by enabling the explicit inclusion and weighing of multiple criteria. In doing so, it supports decision makers in making trade-offs between traditional HTA criteria (such as efficacy and safety) and broader criteria (such as unmet need, severity, etc.), while making these trade-offs more transparent (Reference Friedmann, Levy, Hensel and Hiligsmann36;Reference Thokala, Devlin and Marsh43–Reference Guarga, Badia and Obach45). Furthermore, MCDA facilitates structured deliberation around a predefined set of key criteria, while allowing for the inclusion and transparent reporting of stakeholders’ individual considerations. As a result, MCDA may provide insights into which elements have the greatest influence on final decisions (Reference Gilabert-Perramon, Torrent-Farnell and Catalan35;Reference Baran-Kooiker, Czech and Kooiker37;Reference Thokala and Duenas42;Reference Guarga, Badia and Obach45–Reference Angelis and Kanavos48).
Despite extensive discussion of MCDA frameworks in the literature (Reference Paulden, Stafinski, Menon and McCabe21;Reference Blonda, Denier, Huys and Simoens23;Reference Hughes-Wilson, Palma, Schuurman and Simoens44;Reference Fedyaeva, Omelyanovsky, Rebrova, Khan and Petrovskaya49;Reference Kolasa, Zwolinski, Kalo and Hermanowski50–Reference Wagner, Samaha and Casciano55;Reference Gasol, Paco, Guarga, Bosch, Pontes and Obach56), practical experience with their application at reimbursement level remains limited, with examples at the European level (Reference Sussex, Rollet, Garau, Schmitt, Kent and Hutchings57) and in Portugal (Reference Vandewalle, Amorim and Ramos58), Spain (Reference Guarga, Badia and Obach45;Reference Jimenez, Ais, Acuña, González, Paco and Gil59–Reference Gil-Nagel, Falip and Sánchez-Carpintero62), and the United Arab Emirates (Reference Alnaqbi, Elezbawy and Fasseeh63).
This study aimed to develop and pilot a Belgian MCDA framework for the appraisal of orphan drugs through a bottom-up, multi-stakeholder process. Specifically, we sought to (i) identify and define a parsimonious set of criteria considered most relevant for orphan drug reimbursement in Belgium, (ii) elicit stakeholder weights for these criteria, and (iii) test the framework in a pilot assessment of four orphan drugs previously appraised in the Belgian context. In doing so, we documented practical challenges and benefits to inform future implementation, both in Belgium and other jurisdictions.
Methods
Study design
We adopted a multi-methods design to develop and pilot an MCDA framework for the appraisal of orphan drugs in Belgium. This included (i) identification of candidate appraisal criteria through a targeted literature review, (ii) multi-stakeholder focus groups to select and define appraisal criteria, (iii) a Delphi process to elicit criteria weights, and (iv) pilot application of the framework to four orphan drugs previously assessed in the Belgian reimbursement context.
Identification of appraisal criteria
A targeted literature review was conducted in PubMed and Embase to identify appraisal criteria used in MCDA frameworks for orphan drugs. The review focused on peer-reviewed reporting original MCDA frameworks or empirical applications in the context of rare diseases, published between January 2020 and January 2023. A list of candidate criteria, definitions and corresponding preliminary score scales was extracted and served as input for subsequent phases. Full details on the search strategy, inclusion criteria, and data extraction are provided in Supplementary Appendix A.
Expert panel and selection of criteria
A multidisciplinary expert panel was convened, comprising representatives from the national reimbursement agency (RIZIV/INAMI), sickness funds, patient organizations, pharmaceutical companies, clinicians, and academics in bioethics. The same panel was involved throughout the entire study to ensure continuity of deliberation.
Four focus group discussions were organized to guide the selection and operationalization of appraisal criteria and to support the pilot application of the MCDA framework. Two meetings were held in person and two online. Before the first meeting, panel members received a guidance document summarizing the candidate criteria derived from the literature review, together with proposed definitions and score levels. During the first two focus groups, participants were asked to assess the relevance, clarity, and feasibility of each criterion, propose refinements, and suggest additional criteria where necessary. Criteria were retained when supported by at least half of the participants. For each retained quantitative criterion, participants agreed on a definition and an ordinal score scale to support consistent application during the pilot phase. In addition to quantitative criteria, participants could identify contextual criteria to be considered qualitatively during appraisal.
Criteria weighing
Following the selection and definition of criteria, a Delphi-based consensus technique (Reference Dajani, Sincoff and Talley64;Reference Habibi, Sarafrazi and Izadyar65) was applied to elicit relative criterion weights. Participants independently allocated a total of 100 points across the quantitative criteria using the point-allocation method (0 = lowest importance; 100 = highest importance). After receiving anonymized feedback on group-level results, two additional rounds were conducted to refine weights and achieve consensus, with participants asked to justify outlying weights. Full details on the Delphi process are reported in Supplementary Appendix B.
Pilot case selection
Four orphan drugs previously assessed in the Belgian reimbursement context were selected to pilot the MCDA framework: ponatinib (oncology), pirfenidone (respiratory), onasemnogene abeparvovec (neuromuscular), and metreleptin (metabolic). The cases were chosen to reflect heterogeneity in clinical benefit profiles, evidentiary maturity, and availability of comparators, thereby allowing us to test the framework across various appraisal challenges commonly encountered in orphan drug reimbursement (see Table 1).
Profiles of the four orphan drugs assessed in the MCDA pilot
Table 1. Long description
The table compares four products across six criteria.
1. Ponatinib (Iclusig): Oncology area; first in class synthetic; reimbursed; alternative treatment exists for most; goal is disease progression inhibition; data includes one open-label trial, one indirect comparison, and no P R O data.
2. Pirfenidone (Esbriet): Respiratory area; second in class synthetic; reimbursed; alternative treatment slows progression; goal is slowing disease progression; data includes meta-analysis of nine R C T s, one retrospective cohort study, and P R O analyses.
3. Onasemnogene abeparvovec (Zolgensma): Neuromuscular area; A T M P gene therapy; reimbursed; alternative treatment is symptomatic; goal is to halt disease progression (cure); data includes systematic review, meta-analysis, indirect comparisons, and Q A L Y comparison.
4. Metreleptin (Myalepta): Metabolic area; first in class synthetic; not reimbursed; no effective alternative treatment; goal is glycemic control and prevention of metabolic abnormalities; data includes two open-label trials, one matched cohort study, and a retrospective study on P R O s.
ATMP, advanced therapy medicinal product; MCDA, multi-criteria decision analysis; PRO, patient reported outcomes; QALY, quality-adjusted life year; RCT, randomized controlled clinical trial.
Data sources for the pilot application
Evidence for the three reimbursed products was extracted from European Public Assessment Reports and Belgian HTA reports. For metreleptin, which was not reimbursed in Belgium, evidence was derived from the joint BeneluxA HTA report. For each product, a structured, high-level evidence matrix was compiled, mapping available evidence to each appraisal criterion, including criterion definitions and score scales. A template for such an evidence guide is provided in Supplementary Appendix C.
Data analysis
All quantitative analyses were performed using Microsoft Excel. Weights for all quantitative criteria
$ \left({W}_i\right) $
were normalized
$ \left({W}_{\mathit{\operatorname{norm}},i}\right) $
to a 0–1 scale (the sum of the individual weights equal to 1) and summarized using medians and interquartile ranges. Then, the median weights were re-normalized to 1. In agreement with participants, blank scores were replaced by the mean of non-missing scores for that criterion. All scores
$ \left({S}_i\right) $
were standardized to a 0–1 scale using min–max normalization.
This ensured that differences in raw score ranges did not mechanically determine their contribution to the final aggregated MCDA score. For each criterion, we calculated the mean and standard deviation (SD) of the normalized scores
$ \left({S}_{\mathit{\operatorname{norm}},i}\right) $
. The value contribution of each quantitative criterion
$ \Big({C}_i $
) was calculated as the product of its normalized weight
$ \left({W}_{\mathit{\operatorname{norm}},i}\right) $
and normalized score
$ \left({S}_{\mathit{\operatorname{norm}},i}\right) $
. The overall MCDA outcome (O) is calculated as the sum of the value contributions of all
$ (i) $
criteria.
Focus group discussions were transcribed verbatim, pseudonymized, and analyzed using the QUAGOL framework (Reference de Casterle B, Gastmans, Bryon and Denier66). A combination of deductive coding (based on predefined topics such as reimbursement criteria) and inductive coding was applied. Coding and thematic synthesis were supported using NVivo 14 software. More information on the qualitative analysis is provided in Supplementary Appendix D.
Ethical considerations
The research protocol was approved by the Ethical Committee (EC) Research University Hospitals Leuven (S66782). All participants provided written informed consent, and data were handled confidentially and pseudonymized.
Results
Expert panel
Between 17 January 2023 and 23 April 2023, eight experts (three males and five females) participated in four focus group discussions and the weighing exercise. Our panel included one clinician, two patient representatives, one ethicist, two representatives from the largest insurance funds in Belgium, and two from the pharmaceutical industry. A ninth expert, a clinician, participated in the first two focus groups, but withdrew due to personal reasons. All experts were senior professionals (> 10 years of experience) with substantial experience in orphan drug appraisal and familiarity with the Belgian reimbursement process.
Final MCDA framework: criteria and definitions
The literature review identified twenty-one unique reimbursement criteria from fourteen peer-reviewed publications (Supplementary Appendix A). Through the multi-stakeholder deliberative process, seven quantitative and three contextual criteria were selected for inclusion in the MCDA framework, presented in Table 2. Table 3 elaborates on the criteria and aspects that were most extensively discussed, highlighting the principal challenges of selecting and defining the criteria within the MCDA framework for orphan drugs. The scoring functions were carefully developed. For comparative criteria (comparative effectiveness, comparative safety, and impact on PROs), a negative starting point was used to distinguish a treatment performing worse than the comparator from one performing similarly. At the same time, the expert panel preferred scoring approaches that would explicitly reward added benefit, without automatically penalizing uncertain or heterogeneous evidence. More broadly, the panel sought to design scoring scales that would encourage transparency and the submission of relevant evidence rather than create incentives to withhold data, which is particularly important for rare disease therapies, where evidence is often limited.
Overview of quantitative and contextual criteria included in the proposed Belgian MCDA framework for orphan drugs
Table 2. Long description
The table is divided into two main sections.
Section 1: Quantitative criteria and Score scale.
- Unmet medical need: Availability and shortcomings of alternatives. Scores range from (0) no medical need to (2) major medical need.
- Social disease burden: Impact on autonomy and lifestyle. Scores range from (1) marginal to moderate to (3) substantial burden.
- Comparative effectiveness: Beneficial change in signs or symptoms. Scores range from (-1) less clinical benefits to (2) moderate to substantial benefits compared to alternatives.
- Comparative safety: Reduction in harmful effects. Scores range from (-1) less favorable to (2) substantially favorable safety profile.
- Impact on patient reported outcomes (P R O s): Changes in quality of life. Scores range from (-1) less favorable to (2) moderate to substantial favorable changes.
- Type of health benefit: Potential for cure. Scores are (0) no potential or (1) potential for cure.
- Quality of evidence and level of certainty: Validity and reliability of data. Scores range from (0) substantial uncertainty to (3) reliable data.
Section 2: Contextual criteria (assessed qualitatively) and Impact on appraisal.
- Value for money and affordability: Includes budget impact (Levels 1, 2, and 3) and cost-effectiveness (I C E R). Impact is categorized as favorable, neutral, or opposing reimbursement.
- System capacity and appropriate use of the intervention: Infrastructure and organizational requirements. Impact is categorized as favorable, neutral, or opposing reimbursement.
- Expert consensus/clinical practice guidelines: Alignment with state-of-the-art practices. Impact is categorized as favorable, neutral, or opposing reimbursement.
ICER, incremental cost-effectiveness ratio; PROs, patient-reported outcomes.
Deliberative and pilot-informed insights on the operationalization and scoring of MCDA criteria
Table 3. Long description
A table with four columns: Criterion and core definition, Discussion, Scoring Issues encountered, and Scoring Solutions considered.
- Unmet medical need: Discussion focuses on remaining patient needs and inclusion of off-label products. Issues include ambiguity in definition and difficulty distinguishing from social burden. Solutions include considering impact on physical appearance and using aggregated scores for small subgroups.
- Social disease burden: Focuses on impact beyond medical burden. Issues involve quantification of caregiver burden and lack of benchmarks. Solutions suggest more quantitative measures and patient perspectives.
- Comparative criteria: A header spanning all columns.
- Comparative effectiveness, safety, and P R O: Discussion notes these are essential but endpoints may not reflect patient benefit. Issues include favorable placebo comparisons by default and lack of active comparators in single-arm trials. Solutions prefer indirect comparisons over placebo data.
- Quality of evidence and level of certainty: Addresses rare disease research limitations. No specific scoring issues or solutions were commented on.
- Type of health benefit: Focuses on potential to cure. Issues include difficulty defining cure at appraisal and overlap with effectiveness. Solutions involved simplifying the scale from a graded negative 1 to 3 scale to a binary 0 or 1 scale.
- Prioritized populations, rarity, and severity: Not retained as standalone criteria due to ethical concerns. Issues include age implicitly influencing other scores. Solutions emphasize shared decision making.
- Cost-related criteria: Recognized as essential but excluded from quantitative assessment due to discomfort in trading costs against benefits. Issues include lack of long-term data and budget impact being context-dependent.
HTA, health technology assessment; MCDA, multi-criteria decision analysis; PRO, patient-reported outcome; QoL, quality of life; SMA, spinal muscular atrophy.
Overall, the panel raised concerns about combining factors that measure benefits and costs into a final, aggregated MCDA score. They concluded that the decision on whether to reimburse an orphan drug should depend on the benefits it provides to the population covered by the reimbursement claim. Meanwhile, cost criteria, such as “budget impact” and “cost-effectiveness”, which depend on the price requested by the applicant, should determine the level of reimbursement and may restrict reimbursement to a narrower population than requested in the reimbursement claim. These factors are therefore included in a second, contextual (and qualitative) assessment. The key considerations underlying the selection of the retained criteria are summarized in Table 3, which also includes insights from the pilot application that are discussed later in this section.
Criteria weights
Table 4 presents the distribution of weights assigned to each quantitative criterion. Unmet medical need received the highest weight, followed by comparative efficacy, impact on PROs, and comparative safety. The lowest median weight was assigned to the type of health benefit. For the weight of unmet medical need, the degree of consensus between participants–indicated by the IQR–was the smallest.
Weight distribution and consensus levels for orphan drug MCDA criteria a
Table 4. Long description
The table contains 14 columns and 7 rows of data. The columns are: Criteria, E T 1, P A 1, P A 2, H I 1, H I 2, C L 1, P I 1, P I 2, Mean, Median, Q 1, Q 3, and I Q R.
* Unmet medical need: Weights range from 17 to 30. Mean 21.6, Median 19, Q 1 18, Q 3 28, I Q R 10.
* Social disease burden: Weights range from 7 to 20. Mean 11.9, Median 10.5, Q 1 7.8, Q 3 15.5, I Q R 7.8.
* Comparative effectiveness: Weights range from 15 to 20. Mean 16.9, Median 16.5, Q 1 15.3, Q 3 18.5, I Q R 3.3.
* Comparative safety: Weights range from 10 to 16. Mean 13.3, Median 14, Q 1 10.5, Q 3 15, I Q R 4.5.
* Impact on patient reported outcomes: Weights range from 10 to 20. Mean 16.8, Median 17, Q 1 15, Q 3 20, I Q R 5.
* Quality of evidence and level of certainty: Weights range from 0 to 20. Mean 11.5, Median 11, Q 1 10, Q 3 16.5, I Q R 6.5.
* Type of health benefit: Weights range from 0 to 10. Mean 8.1, Median 9.5, Q 1 8, Q 3 10, I Q R 2.
Stakeholder identifiers: E T is Ethicist, P A is Patient Representative, H I is Health Insurance Representative, C L is Clinician, and P I is Pharmaceutical Industry Representative.
a Weights were assigned to each criterion using the 100-point allocation method.
Q1, first quartile; Q3, third quartile; IQR, interquartile range. Identifiers: ET, ethicist; PR, patient representative; HI, health insurance representative; CL, clinician; PI, pharmaceutical industry representative.
Pilot MCDA outcomes
Figure 1 presents the overall weighted MCDA scores for each of the orphan drugs across the seven reimbursement criteria. Onasemnogene abeparvovec achieved the highest overall score (0.77), followed by metreleptin (0.64), pirfenidone (0.47), and ponatinib (0.39).
Mean total MCDA scores for four orphan drugs.
Figure 1. Long description
A legend at the top identifies four drugs: Pirfenidone (0,47) in light green, Ponatinib (0,39) in purple, Onasemnogene abeparvovec (0,77) in blue, and Metreleptin (0,64) in dark green. The radar chart uses a scale from 0.00 at the center to 0.20 at the perimeter. The six axes, starting from the top and moving clockwise, are:
* Unmet medical need: Metreleptin and Ponatinib score highest, near 0.18 and 0.15 respectively.
* Social disease burden: Onasemnogene abeparvovec leads, followed by Metreleptin.
* Comparative effectiveness: All drugs show a significant peak here, with Onasemnogene abeparvovec and Metreleptin reaching approximately 0.18.
* Comparative safety: Onasemnogene abeparvovec has the highest score, while others are closer to the center.
* Impact on patient reported outcomes: Metreleptin and Onasemnogene abeparvovec show the most significant extension.
* Type of health benefit: Onasemnogene abeparvovec shows a distinct peak compared to the other three drugs.
* Quality of evidence and level of certainty: All drugs cluster at lower values near 0.05 to 0.10.
For onasemnogene abeparvovec, criteria that contributed most to its overall performance score were unmet medical need for patients with spinal muscular atrophy (SMA) (0.18), impact on PROs (0.14), and comparative effectiveness (0.12). Criteria that scored the lowest were quality of evidence and level of certainty (0.03) and type of health benefit (0.10). Overall, metreleptin scored similarly to onasemnogene abeparvovec on all but two criteria. It received the same score for unmet medical need (0.18) and received similar high scores for its impact on PROs (0.15) and comparative effectiveness (0.12). However, metreleptin scored remarkably lower for type of health benefit (0.05 for metreleptin vs. 0.10 for onasemnogene abeparvovec) and comparative safety (0.03 vs. 0.10), which was its lowest-scoring criterion, apart from the quality of evidence and level of certainty (0.02). The total MCDA scores for ponatinib and pirfenidone were similar across the majority of seven criteria, and were attributed mostly to their individual performance scores for unmet medical need (0.15 vs. 0.12, respectively), “comparative effectiveness (0.10 vs. 0.08) and impact on PROs (0.06 vs. 0.08). Ponatinib received its lowest scores for quality of evidence and level of certainty (0.02), comparative safety (0.03), and a zero score for type of health benefit. Pirfenidone received the lowest performance scores for type of health benefit (0.02), comparative safety (0.04), and social disease burden (0.06), although these were similar to ponatinib (0.00, 0.03, and 0.04, respectively). Supplementary Appendix E presents the final MCDA scores and a spider chart illustrating the unweighted mean scores across all criteria.
Pilot application and methodological insights
The pilot application generated important methodological insights into the feasibility, interpretability, and limitations of individual criteria when applied in a real reimbursement context. A synthesis of these criterion-level insights is provided in Table 3.
Discussion
Although several MCDA frameworks have been developed for the appraisal and assessment of orphan drugs, pilot experiences remain limited (Reference Guarga, Badia and Obach45;Reference Sussex, Rollet, Garau, Schmitt, Kent and Hutchings57–Reference Alnaqbi, Elezbawy and Fasseeh63). This study presents the first MCDA framework to structure orphan drug appraisal in Belgium, developed through a bottom-up, multi-stakeholder process. It provides empirical insight into the applicability of selected MCDA criteria within the Belgian reimbursement context and identifies where further clarification, refinement, or contextual guidance is required before implementation. The lessons learned may inform HTA and reimbursement agencies seeking to operationalize or adapt similar MCDA frameworks within their own drug appraisal process.
Key criteria for the assessment and appraisal of orphan drugs in Belgium through MCDA
The proposed MCDA framework comprises seven quantitative and three qualitative, contextual criteria. The criteria prioritized by the expert panel–particularly unmet medical need, comparative efficacy, comparative safety, and impact on PROs–largely align with those commonly reported in published MCDA frameworks for orphan drugs (Reference Zelei, Mendola, Elezbawy, Németh and Campbell67). While disease severity is typically considered in MCDA frameworks, it was excluded as the panel considered its assessment insufficiently operationalizable and its aspects are covered by unmet medical need (Reference Zelei, Mendola, Elezbawy, Németh and Campbell67). Nevertheless, one may argue that the assessment of unmet medical need is similarly difficult to operationalize, as illustrated by the panel discussions.
Although the criteria considered by the Belgian reimbursement committee are not formally codified in legislation (The Royal Decree of 1 February 2018, the “RD Reimbursement”), existing guidance indicates that similar elements are assessed in practice (68). By explicitly structuring these elements within an MCDA framework, the proposed approach enhances transparency and supports more consistent and accountable appraisal. A detailed mapping of the Belgian reimbursement guidance to the MCDA criteria is provided in Supplementary Appendix F.
Evolving preferences for evidence
The panel’s preference for indirect comparative evidence against an active comparator, rather than direct comparative studies versus placebo, highlights the need for greater acceptance of, and clearer methodological guidance for, indirect treatment comparisons (Reference Freitag, Gurskyte and Sarri69;Reference Macabeo, Rotrou, Millier, François and Laramée70), especially in contexts with small and heterogeneous patient groups. More broadly, accepting less stringent evidence may be justified to avoid unduly penalizing therapies for which more conventional evidence generation is not feasible. Howeve, future MCDA frameworks should specify in advance under which conditions this is acceptable, and require that its application in individual appraisals is explicitly justified and transparently reported.
Including novel criteria for the reimbursement of orphan drugs
While the inclusion of patient-centered and social criteria (social disease burden, impact on PROs, and unmet medical need) allows MCDA to capture the broader value of orphan drugs, approaches to quantify them are more uncertain and experience with their scoring remain limited (Reference Zelei, Mendola, Elezbawy, Németh and Campbell67). While explicit inclusion of these criteria provides information on how they are defined and traded off with classic criteria, further work is needed to refine their scope and operationalization and to strengthen their validity for inclusion in MCDA.
Unmet medical need remains a key consideration in reimbursement decisions, and its normative importance is supported by various societal preference studies (Reference Dabbous, Chachoua and Aballéa71). Remarkably, the orphan drug that scored highest for unmet medical need (onasemnogene abeparvovec) also had the greatest number of available alternatives. This was because experts considered the burden of the disease in young children and the added benefit of the new treatment over existing options. However, his may create potential overlap with criteria such as social disease burden and type of health benefit. Furthermore, broad criteria such as unmet medical need may unintentionally embed value judgments (Reference DiStefano and Krubiner72)–for example, a tendency to prioritize younger patients–that fall outside the criterion’s current definition and are not made explicit in the reimbursement report. Future refinements should therefore further define the concept of unmet medical need (Reference Vreman, Heikkinen and Schuurman73) and make such normative elements more transparent.
Budget impact and cost-effectiveness
Overall, there seems to be no consensus within the HTA community on whether and how to incorporate cost criteria into MCDA frameworks for orphan drugs, as published approaches vary considerably (Reference Zelei, Mendola, Elezbawy, Németh and Campbell67). In the present framework, budget impact and cost-effectiveness were excluded from the final aggregated (quantitative) MCDA score, but retained as qualitative contextual criteria due to their importance in appraisal and price negotiations. This approach is also consistent with ISPOR task force guidance, in which MCDA is framed primarily as a tool to assess benefit (Reference Baltussen, Marsh and Thokala38;Reference Phelps, Lakdawalla, Basu, Drummond, Towse and Danzon74). In addition, scoring cost-effectiveness within MCDA would require a clearly defined willingness-to-pay threshold (Reference Phelps, Lakdawalla, Basu, Drummond, Towse and Danzon74), which remains undefined in Belgium.
The proposed framework therefore mirrors Belgian practice by preserving a clear distinction between assessing the added value of a therapy and subsequently determining whether, and under which conditions, reimbursement is feasible and acceptable. Cost considerations are reserved for a second, qualitative appraisal step, in which decision makers may adopt a favorable, neutral, or unfavorable position toward reimbursement based on affordability, value for money, and other contextual factors that cannot be meaningfully captured in the aggregated MCDA score.
At the same time, excluding cost criteria from the quantitative MCDA does not imply that cost-effectiveness should be disregarded. Despite the well-recognized uncertainties surrounding cost-effectiveness analyses for orphan drugs, retaining cost-effectiveness analysis within the appraisal process remains essential to ensure applicants pursue cost-effective pricing strategies (Reference Blonda, Denier, Huys, Kawalec and Simoens28).
Leveraging MCDA to optimize post-launch evidence generation and health technology reassessment
A central challenge in orphan drug appraisal is the scarcity of robust clinical evidence at the time of assessment. For this reason, the proposed framework explicitly incorporates the criterion quality of evidence and level of certainty, consistent with several published MCDA frameworks for orphan drugs (Reference Gilabert-Perramon, Torrent-Farnell and Catalan35;Reference Guarga, Badia and Obach45;Reference Schey, Krabbe, Postma and Connolly52;Reference Kolasa, Zwolinski, Zah, Kaló and Lewandowski54;Reference Wagner, Samaha and Casciano55;Reference Roldan, Badia and Marcos-Rodriguez60;Reference Morgan, Vogler and Wagner75). A low score on this criterion signals substantial residual uncertainty and should be interpreted as a key consideration alongside the aggregated MCDA outcome.
Importantly, high uncertainty does not necessarily justify refusal of reimbursement, but may warrant conditional acceptance. In line with previous literature, uncertainty should only be accepted temporarily (Reference Wagner, Samaha and Casciano55), and may therefore prompt the use of managed entry agreements (MEAs), which allow temporary reimbursement while additional evidence is generated (Reference Wenzl and Chapman76). These may include financial or outcomes-based components, such as payback mechanisms, discounts, or reimbursement linked to predefined clinical or patient-relevant milestones (Reference Dabbous, Chachoua and Aballéa71;Reference Garrison, Towse and Briggs77–Reference Facey, Espin and Kent79).
In this context, MCDA can support reimbursement committees by identifying the dimensions of value in which the evidence remains weak, thereby helping to inform the type of evidence generation required. For instance, orphan drugs scoring poorly on comparative safety or on quality of evidence and level of certainty may warrant MEAs focused on collecting more robust post-launch safety data.
To optimize the use of MCDA for informing subsequent MEAs, future iterations of the framework could further refine the criterion quality of evidence and level of certainty by incorporating sub-scores linked to the comparative criteria included in the MCDA. This may result in a single composite score for this criterion, in which each sub-score is weighted according to the weight of the corresponding comparative criterion. In this way, uncertainty in more highly weighted criteria would count more heavily.
MCDA as a tool to structure the assessment and appraisal process for (orphan) drugs
The expert panel agreed that the main merit of MCDA lies in its ability to structure deliberation and make the appraisal process more transparent by making underlying trade-offs explicit. MCDA scores, both at criterion level and in aggregated form, support appraisal by visualizing these trade-offs,e.g., via the spider graph in Figure 1. The criterion-level scores and weights are particularly informative in showing these trade-offs, whereas the aggregated score should be interpreted flexibly within the broader deliberation (Reference Baltussen, Marsh and Thokala38;Reference Oortwijn, Husereau and Abelson80). Together, these scores inform the assessment of the overall value of the intervention, which constitutes only the first step of appraisal.
For example, an orphan drug with a relatively low overall MCDA score may still be reimbursed in a refractory subgroup if committee members judge that the score does not fully reflect subgroup-specific considerations, such as a higher unmet medical need than in the overall target population. In such cases, the committee may depart from the score, provided that its reasons are clearly documented. The subsequent contextual step may then still provide additional arguments for or against reimbursement, irrespective of the outcome of the initial value assessment.
We would therefore advise against adopting a strict minimum MCDA score for reimbursement, also because small differences in scores around such a threshold could lead to unequal decisions for interventions of near-equivalent value. Over time, however, comparisons with previous appraisals may help indicate whether an intervention provides relatively good value. In the context of (MEA-related) reassessments, aggregated MCDA scores may also be compared with previous scores for the same intervention and linked to price expectations or negotiation ranges (Reference Ollendorf, Rand, Mooney, Phillips and Basu81;Reference Ollendorf, Rand, Blonda, Mooney, Phillips and Basu82). Nevertheless, any score should be interpreted in light of the limitations of quantitative MCDA, as discussed in detail by DiStefano and Krubiner (Reference DiStefano and Krubiner72). Human judgment therefore remains essential.
In a potential real-world application, applicants would submit the relevant evidence in a structured format aligned with the MCDA criteria, after which the HTA/reimbursement agency would conduct its standard assessment. A multi-stakeholder drug reimbursement committee could then appraise the evidence, taking into account the HTA body’s critique, with each member individually assigning a substantiated score for each criterion. The main arguments underlying each score as well as those raised during the contextual deliberation, would then be documented in the final appraisal report.
Limitations and directions for future research
While the criterion type of health benefit was included to recognize innovative therapies, the pilot revealed that claims related to curative potential are often uncertain and interpreted differently across stakeholders. Future work should therefore assess whether this criterion can be robustly and consistently operationalized, or whether it risks introducing ambiguity into appraisal. In addition, the wording of the score labels for quality of evidence and level of certainty may warrant further refinement, as their framing in terms of uncertainty, rather than certainty, may have been counterintuitive.
Individual scoring was used to capture divergent perspectives and ensure that all expert views were reflected, including those less likely to be voiced during group discussion. Variation in scores was therefore expected and reflects the multidisciplinary nature of reimbursement decision making (Reference Broekhuizen, Groothuis-Oudshoorn, van Til, Hummel and IJzerman83). If MCDA were to be formally implemented, decision makers should consider whether to rely on individual or consensus scoring. In either case, clear and consistently reiterated criteria definitions are essential to ensure that score variation reflects genuine differences in judgment rather than divergent interpretations (Reference Cleemput, Devriese, Kohn and Westhovens84).
Lastly, although we aimed to elicit weights from a balanced range of stakeholders, our eight-member expert panel may not fully representat the range of stakeholder perspectives in Belgium. Further studies should therefore test the external validity of the criteria weights by surveying a broad range of professional stakeholders as well as the wider public.
Conclusion
This study developed a multi-stakeholder MCDA framework for orphan drug appraisal in Belgium and provides the first empirical insights from its pilot application. The findings underscore the importance of complementing comparative clinical evidence with patient-centered criteria (such as unmet medical need) and contextual criteria, while also highlighting areas for further refinement before formal implementation. Nonetheless, this work offers an initial foundation for applying MCDA to orphan drug reimbursement in Belgium, with a view to more consistent and transparent decision making, and may inform similar initiatives in other jurisdictions.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/S0266462326104127.
Acknowledgments
The authors express sincere appreciation to the experts who participated in the focus groups for their time, commitment, and invaluable contributions that significantly enriched the research. Special recognition is extended to Lukas Verhamme for his meticulous work in transcribing the focus group discussions, and to Marc Dooms for his insightful contributions throughout the study process.
Author contribution
AB conducted the literature review, developed the study protocol, prepared the submission for ethical approval, and conducted the study: inviting stakeholders, developing the evidence guides, conducting focus group discussions, analyzing transcripts, and writing the manuscript. SS developed the concept of the study and provided support for selection of the study methods and the final selection of orphan drugs. All authors provided detailed feedback at various points throughout this study: during the development of the study protocol, the study conduct, as well as during the writing of the manuscript. All authors approved the final manuscript.
Funding statement
This research was funded by the Scientific Research Foundation Flanders (FWO) (grant number G0B9819N).
Competing interests
The authors declare none.
Open access
