Healthcare decision makers are responsible for resource allocation decisions with the primary objective to maximize health and social welfare in the whole population (Reference Brouwer, Culyer and van Exel1). Health technology assessment (HTA) helps make such decisions about whether to reimburse a new treatment by providing guidance on the efficient use of resources, ultimately, optimizing patient access to these. It relies on systematic approaches to appraising evidence about the value of using this treatment in terms of benefits (and costs) in real-world settings, while including considerations of social, ethical and legal aspects to inform coverage for this technology (Reference Luce, Drummond and Jonsson2).
Routine HTA methods that rely on clinical (and economic) evidence may not adequately capture all the important considerations of a treatment's value and the impact of the condition on patients in real-world settings (Reference O'Donnell, Pham and Pashos3). This is partly because HTA is undertaken at the time of the treatment's launch onto the market when evidence is often incomplete because real-world evidence is generally not available. HTA bodies also tend to rely on experimental evidence collected within controlled environments (e.g., randomized clinical trials) (Reference Rawlins4), despite their limitations in capturing effectiveness (Reference Black5). In such cases, scientific judgments about the reliability, generalizability, and meaningfulness of this evidence in the clinical context are made (Reference Rawlins and Culyer6;Reference Rawlins7).
Additionally, these processes also account for elicited societal preferences that refer to cases when society agrees to forego health to treat specific populations. Yet, decision makers may grant preference for a treatment despite this preference not having been previously elicited by the general population; these would be considered as social value judgments (Reference Rawlins and Culyer6;Reference Rawlins7). These judgments are usually made as part of the deliberative process of HTA, during which experts and key stakeholders are consulted and the evidence is discussed until a decision is taken (Reference Garau, Shah and Towse8). The main criticisms of this process is the lack of “accountability for reasonableness” given that there is not always a clear process to account for the inclusion of these forms of evidence in the assessment process, as well as the lack of consistency in accounting for these “other considerations” (Reference Garau, Shah and Towse8–Reference Earnshaw and Lewis10).
Drugs used to treat rare conditions with an orphan designation are often characterized by uncertainty and high incremental cost-effectiveness ratios (ICERs), and are usually not cost-effective (Reference Drummond, Wilson and Kanavos11). This is a consequence of the difficulties in producing robust evidence due to the small patient populations and the heterogeneity of these conditions, as well as their high prices. These reimbursement decisions, therefore, rely on whether society is willing to forego health to the whole population to treat fewer patients with a rare condition (Reference Drummond and Towse12). Little evidence in support of a societal preference for rare conditions exists, and the few studies that attempted to elucidate this suggested the contrary when patients with more common diseases were denied treatment to treat fewer patients with a rare condition (Reference Drummond and Towse12–Reference McCabe, Stafinski and Menon15). In such cases, these decisions partly rely on the decision-makers’ willingness to accept high ICERs based on additional factors that influence their judgment of (scientific and social) value, such as, for example, disease severity, the treatment's orphan status, or to what extent evidence characterized by uncertainty is acceptable (Reference Simoens and Dooms16).
They also rely on the flexibility of these processes in, for example, their ability to implement managed entry agreements or the availability of separate funding programs (e.g., Cancer Drug Fund in England). It is somewhat different in France, where a procedure has been set up to expedite access to drugs for rare diseases, as a means to support development and dissemination of treatments for populations suffering from rare conditions.
This study goes beyond the assessment of clinical and economic evidence into other areas that help explain value. Its purpose is to explore how broader aspects of a treatment's value and the impact of the condition on patients, not captured by routine HTA methods, influence these HTA processes in different countries. This is all the more important given that expectations from HTA bodies in terms of relative effectiveness may differ depending on drug and disease characteristics (Reference Nicod and Kanavos17). The subject of analysis was a sample of orphan drugs in four countries (England, Scotland, Sweden, and France), because of the greater uncertainty characterizing these. We then examined whether the social value judgments revealed pertain to orphan drugs and under what circumstances do they have a preferential status.
METHODS
Study Sample
Purposive sampling was used to select the study countries with the aim of having a good representation of different types of decision-making characteristics, in terms of: (a) the criteria used; (b) the perspective adopted; and (c) any existing elicited preferences (Table 1). The HTA agencies and their decision-making Committees included were: the Appraisal Committee of the National Institute for Health and Care Excellence (NICE) in England, the Scottish Medicines Consortium (SMC) in Scotland, the Dental and Pharmaceutical Benefits Board (TLV) in Sweden, and the Transparency Committee of the Haute Autorité de Santé (HAS) in France. HAS assesses the drug's medical benefit (Service Médical Rendu, SMR) to inform its coverage rate and the relative improvement in medical benefit (Amélioration du Service Médical Rendu, ASMR) to inform the pricing negotiation, ranking treatments in five levels. Both the drug's medical effectiveness (risk-benefit ratio) and its interest in terms of public health (ISP) are accounted for in the SMR assessment. No economic modelling was done by HAS at the time of the sampled drugs’ appraisals. For cost considerations, NICE and SMC agencies adopt a health service perspective and TLV a societal perspective.
HTA Bodies Sampling and Classification Framework of Scientific Assessment and of Social or Societal Preferences
Ten drug and indication pairs were selected, with the following criteria: (a) with an orphan drug designation from the European Medicines Agency, (b) appraised by the NICE Single Technology Appraisal process until December 2012, and (c) by at least two other study countries.
Data Collection and Analysis
This empirical study applied an existing methodological framework enabling the systematic identification and comparison of the criteria driving HTA decisions for the same drugs in different countries through a mixed methods research design comprising three key stages: the evidence appraised, its interpretation and its influence on the final recommendation (Reference Nicod and Kanavos17). In this study, we focus on the results from the interpretation of the evidence component. Specifically, we wanted to see what elements beyond cost-effectiveness, cost, effectiveness, and safety were raised by the HTA agencies and whether these played a role on the decision making.
Thematic analysis was conducted to identify and code all the “other considerations” accounted for during the appraisal process and recorded in the appraisal reports. Bottom-up coding was performed, where codes were inductively created while examining the data to summarize what was put forward and categorize this data depending on the type of information provided (Reference Onwuegbuzie, Combs, Tashakkori and Teddlie18). The section of text coded included all the text referring to the “other consideration.”
For example, the whole section of text referring to the scarcity in the treatments alternatives available would be coded as “few treatment alternatives.” Codes were then categorized into subcategories depending on the type of information provided, and recorded in a coding manual (Reference Nicod and Kanavos17). These were clustered into two groups: disease characteristics and treatment characteristics. Coding was iterative and flexible to ensure the transferability of codes to other drugs and countries, and additional codes were created with newly identified “other considerations.” Coding was conducted by the lead author. Coding reliability was tested by a colleague expert in Health Policy, who re-categorized each individual code into one of these. Where differences were observed, adjustments were made and documented. The validity of the data collected was established through feedback from external experts, including from HTA bodies (who have been presented most of this work), HTA experts from the Advance-HTA Consortium, and fellow peers when presented at conferences.
Coding was performed vertically and horizontally. In the former, all “other considerations” were coded in a systematic manner as prescribed in the coding manual (Reference Nicod and Kanavos17). In the latter, all “other considerations” were double coded with: (a) if it was put forward as one of the main reasons for the decision, (b) source of the information provided (e.g., experts), and (c) if it was accounted for in the other countries. The data collected qualitatively were then quantitatively analyzed to determine: (a) the type and frequency of “other considerations” accounted for; (b) cases when these were one of the main reasons for the decisions; (c) how they were provided; and (d) how they compared across agencies. The qualitative statistical software NVivo 10 was used for the data collection and analysis (19), and Excel for further data analysis. Data sources consisted in the HTA reports publicly available from each HTA body, complemented with a selected review of the literature and input from key stakeholders (HTA bodies, Advance-HTA Consortium, other peers).
For each sub-category of “other considerations,” we then explored whether they are more likely to pertain to orphan drug and rare disease characteristics, identified from key reports and official documents defining rare diseases and highlighting their common characteristics (20;21).
RESULTS
Value Judgment Classification Framework
When evidence is uncertain or incomplete, scientific value judgments are made about its acceptability. Societal preferences are also accounted for by HTA approaches. These pertain to giving preference to certain (nonquantifiable) aspects of living with a disease or taking a treatment, which are translated into prioritizing certain groups of patients over others, which can be elicited or not (Reference Rawlins and Culyer6;Reference Rawlins7). These preferences are typically elicited by a group of representative citizens (e.g., NICE's Citizens Council) or are enshrined within legislation. Examples of elicited preferences include the “SMC modifiers,” or disease severity in Sweden defined “on the basis of the relevant, initial condition and risk of permanent injury, ultimately death without treatment. All the positive effects the medicine has on people's health and quality of life are accounted for” (22). Nonelicited preferences, referred to as social value judgments, originate from the individual appraisal committee member's value judgment based on their experience or on what they believe society would prefer, and are usually made as part of the deliberative processes of HTA. Both scientific and social value judgments are defined as “other considerations” within the scope of this study (Table 1).
Study Drugs and HTA Recommendation
The study included ten drugs for specific indications. The five inpatient drugs were not appraised by TLV, whom only appraised outpatient drugs at the time of the study. Based on the indicative cost-effectiveness thresholds (Table 1), some treatments with an ICER greater than the respective threshold received a positive recommendation: mannitol dry, azacitidine, lenalidomide, mifamurtide, and trabectedin for NICE; azacitidine, lenalidomide, mifamurtide, and imatinib for SMC; everolimus, mifamurtide, and romiplostim for TLV. In some instances, the ICERs were improved with a Patient Access Scheme that provided a confidential discounted drug price. In France, where coverage is disconnected from the ICER and no threshold exists, only one case was rejected for reimbursement (mifamurtide), three drugs received an ASMR V where no additional benefit was recognized (ofatumumab, mannitol dry, trabectedin), and the remainder were considered to provide additional benefits (Table 2).
ICER and Coverage Decisions (SMR and ASMR in France)
Legend: ✔: acceptable ICER; ★: acceptable ICER accounting for other factors; ✗: high ICER, acceptable only if exceptional circumstances; PAS: Patient Access Scheme; EoL: End-of-Life treatment; severity: disease severity considered high; QALY: quality-adjusted life years; ICER: incremental cost-effectiveness ratio; NICE: National Institute for Health and Care Excellence; SMC: Scottish Medicines Consortium; TLV: Dental and Pharmaceutical Benefits Board; HAS: Haute Autorite de Sante; EU: pricing based on European-price levels; EL: prices equal or lower to existing treatment options; *1 SEK = 0.110202 EUR; ** coverage rate not specific in HTA report, but determined based on SMR rules.
Note. In France, the assessment relies on the Medical Benefit (SMR) and Improvement in Medical Benefit (ASMR), which rank from insufficient benefit (SMR) or no improvement (ASMR V), to major benefit (SMR) or major innovation (ASMR I), respectively. The former drives the coverage rate and the latter the pricing scheme applicable.
“Other Considerations”: An Overview
A total of 125 individual “other considerations” were coded and grouped into ten categories (Figure 1). These were either provided as background information, by experts, or were considered important for the decision. Ninety-four of these 125 codes were included by NICE and used 173 times across all ten cases (e.g., one may have been coded for more than one drug), followed by twenty-four codes used sixty-seven times by HAS, twenty-three codes used fifty times by SMC, and thirty-three codes used fifty-six times by TLV. The most commonly reported disease characteristic related to the nature of the disease, its rarity, and unmet need. The most common treatment characteristics related to its type of benefit, innovative nature, indirect benefit, or the nonsignificance of its adverse events.
Proportion of drugs that accounted for a category of “other considerations,” per cluster.
“Other Considerations” as Pivotal Factors in the Decision Processes
A proportion of these 125 “other considerations” were also put forward as one of the main reasons for their decisions. These represent 18 percent of those put forward by NICE (32 of 173), 24 percent by SMC (12 of 50), 34 percent by TLV (19 of 56), and 100 percent by HAS (67 of 67) (Table 3). For the purpose of HAS, these “other considerations” were mainly discussed in the conclusions of the Transparency Committee when assessing the ISP and have all been considered as main reason for the final recommendation.
“Other Considerations” as Pivotal Factors in the Decision
Legend: end: end-of-life supplementary advice; sev:severe disease; mod: SMC modifiers; dis: lower discount rate; NICE: National Institute for Health and Care Excellence; SMC: Scottish Medicines Consortium; TLV: Dental and Pharmaceutical Benefits Board; HAS: Haute Autorite de Sante.
A proportion of these (e.g., cases with a superscript in Table 3) pertained to those preferences elicited by each HTA body (Table 1), where higher ICERs or uncertain evidence may be accepted. Four drugs were eligible under the NICE end-of-life supplementary advice, three of which were considered cost-effective with an ICER ranging between £34,000 and £47,000/quality-adjusted life-year (QALY) (lenalidomide, azacitidine, trabectedin), and the fourth (everolimus) not cost-effective with £51,700/QALY. Weaknesses in the economic model were deemed acceptable because of the SMC modifiers in four cases (eltrombopag, imatinib, azacitidine, lenalidomide). For HAS, all study drugs were recognized as targeting patients with rare diseases and assessed within the framework of one or more ministerial plans. In Sweden, higher ICERs were accepted due to the severity of the conditions.
Cases without a superscript in Table 3 represent the additional (nonquantified or nonelicited) “other considerations” put forward as one of the main reasons for the decision, relating to the scientific and social value judgments made. For NICE, these included the treatment's unmet need for lenalidomide, mifamurtide, and mannitol dry; its innovativeness for azacitidine and mifamurtide; and the severity of the disease for mannitol dry. Additionally, the impact on families’ and friends’ quality of lives, the rarity of the disease, and the ability to contribute to society and live an active and fulfilling life were also highlighted for mifamurtide.
For SMC, these included the oral administration benefit, the orphan status, and unmet need (e.g., additional treatment option) for eltrombopag, the potential reduction in resource use for romiplostim, and the life-extending nature of the treatment for mannitol dry and azacitidine. Similarly, TLV also valued certain treatment characteristics, such as its oral administration benefit (e.g. eltrombopag), novel mechanism of action (e.g., eltrombopag, romiplostim), orphan status (e.g., eltrombopag), and the impact of the disease on quality of life and daily activities (e.g., eltrombopag, romiplostim, lenalidomide). Unmet need was also recognized (e.g., eltrombopag, romiplostim), and in one case, TLV acknowledged the changing environment in clinical practice (e.g., lenalidomide). For HAS, both disease and treatment characteristics were put forward for all drugs, namely around the nature of the disease, the need for treatment alternatives, and the direct or indirect benefit from taking the treatment. In France, orphan drugs are presumed to be innovative and thus subject to fast-track HTA consideration. In the assessment, the innovativeness of a drug is recognized for those drugs with ASMR I-III.
Stakeholder Input
No mention of stakeholder input was found for TLV given that this is done informally and generally not documented. In contrast, formal channels exist to collect stakeholder input during the technology appraisal processes through the Public Involvement Programme (PIP) at NICE, the Patient and Public Involvement Group (PAPIG) at SMC, and the procedures for rapporteurs at HAS. In the latter case, the Transparency Committee meeting minutes note how many outside experts provided input but not the content of their advice.
“Other considerations” were provided by stakeholders in 116 of the 175 identified in the NICE appraisals. Forty-one percent of these (n = 116) were provided by clinical experts, 21 percent by patient representatives, and 35 percent by both clinical experts and patient representatives. Clinical experts provided information about the nature of the disease affecting the patient (27 percent), issues around current treatment alternatives (13 percent), the treatment's unmet need (11 percent) and innovativeness (10 percent), and the nonsignificance of adverse events (10 percent). Patient representatives provided information about the nature of the disease affecting the patient (33 percent), the nonsignificance of adverse events (14 percent), the indirect benefits from taking the treatment (12 percent) and the patient's unmet need (11 percent). In Scotland, all drugs except trabectedin and imatinib received a Patient Interest Group submission. The detail of these submissions was not publicly available. Clinical input was recorded in the reports for two cases, commenting on the treatment pathways in terms of symptoms (e.g., eltrombopag) and unlicensed comparators (e.g., romiplostim).
Orphan Drugs and Special Status
Table 4 represents the subcategories of “other considerations” identified in the sampled drugs (rows) and whether they pertain to certain characteristics specific, but not limited to, rare disease and orphan drugs (columns). Unmet need is more likely to characterize rare diseases given the scarcity of relevant knowledge and expertise and the fact that often no effective cure exist. This is due to issues around the diagnosis of some of these rare diseases, the complex and unknown nature of these conditions, together with the lack of coordination among centers of expertise at EU and international levels, and the lack of knowledge around best practices (23). Furthermore, given that orphan drugs often do not have effective cure, treatments for rare diseases are more likely to be innovative. On this basis, the “other considerations” that were put forward as one of the main reasons for the final decision identified previously, therefore, influencing the final decision, may favor orphan drugs compared with drugs to treat normal conditions. This was seen, for example, with “unmet need” for lenalidomide, mifamurtide, and mannitol dry by NICE, and for eltrombopag, romiplostim by SMC.
Special Status of Orphan Drugs
DISCUSSION
This study identified the value judgments made for ten orphan drugs in four countries to understand how they influenced the assessment process. The study also identified cases when these “other considerations” were provided by different stakeholders, by type of information provided, and when they related to rare disease characteristics. Implications are discussed here and focus on five topical areas: (a) the classification framework, (b) existing literature, (c) determinants of social value, (d) accountability for reasonableness, and (e) orphan drugs and special status.
One of the significant contributions of this study is the proposed classification framework of the value judgments made during HTA processes (Table 1). It allows to highlight needs for further research (when evidence is incomplete or preferences are nonelicited). If they continue not to be elicited or quantified, retrospectively identifying these to prospectively create a taxonomy of criteria may facilitate their being used more consistently when similar scenarios are encountered in the future. For example, NICE emphasized the impact of osteosarcoma on families’ and friends’ lives when assessing mifamurtide, or SMC and TLV recognized the “oral administration benefit” when assessing eltrombopag. These are nonquantified or nonelicited criteria for which preference could be given in future cases by their inclusion in the taxonomy of criteria to be accounted for. This is all the more important when considering the extent to which these considerations are different across countries and likely also across decision-making bodies within one HTA agency. These differences are either a consequence of agency-specific value preferences (Reference Nicod24), or of committee-specific preferences reflecting the composition of the decision panel and their individual judgments driven by their experiences, and it is, therefore, important to improve the consistency in their use.
The different “other considerations” identified and their classification into sub-categories and clusters are in line with findings from the literature on (social) value judgments. Schwappach (2002) divides the determinants of social value into patient and treatment characteristics (Reference Schwappach25). Our study clustered these determinants in a similar manner and takes one step further by applying this same classification to both social and scientific value judgments. Second, several individual social values were identified in the literature. One study in England used qualitative techniques to define these, where respondents agreed to favor need, preventive care, quality of life, health improvement and life expectancy, in addition to not favoring certain populations according to age or socio-economic status (Reference Baker, Bateman and Donaldson26).
Generally there is agreement about what these social values are, but the determinants of social value remain broadly defined and no exhaustive list of these exists. When comparing these results to the topics defined in several of the EUnetHTA Core Model® domains, commonalities and differences are seen (27). The topics included in the ethical domain relate to societal preferences and the norms or values from using a technology, which corresponds to, in this study, the disease nature affecting patients and their surroundings, and recognized unmet need in terms of how the introduction of a new technology affects the distribution of health care resources. However, the judgment about the magnitude of this unmet need was clearly captured in our results, but does not seem to be explicitly accounted for in the Core Model®.
Topics in the social domain relate to the types of resources required and the experiences, actions and reactions from patients when using the technology, and correspond to the elicited societal preferences (e.g., rare diseases), and to the treatment's direct and indirect benefits, including if the adverse events are manageable (e.g., patients’ experiences). Topics in the organizational domain relate to the consequences on resources or the organizational aspects from using the technology, and would correspond to issues with current treatment alternatives or around clinical practice. The innovative nature of the treatment, identified in our study, is captured within the technical characteristics of the technology in the Core Model®, although no explicit definition is provided (27). The value judgments identified in this study, therefore, correspond to the different domains included in the Core Model®, and further contribute to understanding their determinants in how they are expressed in practice.
Given that the determinants of social value are only broadly outlined, this study contributes to better defining these. For example, “unmet need” is a determinant of social value. It is accounted for in the weighing of disease severity by TLV, as one of the SMC modifiers, and discussed by a NICE Citizen Council meetings. Nevertheless, no clear definition of unmet need exists. Our results captured the variety of ways of expressing “unmet need” (Table 3), which can be used to define it. Another example is disease severity, for which no single definition exists. It is characterized by several determinants, which include the impact on quality of life and mobility, or considerations of life expectancy (Reference Garau, Shah and Towse8;Reference Dolan and Shaw28).
Severity is included into HTA either through a weighing of the QALY (or of other measures of HTA) or as part of the deliberative process (Reference Garau, Shah and Towse8). The latter would apply to our study countries because no specific weighing for severity was seen, including in Sweden where it is explicitly accounted for despite the definition of disease severity being broad (as noted earlier). Our results identified the various forms of expressing severity, which can be used to better define severity for future cases. For TLV, these included: the life-threatening nature of the disease, the negative impact on daily activities including functional capacity and on quality of life, and the short life expectancy from having the disease. In France, where no ICER or threshold exist, informal methods are used to incorporate societal and political values into the assessments. This is explicit in the evaluation of the public health value (intérêt de santé publique) of drugs as part of the coverage evaluation (SMR); however, whether these determinants of (social) value are accounted consistently across cases is another question, which could be partly addressed by applying the taxonomy of criteria.
For a resource allocation decision to be accountable for reasonableness, the process should be transparent and public, based on reasons that are relevant, decisions should be revisable when new evidence is available, and the process should allow for these conditions to be enforced (Reference Daniels and Sabin9;Reference Daniels29). This usually takes place during the deliberative process of HTA, during which the Committee discusses the evidence and accounts for stakeholder opinion until a decision is made. The decision and reasons for the decision should then be documented in the HTA report, most often publicly available, as is the case with our study countries. In terms of stakeholder input, a clear process exists at NICE and SMC where they are given the opportunity to voice their concerns or opinions. Our analysis confirmed that this is well-reported for NICE (given the high number of “other considerations” provided by different experts), but is not as detailed in SMC's summary of advice, probably because it is a less detailed report.
HAS specific procedures governing outside experts (rapporteurs) who provide advice and input in the evaluation process. For TLV, no official procedures exist, although some of the key stakeholders are represented within the Appraisal Committees (e.g., clinical experts). Generally, the type of input from these stakeholders could be better documented or transparent. Some argue that it is not sufficient to have a formal procedure to account for stakeholder input and value judgments, but that it should also be clear how these have influenced the decision, which is often lacking (Reference Garau, Shah and Towse8). Our results further confirm this in the number of “other considerations” (from stakeholders or not) identified, where it is not entirely clear how these factors contributed to the decisions particularly in those cases where these were (nonelicited or nonquantified) value judgments. The taxonomy of criteria developed together with the type of input from different stakeholders may help understand the criteria that are relevant to decision making and their sources that go beyond routine methods of assessing clinical benefit and ICERs.
Little agreement exists on whether patients with rare diseases requiring orphan drug treatments deserve a preferential status (13–Reference McCabe, Stafinski and Menon15). Nevertheless, governments recognize the difficulties in appraising these treatments and the fact that they should be treated differently. In France, patients with orphan diseases have a preferential status, but their needs go beyond drugs. Only recently, NICE and SMC have implemented new procedures for end-of-life and ultra-orphan drugs. The treatment's additional benefit and other elements not captured by the ICER (e.g., unmet need, disease severity, added value to the patient and their surroundings) are now accounted for by SMC, together with patient and clinical engagement. These other elements correspond to the “other considerations” identified in this study.
Similar questions are arising in Sweden, where a consultation on how to appraise orphan drugs has recently been issued. Furthermore, in NICE's recent consultation on value-based pricing, they attempted to find novel approaches to capturing burden of illness and other issues. They concluded that approaches to adjusting the QALY were insufficient, whereby explicitly accounting for these additional criteria is essential for decision-making. This study provides an alternative to the issue of preferential status by accounting for the nonelicited or nonquantified “other considerations” that influenced previous decisions, and query whether it would be worth eliciting preferences for these. This could then feed into novel approaches in assessing orphan drugs (e.g., MCDA).
CONCLUSIONS
This study systematically identified the scientific and social value judgments made in four countries for a sample of orphan drugs, and explored how they influenced the deliberative process of HTA. The proposed classification framework of these value judgments was used to identify needs for further research and to improve consistency in their use across cases. This was then used to address different issues around identifying and better defining the determinants of social value or how to improve the lack of accountability for reasonableness particularly in cases when it was not clear how the “other considerations” identified influenced the decisions.
It also provided a way forward to eliciting whether these orphan drugs deserve a special status by eliciting preferences around some of the social value judgments made which are more likely to pertain to orphan compared with nonorphan drugs, rather than focusing on the opportunity cost of these. Given the challenges in producing robust evidence for orphan drugs due to the small patient numbers and heterogeneity of the diseases, scientific and social value judgments are unavoidably part of the decision processes for these drugs. Their identification through the application of this framework enables us to create a taxonomy of criteria relevant to these decision-making processes, which go beyond routine methods for HTA.
SUPPLEMENTARY MATERIAL
Supplementary Figure 1: http://dx.doi.org/10.1017/S0266462316000416
CONFLICTS OF INTEREST
Dr. Nicod reports grants from European Commission, 7th Framework Programme during the conduct of the study, and personal consulting fees outside the submitted work. Dr. Kanavos reports grants from European Commission, 7th Framework Programme during the conduct of the study.
