The National Health Systems (NHSs) among European countries and UK are diverse, as well as their evaluation systems on drug innovations to be reimbursed (Reference Angelis, Lange and Kanavos1;2). These countries have distinct approaches to evaluate new treatments, and the criteria assessed might have different relevance in the price and reimbursement (P&R) decision-making process (Reference Angelis, Lange and Kanavos1;2). Commonly, criteria used to determine the P&R of new drugs are based on efficacy, safety, and cost, although some countries include additional criteria. For instance, in the UK (NICE), main criterion for drug reimbursement is based on the cost per quality-adjusted life years (QALYs) (3). Currently, the threshold for reimbursing new drugs in the UK ranges between £20,000 and £30,000 per QALY but with important exceptions, for example, for the HST process (Reference Angelis, Lange and Kanavos1). In the French Health System (HAS), relevant criteria are based on the ASMR classification (in English, ASMR stands for improvement of medical benefit), which can be classified in five different ASMR levels according to how the drug improves patients’ clinical situation, being ASMR I a major therapeutic progress, and ASMR V lack of clinical improvement (4). ASMR classification from I to IV indicates the possibility of a higher price compared to the available alternatives (4). In Germany, evaluation of a new drug is based in early benefit assessment (by GB-A/IQWiG), which will determine the price. If no additional benefit is provided, the price will be the same or lower than its comparators. Additional benefit assessment takes into consideration the improvement of patients’ quality of life (Reference Ruof, Schwartz, Schulenburg and Dintsios5;6). In Italy (AIFA), a new P&R system has been recently established that assesses the additional clinical benefit compared to the standard of care (like in France or Germany). Benefit is ranked in five different categories (from Maximum to Absent), and if no additional therapeutic benefit is provided by the new drug, the price will be the same or lower than its comparators (7–9).
Even though the criteria vary across countries, overall they assess similar aspects: unmet needs, severity of the disease, therapeutic and safety impact, innovation degree, and socioeconomic impact among others (Reference Angelis, Lange and Kanavos1).
In Spain, unlike other countries, there is no clear delimitation of the reimbursement and pricing decisions (Reference Epstein and Espín10). The P&R decision by the Spanish NHS is based on a set of criteria listed in the Royal Legislative Decree 1/2015 (RDL 1/2015) of the law on guarantees and rational use of medicines and health products (11). Once a new medicine is approved by the European Medicines Agency, the marketing of this medicine in Spain is authorized by the Spanish Agency of Medicines and Medical Devices (AEMPS) (12). Then, a Therapeutic Positioning Report (TPR) is issued by REvalMed network (13). The REvalMed network was set up in 2019 and is formed by three main bodies: the AEMPS, responsible for therapeutic evaluation; the General Directorate for Common Portfolio of the NHS and Pharmacy Services (DGCCSF), responsible for the economic evaluation; and the expert reviewers appointed by the autonomous communities, divided into seven therapeutic nodes (13). The TPR informs of the added therapeutic value of the new drug compared to the current treatments to provide information for the P&R decisions. Finally, the Interministerial Committee on Pricing of Medicines and Healthcare Products (CIPM) is responsible for the final P&R resolution, establishing the maximum price of the medicine in Spain (14). Currently, the CIPM specifies the criteria of the RDL 1/2015 by which the reimbursement of a drug is approved or denied, but the information provided is neither defined nor explicative, making difficult to comprehend the decision relative to the value of a new drug for the Spanish NHS.
This study aims to characterize the criteria listed in the RDL 1/2015 based on the definitions of P&R criteria used in other countries. Afterwards, a feasibility study is performed to evaluate a new drug using the characterized RDL 1/2015 criteria including a reflective discussion of the results.
Methods
Study Design
The study was planned in two phases (Supplementary Figure 1). In the phase one, the objective was the characterization of the criteria listed in the RDL 1/2015 of the Law on Guarantees and Rational Use of Medicines and Healthcare Products (11) (Table 1) through a literature review to identify similar P&R criteria used in other European countries, UK and Canada Health Technology Assessment (HTA) Agencies. For each of the P&R criteria identified in the literature review, their subcriteria and definitions were included as potential subcriteria of the RDL 1/2015 criteria. Then, a multidisciplinary expert panel agreed on the relevance/suitability of the subcriteria and their definitions for each of the RD1/2015 criteria using a qualitative consensus approach. Next, the expert panel ranked the characterized criteria according to its relative relevance in the P&R decision-making process. In the phase two, a drug currently under evaluation by the new REvalMed network was used as a pilot study to assess the feasibility of the characterized criteria.
Reimbursement Criteria Listed on the RDL 1/2015 of the Law on Guarantees and Rational Use of Medicines and Healthcare Products
Phase I
Literature Review
A literature review was carried out for each of the criteria listed in the RDL 1/2015 to ascertain if and how other countries define and characterize such criteria within their P&R system. The inclusion criteria for literature review were country legislation, documents from the most relevant HTA agencies, methodological guidelines, or articles published from January 2010 to November 2020 that included the P&R criteria or the evaluation methodology in Germany, Canada, Spain, France, Italy, and the UK. Included documents were in Spanish, English, French, Italian, or German language. Outdated legislation, documentation, methodological guidelines, and articles were excluded.
Expert Panel
The study required a multidisciplinary group of experts with broad practical experience in the public sector at national, regional, and local (hospital) level for the evaluation and decision-making process of new medicines in Spain. The multidisciplinary panel of experts was formed by five Head of Hospital Pharmacy (including Spanish Society of Hospital Pharmacy (SEFH) former Director and current Balearic Autonomous Delegate), the Head of the Department of Medicines for Human Use of the AEMPs, the former General Director and former Subdirector of Pharmacy of Spanish Ministry of Health and the former General Director of Economic-Financial Management of the Madrid Health Service (SERMAS), some of them with more than 20 years of experience in the public sector in drug evaluation and decision making. M.A.C. acted as a group coordinator and X.B. provided technical support (methodology development, discussion moderator, and matrix fulfillment).
Characterization of Criteria Listed in the RDL 1/2015
A list of subcriteria and its definitions identified in the literature review was proposed to the expert panel for each of the criteria listed in the RDL 1/2015. Experts evaluated each of the subcriteria using a three-point scale (0—subcriteria does not apply to the RDL 1/2015 criteria and should not be included; 1—subcriteria does define the RDL 1/2015 criteria, but a modification or adaptation is needed to be included; 2—subcriteria does clearly define the RDL 1/2015 criteria and should be included). The experts could propose additional subcriteria not identified in the literature review that they considered relevant. Results were then discussed and agreed on an online session.
Ranking the Criteria Listed in the RDL 1/2015
Experts ranked the characterized RDL 1/2015 criteria according to their relative relevance in the P&R decision process using a 5-point scale (one for lowest relative importance; five for highest relative importance.
Phase II
Feasibility Study
In the second phase of the study, the experts selected from the current drugs being evaluated for the NHS, a novel antibiotic (cefiderocol) for the treatment of adults with infections caused by multidrug-resistant gram-negative bacteria with limited treatment options.
Fulfillment of the Evidence Matrix and Expert Scoring
An evidence matrix was developed using the characterized P&R criteria listed in the RDL 1/2015 and was populated with available information of the drug under evaluation. A five-point scale was used to evaluate the contribution of a new medicine in each criterion, where one represents the lowest score and five the maximum. For example, in the criterion A—“Severity, duration, and after-effects of the different pathologies for which they are indicated”: one point means lack of severity, duration, and after-effects while five points mean maximum severity, duration, and after-effects of the different pathologies for which they are indicated.
The fulfilled matrix was then sent to the experts, which scored individually each of the criteria including their rational behind the score.
Presentation and Reflective Discussion of the Results
The results were presented and discussed online by the expert panel. During the discussion session, the experts also analyzed the adequate content of the information of the drug under evaluation according to the criteria and subcriteria definitions, the need to modify or redefine any of the subcriteria included, and the necessity to modify or redefine the response scale used.
Data Analysis
Phase I—Experts’ scoring was collected individually and transferred to a database in the Microsoft Excel software. To validate the subcriteria and its definition, the percentage of experts who considered the subcriteria valid (1 or 2) and those who considered the subcriteria invalid (option 0) was calculated. If more than 55 percent of the experts scored a subcriteria with the option 0—“subcriteria does not apply,” then this subcriteria was excluded for further discussion. Phase II—For the feasibility study, scoring results were analyzed by calculating the mean, standard deviation, and minimum and maximum value. A final score for drug reimbursement by the Spanish NHS was calculated and presented using two different approximations: 1—sum of the mean score obtained for each criterion, 2—the sum of the value of each criterion (VC) obtained after standardization (S) and weighting (W) using the following formula 
$ {\sum}_0^i({\mathrm{VC}}_i)\hskip0.35em =\hskip0.35em {S}_i\times {W}_i $. The weight for each criterion was the scores obtained in the ranking assessment.
Results
Literature Review
Out of eighty-six documents identified in the literature review, sixty-seven were included. These documents came from the most relevant and recognized HTA agencies (members of the EUnetHTA) (15) at the European level. The HTA agencies and the national competent authorities identified in the review were French National Authority for Health (HAS) and National Agency for the Safety of Medicines and Health Products (ANSM) in France, Institute for Quality and Efficiency in Health Care (IQWIG) in Germany, Unità di Valutazione Technology Assessment (UVTA/AOP) and Italian Medicines Agency (AIFA) in Italy, National Institute for Health and Care Excellence (NICE) and Medicines & Healthcare products Regulatory Agency (MHRA) in the UK, Canadian Agency for Drugs and Technologies in Health (CADTH), and Spanish Agency of Medicines and Medical Devices (AEMPS), Health Technology Assessment Agency of Carlos III Institute (AETS-ISCIII), Andalusian Health Technology Assessment Agency (AETSA), AQuAS and AVALIA-T in Spain. Other information sources were documents from international and national research organizations and societies in Health Economics such as the Organization for Economic Co-operation and Development (OECD), European Network for Health Technology Assessment (EUnetHTA), International Network of Agencies for Health Technology Assessment (INAHTA), and the International Society for Pharmacoeconomics and Outcomes Research (ISPOR). The PRISMA flow diagram of literature review results is shown in Figure 1.
PRISMA diagram of the literature review results to identify the price and reimbursement criteria and methodology of drug evaluation used in Canada, France, Germany, Italy, UK, and Spain.
Criteria listed in the RDL 1/2015 and their subcriteria identified in the literature review results are described in Table 2. The results of the literature review showed that in European countries and in UK, the evaluation of the therapeutic benefit of a medicine is assessed before the economic evaluation and price negotiation (16–18). Based on that, the panel of experts decided to split the criterion C into two criteria: C1 “Therapeutic value of the drug” and C2 “Social value of the drug and its incremental clinical benefit, considering its cost-effectiveness ratio.”
Results of the Expert Panel Scores of the Potential Subcriteria Identified in Literature Review for Each of the P&R Criteria Listed on the RDL 1/2015 using a 3-Point Scale (from 0 to 2)
Characterization and Discussion of Criteria and Subcriteria Definitions
The subcriteria and its definitions to be included within each criterion listed in the RDL 1/2015 were scored by the expert panel using three-point scale (Table 2). After discussion, thirty-two out of forty-two initial subcriteria were finally included, some of them with adaptations (Table 3).
Final Consensus on the Characterization of the P&R Criteria Listed in the RDL 1/2015. For Each Criterion, the Agreed Subcriteria and its Definitions Are Shown
The subcriteria that were redefined by the expert panel were “Impact of the disease on the quality of life of family members/caregivers” as “Impact on the quality of life of family members/caregivers”; “Unmet needs in patient reported outcomes” as “Unmet needs in patient’s quality of life”; “Place occupied in therapy” as “Therapeutic positioning”; and “Safety and tolerability” as “Incremental safety and tolerability”; and the additional subcriteria proposed by expert panel was “Technological innovation of the drug” within “Degree of innovation of the drug” criteria. In addition, the descriptions proposed for each subcriteria were also adapted by the expert panel. The final proposal of RDL 1/2015 characterized criteria, subcriteria, and their definitions is shown in Table 3. Expert panel agreed to use a five-point scale to evaluate the contribution of a new medicine in each criterion (one = minimum value, five = maximum value).
Criteria Ranking
The expert panel was invited to perform a ranking of the RDL 1/2015 criteria according to their relevance in the P&R process using a five-point scale. Mean results showed that most important criteria for P&R of medicine were (mean ± SD): “C.1. Therapeutic value of the drug” (4.3 points ± 0.49) and “A. Severity, duration, and after-effects of the different pathologies for which they are indicated” (4.3 points ± 1.04), followed by “D. Rationalization of public spending for pharmaceutical services and budget impact on the National Health System” (4.1 points ± 0.83). Criteria ranked with medium importance were “B. Specific needs of certain groups” (3.9 points ± 1.21), followed by “E. Existence of drugs or other therapeutic alternatives for the same conditions at a lower price or lower cost of treatment” (3.8 points ± 1.28) and “C.2. Social value of the drug and its incremental clinical benefit, considering its cost-effectiveness ratio” (3.8 points ± 0.89). The least important criterion was “F. Degree of innovation of the drug” (3.0 points ± 1.31).
Feasibility Study
A feasibility study using cefiderocol was performed to validate the characterization of the criteria listed in the RDL 1/2015 used in P&R decision-making process of new medicines in the Spanish NHS, obtained in study’s phase one. Scores were mainly used to define the benchmark and drive the discussion as in multicriteria decision analysis methodology (Reference Goetghebeur, Wagner and Khoury19). The score for drug reimbursement by the Spanish NHS obtained by cefiderocol was 24.6 out of 35 (obtained by summing individual criteria scores), with all individual criteria reaching an average score of three or above out of five (Figure 2).
Scores for each of the characterized criteria listed in the RDL 1/2015 in the feasibility study of cefiderocol performed by the expert panel. Scores go from one to five, where one represents the lowest contribution to the criterion and five the maximum.
When standardizing and weighting the scores according to their relevance in the P&R process, the score for drug reimbursement by the Spanish NHS of cefiderocol was 0.7 out of 1 (min: 0.2; mean: 0.6; max: 1), a score that was considered moderate to high.
The highest score and consensus among experts were observed in criterion A. “Severity, duration, and after-effects of the different pathologies for which they are indicated” (4.2 ± 0.4), followed by criterion B (4.1 ± 0.8), reflecting the high unmet need for multidrug-resistant bacterial infection. The highest dispersion was observed in the economic criterion D. “Rationalization of public spending for pharmaceutical services and budget impact on the National Health System” (3.3 ± 1.2), reflecting in part the uncertainty about the reimbursed price and budget impact (BI) for the NHS in Spain. In addition, a similar consensus among experts was observed in other criteria: in criterion C.1. “Therapeutic value of the drug” (3.6 ± 0.9), criterion C.2 “social value of the drug and its incremental clinical benefit, considering its cost-effectiveness ratio” (3.0 ± 0.9), criterion E. “Existence of drugs or other therapeutic alternatives for the same conditions at a lower price or lower cost of treatment” (3.2 ± 0.8), and criterion F. “degree of innovation of the drug” (3.1 ± 0.9).
During the assessment of the pilot study, experts analyzed and discussed about the need to modify or redefine any of the subcriteria, and the requirement to modify the response scale used for each criterion.
First, the experts agreed to develop a protocol to guide the evaluators when scoring each of the criteria, mainly to address the following two situations: framing scoring within the context of P&R; and how to assess a criteria when information is not provided for all subcriteria. For instance, in criteria F. “Degree of innovation of the drug” experts discussed on how the score should be approached when the drug provides innovation only in one or some of subcriteria, and how to define the relative weight for each of them.
Moreover, for criterion “B. Specific needs of certain groups” experts agreed to modify the definition of subcriteria to adapt it better to the criteria as follows: “Specific needs of certain groups in relation to the efficacy,” “Specific needs of certain groups in relation to safety,” and “Specific needs of certain groups in relation to the quality of life of patients.”
For criterion D. “Rationalization of public spending for pharmaceutical services and budget impact on the National Health System,” experts considered important to narrow the criteria in order to reduce the degree of freedom of the decision maker, as this criterion can be scored from many perspectives. Also, it is necessary to clearly define what is considered “rationalization of pharmaceutical spending.” Some ideas were proposed by participants in this regard, such as the inclusion of a subcriterion performing a quantitative analysis to determine the rationalization of the budget impact (BI) of the evaluated drug.
Regarding the drug contribution scoring for each criterion, experts agreed to modify the response scale for economic criteria including negative values (from −5 to 5). The reason was to extend the threshold between reimbursed and nonreimbursed drugs final scores, by reducing the bias usually provided by the high scores obtained in the criteria evaluating the disease, unmet needs, and therapeutic benefit.
In addition, experts considered that the final score should be contextualized. For instance, a score of 0.7 with available therapeutic alternatives could be different than a score of 0.5 being the only therapeutic option.
Discussion
This study aims to perform a feasibility study of the characterized criteria listed in the Royal Legislative Decree 1/2015 of the Law on guarantees and rational use of medicines and health products for the inclusion of new drugs in the Spanish NHS, making progress in understanding the P&R evaluation of medicines in Spain.
The study was conceived due to the lack of explanation of the current CIPM resolutions, which only indicates the criteria by which the reimbursement of a drug is approved or denied but does not include the information behind its reasoning and final decision which, to our believe, would be indispensable for a more transparent P&R decision-making process.
Potential subcriteria and their definitions, based on criteria definitions used in other countries identified in literature review, were presented to an expert panel. After scoring and discussion, the expert panel achieved consensus in all the criteria, subcriteria, and their definitions, considering it a potential tool for an objective P&R evaluation in Spain. Criteria A and B evaluate the disease and the unmet needs regarding its current treatment, while criterion C.1 refers to drug’s contribution to the first two. Criteria C.2, D, and E evaluate the new drug from different economic perspectives. Criterion C.2 evaluates the drug efficiency, and Criterion D considers the drug’s affordability by the NHS. Criterion E complements the previous ones by comparing the costs to other therapeutic alternatives.
Based on ranking results made by the expert panel, criterion C.1 “Therapeutic value of the drug” was considered to be the most important criteria, followed by the criteria A and D, that evaluate the disease severity and the BI of the new drug for the NHS, respectively. Interestingly, these results might be in line with the current P&R decision-making process in countries like France, Germany, or Italy where the P&R process is linked to the therapeutic benefit of the drug, which is assessed in first place and that will determine the price of the drug in comparison to alternative options (4–7). In Spain, the new TPR is developed prior to the P&R decision-making process and includes the economic assessment based on the therapeutic benefit of the drug. Nevertheless, the structure and criteria included in the TPR do not currently follow the criteria established by the law.
In the phase two of the study, an evidence matrix was developed using the characterized criteria listed in the RDL 1/2015 and populated with available information of cefiderocol, a new antibiotic currently in P&R process. Noteworthy, the assessment of new antibiotics is a topic of discussion in Europe—for instance the emergence of specific evaluation pathways for antibiotics in Germany (20) or new financial agreements in UK (21)—nevertheless, in Spain there is no specific pathway for antibiotics assessment and the same evaluation process is applied to all kinds of medicines. A reflective discussion based on the scoring results and matrix content was carried out by the expert panel to assess the feasibility of the study, as well to identify necessary adaptations to the criteria, subcriteria, and their definitions or the scale used to evaluate each criterion. It is important to note that scores were mainly used to define the benchmark and drive the discussion (Reference Goetghebeur, Wagner and Khoury19).
Cefiderocol obtained moderate–high scores in criteria A and B and C.1, reflecting the high unmet medical need in multidrug-resistant gram-negative infections (particularly those that are resistant to carbapenems) and the additional benefits in terms of efficacy and safety of this drug in infections caused by carbapenem-resistant gram-negative pathogens, for which there are limited safe and effective treatment options. Some of the experts mentioned that the therapeutic benefit (criterion C.1) of the new drug was difficult to measure without efficacy data of the comparators, explaining at least in part, the dispersion observed in these criteria. High dispersion was observed also for the economic perspective criteria, particularly for criterion D. In this regard, experts considered necessary to define more precisely the term “rationalization of pharmaceutical spending” and proposed to include quantitative analysis for the BI of the drug evaluated. Several options were considered, for instance by calculating the impact of the drug in the NHS budget, by calculating the percentage of the budget that would represent the inclusion of the drug within its therapeutic area (by ATC group), or by defining a BI threshold per therapeutic area. Interestingly, countries such as UK or France have already defined additional steps (price negotiations or economic evaluations) when medicines exceed established BI thresholds (3;Reference Epstein and Espín10;22). It is important to note that contrary to what happens in Spain, in those countries the reimbursement and pricing are two different processes (Reference Epstein and Espín10). In addition, experts also considered that scores given by the evaluator in each criterion must be driven from a P&R point of view (not a clinical point of view), and that economic criteria should be more decisive in the P&R final score.
In summary, information of cefiderocol included in the evidence matrix was considered appropriate and adequate, and the overall score achieved by cefiderocol in the P&R process to be moderate to high. The main changes or adaptations proposed by the experts were developing a protocol to guide the evaluator during the scoring process; the modification of the response scale for economic criteria by including negative values; and the need to carry out a study to operationally define “rationalization of pharmaceutical spending”.
We acknowledge some limitations of this study. First, the nature of the study, designed as a feasibility study, therefore scores and results are preliminary needing further validation studies. Second, expert panel was formed by a small number of participants (n = 10). Nevertheless, the panel was multidisciplinary, and all members had relevant experience in the evaluation of medicines in Spain at national, regional, and local level. Furthermore, the number of experts is approximately the same as the number of experts who form the CIPM in Spain (13). Eventually, the information available of the evaluated drug to fulfill the P&R matrix was reflected in the discussion of the feasibility study.
The present study explores the characterization of the criteria listed in the RDL 1/2015 for the inclusion of new drugs in the Spanish NHS. It is important to note that the subject of this study is under discussion in Spain and is included in the strategic working lines of the Advisory Committee on Pharmaceutical Provision (CAPF) and the Spanish NHS (23).
Experts considered criteria and subcriteria adequate for the P&R decision-making process in Spain, indicating its possible applicability, although additional adjustments were proposed. Further studies to refine the current results and validate it as a potential tool for P&R decision making in Spain are needed. Furthermore, a retrospective evaluation of reimbursed and nonreimbursed drugs would be necessary to define P&R scores meaning according to each drug context.
We believe that the present study could be used as a framework for the development of standardized, objective, and holistic evaluations of new medicines, if adapted to each country legislation. The weigh-in of each of the criterion on the final P&R decision could be easily adjusted according to each country criteria prioritization. As well, considering a multidisciplinary evaluation panel would provide a valuable insight from different stakeholder’s point of view.
To our knowledge, this study represents the first step toward a more objective, transparent, and structured evaluation for the drugs to be included in the Spanish NHS.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/S0266462322000332.
Conflicts of Interest
M.A.C. received consulting fees from Shionogi S.L.U. X.B. received funding from Shionogi S.L.U to develop the study.
Funding Statement
This study has been funded by Shionogi S.L.U.
Open access
