In 2012, the World Health Organization Regional Office for Europe published the Health 2020 program (Reference Rys1), a European political framework for governmental and social action in health promotion and protection. One of the specific objectives defined in this program was the empowerment of both citizens and patients. Patient empowerment could improve health outcomes, increase patient satisfaction, and improve communication between professionals and patients. In addition, the use of resources and costs of healthcare services would be optimized (Reference Rys1).
The increasingly used concept of “patient empowerment” was defined as a continuous process through which patients (and patient associations) work in partnership with the healthcare system to enable patients to become more responsible for (and involved in) their treatment and healthcare. This process requires the development and implementation of policies, strategies and healthcare services that increase patients’ awareness and promote a more balanced partnership in clinical decision making (Reference Loukanova, Molnar and Bridges2;3). The concept of patient empowerment has also gained in relevance in relation to cancer patients, specifically in patient involvement in treatment decision, adherence and management of symptoms, among others (Reference Aslani4–Reference Te Boveldt, Vernooij-Dassen and Leppink6).
The assessment of new cancer therapies is a great challenge to health authorities. This is due to the emergence of many new cancer therapies and diagnostic methods that may suggest a small increase in patient survival and the high costs that they have to National Health Systems affected by fund restrictions. A retrospective cohort study has shown that most of new oncology drugs authorized by the European Medicines Agency (EMA) from 2009 to 2013 came to the market without straightforward evidence that they improved survival in cancer and only thirty five percent of the new treatments showed significant prolongation of survival (Reference Davis, Naci and Gurpinar7). The number of approved cancer therapies is rising quickly, and this fact is reflected in the number of medicines recommended for approval by EMA: seventeen (21 percent) of eighty-one drugs recommended in 2016 were targeted to cancer treatment and five of them were evaluated through fast track (8). The emergence of a great number of oncology therapies, such as immuno-oncology therapies, and the need to provide fast access to appropriate patients, will be increasingly challenging in our resource-constrained healthcare systems (Reference Aronson, Ferner and Hughes9).
In the cancer field, the importance of involving patients in management and decision making in drug development and assessment process is increasing. The European Commission agreed a public-private partnership with the European Federation of Pharmaceutical Industries and Associations (EFPIA) called Innovative Medicines Initiative (IMI1 [2008–13] and IMI2 [2014–20]). This initiative is aimed at facilitating open collaboration in research to advance the development of, and accelerate patient access to, personalized medicines for the health and wellbeing of all, especially in areas of unmet medical need. Input from patients and patient groups is essential to making this process successful, therefore, IMIs actively encourage patients to participate as members of its projects (10).
To involve patients in making decisions about oncology drug treatments, relevant institutions such as American Society of Clinical Oncology (ASCO) and European Society of Medical Oncology (ESMO) have developed frameworks for assessing the value of new cancer treatment options. The frameworks will serve as the basis for standardized tools that physicians can use with their patients to discuss the relative value of new cancer therapies compared with established treatments (11;Reference Cherny, Dafni and Bogaerts12). The development of these frameworks has considered the perspective of all stakeholders, including representatives of patient associations. Recently, the National Health Council's Rubric has also developed a tool that includes criteria for evaluation of value frameworks with respect to patient-centeredness and meaningful patient engagement (Reference Schnipper, Davidson and Wollins13).
To our knowledge, no standardized approached has been explored to involve patients and patient associations to assess the value of new drugs. MCDA is a method that allows for considering a comprehensive set of criteria that are relevant for establishing the value of healthcare interventions in different contexts, under a systematic and transparent process and incorporating a wide range of stakeholder views. In addition, MCDA has been proposed as a useful tool to elicit individual values and facilitate sharing of diverse perspectives (14).
An MCDA framework to assess oncologic drugs from the patient perspective has two main purposes: (i) to identify the most valued criteria from patient perspective when patients or patient associations position themselves on the value of a new oncologic treatment and (ii) to identify criteria in which patients need additional training to be able to provide systematic, structured and evidence-based input to health decision makers and committees, with their evaluation of a new oncologic drug in different evaluation committees.
The aim of the current study was to involve patients’ representatives to obtain a patient perspective in drug assessment by understanding which criteria would be important for them and how they would use these criteria in the assessment of drug value.
Methods
Study Design
The study focused on developing a pilot MCDA framework to assess drugs from the patient perspective. We define an MCDA framework as a set of criteria structured in domains that are weighted and scored with the purpose of assessing new drugs. The first phase of the study consisted of a literature review to identify and define the MCDA framework criteria. The second part of the study consisted of the evaluation of these criteria and the assessment of their relative importance by a panel of representatives of patient associations. The MCDA framework was tested through the assessment of a hypothetical oncologic treatment. The framework development and validation was carried out using the Evidence and Value: Impact on Decision Making (EVIDEM) methodology (Reference Goetghebeur, Castro-Jaramillo, Baltussen and Daniels15;16), which uses a mixed quantitative and qualitative approach, structuring criteria in a set of quantitative criteria, and qualitative criteria. Quantitative criteria are weighted when using the MCDA framework to elicit the individual preferences. After the weighting of criteria, the reflective MCDA methodology analysis encourages the user to reflect on the evidence and make a judgment on its meaning using an interpretive scoring scale and also to provide a narrative to explain the reasoning that underlies the score. Qualitative criteria are criteria that are not suitable for scoring (e.g., cultural and historical context) but nonetheless are an integral part of the reasoning, the framework provides a simple qualitative assessment tool to consider the impact of these criteria (positive, neutral, or negative) on the value of interventions.
Identification and Definition of Criteria
A literature review (LR) was carried out. The aim of this review was to identify the available publications on the assessment of the value of oncologic treatments from the patient perspective in the field of healthcare decision making, to extract the criteria used in these evaluations and their definitions. Four questions were formulated to be answered by the LR: (i) What is the involvement/participation of patients in decision making related to the incorporation new cancer treatments in Europe?; (ii) Which criteria are relevant for patients in relation to the innovation and incorporation of new cancer treatments?; (iii) Which criteria are relevant for evaluation and decision making related to oncology innovation in Europe?; (iv) Which criteria from MCDA frameworks are relevant to patients?
PICOT (population, intervention, comparison, outcome, and time) methodology (Reference Riva, Malik, Burnie, Endicott and Busse17) was used to define inclusion and exclusion criteria for each research question. These four questions were used to disaggregate the components of the question and to define the inclusion criteria. The search was performed using keywords combined with Boolean operators (“AND” and “OR”) in MEDLINE & MEDLINE In-Process databases. Inclusion criteria for the LR were: the publication had to answer the specific question, the publication had to be published between January 2010 and March 2017, and it had to be published in English or Spanish.
The LR was complemented with reports searched on the web site of the European Medicines Agency (EMA) and the Spanish Agency for Medicines and Medical Devices, as well as by searches in gray literature sources such as Google and MEDES. EVIDEM MCDA framework criteria (Reference Goetghebeur, Castro-Jaramillo, Baltussen and Daniels15) were also included, because this MCDA framework covers criteria used in healthcare decision making. Some of these criteria are not frequently used by patients and, therefore, may not appear in the literature review performed, so it was decided to incorporate them to create a broad set of criteria that reflects the patient perspective, that could be taken into account by decision makers in the assessment of a new drug.
Selection of publications for data extraction was carried out by two independent reviewers. It was based on the titles and abstracts according to the established inclusion criteria and the formulated questions. The full texts of the selected publications was reviewed to extract the relevant criteria from the patient's perspective. After identifying the criteria, it was evaluated whether the publication recommended the use of each criterion for the drug evaluation. Criteria were included if more than 50 percent of publications recommended their use, or excluded if more than 50 percent of publications did not report on a criterion, or if a rationale was given for its noninclusion. In case a criterion was recommended for its use in some publications and other publications recommended its noninclusion, the criterion was included to be validated by the expert panel later on.
The identified criteria from the literature, criteria from the EVIDEM framework and their definitions were presented as a structured pilot MCDA framework (Reference Goetghebeur, Castro-Jaramillo, Baltussen and Daniels15), in which criteria were classified into normative or contextual criteria and grouped within domains.
Criteria Assessment
The criteria were assessed by a sample of representatives from different cancer patient associations in a first face-to-face workshop. Representatives of general patient associations, general cancer patient associations and patient associations of specific types of cancer were invited to provide the widest possible scope. We define a patient representative as a member of a patient association that acts on behalf of the interest of all patients of a given disease, but not representing patients as a scientific sample, rather than experts that bring their personal opinions from the view of general patient's interests. First, the panel was trained on MCDA methodology and then validated the criteria. To consider the inclusion, exclusion or adaptation of the criteria in the MCDA framework, the panel were asked, for each criterion, whether the criteria was suitable for assessing the value of an oncologic treatment from the patient perspective. Patients could modify the criteria definitions, eliminate, or add new criteria. The criterion was considered excluded if more than 50 percent of respondents answered “no,” included if more than 50 percent answered “yes,” and adapted if there was a tie or other combinations of answers.
Criteria Weighting
Following the EVIDEM methodology (Reference Goetghebeur, Castro-Jaramillo, Baltussen and Daniels15;16), the weighting of quantitative criteria included in the pilot MCDA framework was done using a 5-point weighting technique. Each participant assigned a relative weight per criterion using a nonhierarchical simple 5-point scale (1 = lowest relative importance, 5 = highest relative importance).
Matrix Development and Scoring
To test the MCDA framework criteria, an evidence matrix, which included available evidence for each criterion of the framework, was developed to assess the value contribution of a hypothetical oncologic drug to each criterion of the framework and its total value contribution. A literature review was carried out to find the required information on a treatment for non-small cell lung cancer (NSCLC) to complete the matrix. The information was collected, summarized and presented as a comparison between hypothetical treatments A and B. The value contribution of treatment A versus treatment B for each criterion was scored by the patient representatives in a second workshop and the results were analyzed.
Value Framework Agreement
A discussion based on the reflections behind the assigned scores by the patient representatives to each criterion in the evaluation of a hypothetical NSCLC drug was carried out to assess the suitability of the criteria included in the matrix. The same rules for criteria inclusion, exclusion, or adaptation used for the criteria assessment during the first workshop were followed. An agreed final pilot MCDA framework was obtained through this discussion.
Data Analysis
Data were collected individually, transferred to a common database, and analyzed with Microsoft Excel software. Data obtained from criteria weighting were analyzed. Mean, standard deviation (SD), and minimum and maximum values were calculated. Criteria weights were normalized to sum up to 1 for each participant: each weight was divided by the sum of weights across all criteria. Scoring of quantitative noncomparative criteria was performed on a scale from 0 (meaning that the new drug does not add any value when measured by that specific criterion) to 5 (meaning that the drug adds a lot of value measured by that specific criterion) and quantitative comparative criteria was performed on a scale of −5 (meaning that the new drug is much worse than the existing drug) to +5 (meaning that the new drug is much better than the existing drug). The mean, SD and range of minimum and maximum scores were calculated. The value contribution (VCx) of each quantitative criterion was calculated as the product of its normalized weight (Wx, ∑ Wx = 1) and standardized score (Sx = score/5). The overall MCDA value estimate (VE) is the sum of all criteria value contributions.
$${\rm VE}=\sum\limits_{x=1}^{n}{\rm VC}_x=\sum\limits_{x=1}^{n}\lpar {\rm W}_x\times {\rm S}_x\rpar$$The evaluation of contextual criteria was performed on a qualitative scale with 3 options (positive, neutral, or negative impact), and scores were transformed into a numerical scale (+1, 0, and −1 points, respectively). Different scores were adjusted in percentage. Thus, scores were represented as a percentage of experts who would consider that the drug would have a negative, neutral, or positive impact for each evaluation context. A descriptive analysis of the value of each criterion was conducted separately.
Results
Identification and Definition of Criteria
Seven hundred seventy-one publications were identified in the LR. After duplicate deletion and exclusion by title, abstract, and full text, forty-one publications were selected for retrieval of criteria. Sixteen criteria and their definitions were extracted from the literature (Figure 1).
Criteria retrieved from the Systematic Literature Review. Forty-one publications were selected after reviewing the literature. Sixteen criteria and their definitions were extracted from the literature review. The figure shows whether the publication recommended its use (green), did not recommend it (red), or remained neutral (orange) for each criterion.
The criteria extracted from the LR together with those of the EVIDEM framework formed the pilot MCDA framework, which consisted of fifteen quantitative and seven contextual criteria (Figure 2A).
Initial pilot framework included twenty-two criteria retrieved from the Safety, Liquidity, and Returns and Evidence and Value: Impact on Decision Making frameworks. During assessment of framework criteria by patient representatives, some criteria were excluded and others adapted. The final pilot framework included eight criteria: seven quantitative criteria and one contextual criterion.
Criteria Assessment
The criteria were assessed by eight patient associations representatives. Two of them represented the general patient associations (European Patient's Academy and Plataforma de Asociaciones de Pacientes), two represented the general cancer patient associations (Asociación Española Contra el Cáncer, patient representative from Sant Joan de Déu Hospital (Barcelona)), and four represented the associations of patients with a specific cancer type (Myeloma Patients Europe, Melanoma España, Confederación Crohn y Colitis Ulcerosa). Thus, representation of associations of both paediatric and adult patients, from different types of cancer, from the most to the least prevalent, was obtained.
After assessment, sixteen of twenty-two criteria were retained in the following domains (Figure 2B): disease impact (3/3 criteria), comparative results of intervention (3/3 criteria), type of health benefit of intervention (1/3 criteria), economic consequences of intervention (3/4 criteria), knowledge about intervention (0/2), normative criteria (3/4), viability criteria (2/3). From the criteria included in the pilot MCDA framework, three of these were adapted by the participants: (i) the “population size” criterion was redefined as “affected population” and considered the number of drug indications and the number of patients benefiting from the treatment; (ii) “Preventive benefit” was redefined as “type of preventive benefit: preventive risk reduction” and considered the risk reduction produced by the intervention (i.e., reduction of prevalence, reduction of risk factors); (iii) and “Type of therapeutic benefit” was converted into “type of therapeutic innovation,” which considered the mechanism of action and type of drug, the clinical benefit provided by the therapeutic innovation to the patient and the ability to individualize its administration to optimize the drug's therapeutic benefits. The resulting pilot MCDA framework included sixteen criteria, of which eleven were quantitative criteria and five contextual criteria.
Criteria Weighting
Figure 3 shows how the quantitative criteria were weighted by the participants. The criteria most valued by the panel members were “comparative patient-perceived health/PRO,” “comparative of efficacy and effectiveness,” and “severity of the disease,” while the less valued criteria were “preventive reduction of risk,” “comparative cost of technological innovation,” and “comparison of nonmedical costs” (Figure 3). The lowest agreement criteria were “population affected” (SD 1.5), “comparative cost of technological innovation” (SD 1.5), and “comparative of other medical costs” (SD 1.2). The criteria with the highest agreement were “disease severity” (SD 0.5), “comparative efficacy / effectiveness” (0.5), and “comparative health perceived by the patient” (SD 0.5). In general, there were no discrepancies in the importance of different criteria among patient representatives’ profiles. The lowest agreement was in the valuation of nonmedical cost comparison (indirect costs).
Criteria weighting (mean) provided by the patient representatives. The criteria are represented sorted from highest to lowest weight assigned.
MCDA Matrix Development and Scoring
A practical exercise was carried out in a second face-to-face workshop involving the same patient representatives that participated in the previous workshop. When the evidence matrix used to assess the value of a hypothetical drug for each criterion of the MCDA framework was scored, aspects of the MCDA framework that needed to be improved to be correctly understood by patient representatives were detected. Thus, the values of some variables that determine efficacy in oncology (e.g., “hazard ratio”) were not well known by the patient representatives, nor the magnitude of the effect they produce, so it was suggested to present the results in an interpretative way (e.g., the survival difference between treatment A and B is a median of 6 months).
It was also suggested to consider only the most relevant efficacy variables in an interpretative way (e.g., overall survival, progression-free survival, and hazard ratio). Panelists also did not know the name or meaning of some disorders or adverse effects (e.g., aplastic anemia), which limited their ability to evaluate drugs. The incorporation of both grade 1–2 and grade 3–4 adverse events into the evidence matrix was considered important.
Finally, difficulties in the interpretation of the quality of life questionnaire and symptoms scale results incorporated in the trials and the magnitude of the benefit they represent in the quality of life of patients were also observed. In addition, according to the panelists, these questionnaires do not capture relevant results for patients, such as the impact of symptoms, the impact on the autonomy and dignity of patients, and the impact on the socio-economic environment of the patients.
Value Framework Agreement
After testing the pilot MCDA framework, a discussion was held to agree on the final pilot set of criteria that could be potentially used by patients to provide their perspective on the assessment of an oncology drug. After discussion, eight of sixteen criteria were retained in the following domains: disease impact (2/3 criteria), comparative results of intervention (3/3 criteria), type of health benefit of intervention (1/3 criteria), economic consequences of intervention (1/4 criteria), knowledge about intervention (0/2), normative criteria (1/4), viability criteria (0/3). Two of these criteria were adapted to include them in the MCDA framework: “cost-consequence comparison: other medical costs” and “cost-consequence comparison: other nonmedical costs” were transformed into a single criterion “use of medical and nonmedical resources,” and a new criterion “contribution of oncologic innovation” was included. This last criterion was included to integrate the concepts of the criteria “type of preventive benefit,” “type of therapeutic innovation,” and comfort / ease of use / mode of administration of cancer innovation (criteria that were excluded during this workshop). Figure 2C shows the final pilot MCDA framework.
Discussion
Analyses of studies on interventions to better inform and involve patients in their healthcare showed that they lead to improved experiences, lower dependence on health services, better adherence to treatment and, in some cases, measurable improvements in health outcomes. The empowerment of patients in the management and decision making related to their disease and its treatment is being increasingly demanded, both by professionals and health authorities in some countries and by the patients themselves. This fact is reflected in the increase of new initiatives that aim to empower the patient (Reference Rys1;Reference Aronson, Ferner and Hughes9), as well as patient associations focused on patient empowerment (18;19).
The present study developed a pilot MCDA framework that enables patients to provide their perspective through criteria that are important for the assessment of an oncologic drug from the patient perspective. This adapted MCDA framework is presented as a useful tool to enable associations of cancer patients to participate in these processes. Thus, MCDA methodology, on the one hand, presents all the information required for the drug evaluation in a structured, transparent way. It helps to interpret the information and makes possible the comparison between different drugs in an objective and systematic way. On the other hand, MCDA provides a common tool for different stakeholders, integrating the patient perspective into drug assessment and healthcare decision making.
Moreover, this methodology allows the generation of a qualitative discussion based on the reflection behind the scores of each participant and considers all different points of view, leading to a transparent and multidisciplinary assessment and decision making. Thus, MCDA could help the patients to participate in different stages of the drug development process such as the design of clinical trials, drug evaluations or participation in health technology assessment (HTA) drug evaluation committees (18–Reference Postmus, Mavris and Hillege20). Therefore, some institutions such as the Catalan Health Service (CatSalut) in Spain have already begun to incorporate the reflective MCDA into their health plans and to incorporate the presence of a patient representative in their evaluation committee (21).
This study showed the need for the participants to be “expert” patients, because some variables and terms of clinical trials were not understood correctly. Thus, this framework could only be used by patient representatives who have knowledge of scientific terminology and its meaning. This could be solved through specific training on the terminology used in clinical trials such as some outcome variables (i.e., hazard ratio) or adverse events as well as on methodology and interpretation of questionnaires of symptoms and health-related quality of life. Also, the information included summarized in some criteria would be adapted, for example, as suggested by Postmus et al. (Reference Postmus, Mavris and Hillege20), adverse events would be aggregated into two generic categories based on the consequences of toxicity, which are more understandable by patient representatives.
The existence of a standardized framework that follows a systematic methodology could contribute to the identification of training gaps that patients could have when positioning themselves on the value of an oncologic drug. Patients could identify the information on which additional training is required when synthetizing available evidence related to each criteria, and then could provide better input to evaluation committees and decision makers.
The results of this study demonstrated that the patient perspective might not be aligned with the perspective of stakeholders in healthcare, due, probably, to the different priorities and needs that each one has. This was evidenced when comparing the relative importance assigned to the criteria by HTA evaluators and decision makers (Reference Aguarón22) and patient representatives. Criteria such as “comparative perceived health/PRO” or “unmet needs” would be more valued by patients than by evaluators, while patients would not consider drug's cost when assessing drug value.
Patient input at the scoping stage helps to identify questions to be addressed that differ from those typically formulated by HTA agencies, governments, and payers. Patients’ views on what constitutes “value” may not be the same as those of clinicians or those who conduct clinical trials. In addition, as the patients pointed out in the face to face meeting, patients do not perceive that quality of life questionnaires capture the real impact that the disease and the treatments have on their daily life. Thus, an improvement of patient input may lead to the identification and selection of outcome measures that capture critical aspects of “benefit” to patients that cannot be captured in clinical trials. Also, involving patients in HTA increases transparency in the public decision-making process (Reference Facey, Hansen and Single23;24) and adds value in scientific evaluation activities, as has been recognized by the Committee for Human Medicinal Products (CHMP) of EMA (11).
The obtained pilot MCDA framework allows to engage patients in the decision-making process in a specific phase of the product's development, but patient's engagement can take place in earlier stages of medicine development and inform or impact HTAs (i.e., NICE in England and SMC in Scotland) through all along the lifecycle of medicines, from early development, throughout evaluation and postmarketing surveillance (25). Although the MCDA framework presented in this study does not cover the methodology on how to engage patients, several frameworks and tools have been developed to help stakeholders to successfully engage patients in a transparent manner throughout all phases of medicines lifecycle like the European framework between the European Medicines Agency and patients and consumers and their organizations (25) or the Patient Engagement Quality Guidance (PEQG) tool, which has been developed to plan, develop, and assess the quality of patient engagement activities (26).
This pilot study had some limitations, mainly due to the small number of participants. The eight-member panel was too small to be regarded as representative of all patients. Therefore, studies should follow-up with a greater number of participants in which the framework developed should be re-validated and weighted.
A standardized MCDA framework would increase patient's ability to participate in healthcare decision making by allowing patient associations to position themselves in the value of a cancer therapy, to support in the dialogue with different healthcare institutions (such as national, regional, or hospital commissions) and decision makers and to incorporate the patient's viewpoint in decision making. It could also be used as a supporting document in claims or reviews of therapeutic positioning reports. In addition, it could be used as a tool to disseminate the value of treatments among patients, to spread the positioning of the value of an oncologic drug, or to raise the patient's knowledge and awareness when making shared decision making with physicians regarding the election of a new treatment. However, to make this possible, patients should be integrated by stakeholders into their drug evaluation committees.
Future studies aiming to replicate this work in other European Union countries and including a higher sample of patient associations should be performed, and the potential implications in HTAs decisions derived from patient's positioning about an oncologic drug derived from the use of an MCDA framework has to be further investigated.
Acknowledgements
We thank the anonymous referees that helped us with their comments to improve the study.
Financial support
Omakase Consulting S.L and ECO Foundation thank Bristol-Myers Squibb for supporting this study. Bristol-Myers Squibb supported the original study, but was not involved in the writing of this manuscript or the results of the study.
Conflicts of interest
The authors have nothing to disclose.
