Multiple sclerosis (MS) is a chronic, autoimmune, neurodegenerative disorder of the central nervous system that is characterized by inflammation and axonal degeneration (1;2). Relapsing-remitting multiple sclerosis (RRMS) is the most frequent presentation of the disease (1;3), and consists of periodic exacerbations of disease symptoms followed by periods of remission (Reference Loma and Heyman1). Though there is no cure for MS, a number of disease-modifying drugs (DMDs) are available to help control the condition and delay disability progression, particularly if initiated early (Reference Farber and Sand4–Reference Johnson, Zhou, Lin, Ko and Herrera6). The recommendation for early treatment initiation, coupled with the chronic and progressive nature of MS, means patients remain on a DMD indefinitely (Reference Grand'Maison, Yeung, Morrow, Lee, Emond and Ward7). Over the course of their lifetimes, it is very common for MS patients to switch to alternative DMDs due to a reduction in the effectiveness and/or the occurrence of adverse events (7;8). In fact, the proportion of patients that discontinue their first DMD ranges from 30 percent to 80 percent (Reference Menzin, Caon, Nichols, White, Friedman and Pill9).
Unfortunately, most cost-effectiveness models of DMDs for RRMS only evaluate a single line of treatment and assume patients are treated with the best supportive care (e.g., symptomatic treatment) after they discontinue that DMD (10;11). Consequently, these cost-effectiveness models do not represent clinical reality (Reference Allen, Montgomery, Maruszczak, Kusel and Adlard10). Although treatment-sequencing models have not been used for health technology assessments (HTAs) in RRMS, they have been used in other chronic diseases such as rheumatoid arthritis (RA) (12–Reference Tosh, Stevenson and Akehurst14). This has prompted HTA bodies such as the National Institute for Health and Clinical Excellence (NICE) in the United Kingdom (UK) to suggest that models in RRMS should consider incorporating multiple lines of treatment, thereby enabling a more accurate estimation of the cost-effectiveness of treatments within their position in a treatment sequence, leading to potentially better pricing and reimbursement decisions (15).
The objective of this study was to gather the insights of HTA experts from various countries on the desirability and acceptability of treatment-sequencing models in general, as well as when specifically applied to RRMS, to better understand their perspectives on the trade-off between accuracy of cost-effectiveness estimation and model complexity. Using qualitative interviews, the following topics were explored from an HTA perspective: (a) the desirability of evaluating treatments in RRMS as part of a treatment sequence; (b) the acceptability of and requirements for complex models by HTA bodies; (c) the use of sequence models in RRMS; and (d) incorporating physician and patients' perspectives on treatment switching in RRMS.
Methods
Study Design and Sample
This qualitative study was conducted in the summer of 2018 and involved in-depth double-blind interviews (interviewers did not know the identity of the participant and participants did not know the sponsor of the study) with stakeholders who had experience with HTAs in France, the Netherlands, Sweden, the UK (England and Scotland), or the United States (USA). Each participant had either participated in their local appraisal committee, advised the appraisal committee, or been involved in the establishment of the appraisal process. Participants also had experience with health economic models and were able to judge the methodology of a pilot treatment-sequencing model. A synopsis of the interview guide was provided to the participants before the interview.
Data Collection
Interviewers conducted one-to-one telephone interviews with predefined questions and prompts (interview guide provided in Supplemental Materials) that lasted up to 90 minutes. Interviews were audio-recorded with the participant's permission. All interviews were conducted in English, or in the interviewee's native language by a native-speaking interviewer. Data collected during the study were de-identified and the recordings of the interview were replayed and summarized with transcripts made of key aspects to accurately capture the exact meaning of the discussion. Questions were all open-ended and were meant to obtain spontaneous responses from the HTA expert. Table 1 provides an overview of the interview process.
Interview conduct and procedures
HTA, health technology assessment.
Results
Characteristics of the HTA Experts
Twenty potential study participants were invited to participate in the study via email; six individuals agreed to participate, nine did not respond, one participant was unavailable, and four declined. A total of six participants were interviewed, all of whom had several decades of experience in health economics and extensive experience with HTAs in their country. These experts represented the UK (n = 2), the Netherlands (n = 1), France (n = 1), Sweden (n = 1), and the USA (n = 1).
Desirability of Treatment-Sequencing Models
Participants agreed that consequences of treatment discontinuation are not usually accurately captured in the current MS economic models that are submitted to HTAs. Actual treatment-sequencing models that explicitly model several subsequent lines of treatment are rare. The experts estimated that 5 percent of the models submitted include treatment-sequencing; 80 percent include only one line of treatment or patients moved to best supportive care after discontinuation. In the remaining 15 percent, after initial treatment, a weighted average of subsequent drugs is assumed as a substitute for subsequent treatment.
When asked about the discrepancies and similarities in handling treatment discontinuation and sequences across different disease areas, all participants spontaneously mentioned the same disease areas (RA, diabetes, multiple myeloma, and oncology in general) where sequences play a major role but are not always modeled. Three possible reasons were provided as to why treatment-sequencing models are more common in certain disease areas compared to others. First, a sequencing model is not needed when an appropriate sequence is established by existing guidelines and the decision under consideration is therefore focused on a comparison of treatments in a specific position in that sequence. A sequencing model is useful in the presence of uncertainty about disease management in general. Second, modeling a disease for an HTA submission is often based on precedents of previously published models and submissions which either have or have not included sequences. Third, for some diseases real-world evidence is available, which facilitates modeling of sequences.
The participants were asked what research questions they would assess if a treatment-sequencing model, without limitations regarding effectiveness data, were to become available. It was noted by the experts that European HTA bodies conduct drug evaluations within the scope of their decision problem, which is to decide on the reimbursement of a single specific drug in a certain indication and treatment line and not its comparators. Although NICE occasionally evaluates multiple drugs at the same time in a “multiple technology appraisal,” the experts stated that European HTA bodies usually do not have the mandate to prescribe what sequences the clinical community can use, as the optimal treatment sequence is usually included in clinical guidelines and is based on effectiveness rather than cost-effectiveness.
There was no clear consensus regarding the desirability of incorporating treatment sequences for a reimbursement submission in the EU or coverage decision in the USA. Most participants thought that all evaluations should incorporate the consequences of treatment discontinuation to enable better decision making. Yet, they acknowledged risks associated with submitting a treatment-sequencing model for a reimbursement request. Given that the current HTA framework in most European countries is not designed to evaluate sequences, an HTA body might be unwilling to accept a sequencing model which may contain more assumptions and/or data gaps than a model without sequencing. The experts noted that HTA bodies would probably accept sequencing models if the body was convinced of the need to consider the drug under evaluation within the context of an established sequence.
Requirements of Complex Models
All participants indicated that the requirements for complex models are not different than those of simpler models, but the requirements are usually harder to reach using complex models. Justification of the design, especially the need for additional complexity, was the crucial issue for acceptance of complex models raised by all experts. Transparency was also noted to be of key importance by the experts. The use of real-world evidence was the preferred source to resolve data gaps when clinical trial data are not available. The experts stated “the burden is on the manufacturer” to handle uncertainty and sensitivity and scenario analyses should be proactively used. Finally, validation of modeling results using real-world data may increase the acceptability of complex models. Table 2 provides a summary of participants' views on the most challenging key requirements of complex models.
Key requirements of complex models
HTA, health technology assessment.
Sequencing Modeling in RRMS
Four participants reported previous experience with RRMS cost-effectiveness models. Two of these participants indicated that fully centering on the Expanded Disability Status Scale (EDSS) is a weakness of these models, whereas one participant thought the EDSS was the most logical measure to use. According to another expert, one strength of a previous submission that used a discrete event simulation was that it reflected clinical reality. None of the participants could recall having evaluated a submission using a treatment-sequencing model in RRMS. However, one expert indicated having developed a treatment-sequencing model in RRMS and reported that it was more complicated than anticipated; with the main challenge being that the results generated by the model did not resemble real-world data.
All participants agreed that variability among patients, as captured in a patient-level simulation model, is relatively unimportant from a policy maker perspective. Payers’ primary interests are mean outcomes and uncertainty around the mean. Participants agreed that heterogeneity (i.e., variability that can, in part, be explained) is very important to capture in a model because it enables different decision making for different types of patients. The most practical way to assess different patients is through subgroup evaluations. All participants except one indicated that their HTA bodies would be very interested in subgroups.
The scarcity of data to inform the effectiveness of ordered treatments was identified as a challenge with modeling treatment sequences in RRMS. All participants indicated that clinical trial data are insufficient to inform a treatment-sequencing model in RRMS. Although effectiveness data may be limited, the participants still advised attempting to model treatment sequences by using existing real-world data as a simple starting point.
Incorporating Physician and Patient Perspectives
A model that explicitly models sequences might incorporate the physician and/or the patient perspective in the decisions on when to switch and what treatment should be next. None of the participants reported ever encountering a model that explicitly models the physician's decision making process. Moreover, only one expert affirmed the benefit of including a physician decision rule in an RRMS sequencing model to determine the subsequent treatments. The other experts stated that treatment patterns derived from real-world data would implicitly capture the results of the shared decision making by physicians and patients, foregoing the need to explicitly model this.
All participants affirmed that the patient perspective is very important and should not be ignored in drug evaluations. However, they did not see the relevance of incorporating it in an explicit decision rule, instead proposing the use of real-world data to derive treatment patterns.
In situations where no real-world data are available, it was proposed to have physicians indicate the expected proportions of patients they would treat with each drug or sequence and calculate a weighted incremental cost-effectiveness ratio for the average sequence based on those proportions.
Discussion
Treatment-sequencing in RRMS, although common in clinical practice, is not typically included in cost-effectiveness models, potentially leading to inaccurate cost-effectiveness estimates and inefficient pricing and reimbursement decisions. Consequently, the goal of this qualitative study was to capture the perceptions of an international sample of HTA experts on the desirability and acceptability of treatment-sequencing models in general and specifically within the context of RRMS, to better understand their perspective on the trade-off between accuracy of cost-effectiveness estimation and model complexity. Each HTA expert indicated it was important to evaluate clinical and cost-effectiveness of treatments as part of a sequence or as a sequence in its entirety. However, they indicated that the current framework for HTA decision making is not conducive to evaluating treatment sequences. Consequently, HTA bodies usually have not requested a treatment-sequencing model and are unlikely to challenge a model that does not contain the explicit modeling of treatment sequences, unless the novel drug will have major implications for the rest of the sequence; however, there are some indications that this trend may be changing (12;15).
The HTA experts also highlighted that developing an RRMS treatment-sequencing model that meets all requirements by HTA bodies will be difficult, mainly due to the scarcity of effectiveness data in later lines of treatment. In the current assessment framework, HTA bodies generally do not need a treatment-sequencing model that explicitly models every line of treatment. HTA bodies only make decisions regarding the novel drug under assessment and do not have the mandate to assess comparator drugs in parallel or to holistically define the optimal treatment sequence. For the current purpose of HTA evaluations, cost-effectiveness models that use simplifying assumptions for later line treatments, grounded and validated in clinical reality, might be sufficient for HTA bodies.
Explicitly modeling individual patient trajectories and multiple lines of treatment require many data inputs that are potentially not available from real-world data. The European HTA experts were unconvinced of the need to model RRMS in such detail and might not be willing to accept data gaps due to the design of the model. However, the amount of data gaps and the level of uncertainty may not necessarily be greater with a more complex model; they may simply be more apparent because the required assumptions are more explicit. It is, therefore, critical for an RRMS sequence model to be transparent and provide sufficient rationale for the model design and each input.
Besides the HTA setting, another interesting opportunity for a treatment-sequencing model lies with a clinical target audience. The experts suggested that HTA bodies are increasingly considering treatment guidelines to define the decision problem, whereas the more clinically oriented experts indicated that guidance on treatment sequences in terms of effectiveness is much needed. All experts indicated that in their countries, the HTA bodies decide upon the reimbursement of treatment, whereas the clinical guidelines have more room to describe the relative value of treatments and to advise on optimal treatment sequences. It should be noted though that these observations are based on a rather limited number of HTA experts and additional research is still needed.
In light of the HTA decision framework and its requirements, the benefits of modeling multiple treatment lines do likely not outweigh the perceived limitations regarding complexity and data availability. These limitations and the resulting uncertainties in the generated outcomes may be a disadvantage to the manufacturer when a treatment-sequencing model is submitted. Nevertheless, when evidence gaps are appropriately handled, HTA bodies are accepting and reviewing sequence models for submission. Currently, a treatment-sequencing model in RRMS may be of use in a clinical setting, where there is potential to support the development of clinical guidelines. This application should be further explored to define additional target audiences of a potential novel treatment-sequencing model in RRMS.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462320000112.
Acknowledgments
The authors wish to thank Jason Allaire, PhD of Generativity Solutions Group for his assistance with editing the paper. This assistance was funded by EMD Serono.
Financial Support
Financial support for this study was provided by a contract between EMD Serono, Inc., a business of Merck KGaA, Darmstadt, Germany and Pharmerit International.
Conflict of Interest
The authors declare the following financial disclosures: MAP and OS are employees of Pharmerit International who received financial support for this study through a contract with EMD Serono, Inc., a business of Merck KGaA, Darmstadt, Germany. GTH and SLW are employees of EMD Serono, Inc., a business of Merck KGaA, Darmstadt, Germany.
