Recent advances in our understanding of cancer biology have greatly broadened the scope of how we diagnose, characterize, and treat different types of cancer. It is thought that approximately 20–25 percent of all oncology patients might be “classified” as having a rare cancer diagnosis, although this may now be an underestimate (Reference Komastubara and Carvajal1). As advances in molecular screening tools and techniques help us identify these molecular aberrations specific to certain tumor types, the frequency at which they can be readily detected also increases (Reference Boyd, Dancey, Gilks and Huntsman2). In turn, clinicians are able to treat patients more precisely with drugs that target specific genetic mutations and kill or halt the growth of tumor cells. Drugs specific to these targets are often more effective and can be less toxic than conventional systemic treatment, but are consequently effective on a smaller subset of patients (Reference Komastubara and Carvajal1).
Paradoxically, these advances in precision medicine have created new challenges in demonstrating the clinical efficacy of these novel targeted therapies. As the proportion of patients with cancers harboring these targets often represents a very small fraction of the population, it is often challenging to recruit sufficient patient numbers to conduct randomized controlled trials (RCTs). This creates challenges when evaluating these drugs through the health technology assessment (HTA) process, and by extension, deciding whether they should be eligible for public reimbursement (Reference Boyd, Dancey, Gilks and Huntsman2).
Traditional HTA methodologies that focus on comparative clinical and/or economic evidence can be difficult to utilize for drugs for rare indications (3;4). Firstly, the magnitude of uncertainty around the clinical efficacy may be largely due to low disease prevalence, and the use of early phase data or surrogate end points. Comparative clinical and cost-effectiveness is challenged by the lack of relevant comparable treatments and the often high costs of drugs (2;4). Beyond these clinical and economic factors, HTA agencies often consider other factors such as patient values, unmet need, and severity of disease (Reference Guindo, Wagner, Baltussen, Rindress, van Til and Kind5–Reference Sandman and Gustavsson7). To incorporate these additional values into HTA decisions, more flexible approaches such as multi-criteria decision analysis (MCDA) and/or deliberative processes have been proposed as part of HTA decision making (Reference Baltussen, Marsh, Thokala, Diaby, Castro and Cleemput8). Internationally, several multi-criteria decision models have been developed, often with rare diseases in mind, to mitigate this uncertainty. However, consensus on the utility of these models is still lacking (Reference Baran-Kooiker, Czech and Kooiker9). Furthermore, some HTA agencies and public payers have established separate or modified processes to review the evidence and make funding recommendations for drugs for rare diseases (10).
Health service funding of ultra-orphan drugs, which varies across the EU and within the UK, has led to geographical inequities in patients' access to treatment. In some instances, support for these drugs would appear to have been approved on the basis that diseases that are rare and severe are a special case (Reference Hughes, Tunnage and Yeo11). The importance of examining the impact of rarity and smaller patient populations in HTA shift has been observed in the nononcology HTA community. For example, a study that examined positive listing rates for orphan drugs reviewed by various HTA bodies including Australia, Scotland, and New Zealand found that positive recommendation rate increased from 50.0 percent for drugs reviewed between 2004 and 2009 to 86.7 percent in 2016; however, 84.6 percent of the latter were conditional on a price reduction (Reference McCormick, Berescu and Tadros12).
In Canada, there is no consistent criterion that defines a rare cancer (or disease), nor is there an alternative HTA review process to assess public funding for drugs for rare diseases. The Canadian regulatory body, Health Canada (HC), and the primary HTA agency, Canadian Agency for Drugs and Technologies in Health (CADTH), do not have a common definition of rarity (Appendix Table 1) (10;13). CADTH, Canada's primary HTA organization, provides evidence-based recommendations to provincial and territorial governments to guide public funding decisions (with the exception of Quebec) (14). Through its pan-Canadian Oncology Drug Review (pCODR), new cancer drugs are evaluated using its deliberative framework wherein overall clinical benefit, patient values, cost-effectiveness, and adoption feasibility are assessed (3;14). While rarity is not explicitly addressed in pCODR's deliberative framework, overall clinical benefit encompasses criteria of effectiveness, safety, burden of illness, and need, and includes considerations for exceptional circumstances (14). In a guidance document issued to its expert review committees in 2016, CADTH illustrated rarity as one of the considerations that should be accounted for when assessing a significant unmet need (15). pCODR notes that the current deliberative framework can be applied to all oncology drugs and situations, including rare cancers and end of life care (15).
The objective of this paper was to explore the impact of rarity on oncology HTA recommendations and public funding decisions in Canada. Given that rarer indications are often associated with less robust clinical and economic evidence, significant cost, high uncertainty, and high unmet need, we also evaluated the cost-effectiveness and strength of supporting clinical trial data for submitted rare and nonrare indications to pCODR (1;16). We investigated the influence of rarity on two main outcomes, type of HTA recommendation issued and drug funding. For the first outcome, we explored the difference in supporting clinical and economic evidence by rarity status. For the second outcome, we explore the difference in time to funding by rarity status (Reference Sandman and Gustavsson7).
Methods:
Definition of Rarity
Although many international HTA agencies have not established a definition of rarity (including an incidence and/or prevalence threshold), there is consistent recognition that rare diseases are often severe, chronic, seriously debilitating, degenerative, life threatening, with no real alternative treatment (17–Reference Clarke, Coyle, Evans, Martin and Winquist19). However, as there is no universally accepted definition, we used incidence as a mechanism to operationalize rarity since it can be measured and quantified. Two incidence-based thresholds from the Recommendation Framework for CADTH Common Drug Review and pan-Canadian Oncology Drug Review Programs: Guidance for CADTH's Drug Expert Committees were selected (15). The two thresholds are: (1) incidence of less than 5 in 10,000 new diagnoses per year and (2) incidence of less than 1 in 100,000 new diagnoses per year (15). These definitions were chosen because CADTH is the governing body that makes the HTA recommendations included in our study.
Identification of Submissions
We reviewed all publicly available reports from CADTH-pCODR's drug submissions between 1 January 2012 and 31 December 2017. Submissions without a final recommendation (withdrawn or suspended) and requests for advice (RFA) were excluded from the analysis. Recommendations issued in 2018 and onwards were excluded as many of these have not yet received final funding recommendations or have not had public funding decisions made yet.
Data Extraction
For each pCODR drug review, two independent researchers reviewed the corresponding evidence provided from the pCODR Expert Review Committee's (pERC) final recommendation, final clinical guidance report (CGR) reports, final economic guidance report (EGR), and provincial funding summary. Each pCODR final report was examined for the funding recommendation/justification, level of supporting evidence (presence of a RCT), incremental cost-effectiveness ratio (ICER), and time to funding (if applicable). From the final recommendation and CGR, data were collected including indication, final recommendation and conditions (if applicable), incidence, and characteristics of the trial(s) accompanying the submission. From the EGR, the ICER estimate or range was extracted. When a range was reported for the ICER, the upper range was used. From the provincial funding summary, the funding status, date of notice to implement, and funding date in each province were extracted. For indications where incidence rates were not available, data were extracted from Canadian Cancer Society statistics (2017 or 2018) or pivotal trials (20). All incidence estimates provide the number of new diagnoses in Canada for a particular year. This incidence was converted to the number of new diagnoses per 10,000 and 100,000.
Assessment of Rarity and Outcomes
Based on the incidence of the requested indication (e.g., tumor type or tumor subtype when applicable), we classified each submission as either a drug for a rare or nonrare indication. We then examined the impact of rarity on four outcomes. First, we explored the frequency of positive funding recommendations (which included conditional positive funding recommendations) by rarity status. Second, we compared the frequency of RCT evidence used for nonrare versus rare indication. Third, using the ICER, we compared whether drugs for rare indications are less cost-effective than drugs for nonrare indications. Lastly, we compared the time from pCODR's “notice to implement” to public funding for rare and nonrare indications in nine provinces.
Statistical Methods
Descriptive statistics were used to summarize the outcome variables for the two different definitions of rarity. Median and interquartile range were reported for continuous variables. For continuous outcomes, the difference between drugs for rare and nonrare indications was assessed using Mann–Whitney U tests to account for non-normality of the data. Frequency and percentage were reported for categorical variables. For categorical outcomes, difference between drugs for rare and nonrare indications was assessed using Fischer's exact tests to account for the small sample size. Odds ratio (ORs) and relative risks (RRs) were used to assess the association between rarity status and the recommendation type. Stratified Cochran–Mantel–Haenszel (CMH) tests were used to assess the association between the presence of supporting RCT evidence and rarity status, controlling for recommendation type.
We did not adjust for multiple comparisons. All analyses were conducted using SAS version 9.4 (SAS Institute, Cary, North Carolina, USA) and two-sided p-value <.05 was considered statistically significant.
Results
Between 2012 and 2017, a total of 104 pCODR reviews were conducted, of which 96 (92 percent) were eligible for our study. Eight (8 percent) reviews were excluded on account of being withdrawn (five), suspended (two), or an RFA (one). Of the ninety-six final funding recommendations reviewed, seventy-five (78 percent) were positive while twenty-one (22 percent) were negative. Amongst the positive recommendation, 88 percent of them were provided with a condition of lowering cost-effectiveness. Using the first rarity threshold (incidence of less than 5 in 10,000 diagnoses per year), all ninety-six submitted indications were classified as rare.
Using the second threshold (incidence of less than 1 in 100,000 diagnoses per year), we found that sixteen of the ninety-six (16.6 percent) reviews qualified as rare (Appendix Table 2). As a result of this finding, we used the second threshold of rarity for the remainder of the study. Positive funding recommendations were provided to 63/80 (78.8 percent) nonrare indications and 12/16 (75 percent) rare indications. When the reviews were stratified by rare and nonrare indications, no significant difference in the rate of positive funding recommendations was observed (p-value = .5) (Table 1).
Frequency of Recommendation Type and Randomized Controlled Trial Submission for Rare and Nonrare Indications Submitted to the pan-Canadian Oncology Drug Review
RCTs Inclusion in Submissions
Nonrare indications were more likely to be submitted with RCT evidence than rare indications (p-value <.01). In total, 8/16 (50 percent) rare indications and 8/80 (10 percent) nonrare indications submitted without any accompanying RCT evidence (Table 1).
Table 2 presents the number of rare and nonrare indications that include RCT evidence and stratifies by recommendation outcome. Of the 75 reviews that received positive recommendations, nine (12 percent) did not include RCT evidence. In total, 6/9 submissions were for rare indications and 3/9 were nonrare. The likelihood of not having an accompanying RCT submitted is six times greater for submissions for rare indications than nonrare indications, stratifying for recommendation type (CMH RR 6.3; 95 percent CI, 2.59−15.47). pERC deemed it feasible to conduct an RCT in two of the nine submissions without RCTs (22 percent). Of the two submissions, one was for a rare indication. Of the twenty-one reviews that received negative recommendations, seven (33.3 percent) did not include RCT. Of the submissions without RCTs, two were for rare indications and five of them were nonrare indications. pERC deemed it feasible to conduct an RCT in all submissions without one that received a negative recommendation (Table 2).
Frequency of Randomized Controlled Trial (RCT) Inclusion for Funding Submissions to the pan-Canadian Oncology Drug Review's Expert Review Committee (pERC) and Incremental Cost-Effectiveness Ratio for Rare and Nonrare Indications by Recommendation Type
Economic Evidence
Table 2 also shows the average ICER for rare and non-rare indications by recommendation type. For positive recommendations, the difference between the mean ICER for rare ($324,493/QALY) and nonrare ($269,055/QALY) indications was not statistically significant (p-value = .49). Similarly for negative recommendations, the difference between the mean ICER for rare ($370,001/QALY) and nonrare ($312,096/QALY) indications was not significant (p-value = .81).
Time to Funding
Following a positive conditional pCODR recommendation, drugs can enter pricing negotiations at the pan-Canadian Pharmaceutical Alliance prior to implementation. Between nine provinces, the median time to funding for rare (34–591 days) and nonrare indications (269–772 days) significantly varied. Within each province, the difference in the median time to funding between rare and nonrare indications was not statistically significant (Figure 1).
Time to funding for rare and nonrare indications by province (excluding Quebec).
Discussion
Using CADTH's recommendation guideline for defining rarity, we found that there is consistency in pCODR recommendations for rare and nonrare indications. In particular, rarity does not appear to have a significant effect on the recommendation outcome. Furthermore, for the nine provinces included in this analysis, the time to funding between rare and nonrare indications did not differ intraprovincially. When using pCODR's common definition for rarity, we found that all cancer drugs were considered rare. This suggests that cancer, as a disease, is often considered more “rare” then noncancer disease.
Our study also found that rare oncology indications are less likely to have RCT data included in the HTA submission compared to nonrare indications. pCODR acknowledges that there are exceptional circumstances in which there may be practical challenges in conducting RCTs and robust pharmacoeconomic evaluations in the presence of significant unmet need (14). Although RCTs are considered the gold standard due to their high internal validity, conducting them in small patient populations can be extremely challenging. As a result, public funding decisions may need to rely on submissions with nonrandomized and noncomparative studies. For these rare cancer populations, pCODR may determine whether it is feasible or not to conduct an RCT. However, this process is not yet defined, and is instead determined through deliberative discussion by members of the pERC and Clinical Guidance Panel. This mechanism does have the potential for inconsistencies in determining whether an RCT is feasible. In previous cases, pERC has deemed an RCT feasible when one has been conducted in a similar patient population for a different drug. Interestingly, the literature suggests that consultation with patient advocacy groups may be important to assess whether an adequate sample of patients can be recruited and retained for a large, randomized trial (Reference Clarke, Coyle, Evans, Martin and Winquist19). Consultation with patient groups may help HTA committees get patient input on the number of patients that would be considered eligible for enrollment in the trial. This will vary with age, geographic distribution, and willingness of a patient to participate (Reference Clarke, Coyle, Evans, Martin and Winquist19). Our results demonstrate pCODR often issued a negative recommendation for indications submitted without an RCT, if conducting an RCT was deemed feasible in that patient population.
Some policy makers argue that since RCTs may not be feasible in all populations, the research paradigm needs to change. Trial designs for rare cancers should be flexible and innovative to ensure recruitment is maximized and collected evidence can be applied to these smaller populations (Reference Billingham, Malottki and Steven21). Otherwise, if the quality of evidence-based medicine resides solely in numbers, patients with rare cancers will be discriminated against by their very nature (Reference Billingham, Malottki and Steven21). As such, alternative trial designs that address these challenges may become increasingly prevalent, especially for ultra-rare populations. These include trials with N = 1, Bayesian trial designs, umbrella and basket trials (21;22). In Canada, there was an increase in the number of basket trials submitted to pCODR in 2019, and there remains no standardized assessment framework for these cases (23). Furthermore, the use of surrogate end points (e.g., progression-free survival) may benefit rare populations, since there are ways to classify interventions as efficacious or nonefficacious outside of the confines of a pre-specified end point (Reference Billingham, Malottki and Steven21). Although some of these end points are validated, they do not always translate into a survival advantage. Advocates for these trial designs note that it is important for patient advocates to be consulted early in the design process, such that they are able to provide input to maximize participation and optimize trial criteria (Reference Gaasterland, Van Der Weide, Du Prie-Olthof, Donk, Kaatee and Kaczmarek24).
In contrast, others argue that rare indications should not be accommodated by lowering the bar for the type and design of supporting evidence that is required, as patients do not benefit from sub-optimal evidence and may fall under false pretenses of cure or improved quality of life (Reference Prasad and Oseran25). In particular, the use of surrogate end points is scrutinized, as these end points may not translate to real-world or long-term survival benefits. For public systems, this is especially alarming, as these drugs may place a large financial burden on payers/the system in return for minimal patient benefit (26;27). As well, earlier phase (phase I/II) trial evidence may be submitted more frequently for drugs for rare indications; however, HTA agencies should be warry that there can be significant discrepancies in early phase data with long-term outcomes.
Conducting international trials is a strategy that has been identified to maximize patient recruitment, investigator expertise, and reduce diagnostic error (Reference Julkowska, Austin, Cutillo, Gancberg, Hager and Halftermeyer28). For example, one international trial was able to accrue over 300 patients across 6 years to study adrenocortical cancer, which is estimated to have an annual incidence of .7–2/million (Reference Prasad and Oseran25). This may suggest that the feasibility of conducting an RCT lies with investigator expectations versus the actual incidence of the tumour (Reference Prasad and Oseran25). It is important to note however, international trials can be substantially harder to coordinate and ensure consistency in standards between treatment centers (Reference Prasad and Oseran25).
There are different strategies in the literature that discuss how HTAs should be conducted for rare diseases, with more recent approaches such as MCDA models to account for explicit and nonexplicit values including unmet need and disease severity. Furthermore, some MCDA tools incorporate explicit decision rules to account for cost-effectiveness in the HTA decisions (Reference Baltussen, Marsh, Thokala, Diaby, Castro and Cleemput8). We found there was no significant difference in the type of recommendation outcome based on cost-effectiveness between rare and nonrare indications in Canada. This finding suggests that the current system demonstrates the principle of fairness, to ensure that drugs for rare diseases are given equal opportunity for a positive recommendation, while requiring good qualitied supporting evidence such as RCTs. This helps public payers get value for money and patients receive the best possible therapies (18;22). It has been suggested that principles of fairness and equity are valued similarly with effectiveness and cost when it comes to healthcare resource allocation decisions (Reference Guindo, Wagner, Baltussen, Rindress, van Til and Kind5). As such, an equitable approach may require HTA committees to provide special consideration and give different weighting to the potential social values for the health benefits accompanying the shortened life expectancy in those with very rare conditions (Reference Sandman and Gustavsson7).
Nonetheless, these issues will become increasingly pertinent, largely due to increasing prevalence of precision medicine and the high-cost concerns for these drugs. The observed trend is expected to increase due to the small user population/market, but also due to the uniqueness of these drugs which have no competitive offerings (Reference Wood29). Through these discussions, rare indications may become de-siloed, which would be advantageous for the patient as well as the healthcare system (Reference Wood29). Currently, most definitions of rare disease are based (somewhat arbitrarily) on disease incidence or prevalence, for which true values can be difficult to estimate (Reference Richter, Nestler-Parr, Babela, Khan, Tesoro and Molsen30). Due to pressures on public funders to provide reimbursement for new therapies in the absence of unambiguous evidence, some researchers have suggested that the definition of rare be based on the clinical severity of disease in addition to the accepted biochemical or genetic marker (Reference Clarke, Coyle, Evans, Martin and Winquist19). This approach would also ensure that expected evidence requirements are defined a priori. However, for this to be successful, it will require collaboration with several groups, and the willingness of said groups to engage in dialogue (Reference Clarke, Coyle, Evans, Martin and Winquist19).
There were several limitations to our study. First, we did not incorporate the line of therapy or previous treatment(s) into our incidence calculations as there is limited data on the number of patients receiving a specific line of treatment. Funding criteria often specify previous therapies or a specific line of therapy in order for patients to be eligible. Therefore, it may be possible that even more indications would be considered rare when line of therapy and previous treatments are taken into account. Second, there were a limited number of rare indications submitted to pCODR, limiting our sample size. We also recognize there may be an inherent submission bias for cases with weaker evidence. Third, there is no universal definition of rarity, which means our rarity designations may change using alternate criteria. We chose only to use the definitions utilized by CADTH, as CADTH is the HTA body that makes the funding recommendations to Canadian jurisdictions, and therefore felt it would be most appropriate to use when evaluating their own funding decisions. However, we recognize our results may vary based on different sensitivities. Fourth, incidence calculations were done using multiple data sources and often best estimates; however, we aimed to be as consistent as possible in sourcing this data, firstly using the pCODR reports and then using data from the Canadian Cancer Society. We recognize that incidence is merely one way to operationalize rarity, which is a multifaceted concept, but was chosen for its utility and ease of operation. Fifth, we excluded two indications that did not have an ICER included in the pERC report. Additionally, we excluded a rare indication with an ICER range between $40,000/QALY and $4,000,000/QALY. This was determined to be an outlier because of the high uncertainty. This wide range in the ICERs emphasizes the clinical uncertainty associated with many of the drugs submitted, which makes it difficult to ascertain whether a drug is actually cost-effective for different stakeholders. Finally, we were unable to account for many factors (e.g., patient values) that may contribute to funding recommendations and decisions due to their subjectivity in nature and lack of detail in the pCODR final recommendations (Reference Gaasterland, Van Der Weide, Du Prie-Olthof, Donk, Kaatee and Kaczmarek24).
In conclusion, rare indications appear to be associated with weaker clinical evidence. There appears to be no association between rarity, positive funding recommendations, and time to funding. Further work will evaluate factors associated with positive recommendations and the real-world utilization of funded treatments for rare indications.
Acknowledgments
We would like to thank the Canadian Centre for Applied Research in Cancer Control for its support and funding of this project. We would also like to thank the members of Cancer Care Ontario's Provincial Drug Reimbursement Program, as well as Alex Chambers for her support and guidance throughout this research.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462320000483.
Funding
This study was supported by the Canadian Centre for Applied Research in Cancer Control (ARCC). ARCC is funded by the Canadian Cancer Society Grant #2015-703549
Conflicts of Interest
Dr. Trudeau reports that she is the Chair of the pCODR Expert Review Committee. Dr. Wright reports that she received personal fees from Novartis (fees donated) for speaking during the 36-month window prior to the publication of this study. The rest of the authors report no conflicts of interest
