Introduction
Total healthcare spending has been steadily increasing—apart from the years of the recent financial crisis—internationally as well as in Europe, at a rate that outpaces the growth in gross domestic product (GDP) (1;Reference Xu, Soucat and Kutzin2), especially in low- and middle-income countries. However, factors such as epidemiological transition, changing demographics, and the introduction of new and innovative health technologies have further increased the asymmetry between resources and needs, thus necessitating the use of methodologically sound and transparent decision-making systems for the optimal allocation of resources.
The use of health technology assessment (HTA) processes, as a means to support evidence-based decision making for resource allocation, has emerged as a sound and transparent decision-making system. HTA, defined as the multidisciplinary process and method of analysis that assesses the economic, medical, social, and ethical effects of introducing and using a health technology (3), constitutes the mainstay of collective decision making in health, and its development is highlighted as a policy priority, especially in the European setting. Indicatively, and among other policy actions, the European Commission launched in 2018 an initiative for legislative proposal to the European Parliament and the European Council regarding the methodological convergence of HTA processes in EU member states, with the aim of shortening decision-making times, avoiding duplication of work, and enhancing the transferability of outcomes, through the introduction of the possibility of joint clinical assessments and joint scientific consultations on documentation (Reference Migliore4).
In practice and according to the experience of the previous years, the HTA process and its outcomes are used in order to inform government organizations (e.g., social insurance schemes) regarding decisions on pricing and reimbursement for new health technologies, as well as place of an incoming technology in the treatment algorithm (Reference Empoli, Integlia and Eleonora5). The key “mechanics” behind those decisions is based on economic evaluation, that is, a method that compares two alternative interventions in terms of both costs and outcomes, (Reference Drummond, Sculpher and Torrance6) in an effort to assess the value for money of one option between a set of comparators. In this sense, and along with relative clinical effectiveness analysis and the assessment of the socioeconomic and equity considerations associated with the introduction and the use of a technology, economic evaluation is a cornerstone of HTA and a sensitive component of resource allocation decisions. In this context, its methodological structure is of utmost importance—and the characteristics of this structure influence its outcomes and the ability of the latter to be transferable (to a certain degree) among countries.
Taking the above into account, this study aims to perform a qualitative comparative analysis of the guidelines for economic evaluation, when such evaluation is performed as part of the HTA process, and highlight the similarities and differences across European countries.
Methods
Data Sources and Analytical Approach
The search process was performed between October 2018 and January 2019. HTA organizations and working groups in European countries were identified via the official Web site of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR), the International Network of Agencies for Health Technology Assessment (INAHTA), and the European Network for Health Technology Assessment (EUnetHTA). The rationale behind this was to include in the analysis only official guidelines that are issued and currently used by HTA organizations and/or guidelines adopted by government agencies.
Following the identification of official documents, we applied a qualitative comparative analysis in the texts that were retrieved, via the use of content analysis. In brief, content analysis, one of the most popular methods of qualitative research, is primarily used to examine formal documents or other forms of written communication with the purpose of identifying common patterns or differences in a set of themes and report them in a systematic manner (Reference Hsieh and Shannon7). An outline of the process of content analysis is described in Figure 1.
Steps of content analysis
Content analysis is characterized by the coding process. The basic coding process in content analysis is to organize large quantities of text into much fewer content categories. In this study, coding was performed during the analysis and the codes were derived from the study data (Reference Hsieh and Shannon7). The coding process was done manually by the authors. Each of the first two authors defined the codes independently, and the outcomes were compared for consistency. The third author reconciled the differences. In the second step, the codes were grouped to create a set of themes. This grouping of codes created the following themes: the process of the choice of comparators for the intervention under examination through the HTA process, the perspective of the analysis (third-party payer vs. societal), the time horizon of the evaluation, the preferred economic evaluation method, the form/expression of the outcomes of the intervention, the preferred instrument for measuring the quality of life and the choice and acceptance of economic modeling, the methods for handling uncertainty, the application of discounting and the magnitude of discount rates, the acceptable data sources for preparing clinical effectiveness data (efficacy and safety), and the decision-making rules for measuring the outcome of the economic evaluation method (e.g., the use of incremental cost-effectiveness ratios [ICER]).
Eligibility Criteria
Papers and reports were selected only from official sources of health technology organizations and institutes, governmental agencies, and working groups. Papers from working groups were included if they had been adopted and endorsed by governmental agencies. The eligibility criteria of the sources (documents) included in the study were the following:
(1) Type of papers: papers from health technology organizations, institutes, and working groups from European countries as well as reports, all adopted either by HTA organizations or by governmental agencies. Working group papers not adopted by HTA organizations or governmental agencies were excluded.
(2) Data: papers reporting guidance regarding the economic evaluation undertaken within a HTA process were eligible for analysis. Papers and reports with missing data (e.g., time horizon and discount rate) were excluded from the study.
(3) Language: the analysis contained texts written in English.
Results
The selection procedure identified nineteen papers and reports providing comprehensive data regarding guidance on the performance of economic evaluation for HTA in nineteen European countries. All papers included data concerning the economic evaluation of pharmaceuticals and approximately half of them (n = 9/19) for all health technologies—including medical devices and health interventions. These were referred to Austria, Belgium, Croatia, Hungary, The Netherlands, Ireland, Germany, Denmark, and France (drugs and medical devices). Most guidelines provide the methodological guide to set out the economic evaluation of either drugs or health technologies generally, whereas some of them provide an overall approach of methods in the HTA process that also include health economic evaluation (Hungary, Poland, Croatia, and Denmark—one out of two guidelines).
A total number of twenty-eight countries of the European Union was planned for investigation, but due to problems relating to the native language in some countries (Italy, Spain, Slovakia, Slovenia, Czech, etc.) and no issued guidelines in some others (Greece, Bulgaria, Luxemburg, Cyprus, Romania, Malta, etc.), nineteen countries were finally assessed (including Norway as an additional country). Countries for which official documents that contained the data types/categories that were deemed necessary for the purposes of the study included Austria (AT), Belgium (BE), Croatia (HR), Denmark (DK), Estonia (EE), Finland (FI), France (FR), Germany (DE), Hungary (HU), Ireland (IE), Latvia (LV), Lithuania (LT), The Netherlands (NL), Norway (NO), Poland (PL), Portugal (PT), Scotland (SCT), Sweden (SE), and England (ENG).
The results of the qualitative content analysis for each of the countries are presented in detail in the Supplementary material, which also contains the references to the original documents that we used for our analysis. Data in the Supplementary material are presented in the form of Meaning Units, Codes, and Categories used for the content analysis. In the paragraphs that follow, we present summaries of results in the level of themes of the content analysis.
Comparative Analysis: Main Similarities between Countries
Choice of Comparator
Detailed instructions on the choice of the comparator appear to be a consistent recommendation among almost all guidelines of our sample study. More specifically, most countries (n = 16/19) recommend “current clinical practice” or “routine practice” or “most used intervention” as the main type of comparator in economic evaluation—and this is further detailed in terms of criteria in the texts. Countries that do not provide sufficient data concerning current medical practice/most used/routine practice as comparative interventions in their guidelines are Germany, Belgium, and Denmark. In particular, Germany's and Belgian's guidelines recommend the efficiency frontier for the identification of an appropriate comparator. In the case of Germany, all therapeutic alternatives relevant in a particular therapeutic area should be included in a health economic evaluation as comparators.
Methods of Economic Evaluation
Cost-utility analysis (CUA) and cost-effectiveness analysis (CEA) are the preferred methods of evaluation (n = 17/19) and are recommended with the exception of Austria and Denmark (they do not provide a specific recommendation). Guidelines referring to these countries do not clearly state a specific preference in the choice of economic evaluation. There is a wide range of outcome measures, but quality-adjusted life-years (QALYs) and, in many cases, life-years gained (LYG) were found to be the most important outcome measurements in terms of assessing health technologies through economic evaluation for almost all countries. In our study, almost all countries (n = 18/19) explicitly recommend QALYs as the primary outcome measurement method in the case of CUA and LYG in the case of CEA.
Data Sources for Efficacy and Safety
The results regarding data sources for the efficacy and safety of health technologies indicate that for all countries (n = 19/19), randomized-controlled trials (RCTs) constitute the preferred source. The use of observational studies is recommended as an alternate method, whereas expert opinion should be the last option. The use of systematic reviews and meta-analysis is recommended in many countries, whereas it is requested in England, France, and Croatia. Hungary clearly states the use of real-world data (RWD) as a priority and particularly recommends analysts to seek to base the analysis on clinical results achievable in the real world, whereas Norway outlines that “RWD can be used to support evidence of, for example, epidemiology, treatment duration in clinical practice, resource use, survival, or adherence to treatment in the Norwegian clinical practice.“
Quality-of-Life Measure Instrument
Health-related quality-of-life (HrQol) instruments are useful and widely used for economic evaluations of health technologies. HTA organizations have specific preferences when reviewing HTA processes. The EQ-5D generic questionnaire, developed by the EuroQol Group (Reference Devlin and Brooks8), appears as the most suitable instrument for measuring health-related quality of life in the majority of the guidelines. Almost all of them have a stated preference for this instrument (n = 17/19). In particular, most guidelines include the 3L version of the EQ-5D as a preference-based HRQoL measure (one question for each of the five dimensions—mobility, self-care, usual activities, pain/discomfort, and anxiety/depression—with three levels of answers), although some countries have also included the 5L version in their guidelines. In most cases, Standard Gamble (SG) and Time-Trade-Off (TTO) methods are recommended as direct preference methods. It should also be mentioned that the SF-36 instrument is recommended in some guidelines in countries such as Belgium, Ireland, Portugal, and Norway.
Discounting
The need for discounting in order to adjust for future costs and benefits that are predicted to occur at a future time point (9) is mentioned in all the documents that are included in our analysis. An explicit discount rate is mentioned by all guidelines (n = 19/19), usually varying between 3 and 5 percent for costs and benefits, with the exception of Belgium (discount rate of benefits set at 1.5%). For countries with guidance available on the suggested discount rates in the case of sensitivity analysis, the discount rate is recommended to vary between 3 and 10 percent (Austria and Belgium), 0 percent (Poland), 1.5 percent (England), or 4 percent (Scotland).
Time Horizon
Time horizon for the analysis, that is, the period in which the results regarding the relative magnitude of costs and outcomes of a healthcare intervention/technology should be assessed (10), is, in most cases explicitly stated and set at the lifetime horizon. Some guidelines report, with a less strong statement, that the time horizon is recommended to be long enough to highlight the impacts (costs and benefits) on the technologies being compared.
The Use of ICER
The ICER, that is, the cost of one extra unit of effect produced with the new technology compared with the comparator (Reference Bang and Zhao11) is defined and referred in most guidelines. Germany calls for an “efficiency frontier concept (which) is an extension of the standard approach of ICERs.” Most countries do not explicitly state a cost-effectiveness threshold for the ICER, whereas many of them mention that there is no such threshold that is applicable to their countries. Furthermore, in some cases (Belgium), threshold values applied in other countries are taken into account, and this should be avoided. On the other hand, only England, Ireland, and Hungary have set maximum thresholds (the range is between €20,000 and €45,000 per QALY in Ireland, whereas in England, the range is between £20,000 and £30,000, and in Hungary, it is 3× GDP per capita). Although the cost-effectiveness threshold is equal to three times the GDP per capita in Poland, these guidelines do not present any threshold for consideration.
Comparative Analysis: Main Differences between Countries
Perspective
The perspective of the analysis is the defining factor of the costs and outcomes that are to be included in the analytical framework of the evaluation (12). In our sample, we noted heterogeneous results regarding the perspectives suggested in published guidelines. More specifically, (n = 9/19) of the total sample recommend the societal perspective as the most comprehensive approach, but in some cases (Austria, Finland, Croatia, and Hungary), there is the possibility of considering other perspectives too, either payer's or health care, or even both health care and societal. On the other hand, around more than a half (n = 12/19) of the total sample recommend a payer/health system perspective, whereas France adopts a collective perspective that is sufficiently broad to take into account all stakeholders concerned by the treatments studied, in the French health system.
Sensitivity Analysis
Sensitivity analysis is defined as the way of analyzing the effect of uncertainty in an economic analysis or a model. In our study, most countries specifically require a deterministic sensitivity analysis, but just over half opt for a probabilistic sensitivity analysis. Many guidelines explicitly require both types of sensitivity analysis simultaneously (n = 8/19).
Modeling and End Point Preference
In our sample, there were marked differences among settings, both in the use of modeling per se and in the types of models used. There are two main types of modeling requirements: whether modeling is required/recommended and thus be a necessary aspect that must be taken into account for measuring effectiveness and whether modeling is an alternate method and is, therefore, not a mandatory process, unless an analysis cannot be carried out in a different way. Modeling is an acceptable process in all countries (n = 19/19), whether it is mandatory or is an alternative. Although all guidelines in all countries clearly state that modeling is acceptable, some of them in some countries explicitly state that this process is a requirement for the economic evaluation of health technologies (n = 6/19). These countries are Finland, England, Germany, France, Norway, and Scotland (in some cases). On the other hand, some countries, such as Belgium and Poland, state that modeling should be only applied with justification in case of insufficient data without any demonstration of cost-effectiveness outcomes of interventions. Moreover, the recommendation of end points is analyzed in fourteen out of nineteen guidelines, whereas in four countries (Croatia, Denmark, Estonia, Lithuania, and Latvia), there is no clear recommendation. Intermediate and surrogate end points are the most recommended end points, whereas some countries prefer final (Belgium) and clinical end points (England and The Netherlands).
The results of the analysis are summarized in Tables 1 and 2.
Comparative analysis of noneconomic processes of guidelines between European countries
Notes: Haute Autorité de santé (HAS) recommends a "collective" perspective that is sufficiently broad to take into account all stakeholders concerned by the treatments studied, in the French health system. Institute for Quality and Efficiency in Health Care (IQWiG) and Belgian Health Care Knowledge Centre (KCE) recommend the efficiency frontier for the identification of appropriate comparator. In case of Germany, all therapeutic alternatives relevant in a particular therapeutic area should be included in a health economic evaluation, as comparators.
AT, Austria; BE, Belgium; HR, Croatia; FI, Finland; NL, The Netherlands; PL, Poland; PT, Portugal; UK-SCT, Scotland; ENG, England; SE, Sweden; LV, Latvia; LT, Lithuania; EE, Estonia; HU, Hungary; IE, Ireland; DE, Germany; FR, France; DK, Denmark; NO, Norway; RCTs, Randomized-controlled trials.
+: included, −: not included, ?: unclear.
1 Primarily, 2 Other perspectives are possible, 3 Whole duration of impact of the study/effects and costs, 4 Depends on research question, but ranges from weeks to years, 5 All generic instruments are acceptable, generally, 6 EQ-5D-3L, mainly.
Comparative analysis of economic processes of guidelines between European countries
Notes: CBA is not recommended in Belgium, Hungary, and Norway while is acceptable in Finland, Portugal, and Sweden. Baltic countries (Estonia, Lithuania, and Latvia) recommend confidence intervals estimation.
AT, Austria; BE, Belgium; HR, Croatia; FI, Finland; NL, The Netherlands; PL, Poland; PT, Portugal; UK-SCT, Scotland; ENG, England; SE, Sweden; LV, Latvia; LT, Lithuania; EE, Estonia; HU, Hungary; IE, Ireland; DE, Germany; FR, France; DK, Denmark; NO, Norway; CEA, cost-effectiveness analysis; CUA, cost-utility analysis; QALY, quality-adjusted life-year; DR, discount rate; ICER, incremental cost-effectiveness ratio; ICUR, incremental cost-utility ratio.
+: included, −: not included, ?: unclear.
1 CUA and CEA should be performed at the same time, 2 Preference primarily on CUA and alternatively on CBA, 3 All methods are acceptable, 4 Both QALY and effects on life expectancy are essential/should be presented in economic evaluation, 5 Should be applied in case of insufficient data to allow assessment of cost-effectiveness, 6 Requirement, 7 No requirement, 8 Univariate, 9 Discount rate for costs at 3% and for benefits at 1.5%, 10 Discount rate at 5% for both costs/benefits, 11 Discount rate for costs at 4% and for health effects at 1,5%, 12 With threshold value
Discussion
HTA, since its emergence in the late 1970s (13) and its baby steps in Europe in the 1980s (Reference Banta and Jonsson14), has traveled a long distance, both in terms of building up its methodological rigor and in terms of its place in the decision-making process in health care. From being a “tool to improve the management of health resources” (15), HTA is currently the basic regulatory mechanism for the introduction and uptake of new technologies, often considered as one of the key hurdles that a technology must overcome in order to find its place in the healthcare market (Reference Hutton, McGrath, Frybourg, Tremblay, Bramley-Harker and Henshall16).
In this role, and, especially when dealing with public resources and their allocation, it is essential for HTA methodologies to be robust and produce inputs for decision making in a fair and transparent manner (Reference Williams and Cookson17). Under these considerations, a special emphasis on the improvement of the frameworks for HTA was placed in the previous years, with the emergence and recognition of best practices for the process (Reference Oortwijn, Determann, Schiffers, Tan and van der Tuin18). This has led to a gradual standardization of some of the key elements in the process between countries. Nevertheless, the structural characteristics and the different sets of societal values that underline the design of various health systems and of their decision-making processes (Reference Nicod19) do not allow a full uniformity between HTA systems and therefore country-specific differences prevail.
It is in the context of these differences that the present study attempted to comparatively examine the official positions (guidance) of different countries in Europe in one of the key structural elements of HTA processes, economic evaluation. In doing so, we employ a qualitative methodology based on the content analysis of the official guidance documents on economic evaluation, as issued by HTA bodies in the European countries that comprised the study sample. To the best of our knowledge, this is the first international attempt of this kind with the use of the above-mentioned methodology.
The results of the analysis, broadly organized as themes (and detailed according to content codes in the Supplementary material), show that similarities in the guidance for economic evaluations within the HTA framework currently exceed the differences that are observed between countries. The differences mainly refer to the choice of the perspective of the analysis, and, to a lesser extent the preference for the types and characteristics of sensitivity analyses and modeling. In contrast and, probably in recognition and establishment of the typical characteristics of economic evaluation today, almost identical approaches are observed in the preference of QALY-based cost-utility analyses, spanning over an adequate time horizon and discounting costs and benefits in a similar fashion.
The results of this analysis are in accordance with the policy evolutions in the field of HTA in Europe, calling for better harmonization of processes on the basis of extensive similarities and on the basis of previous research efforts, such as the EUnetHTA reports (Reference Heintz, Gerber-Grote, Ghabri, Hamers, Rupel and Slabe-Erker20). These reports, however, are based on a different methodology and refer to slightly older periods.
This study is not without limitations. Some of them pertain to the availability of data and the consequent synthesis of the sample. Due to language constraints, the selected guidelines were only in English. This means that there is a possibility of exclusion of guidelines from other countries that could follow a different approach. The relatively small number of European countries and the smaller number of countries in the sample could imply that the trends observed in this study might not be generalizable across all of Europe.
Apart from the sample drawbacks, some other limitations must be acknowledged that are inherent in the methodology that is used. Specifically, it must be noted that the analysis is descriptive in nature and seeks to highlight differences in similarities not in a quantitative manner of classifications but in terms of content. Moreover, ambiguities or elements that might be omitted from a guideline document on account of being considered common knowledge and therefore redundant cannot be incorporated into the analysis and can be highlighted as differences in approach. However, due to the nature of the technical documents, such omissions seldom occur.
The critical role of HTA today and its importance as a decision-making process has led most European countries to develop specific guidelines on how it should be performed. A close examination of these guidelines under a methodological strategy can identify points of convergence or elements of discrepancy. Convergence could support the comparability of results in establishing a relatively “common” measure of value between European countries. In general, a trend of harmonization is observed regarding the way in which and how economic evaluation is performed in the context of HTA. However, there is a long way to go for differences to be reconciled–calling for closer cross-country collaboration is necessary for reconciling differences. Despite the featuring of achievements regarding cooperation and harmonization of methodologies by the Joint Assessments of EunetHTA, there are still challenges and barriers to deal with. Strengthening of cooperation is needed between states for a feasible harmonization of methods as this process might yield benefits for health systems, industry, and patients in Europe in terms of comparative guidelines with a common value for a transparent HTA process.
In summary, the future of the HTA process in Europe is a controversial topic, whereas a more aligned HTA environment seems to be an urgent issue to deal with due to upcoming health technologies. For many years, the establishment of an overall HTA European Agency for making reimbursement decisions is being debated. Αccording to Drummond (2003), a pan-European HTA Agency could be a possibility, but the harmonization of economic evaluation guidelines, decision-making processes, and societal willingness to pay (WTP) are specific and present difficulties coming in the way of the harmonization of final decisions with regard to reimbursement of a health technology (Reference Drummond21). It is understandable that WTP is the most difficult and challenging problem to solve before proceeding with the establishment of a pan-European Agency and the first step to solve this problem is to take into account a number of other factors that inflence WTP. Last, but not least, Brexit and uncertainty in the HTA process in Europe should worry decision makers and industry. NICE will continue to offer advice through its early dialogues services as part of the EUnetHTA, but it cannot offer early dialogues with the European Medicines Agency (EMA). The question then arises: What will be the role of global leading agencies, such as NICE, in the post-Brexit era for the European HTA process?
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462320002081.
Funding
This research received no specific funding from any agency, commercial or not-for-profit sectors.
Financial Disclosure
None reported
