Introduction
Sixteen years have passed since INAHTA's initial investigation on values and ethical issues in technology assessment in health care (HTA), which revealed “the existence of disparate methods for making values and ethical issues explicit” among its members (1). In 2005, following their initial survey, INAHTA mandated its Ethics Working Group to reflect on how HTA organizations could better fulfill their role in meeting social expectations. This group concluded on the importance to enhance HTA methodology by adding an integrative and context sensitive approach of ethics and argued that ethical analysis should be “integral” to HTA (2). They proposed, as a starting point, a question-based framework for “tackling” the integration of ethics into HTA. This framework is focused on the elicitation of a reflection and considered as a “significant improvement” for handling ethical issues in HTA (Reference Burls, Caron, de Langavant G, Dondorp, Harstall and Pathak-Sen3). The official position of INHATA's member organizations was also surveyed “to explore barriers and facilitators influencing the integration of ethical considerations in HTA” (Reference Assasi, Schwartz, Tarride, O'Reilly and Goeree4). This study identified several barriers, namely: “limited ethical knowledge and expertise”, “difficulties in finding ethical evidence or using ethical guidelines”, and facilitators like involvement of ethicists in HTA and the “simplification of ethics methodology” (Reference Assasi, Schwartz, Tarride, O'Reilly and Goeree4). Another survey conducted with IJTAHC authors (2005–2007) revealed that: “90 percent of HTA professionals agree that healthcare decisions involve value-judgements and that ethical analysis is important to HTA” (Reference Arellano, Willett and Borry5). Despite the reported limitations to integrating ethics into HTA, these studies revealed the importance for HTA-organizations and researchers to consider ethical issues and values in this process.
In a previous publication, we reported that the integration of ethics is relevant to a decision-making paradigm that provides rational grounds for the decision makers' justification of their social choices (Reference Bellemare, Dagenais, Bédard, Béland, Bernier and Daniel6). Because decision making implies value-judgments, answering the question of how to elicit value-judgments in HTA is fundamental. The answer to this question is twofold (resulting from the reflection that followed INHATA's survey): the necessity to render explicit the values involved in this process as described in Burls et al. (Reference Burls, Caron, de Langavant G, Dondorp, Harstall and Pathak-Sen3): “As HTA inevitably is value laden, ethical analysis aims to make these values explicit […] such that decisions can fully be informed” and the necessity for an ethical analysis in HTA so that “by integrating ethical analysis into assessments, findings become more relevant.” In this study, values are considered as the core of value-judgments, where an attribution of a value to something is an evaluation which is distinct from assertions and moral prescriptions (Reference Legault, Bédard, Bellemare, Béland, Bernier and Dagenais7). Ethical analyses can therefore be categorized by three approaches proposed in the HTA literature (Reference Legault, Patenaude, Parent, Béland, K.-Bédard and Bellemare8) and reports: descriptive ethics (Reference Duthie and Bond9), evaluative ethics (Reference Duthie and Bond9) and prescriptive ethics (Reference Giacomini, Winsor and Abelson10). Another question central to the integration of ethics is who should be responsible for ethical analysis because external ethical expertise is perceived as a facilitator for such integration (Reference Assasi, Schwartz, Tarride, O'Reilly and Goeree4). Furthermore, can the involvement of patients, citizens, and stakeholders be useful to resolve conflicting value-judgments in decision making? (Reference Coulter11).
In this study, we aim to identify HTA professionals' perceptions about the purpose and methods of the integration of ethics in the HTA processes. What is the relevant information needed to guide the decision makers' justification of their social choices? Is there a need for eliciting implicit value-judgments in this process, what type of ethical analysis is required and what is its place in the final HTA report? What is the role of external expertise in ethics and the one of patients, citizens and stakeholders in handling the final trade-offs between conflicting value-judgments?
Methods
A research protocol was produced to conduct a Web-based, anonymous survey with LimeSurvey (Version 2.73.1) (12). The questionnaire (see Survey Questionnaire in the Supplementary material) was written in English and validated (internally and externally). This research project was subsequently approved by the Scientific- and Research Ethics-Committees of Centre intégré universitaire de santé et de services sociaux de l'Estrie – Centre hospitalier universitaire de Sherbrooke (CIUSSS de l'Estrie CHUS), Québec, Canada.
Subjects
Convenience sampling was utilized to develop our participant contact list. All HTA organization members from existing lists we had, and organizations having declared HTA activities were included in the list of participants that would receive the survey. Four approaches were used for our sampling to maximize the coverage of HTA producers around the world. First, the names, Web site information, e-mails, and contact persons were collected in 2016–2017 from membership listings of the following HTA organizations' Web sites: International Network of Agencies for Health Technology Assessment (INAHTA), Health Technology Assessment International (HTAi), European network of Health Technology Assessment (EUnetHTA), International Information Network on New and Emerging Health Technologies (EuroScan), “Red de Evaluación de Tecnologías en Salud de las Américas” (RedETSA), and the network of Asian HTA agencies (HTAsia Link). These organizations were also invited to participate. Second, similar information was then collected from a World Health Organization's (WHO) listing (13), the HTA directory of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the 2015 WHO Global Survey on HTA (14). In the latter case, countries with declared HTA activities without an identified HTA agency, their Ministry of Health (or acknowledged governmental department therein) were contacted. Third, a contact list was developed in collaboration with “Institut national d'excellence en santé et en services sociaux” (INESSS) (15) Québec, Canada (n = 61). Finally, personal contacts were made at two HTAi conferences: Rome, June 2017 and Vancouver, June 2018. After eliminating duplicates, a total of 328 different HTA agencies, located in 75 different countries, were identified (see Countries Invited to Complete Survey in the Supplementary material). Complete contact lists were exported in csv format and stored in LimeSurvey™ software (v. 2.73.1).
Questionnaire
A survey consisting of twenty-two questions (see Survey Questions in the Supplementary material) was created based on our previous work on the integration of ethics into HTA (Reference Bellemare, Dagenais, Bédard, Béland, Bernier and Daniel6;Reference Legault, Patenaude, Parent, Béland, K.-Bédard and Bellemare8;Reference Legault, Patenaude, Bédard, Bellemare, Bernier and Dagenais16). Significant citations from the HTA literature were paired with questions to ensure a common understanding and to focus on current considerations linking HTA's social role, decision-making paradigm, and the integration of ethics.
The survey questionnaire was structured in five sections, aiming to survey the HTA practitioners' perception on: (i) the purpose of HTA, (ii) its value ladenness, (iii) the type of ethical analysis (which, by whom and how), (iv) the aim of the process itself, and (v) the usefulness of participatory approaches in decision making. In most of the survey's sections, the participants were answering Likert scales of 4 to 5 choices (see Survey Questions in the Supplementary material) and included a field for narrative comments.
A total of 328 invitations were sent on 16th April 2018 and the survey was carried out until 25th July 2018. Reminders were sent 2 weeks later, and then 1 week after to those who had not already participated. Each of the respondents' country was also categorized according to World Bank's four Income Groups and seven Regions (17) (see Surveyed Population in the Supplementary material), to allow analyses on the basis of per capita-income or regional differences (data not shown).
Results
Survey's sample description
Eighty-nine participants completed and submitted a finalized survey for a participation rate of 27.13 percent (89 surveys submitted/328 invitations sent). From the seventy-five countries invited, thirty-three responded (i.e., 44.0 percent). Countries with more than one respondent include Canada (n = 39; 44.3 percent), Spain (n = 5), Australia and Brazil (n = 3 each), Argentina, China, Germany, Italy, Lithuania, Mexico, the Netherlands, the United Kingdom, and the United States (n = 2 each), whereas the twenty other participating countries each had one respondent.
As shown in Surveyed Population (in the Supplementary material), participating HTA organizations included twenty-six governmental departments (29.2 percent), twenty-nine governmental and quasi-governmental agencies (32.6 percent), twenty-three research & academia (25.8 percent), and ten hospital-based (11.2 percent). The population covered by the participants' agencies was distributed as follows: one at the international level (1.1 percent), forty at the national level (44.9 percent), sixteen at the provincial level (18.0 percent), fourteen at the regional/local level (15.7 percent; two possible initial choices merged afterward), and eighteen at the hospital level (20.2 percent).
Questionnaire section 1: purpose of HTA (social role)
For this section, participants were provided the following citation from the literature to define the purpose of HTA “to offer useful input into this process [for decision makers] so as to increase the quality of the deliberations and of the resulting decisions” (Reference Schokkaert18). They were questioned on the importance of different requirements for HTA reports. Table 1 shows that more than 84 percent of respondents answered they globally agreed on the importance of the requirements covered in this section. There was a stronger agreement on the requirement to base decisions on scientific evidence (73 percent) and analysis of contextual data (65 percent), as compared to the need for patients/citizens to have access to the rationale underlying decision makers' choices (42 percent). Access to justifications for citizens had the least preference.
Survey section on the purpose of HTA nowadays. Question: In order to offer useful input for decision makers, how do you agree on the importance for HTA full reports to consider the following needs?
Questionnaire section 2: value-ladenness of the HTA process
A citation from the literature was provided stating that “Ethics related to the choices made with regard to hypotheses, evaluation design, outcome measures, and so on, when assessing a technology. These certainly are value-laden. For hypotheses and outcome measures are based on a specific notion about the merit of a technology, or what a technology should do to be valuable” (Reference Oortwijn, Reuzel and Decker19). Participants were asked to specify to what degree of explicitness value-judgments should be reported in various report's sections (i.e., scoping, analyses of safety, efficacy, cost-effectiveness and ethics, and recommendations). Table 2 shows that more than 64 percent perceived that all report's sections should be highly explicit on the value-judgments they contain. The efficacy and safety sections were given the highest score (78.7 and 74.2 percent, respectively) and scoping and ethical analysis sections the lowest score (64.1 and 66.3 percent, respectively).
Survey section on value-ladenness of the process of HTA. Question: please specify at what degree of explicitness should value judgments be reported in the following sections of a full HTA report?
Questionnaire section 3: ethical analysis
In this section, the participants were asked to consider three types of ethical analyses found in HTA reports or HTA literature according to the following definitions, for each: (i) Descriptive ethics: to raise ethical issues without analyzing them and expose the different issues decision makers must be aware of and cope with (Reference Duthie and Bond9); (ii) Evaluative ethics: to analyze and evaluate the issues, prioritizing underlying values, identifying conflicting value-judgments and their respective justification (Reference Duthie and Bond9); and (iii) Prescriptive ethics: to draw a line between what is acceptable or unacceptable technologies by means of a moral reasoning (Reference Giacomini, Winsor and Abelson10). They were asked whether the report's ethical analysis should only be descriptive, descriptive, and evaluative or all of the above. Most of respondents showed a preference for a combination of descriptive and evaluative ethics (n = 39; 43.8 percent), followed by that for all three (n = 25; 28.1 percent) or for descriptive ethics alone (n = 14; 15.7 percent).
On the question of who should be responsible for ethical analysis, some considered it should be conducted by a standard team of HTA professionals (n = 9; 10.1 percent) or an expert in ethics alone (n = 12; 13.5 percent), whereas three quarter responded it should be a combination of both (n = 68; 76.4 percent).
The respondents were asked that given ethical analysis is a specific field of inquiry, its results should be presented in a separate section similar to those relating to a technology's effectiveness, safety, and cost. Seventy-eight percent of respondent strongly agreed/agreed on the importance of such separate section (n = 69; 30.3 and n = 42; 47.2 percent, respectively).
Questionnaire section 4: aims of the HTA process and reports
Table 3 shows the survey section's results on HTA process and report, whereby participants were asked whether they agree with various proposed aims for HTA according to three following statements: (i) they should concentrate on one phase: assessment, which includes epidemiological studies, the collection of all available scientific information (e.g., literature review and cost-effectiveness analyses) and then a subsequent analysis and final synthesis; (ii) “Following Blanquaert & Caron the process of HTA can be described as a combination of two phases”: (a) assessment (as described above); and (b) appraisal, in which contextualized data are gathered and recommendations made (Reference Martin, Williams-Jones and de Ortúzar20); and (iii) three phases: (a) assessment (as described above); (b) contextual data, including ethical analyses and qualitative studies on patients and citizens perspective, organizational changes and so on, and a subsequent analysis and synthesis; and (c) recommendations grounded in the first two phases.
Survey section on HTA process and report. Question: Do you agree with the following statements proposing these aims for HTA?
Most respondents disagreed, agreed and strongly agreed with, respectively, the one, the two and the three phase(s) structures for HTA process and reports (Table 3). The results showed that 47.2 percent (n = 42) globally disagreed (i.e., including disagreed and strongly disagreed answers) with a one-phase approach (i.e., assessment only) whereas 16.7 percent (n = 15) of respondents had no opinion. More than half either globally agreed (n = 46; 51.7 percent) with the two phases process proposed (i.e., assessment and appraisal), with a similar proportion having no opinion. Almost half of respondents strongly agreed (n = 44; 49.4 percent) and three quarters globally agreed (n = 68; 76.4 percent) with a three phases approach to HTA process and report (i.e., assessment, contextual data, and recommendation).
Questionnaire section 5: participatory approaches in the production of recommendations
The participants were informed that as the conclusions of the analyses attribute a given value to these findings (i.e., efficacy, safety, cost-effectiveness, and ethical validity), they could be required to trade-off between these values in order to reach specific recommendations. They were asked: to which extent do you think public/citizens-, patients-, and stakeholders-involvement, could be useful for this matter? Table 4 shows that 56, 70 and 78 percent, respectively, found the former participatory approaches globally useful.
Survey section on participatory approaches in the production of recommendations. Question: To which extent do you think the following participatory approaches could be useful in the production of recommendations?
Discussion
Questionnaire section 1: the purpose of HTA in society (social role)
HTA's social role has always been related to guide decision makers. However, the impacts of healthcare decisions are not limited to government agencies they also have an impact on professionals, patients, and citizens. The results are very homogeneous and seem to confirm the prevailing trend considering HTA reports as primarily directed to governmental or hospital agencies rather than to patients or citizens. Of course, if the aim of the report is exclusively aimed at establishing if it is worth investing in a technology or an intervention, then evidence on safety and efficacy are the primary concern. However, if the reports are also destined to patients and professionals to guide their decisions, they must include other considerations such as patients' preferences, professional practice, and organizational setting. Interestingly, HTA professionals also perceive important the need for contextual data which contrasts with the relative importance given to these “domains” (i.e., ethical, organizational, patient, legal, and social), according to a review of EUnetHTA (21) reports.
Questionnaire section 2: value-ladenness of the HTA process
The value-ladenness of the HTA process is widely recognized in HTA literature, but the degree to which HTA reports should render explicit the implicit value-judgments embedded in the process is not settled. Value-judgments are fundamental components of any decision-making process. Every decision rests on value-judgments that are considered to be the best thing to do, and part of the justification requires rending explicit the value-judgment imbedded in the process. The questions were formulated to directly address the decisions made at different stages of an HTA report's production: in the scoping phase, in the choice of analysis (safety, efficacy, cost-effectiveness, and ethics) and in the recommendations. In the final report of INATHA's Working Group on Handling Ethical Issues, there was no clear answer to the question “How far should HTA go in: Displaying values involved in the HTA-Process itself?” Although acknowledging the fact that hidden value assumptions in methodologies could give the impression that the results are more “neutral” than they seem, it nevertheless raises the fundamental lack of conceptual and methodological foundations to treat these aspects. Furthermore, the paradigmatic example given in the report is the value-ladenness of the measures of the “quality of life”. The results of our survey show that 64 percent of respondents perceived that all sections must render explicit value-judgments. This confirms the shift from the values imbedded in the methodology (or scientific discipline) to the value-judgments in the decision-making paradigm. They showed that almost two thirds of respondents perceived that all stages/sections must render explicit the implicit value-judgments involved. It is interesting to note that aside from the ethical domain, not only do HTA professionals agree on the value ladenness of the other sections, they also think that value-judgments therein should be made explicit. This seems to confirm the importance of recognizing implicit value-judgments during the decision-making process of HTA (i.e., methodological, perspective, epistemological, and so on choices) in order to insure the transparency of the process.
Questionnaire section 3: ethical analysis
The fact that there is “no standard procedure in ethics” is, according to our previous publication (Reference Bellemare, Dagenais, Bédard, Béland, Bernier and Daniel6), a main obstacle to the integration of ethics into HTA. Numerous approaches are proposed for ethical analysis in HTA (21), diverging on three levels: the disciplinary foundation defining the framework, the specific nature of the evaluation and the reasoning modes applied to a particular situation (Reference Legault, Bédard, Bellemare, Béland, Bernier and Dagenais7). In the INAHTA Working group's report (2), the preferred approach was based on the distinction between analysis of situations where a normative consensus exists, and situations where moral conflict arises in society, proposing essentially a descriptive ethics' approach. Only 16 percent of the respondents indicated a preference for this approach. The results also showed that 28 percent of the respondents preferred a prescriptive approach whereas most of the ethical frameworks proposed in HTA are prescriptive and grounded on a moral philosophy. Descriptive and evaluative ethics was preferred by 43 percent, indicating a preference for an axiological trend consistent with a decision-making paradigm. This corroborates the reported limitation on ethical analysis knowledge observed when surveying INAHTA member agencies (Reference Assasi, Schwartz, Tarride, O'Reilly and Goeree4).
The preferences for a descriptive and evaluative ethical analysis were confirmed by the results on the questions related to who should produce them. Three quarter of the participants favored a combination of HTA professionals and ethic experts to be responsible for ethical analysis over HTA professionals alone. This contrasts with the previous literature finding, where approximately 60 percent find “usual HTA experts” should be responsible for this analysis, assuming that at least one of these professionals should have training in ethics (Reference Arellano, Willett and Borry5).
There was a general agreement on the importance for ethical analysis to be included as a specific section in HTA reports. We concede there might have been some confusion in the understanding of this question, as expressed in answers and in some of the commentaries given (not shown). Indeed, as some participants mentioned, ethical analysis should be explicit and established throughout the report, which could explain why some excluded the need for a separate ethical analysis' section.
Questionnaire section 4: aims of HTA reports and process
The social role of HTA reports is to guide decision makers in their deliberations. What information is required to maximize the results? The nature of the report implicitly assumes the decision makers' needs for information. Since its beginning, the core of HTA is evidence-based. The main assumption is that decision makers must ground their decisions on solid scientific evidence, suggesting HTA should concentrate exclusively on assessment. However, because decision makers evolve in particular organizational and societal settings, should other information be assessed to guide the appraisal of information in the final decision? Furthermore, is the aim of the HTA process only to give factual data or should it elaborate recommendations for the decision maker grounded on scientific facts and contextual data analysis? In contrast with the question on the type of ethical analysis preferred (first question section 3), the answers given in this section were not exclusive (i.e., one choice vs. multiple selection answers). The respondents globally disagreed with a one-phase scenario (i.e., 47.2 percent). More than half globally agreed with a two-phase scenario including assessment and appraisal, which is the main trend in these past years. Whereas the strongest agreement called for a three-phase scenario, therefore suggesting HTA professionals were inclined to carry appraisals or contextualize findings and make recommendations. This is consistent with the recognized importance of integrating scientific evidence and value-judgments in recommendations to guide decisions makers (Reference Bellemare, Dagenais, Bédard, Béland, Bernier and Daniel6) and adequately fulfilling HTA's requirements for an active social role. Whether these results call for a change in governance for HTA agencies (implying resources and infrastructure allocation to implement these three phases) or if they simply indicate an appeal for more appraisal in HTA reports from this community is beyond the scope of this study. It nonetheless indicates a gap between the perception of HTA professionals and their actual practice, suggesting that the lack of appraisal or contextualization and recommendation should be addressed.
Questionnaire section 5: participatory approaches in the production of the recommendations
Participatory approaches of citizens, patients, and other stakeholders are more and more integrated into HTA. Is there a limited role for contextual data on patient's perspective or should they somehow participate in the decision-process for the production of recommendations? Recommendations require that trade-offs be made between conflicting value-judgments. Trade-offs that can be grounded on different considerations. We explored the possibility of involving patients, citizens, or other stakeholders as a mean of resolving conflicting value judgments. These questions were meant to see what the relative importance given to the input different groups might have in such trade-offs without stating whether they may help resolve dilemmas on heterogeneous or uncertain evidence. Public/citizens' involvement seemed to be considered “very useful” in a larger proportion compared to that of patients and stakeholders, possibly because of their respective personal/private interests involved. The results confirmed that there is no consensus on the nature of the contribution of patients in the decision-making process of HTA. Furthermore, the results also showed that trade-offs could be grounded on considerations other than those of patients, citizens, and stakeholders.
Finally, the authors were not expecting such a high homogeneity in the responses to this survey (except in section 3) and were surprised by such results given the respondents' international span and the distribution of agency types and population covered.
The use of a convenience sampling for our contact lists may have introduced a bias, although we think that its international participants' span, its distribution (agency type, population, and regions) and the responses' homogeneity (except those to section 3) with a low regional variation (data not shown), provide a significant representativeness alleviating the presence of such bias. In addition, perceptions of participants from other countries might have been outweighed by the fact that Canadian participants constituted approximately 44 percent of the sample, which was tested and indicated a similar homogeneity in both subsets (Canadians vs. other countries, not shown). Furthermore, given our intention to have the day-to-day and grounded perspective of HTA professionals, we did not mention in the invitation that professional ethicists should not contribute to this survey, because selecting an ethical expertise would have introduced a bias.
This homogeneity among participants indicates these HTA professionals not only have an awareness of the issues raised by the integration of ethics in the HTA process but are converging on strategies to elicit value-judgments in the process and favor a three phases approach where ethical considerations have an impact on the nature of the recommendations.
Conclusions
This study aimed to clarify how HTA professionals perceived the integration of ethics into HTA. The questionnaire's purpose was to verify if ethical integration was related to a decision-making paradigm. The results suggest that such a paradigm is at the core of HTA's social role and therefore guides the selection of information gathered in the process: scientific evidence and contextual evidence. Furthermore, it illustrates the necessity of eliciting value-judgments made in the decision-making process of HTA as well as in the ethical analysis where descriptive and evaluative ethics were preferred. Finally, it highlights the importance of a three-phase approach in HTA where scientific and contextual evidence (contextual data, including ethical analyses and qualitative studies on patients and citizen's perspective, organizational changes, and so on) are integrated in order to ground recommendations and fulfill the need to address HTA's social role in guiding decision makers.
Our study showed that HTA professionals have high expectations for the integration of ethics into HTA processes and reports. Eliciting the value-judgments imbedded in the decision-making process of HTA is required to clarify the reasons grounding the whole process. Furthermore, ethical analysis should be descriptive and evaluative and be reported in a separate section like safety, effectiveness and cost. Finally, an HTA report should have a three-phase approach making recommendations integrating both evidence and contextual analysis (including ethical analyses, qualitative studies on patients and citizens perspective, organizational changes, and so on) in order to fulfill HTA's social role in guiding decision makers. Only a study of recent reports could should how the integration of ethics into HTA reports has changed according to these expectations.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462320000793.
Acknowledgments
The authors wish to thank: Drs Nazila Assasi and Luis Arellano for full access to their survey's material, Mr. Simon Brière Eng. for technical support with LimeSurvey™, and Ms. Louise Corbeil for editorial work on the manuscript. We also wish to thank the Institut national d'excellence en santé et en services sociaux (INESSS, Québec, Canada) and the Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU, Sweden) for help in reaching our local network and additional HTA agencies in Sweden, respectively.
Author contributions
The authors state that (i) each author contributed to the conception and design or analysis and interpretation of data and the writing of the paper; (ii) each author has approved the version being submitted; and (iii) the content has not been published nor is being considered for publication elsewhere.
Financial support
This work was supported by the Canadian Institutes of Health Research (J.P., grant number 142187).
Conflict of interest
None.
