Governments around the world employ health technology assessment (HTA) to assist them in making healthcare funding decisions. HTA processes evaluate health technologies (medical treatments, medical devices, drugs, and healthcare programs) in terms of their safety, effectiveness, cost-effectiveness, and broader implications for society (Reference Banta1).
Governments have increasingly involved the public in health-funding policy development, and calls for patient and public involvement have been particularly influential in publicly funded healthcare systems (Reference Hailey, Werkö, Bakri, Cameron, Göhlen and Myles2). Some authors argue that patient and public perspectives should be included when a range of different values need to be considered or when more information on social or psychological experiences of living with a disease is warranted (Reference Gauvin, Abelson, Giacomini, Eyles and Lavis3). Multiple involvement methods have been used by HTA agencies and government bodies (Reference Menon and Stafinski4), but more advances have been made in the involvement of patients than the general public (Reference Nilsen, Myrhaug, Johansen, Oliver and Oxman5). Patient experiences can provide additional information on health outcomes during an HTA (Reference Facey, Boivin, Gracia, Hansen, Lo Scalzo and Mossman6). Some purposes for public involvement include identifying when particular social values are worth emphasizing in policy development, increasing the transparency of decisions, and the accountability of government spending (Reference Rosenberg-Yunger, Thorsteinsdottir, Daar and Martin7).
There are methodological challenges in involving the public in HTA (Reference Hunter, Kieslich, Littlejohns, Staniszewska, Tumilty and Weale8); for example, it is not clear which processes are more appropriate for involving a representative sample of the public. Nonetheless, this has not stopped some countries from trialing diverse public involvement processes in HTA (Reference Lopes, Street, Carter and Merlin9). There are also criticisms of how both patients and the public have been involved in healthcare funding decisions (Reference Hailey, Werkö, Bakri, Cameron, Göhlen and Myles2). Stakeholders often fail to appreciate the different roles that patients and the public can play in HTA (Reference Hunter, Kieslich, Littlejohns, Staniszewska, Tumilty and Weale8) and what meaningful involvement constitutes in the eyes of patients and the public (Reference Lopes, Street, Carter and Merlin9).
Public Involvement in Canadian HTA
Canada has a primarily public healthcare system, with a federal level responsible for defining the provision of healthcare services (Reference Marchildon10) for the whole country and a provincial level, which is responsible for the administration of health services. Canada also has a pan-Canadian HTA agency that assesses cancer and non-cancer medicines and some non-drug health technologies. Some Canadian provinces have their own HTA processes that support their provincial healthcare system. Both the pan-Canadian HTA agency and some provincial HTA processes have patient and public members in their review or recommendation committees (11).
In this paper, we answer the following research question: “What do members of the Canadian public think about the rationale and design of public involvement in HTA and related funding decision making?” We do this by reporting the views of members of the Canadian public on public involvement in HTA.
Methods
A Systems Thinking Approach
This paper is part of a more comprehensive project that uses Complex Adaptive Systems (CAS) (Reference Plsek and Greenhalgh12) and Soft Systems Methodology (SSM) (Reference Checkland and Poulter13) to examine patient and public involvement in HTA. CAS proponents treat all manner of phenomena as systems composed of diverse, independent, and interacting agents. In SSM, challenges in a system are constructed through the interplay of stakeholders' perspectives or “worldviews” (Reference Checkland and Poulter13). In this project, HTA stakeholders and we, as researchers utilizing SSM, have constructed public involvement in HTA and healthcare funding decisions as a challenging situation (Reference Checkland and Poulter13). The HTA process is considered a system and its stakeholders are agents (healthcare system staff, HTA staff, health industry staff, patient organizations, and public members, patients, and clinicians). The general public is conceptualized as a distal agent, currently acting at the periphery of the system.
We describe findings from two sets of two focus groups conducted with members of the Canadian public to elicit their views on being involved in HTA and healthcare system funding decisions. The first set was exploratory, took place in December 2016 and, after a presentation about provincial healthcare funding decision making and HTA in Canada, participants discussed their potential participation in such processes, including whether the public could participate, at which stages, and through which involvement methods. The second set of focus groups took place in January 2008 and featured the same topics in addition to participants' views on reasons for public involvement. Overall, this approach served to bring out a significant number of ideas (first set of focus groups), which were later explored in more depth (second set of focus groups).
Sampling and Data Collection
We recruited participants via postal invitation. We sent out letters to 500 residents living in a single province in Canada, asking them to answer screening questionnaires online or over-the-phone to determine their willingness and eligibility to participate. We decline to name the province to maintain participants' anonymity. We selected a demographically representative sample of residents to take part in four focus groups (sixteen people for the first set of focus groups and twenty for the second set). We excluded individuals who were members of patient organizations or who had worked (or had relatives who had worked) for health industry companies, government health departments, or healthcare delivery organizations (see the Appendix for details).
Data Analysis
Focus group sessions were audio-recorded and then transcribed. Transcripts were coded using a combination of inductive and deductive coding. Five initial codes were developed based on elements of CAS and SSM: “HTA process,” “Public involvement,” “Patient involvement,” “Relationships,” and “Other stakeholders” and open coding generated a further 178 codes. Codes were collapsed into first-order themes, which numbered eighteen. Codes and first-order themes were then combined into second-order themes based on the theoretical framework, which resulted in five themes: “System,” “Environment,” “Agents,” “Interactions,” and “PPI processes.” Only these themes are reported under broader headings: “Stakeholders,” “Decision Making,” and “Public Involvement Processes” (see the Appendix for details).
Findings
Stakeholders
Participants in all focus groups talked about their lack of trust in various stakeholders involved in HTA and healthcare funding processes. Some participants were concerned that governments' political biases influence decisions. Governments would only benefit groups that would vote for or fund them. Other participants felt that ministers are chosen based on being media-savvy and the ability to win elections leading to poor decision making. Some participants believed that pharmaceutical and health insurance companies have vested interests in the reimbursement of health technologies and that they disproportionately profit from government funding.
… our private health insurance [companies] find themselves somewhat responsible for a few of these drugs… [T]hey have a vested interest in this. We are paying these huge sums of money that goes to a great big pot instead of the government having to take all these responsibilities. (Focus Group Three, participant K)
Many participants believed that pharmaceutical companies pressure governments to reimburse them. Some participants indicated a lack of trust in HTA processes and stated that trust in HTA organizations would have to be built over time. Some participants believed that media outlets might inaccurately report government actions and information from pharmaceutical companies due to their own vested interests.
I feel that you…we have a situation where we get information through the media that may not necessarily be what's going on behind those doors in that decision making, and so that distorts the public view… (Focus Group Three, participant D)
In general, participants held the view that the public is a stakeholder in healthcare-related decisions because they fund the system through their taxes. However, participants also recognized that public participation might increase the complexity and costs of HTA and funding processes. Some participants discussed the difficulty of defining the public as this would depend on how people are selected to take part (e.g., an open invitation to an online survey or selection based on demographic criteria).
Advantages and disadvantages of the public being more closely involved in HTA processes were discussed. Advantages included: the public is unbiased, has no vested interests in comparison with other stakeholders, and has a broader perspective based on their life experiences. However, disadvantages included: the public can base their views only on their preferences (here, “preference” is used in its quotidian meaning as in something that is liked “best”), neglecting other relevant factors, and are not knowledgeable about health technologies, medical issues, or government processes.
Most participants described groups and individuals differently. For example, the public was regarded as impartial when considered as a collective, however individual members were seen as potentially biased toward benefiting themselves, their family, or friends. Similarly, governments were seen as concerned with balancing budgets and with party ideologies, but individual politicians were seen as biased by their personal desire to be re-elected. In general, when considered as individuals rather than as groups, stakeholders were seen as potentially influenced by a variety of personal factors, such as their own beliefs or relationships.
I think whoever heads up a particular ministry or position is trying to form a legacy for themselves. I have sat on so many [name of the meeting] where they have open houses and so on and they don't want to hear from you, they are just there for a photo op or something like that. (Focus Group Three, participant K)
Participants suggested that institutional mechanisms need to be established to avoid individual preferences interfering with population-wide healthcare funding decisions.
Decision Making
Some participants stated that they did not understand the processes employed by the government to make decisions, suggesting that governments should educate, inform, and involve the public in such processes. Local governments were seen as more approachable and trustworthy than the federal government, and participants' trust tended to decrease as the level of government increased. Many participants said that they did not trust governments because they may only consider the interests of special-interest groups, prioritize short-term outcomes, and give too much power to key individuals who can be swayed by their personal or others' vested interests.
The majority of participants believed that the public should be engaged in funding decisions involving HTA; however, there were some dissident voices. Participants in favor of public involvement argued that public values and priorities could counterbalance the values and priorities of other stakeholders. For these participants, there is a need to increase transparency. Factors presented by the public as important should be considered in decision making, leading to greater public acceptance of controversial decisions.
I think though that one thing that comes in the public involvement that does make a difference, it's like the social license to make those decisions. Because more and more technologies become available that are very expensive and if they are not available, people are very upset about that because it's a life and death thing. (Focus Group One, participant K)
The participants who expressed scepticism over involving the public said that committees with no public membership were more knowledgeable of government processes, medical issues and health technologies.
I'm happy to leave it [healthcare funding decisions] to experts because it's just a lot of education and knowledge that goes behind that assessment with all those different technologies, the cost of it and effectiveness… I don't think the general public understands enough about medical technologies to have valuable input. (Focus Group One, participant G)
According to these participants, committees could still take into consideration the values of the public, gathered through research. Some of the reasons not to involve the public included potential delays in decisions and increased costs and bureaucracy. Also, members of the public might be unwilling to take part in involvement processes because they might not be interested or might feel that they lack the knowledge to make such decisions.
The majority of participants recognized public values as being relevant to funding decisions, especially when research evidence is not clear on whether the benefits of a health technology outweigh its harms. Public values were seen as appropriate to guide decisions because the public was seen as independent of the HTA and government processes, offering impartial views.
However, ascertaining the values of the public was not seen as straightforward. Some participants argued that some value judgments might change over time and depend on the type of health technology, whereas others may remain the same across different contexts. Regular consultations were proposed to determine the values of the public and to check whether government decisions were in line with public priorities.
Personal values were seen to have the potential to negatively impact decision making, particularly where they supported benefits for some groups to the detriment of others. Some participants argued that the personal values of clinicians, researchers, politicians, and health industry staff might be at odds with public values and priorities.
…scientists make their research… they already have a particular idea about what is valuable to research about… you already would have in some sense, thought about disease A if it is eradicated, for example. (…) But, that's a value that the bureaucrats or whoever sets the agenda bring to the table… I think that's problematic because, if they belong to a particular religious group, that might affect the value they place on sickness or the kind of way they think about that… Perhaps that's where public participation is important because the public needs to go and say, “No, that is not the value we want, what we want is this” to prevent the possibility that the bureaucrats will be bringing their own values and take it to be the values of the general Canadians. (Focus Group Three, participant E)
In this sense, the public values were seen as able to offset the personal and group values of other stakeholders involved in HTA and funding decisions. The public was seen as not having a vested interest in the outcome of a specific healthcare funding decision, whereas all other stakeholder groups were seen as having a distinct stake in the outcome.
Public Involvement Processes
Participants discussed the stages at which public involvement would provide meaningful information to HTA processes and health-funding decisions. Some participants suggested that public perspectives could be included in setting priorities for HTA and establishing criteria for making decisions on whether health technologies should be publicly reimbursed. The underpinning rationale was that these areas especially involve value judgments, and the public can contribute more impartial value judgments than the other stakeholders. Additionally, public involvement was seen as useful at the assessment stage for controversial, value-laden health technologies by providing public views on whether these should be publicly reimbursed.
Participants discussed various methods of public involvement with different types of processes considered more appropriate for specific objectives. Face-to-face methods would be most appropriate for value-laden questions or in developing criteria to guide policy makers in priority setting, reimbursement recommendations, and collecting information for the assessment of a health technology. Surveys were seen as appropriate when governments need to elicit public priorities with respect to a pre-determined list of options. Table 1 lists the involvement processes suggested by participants.
Involvement processes suggested by participants
Some participants indicated that survey design factors could impact the results (how they are designed, delivered, and reported). Some participants suggested that the selection of participants for face-to-face involvement processes should be demographically based and able to ensure that the public is not influenced by other stakeholders (monetarily or with misinformation). Various participants talked positively about citizens' juries because their structure is similar to that of legal juries, with procedures that they are familiar with and trust. The timing of involvement was seen as important because it could potentially impact on people's trust in the process (e.g., if it occurs too late, the public may perceive it as tokenistic).
In all of the focus groups, information exchange between the public and HTA and funding decision stakeholders was seen as valuable. The public can provide information to HTA and decision-making processes regarding public values, which can improve decisions. In a reciprocal manner, the public can be informed about government processes, health technologies, health conditions and diseases, and the ethical and financial issues regarding some decisions. However, this two-way sharing of knowledge was considered a challenge by some participants. The main problems identified related to cost and issues in effectively and impartially translating technical information for a public audience.
…[N]o matter how you set up a board or however you make a decision…, you are going to end up pissing somebody off, right? Something is not going to be covered. Potentially, there is going to be room to learn; there is going to be room to grow. I think the more that communities and governments allow for that room to grow and evolve, the more that can be kind of a comfortable thing to know that you can say: “Ok, well maybe we are not making this decision the best way, let's try something else, instead of being like so rigid about it, I think that makes a difference.” (Focus Group Three, participant K)
Discussion
Our findings highlight that public concerns related to being listened to when it comes to specific funding decisions and a lack of trust in institutions and HTA stakeholders depending on how processes are managed. The main issues identified were a lack of understanding of the rationale for funding decisions, a lack of transparency in decisions processes, and distrust with respect to whose interests and values are being considered. Fostering public involvement was seen as a way of providing information to the public and a way to increase the diversity of social values to be included in decision making. In the scholarly literature (Reference Grimmelikhuijsen, Porumbescu, Hong and Im14), trust in government (or political trust) has similarly been linked to the transparency of government processes.
To understand how transparency plays into public trust and the social license that underpins public acceptance of government processes, we can turn to the study of Grimmelikhuijsen et al. (Reference Grimmelikhuijsen, Porumbescu, Hong and Im14). It highlights three features of transparency relating to the information made public: completeness (whether people have access to all of the information available); color (whether those releasing the information have reported it positively or negatively depending on their interests); and usability (how easy the information is to understand). Our findings suggest that, when it comes to healthcare funding decision making, at least some members of the Canadian public are dissatisfied in relation to all three elements: information about processes is too incomplete, too colored, and too hard to understand.
Job (Reference Job15) explains how both “rational” and “relational” trust-building theories may play a role in explaining public trust in government. Job defines political trust as the “attitudes people have towards the future actions of government, government organizations, and the people who administer those abstract systems” (p. 3). Rational trust theories contend that trust is built in response to government performance, whereas some relational theories contend that trust is built on the basis of people's cultural norms, learning experiences and beliefs that are projected onto political institutions. Participants in our study presented reasons for their distrust that were both rational (governments are visibly influenced by others when managing public money) and relational (participants trust the federal government less than provincial and municipal governments because they feel the federal government is more distant from them). Low levels of political trust have been linked to lower levels of law compliance by citizens and potential problems in governability (Reference Marien and Hooghe16). In HTA, for example, people who distrust government healthcare policy-making processes may protest for changes when particular health technologies are not approved for funding.
Participants in our study identified the diverse “worldviews” of stakeholders in HTA and related healthcare funding decisions. “Worldviews” are defined as the assumptions people make about the world based on their experiences, genetics, and contextual influences (Reference Checkland and Poulter13). When a challenge arises, worldviews come into play in how the situation and its implications are seen by stakeholders. Challenging situations cannot be “solved” because the problem is seen differently by diverse stakeholders, but accommodations between the different worldviews can be achieved. Poetz (Reference Poetz17) argues for the use of relationship management skills when it comes to public involvement around controversial issues that include technical or scientific information and cautions against striving for consensus in favor of simply finding ways to move forward.
Our findings suggest that the worldviews and interests of diverse stakeholder groups could potentially be at odds with what the public believes would be best for society. Governing political parties can be viewed as focusing on re-election, whereas bureaucrats can be viewed as overly concerned with cost savings at the expense of other public priorities. Health-related companies are seen as trying to increase their profits at the expense of others and undermining public involvement initiatives. Researchers are seen as having interests in specific topics of research that may not align with the views of other stakeholders.
Moe (Reference Moe18) recognizes that the public is indirectly invested in the decision by having an interest in how taxpayer money is used by the government and by having the potential to benefit in future from health technologies. Citizens' interests are “diffuse” (Reference Moe18) compared to the interests of other stakeholders, who receive direct benefits. In our research, participants presented the public interest as diffuse when the public was considered as a group. For this reason, many participants regarded the public as able to provide an outside perspective, in contrast with the government, clinicians, academics, and the health industry. Participants, however, also depicted the public as naïve and at risk of being manipulated because it generally lacked clinical and political expertise. In this context, public values can be understood as (Reference Bozeman19):
(…) providing normative consensus about (a) the rights, benefits, and prerogatives to which citizens should (and should not) be entitled; (b) the obligations of citizens to society, the state, and one another; and (c) the principles on which governments and policies should be based. (…) Citizens can hold a public value that is not the same as their own self-interested private value.
Our findings indicate that participants looked beyond collective vested interests, pinpointing how individuals' worldviews and interests could affect decision making. To overcome this, participants suggested that decision-making processes needed a “check and balance” procedure and that the design of public involvement processes also needed safeguards against individual interests and values.
Participants in our focus groups reviewed the benefits and drawbacks of diverse methods, as can be seen in Table 1. Based on these considerations, most participants agreed that public involvement processes should employ a mixed-methods strategy to support the participation of a large number of people through surveys as well as informed public debate using deliberative processes. The representative methods (e.g., surveys and online vignettes) allow the views of a large number of people to inform decisions, preventing personal biases becoming prevalent in discussions. Participatory methods (e.g., citizens' juries, focus groups, and community forums) make it possible for members of the public to understand issues in-depth and to contribute more considered views, which may then carry the same weight provided by other well-informed stakeholders. Some challenges can arise relating to each type of democratic rationale when used to underpin involvement processes, according to Bevir (Reference Bevir20,Reference Bevir21). In a representative democracy, the public would not have access to sufficient information to present considered viewpoints; and in a participatory democracy, only a select group would participate, excluding most people from taking part in decisions. In line with modern democratic theory debates, our research highlights the tension between having direct participation from anyone and public involvement in which some procedures are in place to ensure that people can debate issues in facilitated spaces and present informed and considered views. One possible way forward is to use the mixed-methods strategy suggested by our participants.
Lopes et al. (Reference Lopes, Carter, Street, Stafinski and Merlin22) also investigate how public involvement affects a lack of trust in government:
Involving the public in policy development and in making decisions can help foster closer links between citizens and governments. Such processes, however, need to be conducted in a way that is seen positively by participants as their view of the process can influence their opinions of government trustworthiness. Governments would also benefit from developing policies that increase the transparency of policy decisions (p. 21).
Fox (Reference Fox23) explains that social accountability involves citizens monitoring government and private sectors using a series of methods, such as the audit committees suggested by our participants. Social accountability can be divided into two types: tactical (tools are used to give citizens a “voice”) and strategic (various mechanisms are used to foster the necessary conditions for that to happen) (Reference Fox23). Strategic social accountability can be achieved when various stakeholders are involved, including the public: explicit consideration of diverse worldviews would enable deliberation on various issues that could affect government and healthcare funding processes. Our study participants argued that public involvement is a desirable feature of healthcare funding decision processes. However, other stakeholders may object to involving the public because they may be unsure about whether it is helpful, cost-effective, or too time-consuming. These concerns were also raised by some participants, who highlighted that the public was not knowledgeable about government or HTA processes. Martin (Reference Martin24) brings a relevant perspective to this debate when discussing the involvement of AIDS activists during drug trials. As members of the public, these activists were able to bring to the attention of health professionals and researchers “human values” that entailed trade-offs between scientific pragmatism (which could help alleviate suffering for some patients) and the scientific purity (keeping on with strict eligibility criteria for trials) that could still move research and care for patients forward (p. 38). There is little evidence in the healthcare literature about the benefits of involving the public in policy development. However, other fields, such as environmental sciences, have a long tradition of conducting and evaluating public engagement initiatives and have ascertained some benefits of these processes to the development of government policy (Reference Côté and Bouthillier25).
From an SSM perspective, the views of all stakeholders should be considered when implementing changes to arrive at outcomes more likely to be acceptable to all, while building trust between those involved.
Conclusion
Our study found that members of the public in one Canadian province do not understand government decision-making processes and are suspicious of the interests driving the actions of various stakeholders involved in HTA processes and healthcare funding decisions. These findings demonstrate that governments can be seen as mostly just fighting to remain in power and pharmaceutical and insurance companies as mostly just wanting to turn a profit. The public, however, with its diffuse interest in funding decisions, can be regarded as an independent party that can contribute to HTA decision making with impartiality, notwithstanding our participant's recognition of the public's lack of knowledge about health technologies. Differences in levels of trust were also evident when participants talked about individuals as opposed to stakeholder groups, with individuals being viewed as overwhelmingly swayed by their own personal interests and stakeholder groups as being better able to seek broader benefits. Accordingly, participants reasoned that decision making should include public values to counterbalance the values of other stakeholder groups and mechanisms should be put in place to prevent individuals from making decisions that would unduly benefit themselves.
Members of the public believe that public involvement will help to incorporate public values into HTA decision making, although there are some recognized drawbacks, such as potential delays in decisions or increases in expense and bureaucracy.
Participants suggested various public involvement methods, such as surveys, focus groups, citizens' juries, and audit committees. Independent of which method is used, easy and timely access to relevant information and a rigorous selection process were considered essential to ensure diverse perspectives are included. An ideal strategy would include the collection of public priorities from large numbers of people, together with in-depth discussions from smaller groups of people. Another possibility could be to use the findings from an in-depth participatory method to spark wider public debate with a more representative method, such as a large-scale survey.
Public involvement should be comprehensive (occurring at multiple stages of the funding process) and accessible (reaching a wide range of people, providing reliable information, and an opportunity for deliberation) to be able to deliver valuable input to healthcare decision-making processes. Involving the public as one stakeholder of HTA and related funding decision processes can broaden policy makers' awareness of different values to be considered and potential implications of decisions that are not foreseen by other stakeholders. It will also allow the public to take part in government processes that will affect them. As part of a system, HTA could be used alongside social value judgments at different stages of health technology development, such as at the design of research projects and the real-world use of new technologies.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462320000537
Acknowledgments
The authors acknowledge the contribution of Andrea Young for administrative support. EL received funding through an Endeavour Fellowship by the Australian Department of Education and Training, an Australian Post-Graduate Award from the University of Adelaide, and funding from the University of Alberta.
Conflict of interest
Dr. Stafinski reports participation in Health Technology Assessment for the pan-Canadian and provincial processes but declares that she does not have a conflict of interest with this project. Prof. Merlin reports undertaking commissioned research for the Australian Government Department of Health, outside the submitted work. The Department of Health had no role in the funding, conception, design, analysis, or interpretation of the submitted work. Ms Lopes reports grants from Australian Government (Endeavour Award), non-financial support and other from University of Alberta (Health Technology and Policy Unit), during the conduct of the study. Dr. Carter reports grants from Australian Government (MSAC and PBAC), outside the submitted work. All other authors have nothing to disclose.
