A 2005 survey of members of the International Network of Agencies for Health Technology Assessment (INAHTA) suggests that the idea of public involvement in health technology assessment (HTA) is gaining momentum (Reference Hailey and Nordwall12). As evidence, the survey indicates that 57 percent of agencies now involve consumers in some aspect of their activities and 83 percent intend to do so in future. That said, not all HTA agencies have embraced the idea of public involvement and no empirical study has yet explored the factors that affect its current and future prospects.
Drawing on the agenda-setting work of John W. Kingdon (2003), we explore through a political science lens the factors that influence the prospects for public involvement in the activities of HTA agencies. To do so, we conducted and analyzed forty-two semistructured telephone interviews with informants in international HTA networks and/or HTA agencies in Canada, Denmark, and the United Kingdom. For the purpose of this study, we defined public involvement as the passive and active procedures used by an HTA agency to interact with the public and its representatives (i.e., citizens, groups representing citizens, patients, and service users, and groups representing patients and service users).
BACKGROUND
The international HTA community's interest in public involvement can be traced back to over a decade ago. In 1998, the International Journal of Technology Assessment in Health Care devoted an issue to consumer involvement, looking, inter alia, at the evolution of consumer advocacy, patients' perspectives on the evaluation of health care, and materials that presented HTA findings to consumers. In 2005, members of Health Technology Assessment International (HTAi) established a subcommittee to explore how the views of the public and the needs and preferences of patients could be incorporated into HTAs so as to inform decisions about the introduction and diffusion of health technologies (13). In the same year, INAHTA surveyed its members regarding their consumer involvement practices (Reference Hailey and Nordwall12), and subsequently, INAHTA created an ethics working group that explores participatory approaches to address ethical issues in HTAs (14). In 2007, the European Network for Health Technology Assessment established a stakeholder forum to dialogue with stakeholder groups about, among other things, the patients' roles, needs, and demands in relation to HTA.
Several dynamics promoted the idea of public involvement over this period. Scarce resources and rapid technological changes confronted policy makers with increasingly complex and contentious coverage decisions (Reference Chinitz6;Reference Lehoux17), causing public involvement to emerge as a means of achieving more informed, transparent, and politically legitimate decisions about the use of health technologies (Reference Abelson and Giacomini2). Academic contributions from the fields of bioethics, the philosophy of science, and social studies have also legitimized public input into HTA (Reference Coulter7;Reference Lehoux17;Reference ten Have25).
The HTA community itself, however, is divided about the meanings and merits of public involvement (Reference Gauvin, Abelson, Giacomini, Eyles and Lavis11). For example, an international study of organizations producing clinical practice guidelines and conducting HTAs found that they rarely considered relationships with consumer groups to be particularly important or valuable (Reference Moynihan, Oxmand, Lavis and Paulsen21). In contrast, others found that HTA agencies valued public involvement but stated that practical challenges, such as limited resources and pressure to produce their recommendations quickly, impeded them from pursuing it (Reference Hailey and Nordwall12).
METHODOLOGY
To explore the prospects for public involvement in HTA, we conducted a thematic analysis of semistructured telephone interviews with informants who were actively involved with international HTA networks and/or with HTA agencies in Canada, Denmark, and the United Kingdom. These countries were of particular interest because they are pioneers in the field of HTA (Reference Banta3;Reference Jorgensen, Hvenegaard and Kristensen15;Reference McDaid19), they provide universal healthcare coverage, and their recent healthcare reforms and commissions of inquiry have reinforced public involvement (Reference Abelson and Gauvin1;Reference Florin and Dixon9;Reference Strandberg-Larsen, Nielsen and Vallgarda23).
This study is exploratory in nature. It draws evidence from a small number of countries that vary tremendously in terms of cultural, political, and institutional contexts which limits the transferability of our findings to other jurisdictions. However, we believe these limitations are offset by the selection of informants who are part of extensive international networks, which can be helpful in identifying patterns across the HTA community as a whole. Detailed findings specific to each agency and jurisdiction are reported elsewhere (Reference Gauvin10).
Between February 2006 and November 2007, we conducted 42 telephone interviews in English (n = 37) or in French (n = 5). Our informants were all professionals closely associated with HTA agencies: senior HTA staff members, scholars, policy makers within government ministries of health, and members of patient and service user groups (Table 1). We used a purposeful sampling strategy to select these informants. Names were identified from a preliminary literature review, a review of organizational charts, and from the leads of other informants. Interviewing informants from each of the four groups allowed us to achieve the dual goal of interviewing people with different perspectives on the topic and to conduct data source triangulation, that is, exploring the factors that influence HTA agencies' decision to adopt, or reject, public involvement from different perspectives. We used the same interview guide for all interviews, but it was used flexibly to allow us to probe interviewees' responses and follow new leads.
Description of Key Informants
HTA, health technology assessment.
Following the conventions of qualitative research, data collection and data analysis occurred iteratively (Reference Strauss and Corbin24). Interviews were systematically transcribed using word processing software and were managed with QSR NVivo®. We used a coding scheme derived from the agenda-setting framework developed by John W. Kingdon (Reference Kingdon16) to analyze the data. Kingdon suggests that agenda of a government or public organization is influenced by the interplay of three streams of factors that can either push an issue on the agenda, or prevent an issue from reaching that agenda. The three streams are the recognition of problems, political events, and the generation and diffusion of ideas within a policy/research community (Table 2).
Factors in Kingdon's Agenda-Setting Model
HTA, health technology assessment.
The validity of the analysis was assured by following the tenets of induction. We used our prior knowledge of the literature as a framework for exploring agenda-setting dynamics revealed by our data (Reference Morse and Mitcham20). Inductive validity was partially ensured by our selection of informants with diverse perspectives and by the process of saturation, that is, we conducted interviews until themes suggested by informants began to repeat themselves and subsequent informants' interviews yielded no new themes.
In discussing our findings, we provide quotes to illustrate the themes that emerged. To protect confidentiality, quotes only identify informants by group membership. This study was approved by the research ethics board at McMaster University and all participants agreed to participate on a voluntary and informed basis.
FINDINGS
When asked whether the HTA community is heading toward greater public involvement, our informants generally agree that the HTA community is cautious and ambivalent about the prospects for public involvement. Many informants acknowledge, however, that HTA agencies' attitudes and approaches vary greatly.
“I think we, the HTA community, [are] moving cautiously towards greater involvement. There is still quite a diversity in the approaches and the attitudes with the different programs and agencies.” – Scholar 2
To explain why the HTA community is so prudent, our informants refer to several factors, some of which enhance the prospects for public involvement and others which hold them back.
Factors That Enhance the Prospects for Public Involvement
When asked whether the issue of public involvement is gaining prominence in response to specific problems, informants are hesitant, sometimes even reluctant, to discuss problems specific to their agency. Nonetheless, they identify several ways that the public could help solve problems affecting the HTA community as a whole. In light of the increased costs of health technologies and the rapid development of new technologies, for example, they stress the importance of involving the public so as to render coverage decisions more legitimate.
“The need to make tough decisions becomes more and more important. And we don't have a choice. [We need] to involve the public in these tough decisions.” – Member 1 of HTA agency
Informants recognize that because the HTA process often requires complex value judgments, gaps between the perspectives of HTA practitioners and the perspectives of the public could produce unsatisfactory decisions. This highlights the need to find ways to broaden the evidentiary base for HTA and consider the public's perspectives during the assessments.
Another factor is the effect of policies that have created a cultural and political climate favorable to public involvement in the healthcare sector. Informants remark that over the past 10–15 years, the public has been increasingly involved in healthcare governance, resource allocation, research, and decision making about patients' own care.
“It's an international trend . . . how to [involve] the public more in what's going on in healthcare.” – Member 8 of HTA agency
Many informants suggest that this climate has caused international HTA fora to initiate a variety of initiatives to debate public involvement. However, some initiatives have faced significant resistance within the HTA community. As one informant contends, initiatives to create an interest group in consumer involvement within an international network were “sabotaged” (informant's term) for several years.
When asked how public involvement could figure more prominently on the agenda of international HTA meetings, informants suggest that the advocates of greater public involvement need to argue their case more effectively. They also argue that leadership by influential members of the HTA community and government officials is essential to moving public involvement forward.
“[If this governmental committee had] not been there, we would have met nothing but hostility and resistance. [. . .] You do need positive encouragement from the very top of the organization or the government. You need people on the ground who are prepared to champion it and find some money and some time, putting the effort and energy in.” – Member 12 of HTA agency
A few informants worry that the commitment to public involvement could drop after a change in leadership within the HTA community or after a change of government officials. This concern suggests that the idea of public involvement is linked to certain leaders, and that future cohorts may not commit to it.
“I am a little anxious at the moment that [. . .] we are in danger of moving backwards and the commitment to user involvement may back off. [. . .] When the leadership changes, it isn't that the policy changes entirely, but you just see a dip in the priority attached to it.” – Policy-maker 8
A factor that clearly favors public involvement is the adoption of public involvement schemes by influential HTA agencies. Several informants remark that HTA agencies have a long tradition of emulating agencies in other jurisdictions and that HTA practitioners look outside their immediate milieu for ideas and norms. In fact, most informants suggest that the greater HTA community, which is organized around key international networks such as IJTAHC, HTAi, and INAHTA, is vital to legitimizing and popularizing ideas. These networks have been influential in creating a close-knit international HTA community with a strong sense of identity. As evidence, many informants refer to the international HTA community in the first person: “we.” For that reason, they expect public involvement to diffuse more rapidly once a critical mass of agencies has adopted public involvement schemes.
“When you [pass] the first threshold, then it begins to move quicker.”
– Member 9 of HTA agency
Factors That Reduce the Prospects for Public Involvement
Although several factors appear to enhance the prospects for public involvement in HTA, the idea continues to face considerable obstacles. First is the belief, shared by a few informants, that the discipline of HTA is relatively new and, consequently, theoretically immature. They argue that the absence of strong theoretical foundations for HTA generates uncertainty about the purposes and methods of public involvement in HTA.
“What is the theoretical model that we want to guide public participation? We need a theoretical model. However, I haven't seen any reference to a model in the field of health technology assessment.” – Member 4 of HTA agency
Second is the discipline's developmental path. Many informants suggest that HTA agencies have been principally concerned with synthesizing the evidence of a technology's clinical and cost-effectiveness. In this sense, these informants point out that the HTA community has been greatly influenced by the quantitative and positivist paradigm promulgated by the Cochrane Collaboration and the evidence-based medicine movement, both of which emphasize experimental methods, the hierarchization of evidence, and systematic reviews. Because most public input is qualitative and/or ad hoc, therefore, it has been devalued. Consequently, some members of the HTA community are apprehensive or outright opposed to involving the public in what they portray as a specialized domain.
“I think there's a big problem. The general public doesn't understand the sort of analysis that is lying within technology assessment. [. . .] I think it's difficult for the untrained or the general public to actually have a useful view on that.” – Member 9 of HTA agency
A few informants note, however, that there has been a slow and gradual shift in recent years toward incorporating patients' and service users' values and preferences into HTAs, the Cochrane Collaboration, and the evidence-based medicine movement. They believe this shift may be the handiwork of social scientists and ethicists, whose use of experiential knowledge and understanding of social and personal contexts has done much to substantiate the value of patients' and service users' perspectives to HTA.
The third obstacle to greater public involvement cited by some informants was the lack of public awareness of the existence of HTA agencies. They believe that the HTA community is mostly involved in academic activities that garner little public attention. They are not convinced that HTA agencies' creation of opportunities to involve the public would actually result in greater public involvement—that the public would be interested in taking part in the activities of HTA agencies even if they knew about them.
“[The public] doesn't know that there is something called the [name of HTA agency]. They haven't got a clue! I don't blame them.” – Member 7 of HTA agency “Most people are completely unaware that it happens. [HTA] is very much academic.” – Member 5 of patient and user group
That said, some believe that the HTA community is responsible for nurturing public interest.
“[You must create] a public wish, a public commitment, to be part of the process and to understand the importance of making decisions in this way. Life is complicated and many people think: ‘Well, there are lots of things I ought to be interested and involved in but frankly, I'd rather be playing football or watching television.’” – Member 7 of patient and user group
When asked whether the interest in public involvement is a response to pressures from patient and user groups, many informants downplay such influence. In fact, they argue that these pressures could have the contrary effect: HTA agencies that are under such pressures could pull away from public involvement altogether. They explain this because of the fear HTA agencies have of involving groups that are possibly affiliated with the health technology industry, which could threaten their scientific credibility and political autonomy.
“Many of these social changes can be very dependent on a few powerful leaders. [. . .] A lot of the progress has come from researchers and scientists themselves. It's not all been push from the patient side.” – Policy-maker 8
The fourth obstacle, mentioned by many informants, is the expense and time required to elicit public perspectives. Even modest public involvement activities can be very time consuming. Meanwhile, decision makers demand that agencies make assessments rapidly. This pressure for quick and efficient HTAs threatens the viability of public involvement. A few informants expressed the hope that the research cycle would decelerate, failing which, public involvement opportunities would continue to be sacrificed in favor of rapid assessments focused on clinical and cost-effectiveness.
“One of the things that I would like to happen is actually for the research cycle to slow down a little bit. [It] is very difficult to involve the public when everything moves so fast. [. . .] The pace at which they work is just horrific and if you don't have the evidence on the table at one meeting you've missed it. It's too late to bring it to the next. [. . .] It's overwhelming and it means that thoughtful input is lost.” – Scholar 11
In addition, meaningful public involvement requires human and financial resources. Although governments have adopted policies and guidelines to promote public involvement, agencies rarely receive the funds to implement them. Costs must therefore be paid from current budgets, consuming resources usually devoted to conventional HTA activities.
“People are saying, ‘Oh, it's very important you must involve members of the public,’ but they don't say, ‘Oh, and here's some money to help.’” – Member 11 of HTA agency
Fifth, many informants appear to be puzzled about the technical feasibility of public involvement and the most appropriate means of “capturing” (informant's term) public views. The newness of the idea has brought uncertainty and many informants are concerned that it could undermine the efficiency of current processes.
“The practicalities of putting it in place aren't trivial.” – Scholar 2
“The fear of incorporating patients, it's the unknown. . . What's going to happen? Our meeting is going to last twice as long to try and bring patients up to speed? Are they going to be able to hold up their end?” – Member 1 of patient and user group
Many informants lack evidence that current public involvement initiatives make the HTA process, the uptake of HTA findings, and policy decisions more effective. One informant argues that, without this evidence, not only does uncertainty remain, but it becomes more difficult to convince those who are committed to evidence-based decision making and apprehensive or resistant to public involvement.
“Part of the problem is trying to demonstrate that it has made a difference.” – Member 15 of HTA agency
Sixth, the lack of clarity with regard to the concept of public involvement limits HTA practitioners' ability to grasp—and espouse—the idea. Many of our informants grapple with fundamental questions such as who is the “public”? Who is representative? What does “involvement” mean? However legitimate these questions, some informants believe that they are used to avoid public involvement and nurture a never-ending, rhetorical debate.
“I think there can be people who go, ‘Ah, but what is the public?’ It can be a kind of ‘How many angels can dance on the head of a pin’ question. So, let's debate what the public is for the next 10 years so we don't have to involve them in anything.” – Policy-maker 8
In the same vein, a few informants suggest that there is still a “fair amount of lip service” (informant's term) paid to the idea of public involvement. These informants voiced skepticism about the HTA community's commitment to public involvement, portraying it as a symbolic concern that stopped well short of investing the public with meaningful power.
DISCUSSION
Although members of the HTA community expect the idea of public involvement to gain prominence in the future, this study suggests that they remain cautious and ambivalent. The study also suggests that it is difficult to determine the place of public involvement on HTA agencies' agenda. This is consistent with Kingdon and other scholars who observed that issues may drift on and off the agenda, making it difficult to determine their status (Reference Baumgartner and Jones4;Reference Kingdon16). Our findings corroborate these observations in that the practice of public involvement in HTA has proven to be fairly unstable, even in those jurisdictions that have adopted supportive policies and guidelines. Pro-public involvement policies and guidelines create the impression that the issue is firmly established on the agenda. However, our study reveals that the idea is actually unstable when viewed over a longer time horizon and can drift off the agenda.
The findings presented here suggest four key factors that may enhance or reduce the prospects of public involvement. The first key factor from the policy stream relates to the role of the international HTA community in diffusing ideas. Our study suggests that the international HTA community is closely knit but lacks a cohesive paradigm regarding HTA and public involvement. This lack of cohesion is evidenced by the absence of common language, but also common outlooks and orientations around public involvement in HTA (Reference Gauvin, Abelson, Giacomini, Eyles and Lavis11). This finding is consistent with scholars' comments that HTA lacks a coherent theoretical foundation (Reference Lehoux17;Reference Leys18). Thus, the absence of convincing theory to guide public involvement in HTA makes many agencies and practitioners reluctant to address the issue.
Indeed, the idea of public involvement remains contentious among members of the international HTA community. Until the feasibility, acceptability, and potential consequences of public involvement are addressed among their peers, some HTA agencies may be reluctant to address the issue. In contrast, the idea is likely to diffuse much more rapidly once a critical mass of practitioners and agencies has adopted a public involvement scheme and recognized public input as valid and legitimate evidence for HTA, as the international HTA community has historically evolved through emulation.
The second key factor from the problem stream relates to the recognition of problems facing HTA agencies. The study confirms that members of the HTA community are protective of the efficiency of their processes and procedures (Reference Hailey and Nordwall12). They find it difficult to conduct their activities with constrained budgets while responding to government officials' and other stakeholders' demands to produce reports more rapidly than ever. That the time and resources necessary to involve the public could reduce the efficiency of the HTA process, causes some agencies to pull back or avoid public involvement altogether.
This last finding stresses the importance of conducting rigorous and compelling evaluations of public involvement. These evaluations could focus on public involvement processes (e.g., their fairness, flexibility, and transparency) and their impacts on HTAs (e.g., how public involvement affects the HTA process and the time required to develop HTA reports). Only a very small proportion of agencies that involve the public appear to have evaluated their practices in this regard (5;Reference Davies, Wetherell, Barnett and Seymour-Smith8;Reference Hailey and Nordwall12;Reference Oliver, Armes and Gyte22). Such evaluations could inform agencies' decision to adopt, or reject, public involvement practices.
A third key factor from the politics stream refers to the need for political leadership. As Chinitz (Reference Chinitz6) suggested, there is often a “temptation to view HTA as being disconnected from politics or depoliticized” (p. 55). Although HTA agencies evolve in a relatively autonomous policy niche, this study illustrates that governments steer their agenda to some extent through accountability links, funding arrangements, and policies. Governments can encourage HTA agencies to adopt a public involvement scheme by creating a favorable political climate, adopting public involvement policies, and allocating the necessary resources to support HTA agencies to involve the public.
Finally, a fourth key factor relates to the role of people who are credible and in positions of authority within the HTA community. As our findings illustrate, they can influence HTA agencies' agendas by pushing the idea of public involvement forward. The downside of their advocacy is that the idea could become so intimately linked to them as individuals that a change in leadership (e.g., a new appointment, staff turnover, or retirement) can affect the salience of the issue and ultimately cause it to fall off the agenda.
CONCLUSION
This exploratory study suggests that members of the HTA community expect the idea of public involvement to gain prominence in the future. However, given the uncertainty about the technical feasibility of public involvement, the risks of its unacceptability and the possibility of future constraints, the HTA community is expected to move cautiously and incrementally along this path. The evaluation of current experiences with public involvement in HTA would usefully inform these decisions.
CONTACT INFORMATION
François-Pierre Gauvin, PhD (francois-pierre.gauvin@inspq.qc.ca), Research Officer, National Collaborating Centre for Healthy Public Policy, Institut national de santé publique du Québec, 945 avenue Wolfe, Quebec City, Quebec G1V 5B3, Canada
Julia Abelson, PhD (abelsonj@mcmaster.ca), Professor, Clinical Epidemiology and Biostatistics, McMaster University, CRL Building, 2nd Floor, Room 227, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada; Director, Centre for Health Economics and Policy Analysis, CRL Building, 2nd Floor, Room 227, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
Mita Giacomini, PhD (giacomin@mcmaster.ca), Professor, Clinical Epidemiology and Biostastics, McMaster University, CRL-218 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
John Eyles, PhD (eyles@mcmaster.ca), Professor, School of Geography and Earth Sciences, McMaster University, 1280 Main Street West, GSB 217, Hamilton, Ontario L8S 4M4, Canada
John N. Lavis, MD, PhD (lavisj@mcmaster.ca), Professor, Department of Clinical Epidemiology and Biostatistics, and Department of Political Science, McMaster University, 1280 Main Street West, CRL-209, Hamilton, Ontario L8S 4K1, Canada; Director, McMaster Health Forum, 1280 Main Street West, MML-417, Hamilton, Ontario L8S 4L6, Canada
CONFLICT OF INTEREST
All authors report having no potential conflicts of interest.
