The focus of health technology assessment (HTA) has often been on stakeholders with major financial or operational responsibilities for health technologies, including government authorities, manufacturing industry, healthcare professionals, and the health insurance industry. Patients and their families and carers are also a key group with major interests in health technologies, but until quite recently limited attention has been given to involving them in the HTA process (Reference Hailey1). Criticism of the situation included comment that health experts such as physicians, researchers, and economists largely dominate HTA decision making. This was seen as antithetical to the belief that acceptable health decision making requires a process that is transparent, not dominated by any particular interest and reflects the values of all users (Reference Pivik, Rode and Ward2).
In 2005 the International Network of Agencies for Health Technology Assessment (INAHTA) prepared a review to help consideration of how it might encourage more extensive involvement of patients in HTA (Reference Hailey1). Involvement of patients in formulation of questions, in assessments and in dissemination had received some consideration in the literature, although there was little that specifically related to the situation typically facing HTA agencies of routinely producing and disseminating advice for decision makers in health care, often with tight deadlines. Some of the literature on involving consumers with health research provided some issues to consider (Reference Ghersi3–Reference Telford, Boote and Cooper6). Common themes included interaction of consumers and researchers, resources, technical demands on consumers, training and education, nature and extent of consumer representation, matching consumer information and information from the literature, time demands and remuneration, and consumer impact on discussion and decisions.
One of the earlier reports of consumer involvement in HTA gave details of approaches used in England by the National Coordinating Centre for Health Technology Assessment (NCCHTA). These included attending prioritization meetings, and commenting on research need for particular topics, research proposals, and research reports (Reference Royle and Oliver7).
Later that year members of INAHTA were surveyed to obtain information on their involvement with consumers. The term “consumers” was taken to include patients, carers, long-term users of services, organizations representing consumers’ interests, and members of the public, following a definition developed by the English HTA program (Reference Royle and Oliver7). The survey found that most INAHTA members interacted with consumers in some aspects of their programs, although not always routinely. Involvement seemed likely to increase in the future (Reference Hailey and Nordwall8).
These activities of INAHTA overlapped with the creation of the Health Technology international Interest Sub-Group (ISG) for Patient and Citizen Involvement in HTA. The ISG has since developed as a strong forum for promotion of processes for incorporating patients’ perspectives into HTAs and engaging patients in the HTA process (Reference Facey, Boivin and Gracia9). There has been continued contact between INAHTA and the ISG.
Some more recent literature has further explored issues and practicalities associated with the engagement of patients and the community in the HTA process. Lee et al. have drawn attention to issues with HTA reports that included patient-related assessments (Reference Lee, Skött and Hansen10). There was considerable variation in how comprehensively the issues were managed and in the methodological transparency of the reports that they reviewed, giving uncertainty as to their relevance and validity. Gagnon et al., in a systematic review, conclude there are few published examples of experiences of involving patients and the public in HTA (Reference Gagnon, Desmartis and Lepage-Savary11). They see a need to develop more systematic approaches to considering patients’ and the public's perspectives in HTA.
Recent articles suggest there is some progress in the use by HTA agencies of input from patients or the community but that further evaluation of process and effectiveness is needed (Reference Gauvin, Abelson, Giacomini, Eyles and Lavis12;Reference Menon and Stafinski13). From a study based on interviews with people in HTA agencies, Gauvin et al. found that the HTA community is moving toward greater public involvement but remains cautious and ambivalent about the technical feasibility of public involvement, acceptability to policy makers and practitioners, and its impact on agencies’ resources and procedures. They suggest there is a need for rigorous evaluations to inform HTA agencies’ decisions on adoption or rejection of public involvement practices (Reference Gauvin, Abelson, Giacomini, Eyles and Lavis12).
In 2010, a follow-up survey was undertaken by INAHTA to obtain information on whether its member agencies had changed their approaches in involving consumers since the earlier survey. This initiative reflected the increasing interest that has been expressed in gathering evidence about patients’ perspectives, and in ensuring their engagement in the HTA process (Reference Facey, Boivin and Gracia9). In this article, we present the main findings from the survey and a comparison with the earlier results.
METHODS
The questions and the definition of consumers used in the earlier survey were considered to still be appropriate and were used again for the follow up. The opportunity was taken to liaise with the HTAi ISG on Patient/Citizen Involvement in HTA on details of the survey. Following suggestions by the ISG, two items were added to the survey questionnaire, covering the inclusion of consumer perspectives in assessments.
The questionnaire was sent to member agencies by the INAHTA secretariat in November 2010, and followed up by three reminders. Responses were compared with those for the 2005 survey. Comparisons between the surveys for each of the questions were presented as the proportions of positive answers received.
RESULTS
Completed survey forms returned by the following agencies: AHTA, ASERNIP-S (Australia), LBI - HTA, GÖG (Austria), DECIT (Brazil), CADTH, IHE (Canada), DACEHTA, MIDT (Denmark), Finohta (Finland), HAS (France), DAHTA, GBA (Germany), HIQA (Ireland), ICTAHC (Israel), UVT (Italy), CNHTA (Korea), VASPVT (Lithuania), CENETEC (Mexico), CVZ (The Netherlands), MaHTAS (Malaysia), NOKC (Norway), AHTAPol (Poland), AETS, AVALIA-T, CAHIAQ, OSTEBA (Spain), SBU (Sweden), CDE (Taiwan), HITAP (Thailand), and HIS, NETSCC, NHSC (UK).
There were differences between the two surveys in the profile of responses, due to changes in the INAHTA membership. In 2010, there were responses from thirty-three members (63 percent response rate), compared with thirty-seven (90 percent response rate) in 2005. Nineteen agencies (58 percent) participated in both surveys, and fourteen had joined INAHTA since 2005. Distributions of responses by region are shown in Table 1.
Distribution of Agencies Providing Responses, by Region
Twenty-two agencies (67 percent) indicated that consumers were involved in some aspects of their HTA programs, compared with twenty-one (57 percent) in 2005. Of those agencies that do not currently involve consumers, 64 percent intended to do so in the future, the same proportion as in 2005. Four agencies provided comments on use of consumers in the preparation of clinical practice guidelines, rather than with HTA.
Three agencies noted difficulties with involving consumers in their programs and provided the following comments.
“The horizon scanning technology briefings we produce do not lend themselves to the addition of consumer opinions. Consumers are represented on the panels that consider the technology briefings and prioritise them for technology appraisal. The subsequent technology appraisal also includes consideration of consumers and the production of an “easy-read/understandable” version.”
“We consider involving consumers important and relevant, but also time consuming.”
“Patient involvement is not a routine activity, this is intended only to our major projects. Among barriers to expanding patients’ involvement there are time and resources constraints; difficulties in identifying appropriate consumers.”
Responses from Those Agencies That Involved Consumers
In both surveys, 95 percent of agencies that involved consumers reported contact between patient organizations and their programs, and 45 percent also involved individual consumers.
Invitation from the agency remained the most frequently used method of bringing consumers into contact with the HTA process (90 percent). Compared with the 2005 survey, fewer agencies accepted requests from consumers on specific topics (59 percent versus 70 percent), while contact with the HTA process in response to publicity on forthcoming assessments was more common (45 percent versus 25 percent). Only a small proportion of agencies provide a training process for consumers who are involved in HTA programs (23 percent in 2010, 19 percent in 2005).
As before, a minority of agencies indicated that they avoided consumer involvement in some types of assessment (24 percent in 2005, 23 percent in 2010). Examples of where this occurred, provided by five agencies, included commercial in confidence evaluation of pharmaceuticals, assessments done on behalf of some other organizations, rapid assessments, and assessments in which there is no added value or benefit from the involvement of consumers. Horizon scanning and rapid assessments had been identified in the earlier survey as areas where consumers were not involved.
In 2010, 55 percent of agencies involved consumers in the formulation of assessment topics (67 percent in 2005). One agency does not generally seek consumers’ input but involves them in a few projects regarding the assessment of treatments for psychiatric diagnoses. Approaches used by agencies are shown in Table 2. In the 2010 survey the proportion of agencies that considered consumers’ suggestions was lower than in 2005, while that for each of the other (more detailed) approaches was higher.
Approaches Used by Those Agencies That Involve Consumers in the Formulation of Assessment Topics
Responses on involvement of consumers in the preparation of HTA reports are summarized in Table 3. In 2010, higher proportions of agencies contacted consumers or used them for interpretation of data and in drafting reports. Similar proportions in each survey used consumers to review protocols or reports. Routine involvement was higher in the 2010 survey for the first two types of task and lower for the third.
Involvement of Consumers in Preparation of Assessments
In 2010, 19 percent of agencies that involved consumers in their programs had undertaken appraisal of such involvement, compared with 29 percent in 2005. Of the agencies that provided responses in 2010, one recorded numbers of reports where there has been consumer input, three noted the type of input, and three considered the influence of consumer input on product quality and relevance. None of them had considered the potential for conflicts of interest to influence consumer opinions.
Preparation of Summaries for Consumers
All agencies were asked whether they prepared summaries or versions of HTA reports that are intended to be easily understood by consumers. In the 2010 survey, 59 percent of agencies reported providing such material compared with 49 percent in 2005. Most of those that provided summaries of reports also involved consumers in their programs (84 percent in 2010 and 71 percent in 2005). Proportions of agencies that provided different types of summary are shown in Table 4.
Types of HTA Summary Prepared for Consumers
Involvement of Consumers in HTA Dissemination
In the 2010 survey, 42 percent of agencies involved consumer organizations and/or individual consumers in the dissemination of HTA materials, compared with 33 percent in 2005.
Consumer Perspectives in Assessments
In response to the additional questions in the 2010 survey, seventeen (52 percent) of the agencies advised that their HTA reports consider consumer-reported attitudes and preferences, and thirteen (39 percent) that their reports include a section that assesses patient/consumer perspectives.
Comparison of Agency Responses Over Time
The 2010 responses from the sub-group of nineteen agencies that participated in both surveys were similar to those they provided in 2005. When compared with the total responses, The proportion of the nineteen agencies that involved consumers was similar to that for all agencies in 2010 and higher than the total for 2005 (65 percent versus 67 percent versus 57 percent). The proportion of sub-group agencies that prepared summaries for consumers was of the same order as for 2010 and higher than 2005 (63 percent versus 59 percent versus 49 percent). For those agencies that involved consumers, responses for all three groups were almost identical for involvement of patient organizations and methods of contact, involvement in topic formulation and in review of protocols and draft reports. The proportion of sub-group agencies that involve individual consumers was somewhat higher than the totals for both surveys (58 percent versus 46 percent versus 48 percent).
Responses from agencies that participated in only one survey were also compared. The proportion of the fourteen agencies from the 2010 survey that involved consumers was higher than for the eighteen that participated only in 2005 (71 percent versus 50 percent). The proportion of agencies that prepared summaries for consumers was also higher in 2010 than 2005 (50 percent versus 33 percent). For those agencies that involved consumers, responses for both groups were almost identical for involvement of patient organizations and of individual consumers, and methods of contact, involvement in topic formulation and in review of protocols and draft reports.
DISCUSSION
As in the previous survey, reported involvement of consumers in HTA varied among INAHTA members, reflecting differences in responsibilities and administrative arrangements. Comparison of findings from the two surveys is tentative, given the differences in response profiles, but there appear to be indications of increasing involvement in some activities.
A limitation in this comparison is the lower response rate by INAHTA members in the 2010 survey. The reasons for this decreased response are unclear. Possibly it was associated with increases in the workload and commitments of some member agencies. INAHTA projects rely heavily on the input from individual agencies and there are clearly limits to the time and resources that members can offer (Reference Hailey14).
There were also differences between the surveys in the profile of responding agencies due to changes in INAHTA membership. Review of responses from agencies that participated in both surveys and those from single survey participants gave some indication of similarities between the two survey populations, particularly for those agencies that involved consumers. A feature of the 2010 survey was the increase in the proportion of agencies from Asia. It is considered unlikely that this change would have had any substantial effect on the comparison of surveys. The Asian agencies have been influenced by the well established HTA programs in Europe and have adopted the most suitable values for their own settings. Attitudes toward consumer involvement have been guided by these influences.
Of the agencies that provided responses, 67 percent involve consumers in aspects of their HTA programs, although not always routinely, a higher proportion than that in 2005. As in 2005, a large majority of agencies intend to involve consumers in the future. Similar findings were reported on types of consumer and how they are brought into contact with the HTA process. A majority of agencies accept requests on specific topics.
As in 2005, a minority of agencies avoid involving consumers in some types of assessment. Comments from these agencies point to practicalities in their programs that contribute to avoidance of contact. Time and resource constraints, requirements of rapid and horizon scanning assessments and projects undertaken in confidence all present challenges for wider patient involvement.
A much higher proportion of agencies in 2010 used consumer input to data analysis or drafting sections of HTA reports. A small majority in each survey used consumers to review of protocols or reports. Evaluation of consumer input to HTA programs remains uncommon.
In 2010, there was an increase in the proportion of agencies that provide summaries of HTA reports for consumers, and also in the proportion that involve consumers in dissemination. The additional items included in the 2010 survey indicated limited inclusion of consumer perspectives in assessment reports.
Overall, the 2010 survey suggests that there is a trend to increased involvement of consumers by INAHTA members in their programs, continuing the generally positive response found in the earlier survey.
There appear to be similar issues to those for HTA in the involvement of consumers in the development of guidelines. A systematic review of patient and public involvement programs (PPIPs) in developing clinical practice guidelines found that of seventy-one descriptive reports twenty-three reported using PPIPs to “incorporate patients’ values, preferences, knowledge, or perspectives in CPG recommendations” (Reference Légaré, Boivin and van der Weijden15). Eccles et al. note that of the seventy-one reports only twenty-eight were dealing with consumers within the guideline development group itself and only twenty-nine were dealing with consumers involved in crafting recommendations (Reference Eccles, Grimshaw, Shekelle, Schünemann and Woolf16). Barriers to consumer involvement that were identified resemble those that have emerged for health research and HTA. They included the discrepancy between the views of patients and experts, challenges of recruitment, obtaining representative input, consumers’ lack of familiarity with technical issues and the degree of work/time involved. Eccles et al. comment that there are still few analytical empirical accounts of attempts to involve consumers in guideline development, and no robust evaluations of the effectiveness of different methods.
POLICY IMPLICATIONS
The involvement of consumers by many INAHTA agencies offers the potential to broaden the perspective of assessments and of the advice provided to decision makers. It also gives stakeholders further opportunity to gain information on health technologies that may help their own inputs to the policy process. There seems to be a trend toward increased involvement, but there are also barriers to wider routine participation of consumers in the HTA processes of these agencies.
CONTACT INFORMATION
David Hailey, MSc, PhD, Grad RIC, Professorial Fellow, University of Wollongong, Australia (dhailey@ozemail.com.au), Sophie Werkö, PhD. Swedish Council on Health Technology Assessment, Stockholm, Sweden, Rugayah Bakri M.B.B.CH, MPH, Ministry of Health Malaysia, Putrajaya, Malaysia, Alun Cameron, BSc (Hons), PhD. Australian Safety and Efficacy Register of New Interventional Procedures -Surgical, Adelaide, Australia, Britta Göhlen PhD, German Institute for Medical Documentation and Information, Cologne, Germany, Susan Myles BA (Hons), MSc, PhD, Healthcare Improvement Scotland, Glasgow, UK, Jasmine Pwu, PhD, Center for Drug Evaluation, Taipei, Taiwan, Jomkwan Yothasamut, BA, MA, Health Intervention and Technology Assessment Program, Ministry of Public Health, Nonthaburi, Thailand
CONFLICTS OF INTEREST
The authors report no potential conflicts of interest.
