Introduction
In Latin America, Health Technology Assessment (HTA) started to become a part of an important health sector reform in the late 1990s (Reference Banta1). In 2012 and 2014, the Pan American Health Organization (PAHO) and the World Health Organization (WHO) approved resolutions on HTA and universal health coverage, calling for a strengthening of HTA capacities in this region (Reference Pichon-Riviere, Soto, Augustovski, García Martí and Sampietro-Colom2). Currently, Argentina, Brazil, Colombia, and Uruguay have HTA bodies that are members of The International Network of Agencies for Health Technology Assessment (INAHTA). Additionally, many other countries from Latin America and the Caribbean region are a part of The Health Technology Assessment Network of the Americas (RedETSA).
The national HTA body in Brazil, the National Committee for Health Technology Incorporation into Unified Health System—Conitec, is composed of an Executive-Secretariat (SE) and a Plenary group with thirteen members, with voting rights, representing the seven Brazilian Ministry of Health secretariats and six other national health institutions (Reference Silva3).
Stakeholder involvement has been developing in several HTA agencies (Reference Street, Stafinski, Lopes and Menon4) including in Latin America, with Brazil and Colombia implementing formal mechanisms to involve citizens (Reference Pichon-Riviere, Soto, Augustovski, García Martí and Sampietro-Colom2;Reference Pichon-Riviere, Soto, Augustovski and Sampietro-Colom5).
In Conitec, this occurs in response to the constitutional requirements for citizens’ participation and a legislative mandate for “social participation” in all parts of the public health system. With this basis, one of Conitec's plenary members is a representative of the National Health Council (Conselho Nacional de Saúde—CNS), which ensures 50 percent of places for citizen entities, 25 percent for health worker entities, and 25 percent for government representatives and service providers. Besides this, the public, who could be patients or patient groups with an interest in a particular health technology, can be officially involved (Reference Silva3):
• submitting HTA topic proposals;
• via the public consultation issued for each recommendation;
• in public hearings when the Brazilian Ministry of Health Secretariat of Science, Technology and Strategic Inputs determines they are necessary.
This is a good starting point, but it was recognized by the Ministry of Health that there was a need for more extensive patient participation in the various stages of HTA and use of processes that are applied consistently across different HTA topics.
Hahn et al. (Reference Hahn, Hoffmann, Felzien, LeMaster, Xu and Fagnan6) outlined the characteristics of tokenism (i.e., perfunctory or uninformed gestures toward engaging with patients), such as low stakeholder diversity, late participation, lack of role definition and appropriate training, and poor scheduling, time frame and format. On the other hand, meaningful patient engagement is described by Hamilton et al. (Reference Hamilton, Hoens, Backman, McKinnon, McQuitty and English7) as a planned approach that involves patients, with these individuals perceiving it as a rewarding and productive experience.
All this reinforces the need to systematize involvement approaches and evaluate them to promote constant improvement. As a result, this research was commissioned by the Brazilian Government and supported by the Ministry of Health and Conitec to collaboratively develop a comprehensive framework for action that could serve as a vehicle to drive improvement of patient involvement in the Brazilian HTA process, building on the public involvement approaches that already exist.
Methods
The development process of this framework was inspired by the works of Abelson (Reference Abelson, Wagner, DeJean, Boesveld, Gauvin and Bean8) and Greenhalgh (Reference Greenhalgh, Hinton, Finlay, Macfarlane, Fahy and Clyde9) and followed a three-phase mixed-methods approach, involving critical reflection on the descriptive and evaluative results, as shown in Figure 1 and described below.
Three-phase approach to building the framework for action to improve PPI in the Brazilian HTA. Abbreviations: CEPPP, Canadian Centre for Excellence on Partnerships with Patients and Public; HTA, health technology assessment; PPI, patient and public involvement.
The first author (ASS) was a technical advisor to Conitec responsible for patient and public involvement (PPI) activities between 2014 and 2020, which allowed a detailed observation and review of the activities in this context.
Phase A: Review of PPI in the Brazilian HTA Process
Phase A sought to elucidate gaps, feasibility, and implementation considerations for PPI in Conitec based on a desktop review of existing practices since its establishment in January 2012 until December 2017 as described by Silva et al. (Reference Hahn, Hoffmann, Felzien, LeMaster, Xu and Fagnan6). It focused on the following questions: (i) In what stages of HTA in Brazil can patients and the public be involved? (ii) In what stages could there be greater patient participation?
Phase B: Stakeholders’ Perspectives and International Best Practices
This phase aimed to address the following questions: (i) What are the barriers to overcome and suggestions to improve PPI in the Brazilian HTA processes? (ii) What are the values and principles that should guide this involvement in Brazil? (iii) What international strategies could be applied or adapted to this context and how should they be evaluated?
To understand stakeholders’ perspectives on questions (i) and (ii), a workshop and survey with patients and patient groups and semistructured interviews were undertaken with a range of stakeholders. For questions (iii), a rapid review of the international practices of involvement in HTA was conducted.
The methodological details are described below.
Patients and Other Stakeholders’ Perspectives about PPI in HTA in Brazil
A patient workshop organized by Conitec SE in collaboration with Oswaldo Cruz German Hospital (HAOC) was held in Sao Paulo in October 2017 (Reference Silva3;Reference Silva, de Sousa, da Silva and Galato10). It involved 103 patients and representatives from 98 patient groups. An online survey applied at the beginning of this event collected data outlining their perspectives about the existing PPI approaches in the Brazilian HTA. Questions in the instrument were informed by the literature and constructed using a multiple-choice Likert agree–disagree scale.
In addition, suggestions on how to improve PPI were received through a group consensus activity that involved all attendees divided into nine groups. The workshop inputs were documented by a range of approaches, including graphic facilitation. These suggestions were compiled and posteriorly analyzed and described (ASS and DG).
Semistructured stakeholder interviews with diverse stakeholders involved with HTA in Brazil were conducted face-to-face or by phone between May and August 2018 to capture perspectives, values, barriers, and suggestions to improve the PPI. Twenty-five participants were recruited using a convenience strategy determined by theoretical saturation and snowball sampling.
The Research Ethics Board of the University of Brasilia approved this research project, and all participants signed a consent form. Quantitative data analysis (survey) was descriptive, using Microsoft Excel. Qualitative data was transcribed and analyzed using Qualitative Solutions Research (QSR) NVivo 12 and thematic content analysis based on the principles of constant comparison and qualitative description (ASS, DG and SB) (Reference Braun and Clarke11).
Rapid Review of PPI in HTA: Practices and Evaluation
A gray literature search for PPI practices in HTA was undertaken on the Web sites of nine HTA organizations (ASS and SB): Health Technology Assessment International (HTAi) and eight selected agencies from the INAHTA members' list, selected by a consensus of the researchers involved in this rapid review. The search resulted in twelve weblinks for the initial review. Three were finally included, from the National Institute for Health and Care Excellence (NICE), CADTH, and HTAi (Supplementary File 1).
A rapid review to explore how patient involvement practices have been evaluated was undertaken in April and May 2019 (ASS and SB). HTA organizations and relevant journals were also searched. The inclusion criteria were studies that addressed the evaluation of patient involvement in HTA, in English, published from 2008 (Supplementary File 2).
Phase C: Developing a Framework for Action
Phase C developed the framework for action to improve PPI, integrating the findings from Phase A and Phase B applied to the whole Brazilian HTA process mapping.
The framework development followed the assessment criteria of the Canadian Centre for Excellence on Partnerships with Patients and Public (CEPPP), that is, to encompass scientific rigor, incorporation of the perspectives of the public and the patient, scope, and usability (Reference Greenhalgh, Hinton, Finlay, Macfarlane, Fahy and Clyde9;Reference Boivin, L'Espérance, Gauvin, Dumez, Macaulay and Lehoux12).
From the beginning of its development in October 2019, the first author made the draft framework available to the Conitec SE team and received insights from its director and coordinators through personal communications.
Five patient group representatives, previously engaged on the interviews as participants, were invited to review a draft framework and a plain language summary of this study. Three of them, called our patient partners—patients and informal caregivers who provide a patient perspective in health research (Reference Hamilton, Hoens, Backman, McKinnon, McQuitty and English7), sent their revisions in July 2020, which were considered by two researchers (ASS and DG). They gave informed written consent and indicated whether they wanted to be explicitly acknowledged in this paper.
Next, the framework was translated to English and reviewed by two other researchers (SB and KF).
Results
Phase A
Review of PPI in the Brazilian HTA Process
Figure 2 depicts the HTA process in Brazil and indicates the stages in which patients and the public are entitled to be involved according to the legal mandate: (1) HTA topic proposals; (4) CNS as a committee member; (5) public consultations; and (8) public hearings (Reference Silva3).
HTA stages in which PPI can occur in Brazil. Source: Silva (2020). Abbreviations: DOU, Diário Oficial da União (federal government official journal); PC, public consultation; SCTIE, Secretaria de Ciência, Tecnologia, Inovação e Insumos Estratégicos em Saúde (secretariat of science, technology and strategic inputs); SE, executive-secretariat.
To evaluate how these legal entitlements were put into practice, we reviewed and mapped all existing PPI practices during the period analyzed (Reference Silva, de Sousa, da Silva and Galato10). This identified a range of different strategies that had been trialed by Conitec, in addition to those established by the law.
Submission of HTA topic proposals requires a dossier of evidence, and so, it is difficult for citizens, patients, or patient groups to suggest a topic given the complexity of the process.
Conitec's seat for one public member of the CNS committee is valuable, but international best practice suggests that at least two public members should be involved.
Public consultation was the main tool for PPI and almost half of the consultation responses were from people who were directly affected by a particular condition or health technology (patients or their informal caregivers). However, no information is available about the methodology used to review the consultation responses and incorporate them into the HTA deliberation. PPI impact was simply measured according to the number of consultation comments received (Reference Silva, de Sousa, da Silva and Galato10).
Public hearings were instigated but not used during the period analyzed.
Activities additional to those legislated include communication initiatives to support PPI (Reference Silva, de Sousa, da Silva and Galato10), such as plain language summaries of recommendation reports, a guide for patient involvement in HTA launched in 2016, the workshop for patients in 2017, and informative video conferences held between 2016 and 2017. Some PPI initiatives, such as social media accounts and a video conferencing program to the public, were discontinued, but there are no records of evaluation of PPI strategies.
Phase B
Patients and Other Stakeholders’ Perspectives about PPI in HTA in Brazil
A majority of the workshop attendees who completed the survey (n = 82, 90.1 percent) had a disease (55.6 percent) and were not engaged in HTA (53.2 percent), probably because they consider PPI in Brazilian HTA difficult (59.5 percent). However, the majority agreed that Conitec involves patients and public (59.6 percent). From all attendees’ perspectives, PPI in Brazilian HTA is important and can be improved.
The group consensus suggestions on how to improve PPI (Supplementary File 3) were analyzed and synthesized and some were grouped together according to different forms of barriers.
Twenty-five semistructured interviews were conducted and are fully reported elsewhere (Reference Silva3). The participants were grouped into four groups by type of audience: pharmaceutical industry (n = 5), patients or patient group representatives (n = 5), researchers (n = 7), and health professionals, including Conitec members (n = 8).
Drawing on perspectives and themes arising from the workshop and interviews, barriers to overcome and suggestions for improvement were identified (Table 1).
Synthesis of main barriers and suggestions to improve PPI in the Brazilian HTA
Abbreviations: CNS, Conselho Nacional de Saúde (National Health Council); HTA, Health Technology Assessment; PPI, Patient and Public Involvement; SUS, Sistema Único de Saúde (Unified Health System).
In the stakeholder interviews, several of the key themes could be categorized as values or principles, which were used to guide the building of the framework.
Transparency was continuously cited in the interviews and also appears among the suggestions received at the workshop. Participants stated the need to clarify the HTA process as well as how PPI occurs throughout it.
Another key theme was capacity building. In the opinion of several stakeholders, the Brazilian public needs more information about the HTA process so that they can be effectively involved in it.
In both the interviews and workshop, there was a desire to involve patients who would be directly affected by the decision in HTAs. In several interviews, there was concern about the lack of representativeness and disparity within the Conitec plenary (seven of the thirteen members represented the Ministry of Health, and only one represented citizens). These perspectives led to a call for these processes to be based on justice and equity, the latter being one of the principles of the Brazilian public health system itself.
Rapid Review of PPI in HTA: Practices and Evaluation
Information on PPI activities in HTAi, CADTH, and NICE provided important insights for the elaboration of this framework.
The HTAi Interest Group for Patient and Citizen Involvement in HTA (HTAi PCIG) presents values and quality standards for PPI in HTA that are directed to HTA agencies and have been recognized, referenced, and when applicable, translated by agencies in England, Scotland, France, Belgium, Colombia, and Canada (Reference Wale, Scott, Bertelsen and Meade13). The HTAi values to support PPI in HTA are relevance, fairness, equity, legitimacy, and capacity building. Canada has recently reflected on its PPI practice and how it aligns with these values and standards (CADTH Framework for Patient Engagement in Health Technology Assessment | CADTH) (Reference Silva3). Both NICE and CADTH have a published strategy that describes the processes and responsibilities of those working in HTA to effectively involve patients; designate appropriate resources to ensure and support such involvement; HTA staff receive training on how to involve patients throughout the HTA process; patient groups also have training opportunities so that they can contribute optimally; and finally, the processes of patient involvement in HTA are evaluated and reviewed regularly in order to continually improve them.
The rapid review of evaluation processes included three articles that met the inclusion criteria. They not only focused on the evaluation of patient involvement in HTAs but also described international experiences of PPI in HTA, and we observed what could be feasibly adapted to the Brazilian context.
Gagnon et al. (Reference Gagnon, Desmartis, Lepage-Savary, Gagnon, St-Pierre and Rhainds14) reviewed PPI practices and evaluation strategies internationally. In terms of international PPI practices, they found that countries such as the United Kingdom, the United States, the Netherlands, Canada, Denmark, New Zealand, and Australia consult and use data provided by the public and patients. Qualitative methods used to capture these data are interviews, focus groups, questionnaires (willingness to pay or discrete choice questionnaires, for example), and weblogs (gray literature). Several agencies in these countries also allow patient participation in the stages of selection of HTA topics, assessment, and in the dissemination of results. For this, the most frequent methods are interviews, focus groups, questionnaires, document analysis, and citizens’ jury to develop criteria guiding the definition of priorities. Gagnon et al. concluded that in addition to systematizing the approaches of PPI in HTA, it is also necessary to evaluate these strategies.
Dipankui et al. (Reference Dipankui, Gagnon, Desmartis, Légaré, Piron and Gagnon15) evaluated patient involvement in HTA via interviews and literature review and identified that some HTA bodies presented patient input and research into patient aspects in a separate section of their reports, allowing an explicit consideration of those inputs and evidence, which could sometimes be shown to directly influence recommendations.
Weeks and colleagues (Reference Weeks, Polisena, Scott, Holtorf, Staniszewska and Facey16) showed that HTA organizations that support PPI involve both patient and public, but some HTA bodies only involve patients. The study reported that, among the surveyed organizations, PPI was performed in a series of activities, such as involving patients in working groups or committees to provide opinions and perspectives, identifying HTA topics, refining HTA scope, identifying clinical outcomes, making recommendations, and helping to disseminate results. According to the authors, a small but diverse set of HTA organizations evaluate their PPI activities using a range of strategies.
Two studies (Reference Dipankui, Gagnon, Desmartis, Légaré, Piron and Gagnon15;Reference Weeks, Polisena, Scott, Holtorf, Staniszewska and Facey16) mentioned that an evaluation of PPI in HTA could be categorized around three topics: (1) patient satisfaction, (2) process evaluation, and (3) impact evaluations. Evaluation of PPI in HTA processes found that in most instances, patient involvement has enriched the content of the HTA report and its recommendations, introduced additional perspectives, and served as a mechanism to validate the findings from the stakeholder views.
Phase C
Developing a Framework for Action
To build this framework to improve PPI, the following learnings, informed by phases A and B, were integrated to the whole Brazilian HTA process mapping by the researchers (ASS and DG):
• the stages of the Brazilian HTA process in which there could be greater involvement;
• barriers to overcome;
• the values and suggestions identified in the stakeholder's perspectives;
• international strategies that could be adapted/applied to the Brazilian context.
For better organization of the strategies proposed in each HTA stage, the typology of engagement mechanisms by Rowe and Frewer (Reference Rowe and Frewer17) was adopted, which includes three levels of involvement that are commonly cited: communication, consultation, and participation. The resulting framework is presented in Figure 3.
Proposed framework for action to improve patient and public involvement (PPI) in the Brazilian health technology assessment. Abbreviations: CNS, Conselho Nacional de Saúde (national health council); DOU, Diário Oficial da União (federal government official journal); HTA, health technology assessment; KT, knowledge translation; PC, public consultation; PPI, patient and public involvement; SCTIE, Secretaria de Ciência, Tecnologia, Inovação e Insumos Estratégicos em Saúde (secretariat of science, technology and strategic inputs); SE, executive-secretariat. *Based on Rowe and Frewer (2005): communication, consultation, and participation mechanisms.
An analysis of the PPI activities in the Brazilian HTA process up to 2017 identified that there is potential for greater involvement, starting from stages 1, 2, and 3. Some HTA agencies involve patients and the public in HTA prioritization, scoping, and data collection (Reference Gagnon, Desmartis, Lepage-Savary, Gagnon, St-Pierre and Rhainds14;Reference Weeks, Polisena, Scott, Holtorf, Staniszewska and Facey16), which would lead to an earlier involvement in Brazil. Conitec could seek to achieve this through conducting prioritization workshops (using consensus methods) based on SUS health priorities and encouraging HTA topic proposals from patients and public through its Web site. In stages 2 and 3, studies using Patient-Reported Outcome Measures (PROMs) (Reference Bryan, Davis, Broesch, Doyle-Waters, Lewis and Mcgrail18) could be encouraged, as well as a search for qualitative research about patients’ experiences and perspectives with the technology or disease, and adding a separate chapter in the recommendation report on patient aspects (Reference Dipankui, Gagnon, Desmartis, Légaré, Piron and Gagnon15).
The Brazilian stakeholders identified barriers and proposals to support more meaningful involvement. Because transparency was a concern raised in several interviews as a barrier and is an important value for the Conitec HTA process, one suggestion was to broadcast committee meetings or make them public (stages 4 and 6). In addition, there were proposals to make the whole process more transparent, expanding communication strategies and defining how patient and public inputs would be considered, taking account of the international examples. We suggest observing transparency as a transversal value throughout the whole HTA process, as well as capacity building.
Capacity building is reported in the literature as a support factor for participation (Reference Gagnon, Desmartis, Lepage-Savary, Gagnon, St-Pierre and Rhainds14). At CADTH and NICE, staff, committee members, and patients/patient groups receive training on appropriate patient involvement. Training/support for public consultation users and participation instruments improvement (stage 5) are strategies that have already been developed in Brazil. However, these could be expanded to overcome reported barriers such as a lack of information about HTA and quality of public consultation comments.
Listening effectively to the voices of patients and the public in the HTA process was observed as an issue to be addressed. Patients (and other stakeholders) pointed to a lack of public representation on the Conitec plenary and during its meetings. Active participation of patient group representatives in the committee meetings for all HTA, with an appropriately defined methodology and focused invitations, is suggested (Reference Gagnon, Desmartis, Lepage-Savary, Gagnon, St-Pierre and Rhainds14). In addition to this, the participation of the public representative (CNS member) through a careful selection of an individual who can represent the public with credibility should be strengthened. This could be achieved with greater dialogue and interaction with patient groups (stages 4 and 6).
Another proposal is to establish the criteria for when public hearings, stages 7 and 8, should occur. Furthermore, knowledge translation improvement and dissemination of results (stage 9) were suggested by stakeholders and international practices (Reference Gagnon, Desmartis, Lepage-Savary, Gagnon, St-Pierre and Rhainds14).
There was no evidence of PPI evaluation or any analysis of the consultation comments nor of these inputs' impact on Conitec's recommendations. The number of submissions received through public consultation is usually the measure of PPI impact, according to Official Information from Conitec's Web site, “Conitec in numbers.” The findings of the rapid review reinforce the importance of this evaluation (Reference Gagnon, Desmartis, Lepage-Savary, Gagnon, St-Pierre and Rhainds14–Reference Weeks, Polisena, Scott, Holtorf, Staniszewska and Facey16), which was added as a continuous activity throughout the suggested framework. The approaches vary widely, from participation in interviews to something more simplified, such as the participants’ satisfaction survey, with questionnaires being applied regularly during work processes, and even the possibility of using storytelling to reflect on the impact of patient involvement in HTA (Reference Single, Facey, Livingstone and Silva19).
Analyzing Brazilian patients’ and other stakeholders’ perspectives, we identified values (transparency, capacity building, equity, and justice) that align with the HTAi PCIG Values and Quality Standards for Patient Involvement in HTA. As Conitec is an active HTAi member, acknowledgment and observation of the identified values during its PPI process should be encouraged.
Beyond the insights from Conitec SE, the review by patient partners complemented our perspective and added value to this framework, reinforcing the possibility of public and patients to submit HTA topic proposals as an involvement mechanism that should be improved, besides the lack of use and criteria for public hearings.
Discussion
According to Facey (Reference Facey, Boivin, Gracia, Hansen, Lo Scalzo and Mossman20), early involvement, training, an appropriate participation method and support from HTA organizations may help ensure that patients contribute meaningfully to the HTA process and output. For Whitty (Reference Whitty21), even though evaluative processes are relatively recent, encouraging HTA organizations to publish narratives of their experiences with public engagement can support the development of systematic yet pragmatic approaches and frameworks.
Following the CEPPP guidance, this three-phase mixed-methods approach framework was based on a critical reflection of (phase A) the existing PPI practices and feasibility for greater involvement; (phase B) relevant values, barriers, and proposals identified through a thematic analysis of data from different stakeholders and literature review; (phase C) an integration of previous results to the whole Brazilian HTA process mapping in consultation with Conitec SE; receiving final review by patient partners to make sure it was built on Brazilian needs.
In the first phase, Silva (Reference Silva, de Sousa, da Silva and Galato10) showed the gradual implementation of PPI actions by the Brazilian HTA agency. However, like many other HTA bodies, the strategies do not go beyond consultation level (Reference Gagnon, Tantchou Dipankui, Poder, Payne-Gagnon, Mbemba and Beretta22). Considering the current trend to broaden the spectrum of involvement to increase genuine participation (Reference Hahn, Hoffmann, Felzien, LeMaster, Xu and Fagnan6) and the findings from this study, the involvement strategies already used in Brazil should be expanded, for an earlier, more direct, and active participation. Involving patients and patient groups in ways by which, and at times when, their input can influence decision making and generating a dialogue to use their unique experiential knowledge are needed (Reference Facey23). Thus, the development of a broader strategy adapting international best practices, involving patients and the public from the beginning and along all the HTA process could be helpful.
An analysis of different stakeholders’ perspectives in Brazil shows that some identified barriers are similar to those identified internationally (Reference Gagnon, Desmartis, Lepage-Savary, Gagnon, St-Pierre and Rhainds14;Reference Dipankui, Gagnon, Desmartis, Légaré, Piron and Gagnon15;Reference Gagnon, Tantchou Dipankui, Poder, Payne-Gagnon, Mbemba and Beretta22;Reference Facey, Facey, Hansen and Single24). International strategies for stimulating and evaluating PPI in HTA, plus patient and stakeholders’ suggestions, made it possible to identify several promising actions that could feasibly be addressed by Conitec, and bringing these actions together creates the systematic approach presented in this framework. Conitec SE first received these recommendations in June 2020 and subsequently moved toward implementing some of them. The final version was shared in August 2020 with Conitec and other interested stakeholders, including all participants of this research.
To overcome the lack of transparency, pointed as an involvement barrier, we suggested making the committee meetings public, which was started by the agency in July 2020, through making meeting recordings available online.
Patient representatives' participation in the committee meetings was suggested as an improvement of a strategy sporadically used by the agency since 2014 (Reference Silva, de Sousa, da Silva and Galato10;Reference Single, Facey, Livingstone and Silva19;Reference Silva, Petramale, Rabelo, Santos, Facey, Hansen and Single25). Gagnon (Reference Gagnon, Desmartis, Lepage-Savary, Gagnon, St-Pierre and Rhainds14) reported that issuing focused invitations (inviting people with experience related to the topic discussed in HTA) is a success factor in patient engagement. Since December 2020, Conitec started issuing public calls for patient representatives for each topic discussed in the committee meetings.
Also, since the publication of this framework, two public hearings have taken place for the first time. The first one discussed a health technology for spinal muscular atrophy in March 2021 and the second one discussed the Brazilian guidelines for hospital treatment of patients with Covid-19 in July 2021.
The main strength of this study was our collaborative approach to ensure that the framework was grounded on the experiences and views of patients and other stakeholders who engage in the HTA process. However, it had some limitations, such as the rapid review searches, which were conducted in only one database (MEDLINE) and narrowed on the evaluation of patient involvement. Besides these, the search engines on the agencies’ Web sites were not specific. Finally, it was not possible to implement a rigorous validation method for the proposed framework, so we attempted to minimize this limitation by inviting previously enrolled patient representatives to review it.
Conclusion
This article is a response to the reported need for a pragmatic approach and more systematic PPI at the Brazilian PPI in HTA. It reports on the findings of different study phases: a review of Brazilian PPI activities in HTA up to 2017; an elicitation of perspectives and values of patients and other stakeholders captured by surveys, interviews, and workshop; and a rapid literature review.
Using this mixed-methods approach involving all interested stakeholders, it was possible to build a framework for action to improve the involvement of patients and the public during all the HTA stages, considering important values and principles for the Brazilian context.
This framework defines and systematizes involvement actions and highlights the importance of evaluating these strategies and their impact to enhance PPI in the Brazilian HTA process.
We encourage other HTA organizations to consider and demonstrate a systematic and planned approach with regular evaluation when pursuing or strengthening patients and public involvement.
Supplementary Material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462321000647.
Acknowledgments
Our acknowledgments to all Brazilian Ministry of Health collaborators who contributed to PPI in HTA since the beginning of its implementation, and specially, to those who supported the workshop in Sao Paulo and this framework building. Many thanks to all stakeholders involved in this research, mainly the following patient partners, Gustavo San Martin, Simone Arede, and Tiago Farina, for reviewing the presented framework and the plain language summary of this study. We also would like to thank Dr. Kathiaja Miranda Souza (National Consultant of Health Technology Assessment and Management at Pan American Health Organization—PAHO, Brazil) for reviewing this manuscript.
Author Contributions
ASS and DG developed the initial outline. ASS, DG, and SB obtained funding. ASS and DG contributed to the workshop, survey, and interview development. ASS and DG analyzed the free-text data. ASS, DG, and SB contributed to the survey analysis. ASS and SB contributed to the rapid review. SB and KF contributed to the English translation of the study and framework. All authors wrote the manuscript, contributed to the interpretation and narrative of the paper, reviewed, and revised the content, approved the final version, and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of the work.
Funding
The work for this manuscript is a result of the doctorate project entitled “The patient and public involvement in the Brazilian health technology incorporation process” from the University of Brasilia, having the Department of Management and Incorporation of Technologies and Innovation in Health (Conitec SE) from the Brazilian Ministry of Health as a co-partner institution. A part of this study was conducted at the Center for Clinical Epidemiology & Evaluation (C2E2) and supported by the University of British Columbia, Canada. It received funding from the Brazilian Government through the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brazil (CAPES), process number 88881.188553/2018-01 of 2018.
Conflict of Interest
Dr. Silva served as a civil servant at the Department of Management and Incorporation of Technologies and Innovation in Health (Conitec SE), Brazilian Ministry of Health between 2014 and 2020. Dr. Facey reports personal fees from Swii.ch, BMS, Health Technology Wales, NICE, Ministry of Health/Czech Republic, Pfizer/Lilly, Novartis, and imi Paradigm project, outside the submitted work. The other authors declare no conflicts of interest.
