Introduction
The involvement of patient and consumer groups (PCGs) in health technology assessments (HTAs) has been considered important, if not mandatory, for all agencies across borders since the end of the twentieth century. This was raised, for instance, as early as 1995 by Vikky et al. (Reference Entwistle, Sheldon, Sowden and Watt1). HTA agencies have since developed methods and procedures to include patients’ perspectives. These have been the object of numerous studies and published subsequent reviews, as (for instance) those of Rashid et al. (Reference Rashid, Thomas, Shaw and Leng2) or Menon et al. (Reference Menon and Stafinski3). The number of publications reflects a cultural shift: other systemic signs exist, such as the detailed arguments for PCG involvement (4) in the European Network for Health Technology Assessment (EUnetHTA) and the description of a full method in their HTA Core Model Version 3.0 (5). Wale et al. (Reference Wale, Scott, Hofmann, Garner, Low and Sansom6) articulated in 2017 arguments demonstrating the added value of PCGs in HTAs, to be used by those of the agencies that are currently building or strengthening PCG participation. As part of the general move toward more PCG participation, a specific interest group named patient involvement (Reference Wale, Scott, Bertelsen and Meade7) was created in 2005 within the Health Technology Assessment International society (founded in 2003).
All stakeholders acknowledge the wave of these important developments. However, what Gagnon et al. (Reference Gagnon, Desmartis, Lepage-Savary, Gagnon, St-Pierre and Rhainds8) concluded in 2011, “More research is [ … ] needed to explore the necessary conditions to move toward greater patient and public involvement in HTA. [It is] important to explore local HTA stakeholders’ perspectives of patient and public involvement and the feasibility of introducing patients’ perspectives in HTA at the local level,” remains most probably true, as reflected by the many research pathways formulated by Facey (Reference Facey, Facey, Hansen and Single9) in 2017. Furthermore, Single et al. have rightly noted that the use of [practical examples described by HTA agencies] “may enable a wider understanding of different approaches to and impact of patient involvement” (Reference Single, Facey, Livingstone and Silva10).
Please note that for the sake of simplicity, only the term “Patient and Consumer Groups” will be used in this text, even though it is an approximation, because there are many subtypes of organizations in the realm of patient's representation groups (Reference Mamzer, Dubois and Saout11).
The French National Authority for Health (Haute Autorité de santé: HAS) is an independent public and scientific authority. Its missions (12) include HTA. By law, an HTA by the HAS is mandatory, before the Ministry of Health's statuses on of a product. The HTAs in view of reimbursement must be performed at the request of manufacturers, after market authorization (which in France is attained through a different agency: ANSM).
At the HAS, health technologies are assessed in different departments according to their categorization as a drug, a medical device, or a procedure, and whether an economic assessment will be performed. Final appraisal of an HTA is done by one of three dedicated committees at the HAS. In the case of a drug, the assessment carried out by the Drug Evaluation Department is brought before the Transparency Committee (CT) for appraisal. Medical devices are assessed by another department and appraised by the Medical Technology and Interventional Procedure Committee (CNEDiMTS). A number of these products will also go through the process of an economic HTA (efficiency assessment), under the final decision of the Economic and Public Health Committee (CEESP). Technologies may follow dedicated full assessment routes in the case of therapeutic group evaluations or of individual marketing authorization.
Such HTAs (for drug, medical device, or efficiency assessment) are legally required to be completed within 90 days. The time constraints are such that PCG participation was not a systematic step until 2017. When estimated mandatory, PCGs could (and still can) be invited in working groups or submitted documents to comment. These procedures of involvement are, as of now, not systematically applied to each product because of the disproportion between the workload, the available human resources, and time constraint. An additional, feasible, and complementary solution was needed to foster PCG participation.
A new procedure that we named "systematic open e-contribution" (SOEC) was designed in 2016 and officially implemented in 2017 so as to initiate a step to respond to this challenge. The word “systematic” was used because each product going through an HTA (except those that take the fast-track routes) is added online to allow PCGs to contribute; “open” refers to the fact that access to the web page is unrestricted, and that any PCG may contribute. This procedure is described below. The mechanisms in place as of early 2021 are pictured in Figure 1.
Inputs of patients’ organizations in health technology assessments performed by the HAS SOEC mechanism related to this study have the colored background.
For each drug and medical device entering the process, the dedicated nonrestricted institutional web page is updated, and it displays that the product is under review. PCGs can use a standardized, open-question form, translated and adapted from the HTAi PCIG (Health Technology Assessment International Interest Group for Patient and Citizen Involvement in HTA) Patient Group Submission Template. In addition, relevant PCGs (as main prospective users of the product, as representatives of the population within the medical indication, or possibly in contact with former participants in clinical studies) are identified and informed by e-mail that the call for contribution has been published, but other PCGs may contribute as well (open process). PCGs have 30 or 45 days, for medical devices or drugs, respectively, to send back the questionnaire after the call has been published on the HAS Web site, thus allowing the HAS to collect their input despite the limited time frame that is available. When a PCG submits a contribution, it is included in the documentation of the dossier provided to the relevant committee for final evaluation.
This mechanism by which PCG participation is enabled was named systematic open e-contribution (SOEC).
The SOEC form is available for downloading from the HAS Web site (https://www.has-sante.fr/jcms/c_2666630/fr/contribuer-a-l-evaluation-des-medicaments-et-des-dispositifs). At the time of the study, the questionnaire was the same for all HTAs. The PCG is required to provide descriptive information about their organization including the number of members, financial sources, and budget. Respondents are invited to disclose potential conflicts of interest. The PCG is also required to provide the name and position of the person responsible for filling or finalizing the response. It is clearly stated in the public description that only groups (organizations) are invited to contribute. The few persons who had sent individual contributions were kindly asked to contact a PCG that could relay their input, and documents received from individuals were not included.
Following the administrative and author's profile, the remainder of the questionnaire is purely HTA oriented and the questions are similar to those of the HTAi PCIG Template. First, the submitting PCG is asked which methods they used and how information was obtained (as described in the procedure PCG must download). Second, the PCG is asked about the impact of disease on patients, as well as on their families and informal caregivers. The third part seeks qualitative feedback on patient experience (positive or negative) with the product under assessment and/or with available therapies, as well as on the main expectations of patients regarding the new therapy at hand. As an alternative to direct experience with the new product, PCGs can provide indirect evidence, based on their knowledge gained from the literature, from survey results, or from oral communications. Lastly, PCGs are invited to sum up key messages they wish to convey to the committee.
Objectives
The aim of this study is to retrospectively describe the general profile patterns of the PCGs that participated to HTAs via SOEC and of the content of their contributions. We study here written materials related to the participation of PCGs through the new mechanism “SOEC”.
Methods
Institutions that Designed the Study
This study was designed in 2019 in collaboration with the HAS, the Department of Medical Ethics “ETREs” (UMRS1138) of the University of Paris, and Hospinnomics (Greater Paris University Hospitals: GPUH – AP-HP).
Nature and Source of the Data Studied
This is a 2-year retrospective study about the participation of PCGs in the SOEC mechanism. It is based on the analysis of written materials (documents) produced during the HTA process. Two types of documents were systematically included in the analysis for all HTAs, with one or more contribution received by SOEC between 1 January 2017 and 31 December 2018. The first set of documents was the PCG contributions (raw questionnaires). The second set was composed by the officially recorded transcript of commission deliberations, leading to the final HTA report issued by the HAS as published on its Web site.
All data used in this study were extracted from the HAS Web site.
Ethical Aspects: Anonymity and Confidentiality
This study did not raise the necessity to seek consent from PCGs, because it used texts that were free of any personal data and already published when this retrospective work was performed. When they send a contribution for an HTA by the SOEC mechanism, PCGsatient and Consumer Group are aware in advance that their contribution will be published. This is explicitly stated in the Guidance for Patient and Consumer Group contributions in the HTA for drugs and medical devices. This guidance is accessible on the HAS Web site, besides the downloadable questionnaire that PCGs are invited to use to contribute. Additionally, the HAS takes the utmost care not to publish any information that will allow the identification of persons directly or indirectly anywhere on its web pages. This rule is followed scrupulously, and the text versions that we used were those that were online. So , by design, this study does not feature aspects that will require ethical review by an independent board with regard to confidentiality (13;14).
Data Analysis Grids
To analyze data in a standardized way, items to collect need to be chosen. For this study, we constructed an analysis grid. This grid was the list of elements that were collected (featured as “what element” and “how to code it”). To build this grid, several steps were taken. A first draft (draft version 1) was written as an output of a 2-h meeting with all authors. This first draft grid was tested on a ten-random HTA sample by two investigators (CGe and MGu). This test allowed to refine the grid with regard to relevancy (some items needed to be merged and others to be split for more accuracy) for the study and in order to be actually used in a standardized way. This test operation with the once-refined grid (draft version 2) was performed once again on another random sample. This led to a draft version 3. This test-to-refine operation was iterated so as to produce a draft version 5. Finally, another 2-h meeting with all authors was convened to finalize the grid.
The final grid that was agreed upon is as follows:
-
HTA administrative information. The extraction included the reason why the assessment was performed: as “first registration” (a situation where the HAS never assessed the product) or for reassessment that can be either a 5-year mandatory renewal or triggered by manufacturers providing new data (in this latter case, to substantiate a claim for a higher reimbursement rate or for an extension of the product indication).
-
PCG description. In addition to the PCG name, the following administrative information was systematically extracted from the questionnaires: the number of members, total budget, budget from private funders, and the ratio of private resources in the annual budget that has been calculated.
-
PCG contribution methods. The methods that PCGs used were systematically screened and recorded on the following binary basis (identifiable/not identifiable) when noted anywhere in the questionnaire: the presence (or not) of a dedicated work group, or of individual interviews, or of an ad hoc survey. Other items were recorded qualitatively by copying raw sentences: reference to, or mining of, social media (mention or quote from forums or other internet platforms) and the use of scientific or medical references including references to publications from the HAS when present.
-
Disease and quality of life. The following items about a patient's quality of life and disease impact were extracted from the questionnaires as binary (identifiable/not identifiable) information: a disease's physical or psychological impact on patients or on their families or informal caregivers; financial impact on the patient, family, or caregiver; medico-scientific technical input from a PCG perspective regarding public statements or a mention of publications on a product during the appraisal meeting; knowledge of a product's use from direct or indirect experience; expectations regarding therapeutic innovation of the condition at hand.
-
Mention of the contribution during appraisal meeting. The recordings of the commission meetings were assessed on the presence/absence of an oral mention of PCG contribution(s) during the deliberation leading to the final report.
Data Extraction and Blinding
Systematic data (raw questionnaires and officially recorded transcripts of the commission's deliberations) extraction was then performed separately by two investigators (CGe and MGu). Blinding was assured by the use of separated electronic copies and remote work. More precisely, the two investigators were working in offices that were in separate towns, using individual laptops without file sharing at this stage. They filled their local version of the grids (in practice an Excel document, on their local hard drive) with information using the same data source (the HAS Web site). They did not exchange anything until they had finished their individual work.
When both had finished, their versions were compared, and any discrepancies were resolved through discussion at a second stage.
Data Recording
All data were recorded in secured drives of HAS, using Microsoft® Excel® for Office 365 MSO (16.0.12527.21378) 64 bits.
Results
PCG Contributions
Number of PCG Contributions and Related HTA Characteristics
From January 2017 through December 2018, 78 of 592 HTAs eligible for SOEC received a contribution from one or more PCG(s). This represents 13 percent of the overall figure.
This study covers the initial 2 years of the SOEC mechanism. In total, 78 assessments matched the inclusion criteria for this study from 592 candidate HTAs, as described above. For these 78 assessments, 79 PCG contributions were received by the HAS. The CT received 67 contributions for 57 assessments (20.1% of the 283 drugs assessments). The CNEDiMTS received 12 contributions for 10 assessments (3.6% of the 272 medical devices assessments). An economic evaluation was performed by the CEESP for 37 of these products; the CEESP received 13 PCG contributions related to 11 products (29.7% of the 37 assessments), including 9 new drugs and 2 medical devices. These figures are illustrated in Figure 2.
Number of products with PCG contribution.
The main clinical areas of the 57 drug assessments before the CT for which at least one contribution was submitted were as follows: oncology (15 assessments, 26% of the 57 CT assessments), infectious diseases (12 assessments, 21%), neurology (8 assessments, 15%), and endocrinology/metabolism (6 assessments, 11%). Each of all other areas represents less than 10 percent of assessments.
The areas of the ten medical device assessments before the CNEDiMTS were neurology (1 HTA), gynecology (1 HTA), medical imaging (1 HTA), rheumatology (2 HTAs), ophthalmology (1 HTA), cardiology (2 HTAs), oncology (1 HTA), and wound care (1 HTA).
The fifty-seven files submitted for HTA to the CT were mostly market-access registrations (n = 32, 56.1% of the 57 CT assessments). The remaining were reassessments (n = 14, 24.6%) and claims for an extension of product indication (n = 11, 19.3%). The ten submissions to the CNEDiMTS were for first registration (n = 5), reassessments, or an extension of product indication (n = 5).
Among those HTAs for which contributions were received, most had only one contribution. For the three specialist committees, this represented 48 out of 57 (84.2%) drug assessments before the CT, 9 out of 10 (90%) medical devices assessments before the CNEDiMTS, and 10 out of 13 (83.3%) assessments before the CEESP (both drugs or medical devices). Only a small minority of assessments benefited from multiple, that is, two to maximum three, contributions.
About the PCGs
During the study period, forty-four distinct PCGs fulfilled at least one contribution. All were French. Among these forty-four PCGs, two-thirds sent only one contribution (n = 29), the remaining one-third was split between those providing two contributions (n = 7) and three or more contributions (n = 8). Multiple contributors (n ≥ 3 contributions) represented patients treated in various areas. The active contributor submitted ten contributions in the field HIV-hepatitis; another two PCGs active in the hepatitis field submitted a total of seven contributions (n = 4 and 3, respectively). Other PCGs that submitted three or more contributions were in the fields of lymphoma (n = 7), renal diseases (n = 6), multiple sclerosis (n = 6), leukemia (n = 4), myeloma, lysosomal diseases, and melanoma (for each, n = 3).
Of the forty-four PCGs, thirty indicated their membership. An equal share of PCGs had fewer than 500 members (n = 14) or 500 to 5,000 members (n = 14); for the two remaining PCGs, one had between 5,000 and 10,000 members and another between 11,0000 and 15,000 members.
Annual budgets disclosed in the contributions ranged from less than 10,000€ (n = 5), from 10,000 to 50,000€ (n = 6), between 50,000 and 100,000€ (n = 6) and more than 100,000€ (n = 24). Only one PCG did not disclose its annual budget. Nearly three quarters of the PCGs who submitted a contribution (n = 32) disclosed private funding; two-thirds (n = 21) relied on private funding for over 30 percent of their annual budget.
Use of the Open-Question Template/SOEC Form
Of the 79 PCG contributions, 5 (6.3%) did not use the HAS template but contributed documents or free texts. The form was otherwise used as an indicative outline, as intended by the HAS, contributing information in the way the PCG deemed appropriate.
Names of the corresponding author of the submitted contributions were found in forty-seven contributions (59%). In most contributions, extra information was provided compared with the HAS template, especially for the three following categories: status of the author(s) in the PCG (director, president, patient, head of department, etc.); the length of time the author(s) had been active in the PCG; academic titles and other relevant qualifications. For example, “X, Assistant Director, responsible for medical and scientific affairs, employee at XXX for 19 years”; “X, psychologist, responsible for family relations at the association for 5 years and in the field of disability for 15 years; “Patients, relatives, and healthcare teams.”
Methods/Information Used by PCGs to Compile Contributions
Method(s) used to prepare the submission could be identified in sixty-six contributions (84%). Another thirteen contributions did not mention any information on how the contribution had been drawn. Various methods were used, such as circulation of a questionnaire (n = 20), individual interviews (n = 7), and organization of dedicated meetings for the contribution (n = 3). Social networks and Internet Web sites also proved useful sources of information: PCGs mentioned their use in fourteen contributions (by pooling data from previous work, by generating new data using Internet-crawling, or by initiating new discussions in the forums). When they reported the methods used in their contribution, PCGs used a combination of several approaches (questionnaire, meetings, etc.) in half of the cases (n = 34).
Thirty-eight of the seventy-nine contributions (48%) referred to already published materials as pieces of evidence or background information. Among them, half referred to at least one result from previous work or a study performed by the PCG itself (n = 19, 50%). In most cases, explicit reference to scientific literature (Wiley Online Library, Cochrane, NCBI, Journal of Hepatology, Journal of Medical Virology, etc.) was made (n = 31, 81%). References to HTAs from the HAS, national diagnostic or care pathways were also used in rare diseases (n = 9, 24%).
Quality of Life
Information on a patient's quality of life was found in 74 contributions (93.7%), and a patient's psychological impact appeared specifically in 54 occurrences (68.4%). Contributions mentioned further information about the financial impact on patients or on their families (n = 26, 32.9%), physical or psychological impact on families or informal caregivers (n = 58, 73.4%), direct experience and use of the product by PCG members (n = 48, 60.8%), the indirect experience of the product known by the PCG (n = 36, 45.6%), and the PCG's expectations about the new therapeutics at hand (n = 74, 93.7%). The following quotes illustrate the latter topic:
• “A new effective second-line therapy that could target molecular characteristics, [ … ], limit toxicities as much as possible, lead to a complete remission”
• “Avoid overdose situations that can be fatal or lead to heavy consequences”
• “Reduce the delay for intervention access”
• “Having a product that can be immediately handy in case of an emergency at home”
• “A reduced number of daily doses”
• “A substitute drug to be taken orally”
• “Promote outpatient autonomy by enabling use out of hospital”
• “Resuming more quickly a normal life”
• “Providing smaller capsules that are easier to swallow, especially if you have to take four”.
Additional Remarks
In sixteen questionnaires, we found additional remarks unrelated to the evaluation of the drug or device. One PCG complained about the short time window to contribute (one month), thus impeding their ability to perform an in-depth analysis. One association took advantage of the contribution to request a hearing before the CT Committee to allow them to communicate directly. We also expressed the hope that their contributions make an impact: “We hope that our contribution will have been able to shed light on the evaluation of medicines.”
Deliberations of the Appraisal Committees
The officially recorded transcripts of the meetings of the appraisal committees show that PCG contributions received through the SOEC process were presented orally and discussed in 88 percent of the cases. The frequency of these mentions tended to increase monthly. This figure reached 100 percent for the last 2 months of the study period.
Discussion and Conclusions
This retrospective study examines the contribution from PCGs among those that willingly contribute to HTAs carried out at marketing-authorization or re-evaluation stages and from initiation of the SOEC process using an open-questions template available on the Web site of the HAS.
The limited timeframe for contribution (30 or 45 days) is signaled by some associations as too short; however, this timeline is primarily linked to HAS legal time constraints (90 days from the manufacturer's application to committee deliberation). In addition, HAS had been informally warned by several PCGs that human resource constraints within their organizations compromise their ability to effectively respond to such requests. The issue of burden of participation for PCGs was recently raised in the literature (among several other publications) by Facey et al. (Reference Facey, Bedlington, Berglas, Bertelsen, Single and Thomas15). This should be investigated by further research.
When methods are described by PCGs, two trends tend to emerge. The first trend is experiential: individual testimonies, unique or appended to each other, presented in quotation marks, allow PCGs to give a direct and immediate voice to users in the questionnaire itself. In this case, the strengths and weaknesses of the contribution lie in the possibility, or impossibility, of committee members to relate individual testimonies with scientific problems that the technical assessment allowed them to pinpoint.
The second trend is evidence synthesis: a sample of quantitative or qualitative evidence is synthetized allowing PCGs to offer a general and well-grounded overview of patient experience. The results reported accordingly are more akin to those displayed in drug or medical device applications and thus look more familiar to committee members; this can ease their integration in HTAs, but also raises another issue, namely whether such results should be subjected to the same level of technical inquiry as those from the industry. This might increase the workload on the HAS in a regulatory-constrained timeframe, lead to a grading of PCG contributions (possibly reinforcing identified inequalities among PCGs), and arguably alter the nature of the process aimed at revealing patient and user perspectives on health innovations.
From the above results, it appears that the two types of knowledge can coexist within one contribution, and it is worth remembering that the original purpose of instituting SOEC contributions was to bring additional knowledge on a product, which could not be provided by the evidence already available to the HAS. Some narrative, experiential elements and some expectations, preferences, or doubts emanating from PCGs need to emerge and to feed the deliberations of committees. Other pieces of knowledge, more in line with clinical research methods, must also be brought to the attention of these committees to the extent that they are deemed relevant by PCGs. For these reasons, the patient association contribution process set up at the HAS adopted an open and nonprescriptive approach from its inception; the results and the descriptions detailed above about how PCGs actually made the template their own and the process in the two 2 years tend to strongly back this initial orientation. The frameworks and the conditions of interactions between HTA agencies and PCGs probably deserve a fresh look (Reference Facey16).
France does not have a national database listing all PCGs by disease area. HAS communication to patient associations or consumer groups is, thus, based on the HAS's network and collaborations going back more than 10 years with 480 different PCGs to date. Centralizing data and contacts and sharing them with other public institutions are a matter of reflection and shall be pushed forward to comply with the European data protection regulation requirements, but these bring along with them some challenges. In addition, some disease areas might be underserved in terms of advocacy structures or capacity to contribute to the process, even in cases of high prevalence (gastric ulcer being an example). The implementation of SOEC was also the occasion to confirm that some product indications relate to conditions for which virtually no patient (even no peers, nor relatives) experience can be described (innovative pathways, no French patient or PCG member included in clinical trials; the complex use of intensive care products in severe cases can be another illustration).
One of the characteristics of the SOEC process is that it is open to any PCG willing to step in. The results show a great variability among the contributors. In terms of size, membership ranges from very small groups (ten members) to major organizations (more than 1,000 members). Unsurprisingly, financial capacity also varies greatly, with reported yearly budgets ranging from 4,000€ to 158,600,000€. We found some PCGs showing expert ability to express university-type evidence-synthesis and to use suitable quantitative or qualitative methods, other PCGs simply brought up interesting narratives. There is no specific requirement on methods for making contributions, so all raw contributions were brought as such to the committees. It appears from the recorded transcripts of the three committees that received PCG contributions that both kinds of input can be useful for their assessments (this information was not retrieved as a parameter for this study). It is obvious that historically strong or extremely popular TV-promoted consortia have greater human resources and capacity to contribute than smaller PCGs, irrespective of the health needs or of the value of information about patient experience in a given HTA. The improvement that the HAS is presently seeking is to build capacity and to empower PCGs by training staff or volunteers. Transparency in terms of potential conflicts of interest is demanded (specific questions in the form) so that the committees are equipped with all relevant information.
Funding sources also varied greatly, with a significant proportion being private funding (over 30% in a majority of PCGs who reported these data). By design, industry funding is related to the products they manufacture, thus creating possible biases or conflicts of interest (Reference Vanstone, Abelson, Bidonde, Bond, Burgess and Canfield17). These aspects are indeed considered for each HTA by committee members.
This study has several limitations, the main one being that it is not possible to directly infer general rules from its results. However, describing the modalities of PCGs in HTAs remains important, because this topic is rapidly evolving (Reference Huls, Whichello, van Exel, Uyl-de Groot and de Bekker-Grob18–Reference Weeks, Polisena, Scott, Holtorf, Staniszewska and Facey20).
Funding
CGe, JAV, MGu, and MGa were employed by the HAS (public funds); JKD and MFM were employed by the Sorbonne University (public funds). This study was not funded by any other parties.
