Nearly 20 years ago in 2004, the European Commission recognized the need to establish a sustainable European Network for Health Technology Assessment (EUnetHTA) (1). EUnetHTA Joint Action 3 (JA3) followed the EUnetHTA Project, EUnetHTA Collaboration, and two previous Joint Actions with the aim of defining and implementing a sustainable model for scientific and technical cooperation on health technology assessment (HTA) in Europe (2).
Within the framework of JA3, Early Dialogues (EDs) were carried out by work package 5A, led by HAS (France) and co-led by G-BA (Germany). An EUnetHTA ED is a non-binding scientific advice given by European HTA bodies (HTAb), before the start of pivotal clinical trials, to improve the quality and appropriateness of the evidence produced by the developers in view of future HTA or reassessment.
EUnetHTA EDs enabled the exchange between Applicants (health technology developers [HTDs] requesting the ED) and participating HTAbs at an early stage in the clinical development process to allow for the integration of HTA requirements (e.g., choice of patient groups and intervention [with consideration of concomitant treatments, dosage, length of treatment, comparators, and relevant outcomes] in the study design [pivotal trials and postlaunch evidence generation (PLEG)] (Reference Moseley, Vamvakas and Berntgen3)) and the economic evaluation plan across multiple European countries.
The aim of this paper is to provide a narrative summary on the methods, output, and process of EUnetHTA EDs carried out under JA3.
Methods
Relevant documents were identified and evaluated on the EUnetHTA Web site (4) and intranet. Additionally, EUnetHTA ED Secretariat records covering EDs from 2017 to 2021 were reviewed and evaluated. Key guidance documents related to the ED process were analyzed and the essential information was extracted to describe the procedures. The available information on patient participation and the involvement of pharmaceutical companies was evaluated based on feedback questionnaires received by the EUnetHTA ED Secretariat and evaluated. A qualitative analysis for the first twenty-one completed EUnetHTA EDs (three multi-HTAs and eighteen Parallel Consultations [PCs]) was conducted by examining the Briefing Books received, the List of Issues, and Final Written Recommendations including the topics covered, HTAb alignment, and the impact of EUnetHTA Final Recommendations on clinical development (unpublished data, 2020). The qualitative analysis also examined patient input. Reviewing each patient interview allowed for the classification of the main topics where patients/patient representatives contributed to the process and the reflection of their input in the EUnetHTA Final Written Recommendations. This was done by identifying the overlap between HTAb recommendations and the patient’s position for every subdomain and noting if reference to a patient’s input was not made. The analysis method was applied equally for all three approaches for patient involvement and where several approaches were used for one ED, the contribution was analyzed jointly with a merged result as output. The patient’s input was judged as “Full” only if 100 percent of the input was reflected in the HTAb recommendations, “Partial” if their input was taken into account but not in its entirety, and “No” if the patient’s view was not reflected in the HTAb common position.
Results
Process
HTDs could choose from two options offered by EUnetHTA for EDs on pharmaceuticals:
-
1. EMA-EUnetHTA PCs: tripartite meetings involving multiple HTAbs, European Medicines Agency (EMA), and the Applicant allowing for prospective and timely advice to the Applicant to integrate specific HTA and regulatory needs into the development plan and, therefore, fulfil the evidence requirements of both at the same time.
-
2. EUnetHTA Multi-HTA EDs: bilateral meetings involving multiple HTAbs and the Applicant to integrate specific HTA requirements into the development plan of a health technology to fulfil the evidence requirements of HTAb.
Framework of the EUnetHTA HTAb Collaboration and Submission by HTDs
In the 10 months leading up to the official launch of the procedures, templates including the unique dossier on the health technology (Briefing Book) and guidance documents were established, published, and implemented. Where necessary, these were prepared and agreed in cooperation with EMA. The standardization permitted applicants to use one set of documents to request an ED from both EUnetHTA HTAb and EMA simultaneously. The templates were nearly identical for Multi-HTA EDs. Figure 1 reflects the EUnetHTA ED request process and outcome based on the request type and the prioritization by the Early Dialogues Working Party (EDWP).
Figure 1.
European Network for Health Technology Assessment Early Dialogue request process and outcome based on request type and Early Dialogues Working Party prioritization.
All EUnetHTA EDs were supported by the EUnetHTA ED Secretariat at HAS, thereby benefiting from HTA scientific and administrative coordination and a concerted effort to find agreement among the participating HTAbs. The procedure was developed considering the different national procedures existing at that time (G-BA, HAS, and NICE) to avoid duplication. The ED process was approximately 3–5 months in duration starting from reception of the draft Briefing Book and the key steps were as follows:
-
1. Applicant submits Draft Briefing Book.
-
2. If needed, the EDWP requests clarifications (possible at any time during the procedure).
-
3. Applicant submits Briefing Book.
-
4. Applicant receives List of Issues from HTAbs (and EMA for PCs).
-
5. Applicant responds to the List of Issues.
-
6. Face-to-Face Meeting (tripartite with EMA if PCs).
-
7. Applicant receives EUnetHTA Final Consolidated Recommendations (including national specificities if any, e.g., requests for additional comparators based on the standard of care, in an annex).
EDWP and Their Contribution to ED
The EDWP was composed of experienced HTAbs (those who have participated previously in the SEED Collaboration (Reference Harousseau, Pavlovic, Mouas and Meyer5), and/or earlier EUnetHTA ED pilots, and/or EMA Parallel Scientific Advice (Reference Tafuri, Lucas and Estevão6;Reference Wang, McAuslane and Gardarsdottir7)) with a national mandate and national experience in conducting EDs on pharmaceutical products. This was balanced with the addition of less experienced HTAb. The composition of the EDWP at the end of JA3 was as follows: AEMPS representing Spain (with AETSA and CatSalut), AIFA for Italy (RER as an alternate), G-BA for Germany, HAS for France, NIPN for Hungary, and NOMA for Norway. NICE for the United Kingdom participated in the EDWP for Multi-HTA EDs only. Several EDWP members shared their seats with other national or regional HTAb to share the workload. For those sharing a seat, the decision on the participating agencys for an ED was made on a case-by-case basis based on availability.
The role of the EDWP was to evaluate and prioritize all ED requests and to participate in all EUnetHTA EDs. The EDWP thereby ensured the high-quality and consistency of EUnetHTA EDs and contributed, together with the ED Secretariat, to the improvement of the process over time.
Additionally, the members of the EDWP were responsible for acting as Scientific Coordinator (SC) and Rapporteur (R) for all EUnetHTA EDs (8). These central leadership roles assured the scientific coordination of each ED by requesting clarifications from the Applicant, drafting the initial List of Issues for review and comments by the HTAb, interviewing patients/patient representatives fully participating in the ED, drafting the initial Final Recommendations for review and comments by the HTAb, and representing the “voice” of the participating HTAb for all topics on which there was consensus.
ED Prioritization
Due to the inherent resource constraints of EUnetHTA JA3, all requests for EDs could not be accepted. Therefore, a prioritization system was set up based on a set of selection criteria developed by the EDWP. These criteria were published on the EUnetHTA Web site (8) and included in the EUnetHTA ED Guidance documents (9;10). The EUnetHTA Selection Criteria state that the product should aim to bring added benefit to patients, that is, by:
-
• a new mode of action for the indication,
-
• targeting a life-threatening or chronically debilitating disease, and
-
• responding to an unmet need of patients (no treatment or only unsatisfactory treatment available).
The EDWP decision and its basis were provided to the company by the ED Secretariat.
During the first 3 years of JA3, companies submitted requests monthly according to the published timelines. Due to the high number of demands posing capacity challenges and after the pause of ED activities during the first half of 2020 related to the Covid-19 pandemic, the restart of ED activities during the summer of 2020 provided the opportunity to establish the selection of products based on an “open call.”
Patients and or patient representatives were systematically involved in EUnetHTA EDs using the approaches for patient involvement (4)) developed by the lead and co-lead partners.
Conflict of interest management was systematic and ensured for all participants (HTAb and external experts alike), according to the EUnetHTA JA3 Procedure Guidance for handling Declaration of Interest (11).
Quality Management
Procedures and guidelines were created to ensure the quality of EUnetHTA EDs, and four key steps for quality management were established.
-
1. A systemic internal review and agreement process of each ED List of Issues and Final Recommendations by all participating HTAb.
-
2. External review by the ED Secretariat for each ED to monitor the adhesion to the procedure and to insure the scientific content of the Final Recommendations.
-
3. A professional medical editor was used to ensure the language quality and continuity of the document.
-
4. Solicitation from applicants for insight and feedback on the quality and usefulness of the procedure and ED Final Recommendations through a standardized feedback questionnaire complemented by regular meetings with the European Federation of Pharmaceutical Industries and Associations and stakeholder groups.
The EDWP met every 2 months with the participation of the ED Secretariat to review and adjust the procedure considering the feedback received.
Output
From June 2017 to May 2021, 113 requests for pharmaceutical EDs were received. Of the requests, 82 percent (93/113) were for PCs of which 32 were accepted based on the EDWP selection criteria. Twenty requests for Multi-HTA EDs were received, resulting in six EDs. In all, thirty-eight pharmaceutical EDs were conducted in JA3. Table 1 provides an overview of the requests received during EUnetHTA JA3 and the resulting number of EDs. Most EDs were requested by “big pharma” with SMEs representing only 15 percent of requests.
Table 1.
Pharmaceutical Early Dialogues in Joint Action 3: Overview of Requests Received and Early Dialogues Conducted
The most frequent reason cited by the EDWP for refusing an ED request (73) was that the product did not meet the eligibility criteria. In these cases, the product did not represent a new mechanism of action in the indication, and/or the unmet need criterion was not met (i.e., other treatments available), and/or the severity of disease criterion was not met. There were three exceptional cases of requests being outside the remit of some EDWP members, and lastly several refusals based on lack of HTAb resources at the end of JA3 despite the products meeting the selection criteria.
Analyzing the EDs, a Successful HTAb Alignment and ED Impact
The number of HTAbs taking part in each ED procedure ranged from 4 to 9 with an average of 6.7. The mix of experienced organizations and those less experienced allowed for capacity building within EUnetHTA, and by the end of JA3, the newer members of the EDWP had also taken on the SC and R leadership roles. Enlarging participation, capacity, and competence building will likely be vital for the future development of these early interactions (not least by facilitating adoption and buy-in of outcomes/harmonization across Europe).
In analyzing the common HTAb recommendations, the frequency (percent) of different levels of agreement for each domain was calculated. The domain “health economics” was omitted from the analysis as not all EDWP members conduct health economic evaluations.
The comparisons of level of agreement between HTAbs were based on four categories:
-
- Full = Alignment (100 percent common HTA responses);
-
- Full* = Alignment* (100 percent common HTA responses) together with national specificities outlined by at least one HTAb;
-
- Partial = When >50 percent of HTAb agreed on a common response; and
-
- No common agreement = When <50 percent of HTAb agreed on a common response.
When HTAbs reached a clearly stated common recommendation with no sentences in the Final Recommendations stating that some HTAbs did not agree on the consolidated recommendations, full agreement (“Full”) was allocated. The allocation of “full alignment” did not prevent the supplementation of national specifications from HTAbs. In such cases, the category “Full*” was given for the analysis. National specificities, any additional specific request, that is, additional subgroup, additional outcomes, additional comparators, and so on by HTAbs were also collected and analyzed by subdomains.
In summary, the analysis in Table 2 shows:
-
• The population, intervention, comparator, and outcome (PICO) framework provides a standard format for specifying research questions. These criteria and study design were discussed in almost all the EDs, the topic of health economics was addressed in more than half, and PLEG in one third.
-
• A high percentage of alignment between HTAbs with more than 80 percent full alignment on all PICO items, for example:
-
o RCTs clearly remain the gold standard and are requested commonly by HTAbs to obtain high quality data for HTA.
-
o The choice of an appropriate comparator is crucial, especially when the study population has a considerable degree of heterogeneity. A well-defined study population is directly related to the recommendations for the comparator arm.
-
o Quality of Life data and meaningful, patient reported outcomes (PROs) are important for HTAb.
-
o Dealing with missing data is a central topic for HTAb.
-
• Although approaches to recommendations on PROs could differ between agencies performing health economics evaluation and those focusing on clinical evaluation, there was alignment on the need to have PRO data gathered during development.
-
• Modifications of development plans were proposed by applicants in twelve out of twenty-one (57 percent) EDs after receiving the EUnetHTA List of Issues including major changes, for example, design adaptation, primary end point choice, changes of comparator, and the addition of a study, but also population criteria, intervention, and other outcomes. Further clinical development adaptations can be expected after applicants have received the final EUnetHTA recommendations.
In addition to the qualitative analysis, PLEG was increasingly addressed in EDs with the participating HTAbs providing advice on PLEG topics in fourteen of the thirty-two PCs(Reference Moseley, Vamvakas and Berntgen3).
Table 2.
Results of Analysis on Alignment between the HTAbs
Establishing Successful Patient Contribution
Eighty-five percent of the EDs in JA3 involved patients (see Table 3). A total of forty-five patients were involved in the thirty-eight EDs; 100 percent were satisfied with the three approaches proposed (interviews and participation in meetings) according to the analysis of twenty-five patient feedback questionnaires. The majority (38/45) of patients had access to the Briefing Book provided by the applicant and thus provided valuable input regarding the submitted development plan.
Table 3.
Summary of Patient Involvement in EUnetHTA Early Dialogues by Approach
Of the twenty-one EDs analyzed in the qualitative analysis (see Table 4), sixteen (seventy-six percent) were performed with patient involvement. The patient contribution was focused on the choice of population to be included (inclusion/exclusion criteria), comparator and outcomes for almost all EDs, but also on study duration and intervention.
Table 4.
Results of the Qualitative Analysis on Patient Input during EUnetHTA Early Dialogues
The percentage of patient input included for the domains/subdomains was high: 67–100 percent.
Overall, the high percentage of inclusion of patient input for the domains/subdomains in the recommendations highlights the relevance of patient contribution to the ED process.
Patient feedback on intervention, potential difficulties of future treatment usage (frequency of administration, acceptability of injection, convenience of oral treatment, etc.), and issues related to current treatment administration (need for hospitalization or specific test or issue of access to treatment) that could become future concomitant treatment were less often reflected in the HTAb recommendations. This could be explained by the lack of questions/issues initially raised on the topic, but also by the fact that the benefit of the way an intervention is administered comes only into effect if the clinical benefit is proved. Additionally, the importance of specific symptoms was often not taken into account or only done so at the country level (i.e., it was included only in the individual HTAb recommendations). The latter derives from different approaches of the HTAbs for providing feedback on outcomes; for example, some HTAbs proactively provide a list of patient relevant outcomes for which patient feedback was considered.
Involvement of External Experts
Although the involvement of patients and patient representatives in the EDs was successful, the identification and recruitment of European-level clinical experts (i.e., having knowledge of the clinical practices in the different European countries) was challenging. While most of the EDWP members regularly included experts in their work, this was considered national-level input that would not necessarily scale to the European level due to variations in care pathways, and so on, in each country. Although consensus was reached that an expert could be used at the EU level, multiple calls for participation by the ED Secretariat remained unanswered.
Collaboration with EMA
From drafting the guidance document to the creation of shared templates and processes for exchanging information, and the systematic joint chairing of PC meetings, close collaboration with EMA was essential for the conduct of PCs. The joint conduct of thirty-two EDs allowed EMA and HTAbss to better understand their respective remits and offer more efficient scientific recommendations to HTDs. The exchanges between EMA and EUnetHTA HTAbs at multiple stages of the procedure prior to the meeting with the applicant allowed for better preparation for the meeting; this represented added value compared to previous parallel scientific advice.
Process Improvement During JA3
The EUnetHTA ED process evolved during the project period based on feedback received from all stakeholders involved in EDs (HTAbs, EMA, HTDs, and patients). The feedback allowed for the simplification of EUnetHTA internal processes and acceleration of the ED process itself. The organization of the List of Issues document according to PICO and the incorporation of PROs as proposed by the EDWP are also examples of these modifications and common achievements to further improve the efficiency of the process. During the JA3 prolongation period, an ED without a face-to-face meeting was tested, thus allowing for a written-only procedure option targeting less-complex requests within a shorter timeframe.
Some Barriers Encountered During JA3, to be Considered in the Future
Human and Financial Resource Constraints
Projects and Joint Actions function on limited and strict budgets.
One of the hurdles during JA3 EDs was resource constraints for HTAbs and recruitment of SC and R; there was a need for a sustainable ED financial mechanism that could allow HTAbs to run more EDs as requested. Although an Early Dialogues Financing Mechanism (EDFM) was developed and a framework for a fee-for-service model established and agreed upon by all participating HTAb, it could not be piloted during JA3. Nevertheless, the EU HTA Regulation provides the possibility for the creation of a fee-paying mechanism (Article 31.1C) (12) based on the experience of the first 3 years. In this case, the information gathered for the EDFM may be useful in its development.
The Future of EU-Level EDs
Four key areas of recommendations were identified for a future system of European HTA EDs and highlighted in the EUnetHTA White Paper on a Future Model of EU HTA Collaboration (13), specifically the organizational framework, the conduct of EDs, IT needs, and the involvement of experts.
First, the organizational framework as setup during JA3 could be maintained, that is, the structure of the ED activity, particularly with a one-stop for all requests, the EDWP as a dedicated and experienced subgroup of HTAbs, the SC and R roles, and the selection criteria to assure that EDs are conducted for innovative products for which an ED presents an added value. The systematic involvement of patients should be preserved. As the central entry point, the ED Secretariat witnessed the high demand from HTDs and was also able to recruit patients/patient representatives to participate in EDs rather easily, thus demonstrating the high interest of EDs from these two key stakeholder groups. PCs with EMA were also an added benefit for HTDs, and therefore this collaboration should continue in the future.
Anticipation of those areas where advice can be made for several technologies in the same indication at the same time would be ideal and would allow for indication-specific advice that would be applicable to all requests in the same indication. Additionally, such an approach would be more efficient both financially and in terms of time spent.
To further enable for better planning, a rotating schedule of EDWP partners to take over the coordination functions has been proposed. The Open Call provided another approach to select all the EDs that would be carried out during a given period at the same time. This efficiently allowed the HTAb to plan and share the workload. However, capacity needs to be built for future HTA consultations to meet the high demand for early advice. Indeed, capacity is one of the major challenges for the future collaboration and will prove vital to the future success and adaptation. Broader HTAb involvement will also facilitate outcome adaption and mutual understanding (potentially some harmonization) of methodological approaches across the European Union.
Second, in terms of the recommendations resulting from an ED, the further development and use of PROs to increase evidence on patients’ preferred outcomes and patient-based outcomes should be requested in clinical trials. That information should be a criterion to be systematically discussed in the development plans.
Further developing the collaboration with EMA, for instance, the qualification of registries for PLEG should be explored since this advice is often provided by EMA at a much earlier time than the ED, and thus it is often difficult for HTAbs to provide any additional or conflicting input at the time of the ED.
Third, a fluid and efficient IT structure is needed allowing not only for the execution of the project management aspects of an ED, but for the secure and efficient sharing of documents with participants external to EUnetHTA.
Fourth, and lastly, the involvement of patients and clinical experts should be strengthened. Although the involvement of patients and patient representatives was successful, further effort is needed to integrate the patient position on outcomes (especially symptoms and side effects) into the final recommendation. Additionally, work is needed to incorporate clinical experts in the procedure. A transparent approach should be implemented for these experts much as there already is for patients.
Conclusions
On average, more than six HTAbs participated in each EUnetHTA ED and, in the majority of instances, were conducted in parallel with the EMA allowing for HTDs to receive timely and pertinent advice from both HTAbs and regulators on their proposed development plans. Incorporating the advice into the product development to meet the evidence requirements of both Regulators and HTAbs may contribute to gaining European approval and positive HTA assessment. The conduct of EDs was one of the successes of the JA3, in terms of collaboration, common procedures, scientific alignment, common general principles, processes gradually improved, independence, capacity building, and inclusiveness. They allowed HTAbs to exchange and better understand each other’s specificities and common points. This is further reflected in the volume of requests from industry and the positive feedback received from both industry and patients. Considering the EU Regulation on HTA and the Joint Scientific Consultations (JSCs), it seems particularly useful that EDWP members gained detailed knowledge of each other’s positions and frameworks: It will make the implementation of the legal requirements smooth in practice.
The well-established structure for EDs could, with a few adjustments, be reproduced in the future European HTA system. EDs will be continued as JSCs in the next 2 years through “EUnetHTA 21” (14), a service contract for the provision of joint work supporting the continuation of European cooperation on HTA before the HTA Regulation becomes applicable. During this period, an intensive exchange on EU HTA methods development and JSCs is foreseen to advise HTDs on the clinical development plan in preparation for future joint assessments. The resource constraints were serious and must be addressed in the future by creating a sustainable, reliable centrally organized funding system.
You have
Access
Nearly 20 years ago in 2004, the European Commission recognized the need to establish a sustainable European Network for Health Technology Assessment (EUnetHTA) (1). EUnetHTA Joint Action 3 (JA3) followed the EUnetHTA Project, EUnetHTA Collaboration, and two previous Joint Actions with the aim of defining and implementing a sustainable model for scientific and technical cooperation on health technology assessment (HTA) in Europe (2).
Within the framework of JA3, Early Dialogues (EDs) were carried out by work package 5A, led by HAS (France) and co-led by G-BA (Germany). An EUnetHTA ED is a non-binding scientific advice given by European HTA bodies (HTAb), before the start of pivotal clinical trials, to improve the quality and appropriateness of the evidence produced by the developers in view of future HTA or reassessment.
EUnetHTA EDs enabled the exchange between Applicants (health technology developers [HTDs] requesting the ED) and participating HTAbs at an early stage in the clinical development process to allow for the integration of HTA requirements (e.g., choice of patient groups and intervention [with consideration of concomitant treatments, dosage, length of treatment, comparators, and relevant outcomes] in the study design [pivotal trials and postlaunch evidence generation (PLEG)] (Reference Moseley, Vamvakas and Berntgen3)) and the economic evaluation plan across multiple European countries.
The aim of this paper is to provide a narrative summary on the methods, output, and process of EUnetHTA EDs carried out under JA3.
Methods
Relevant documents were identified and evaluated on the EUnetHTA Web site (4) and intranet. Additionally, EUnetHTA ED Secretariat records covering EDs from 2017 to 2021 were reviewed and evaluated. Key guidance documents related to the ED process were analyzed and the essential information was extracted to describe the procedures. The available information on patient participation and the involvement of pharmaceutical companies was evaluated based on feedback questionnaires received by the EUnetHTA ED Secretariat and evaluated. A qualitative analysis for the first twenty-one completed EUnetHTA EDs (three multi-HTAs and eighteen Parallel Consultations [PCs]) was conducted by examining the Briefing Books received, the List of Issues, and Final Written Recommendations including the topics covered, HTAb alignment, and the impact of EUnetHTA Final Recommendations on clinical development (unpublished data, 2020). The qualitative analysis also examined patient input. Reviewing each patient interview allowed for the classification of the main topics where patients/patient representatives contributed to the process and the reflection of their input in the EUnetHTA Final Written Recommendations. This was done by identifying the overlap between HTAb recommendations and the patient’s position for every subdomain and noting if reference to a patient’s input was not made. The analysis method was applied equally for all three approaches for patient involvement and where several approaches were used for one ED, the contribution was analyzed jointly with a merged result as output. The patient’s input was judged as “Full” only if 100 percent of the input was reflected in the HTAb recommendations, “Partial” if their input was taken into account but not in its entirety, and “No” if the patient’s view was not reflected in the HTAb common position.
Results
Process
HTDs could choose from two options offered by EUnetHTA for EDs on pharmaceuticals:
1. EMA-EUnetHTA PCs: tripartite meetings involving multiple HTAbs, European Medicines Agency (EMA), and the Applicant allowing for prospective and timely advice to the Applicant to integrate specific HTA and regulatory needs into the development plan and, therefore, fulfil the evidence requirements of both at the same time.
2. EUnetHTA Multi-HTA EDs: bilateral meetings involving multiple HTAbs and the Applicant to integrate specific HTA requirements into the development plan of a health technology to fulfil the evidence requirements of HTAb.
Framework of the EUnetHTA HTAb Collaboration and Submission by HTDs
In the 10 months leading up to the official launch of the procedures, templates including the unique dossier on the health technology (Briefing Book) and guidance documents were established, published, and implemented. Where necessary, these were prepared and agreed in cooperation with EMA. The standardization permitted applicants to use one set of documents to request an ED from both EUnetHTA HTAb and EMA simultaneously. The templates were nearly identical for Multi-HTA EDs. Figure 1 reflects the EUnetHTA ED request process and outcome based on the request type and the prioritization by the Early Dialogues Working Party (EDWP).
European Network for Health Technology Assessment Early Dialogue request process and outcome based on request type and Early Dialogues Working Party prioritization.
All EUnetHTA EDs were supported by the EUnetHTA ED Secretariat at HAS, thereby benefiting from HTA scientific and administrative coordination and a concerted effort to find agreement among the participating HTAbs. The procedure was developed considering the different national procedures existing at that time (G-BA, HAS, and NICE) to avoid duplication. The ED process was approximately 3–5 months in duration starting from reception of the draft Briefing Book and the key steps were as follows:
1. Applicant submits Draft Briefing Book.
2. If needed, the EDWP requests clarifications (possible at any time during the procedure).
3. Applicant submits Briefing Book.
4. Applicant receives List of Issues from HTAbs (and EMA for PCs).
5. Applicant responds to the List of Issues.
6. Face-to-Face Meeting (tripartite with EMA if PCs).
7. Applicant receives EUnetHTA Final Consolidated Recommendations (including national specificities if any, e.g., requests for additional comparators based on the standard of care, in an annex).
EDWP and Their Contribution to ED
The EDWP was composed of experienced HTAbs (those who have participated previously in the SEED Collaboration (Reference Harousseau, Pavlovic, Mouas and Meyer5), and/or earlier EUnetHTA ED pilots, and/or EMA Parallel Scientific Advice (Reference Tafuri, Lucas and Estevão6;Reference Wang, McAuslane and Gardarsdottir7)) with a national mandate and national experience in conducting EDs on pharmaceutical products. This was balanced with the addition of less experienced HTAb. The composition of the EDWP at the end of JA3 was as follows: AEMPS representing Spain (with AETSA and CatSalut), AIFA for Italy (RER as an alternate), G-BA for Germany, HAS for France, NIPN for Hungary, and NOMA for Norway. NICE for the United Kingdom participated in the EDWP for Multi-HTA EDs only. Several EDWP members shared their seats with other national or regional HTAb to share the workload. For those sharing a seat, the decision on the participating agencys for an ED was made on a case-by-case basis based on availability.
The role of the EDWP was to evaluate and prioritize all ED requests and to participate in all EUnetHTA EDs. The EDWP thereby ensured the high-quality and consistency of EUnetHTA EDs and contributed, together with the ED Secretariat, to the improvement of the process over time.
Additionally, the members of the EDWP were responsible for acting as Scientific Coordinator (SC) and Rapporteur (R) for all EUnetHTA EDs (8). These central leadership roles assured the scientific coordination of each ED by requesting clarifications from the Applicant, drafting the initial List of Issues for review and comments by the HTAb, interviewing patients/patient representatives fully participating in the ED, drafting the initial Final Recommendations for review and comments by the HTAb, and representing the “voice” of the participating HTAb for all topics on which there was consensus.
ED Prioritization
Due to the inherent resource constraints of EUnetHTA JA3, all requests for EDs could not be accepted. Therefore, a prioritization system was set up based on a set of selection criteria developed by the EDWP. These criteria were published on the EUnetHTA Web site (8) and included in the EUnetHTA ED Guidance documents (9;10). The EUnetHTA Selection Criteria state that the product should aim to bring added benefit to patients, that is, by:
• a new mode of action for the indication,
• targeting a life-threatening or chronically debilitating disease, and
• responding to an unmet need of patients (no treatment or only unsatisfactory treatment available).
The EDWP decision and its basis were provided to the company by the ED Secretariat.
During the first 3 years of JA3, companies submitted requests monthly according to the published timelines. Due to the high number of demands posing capacity challenges and after the pause of ED activities during the first half of 2020 related to the Covid-19 pandemic, the restart of ED activities during the summer of 2020 provided the opportunity to establish the selection of products based on an “open call.”
Patients and or patient representatives were systematically involved in EUnetHTA EDs using the approaches for patient involvement (4)) developed by the lead and co-lead partners.
Conflict of interest management was systematic and ensured for all participants (HTAb and external experts alike), according to the EUnetHTA JA3 Procedure Guidance for handling Declaration of Interest (11).
Quality Management
Procedures and guidelines were created to ensure the quality of EUnetHTA EDs, and four key steps for quality management were established.
1. A systemic internal review and agreement process of each ED List of Issues and Final Recommendations by all participating HTAb.
2. External review by the ED Secretariat for each ED to monitor the adhesion to the procedure and to insure the scientific content of the Final Recommendations.
3. A professional medical editor was used to ensure the language quality and continuity of the document.
4. Solicitation from applicants for insight and feedback on the quality and usefulness of the procedure and ED Final Recommendations through a standardized feedback questionnaire complemented by regular meetings with the European Federation of Pharmaceutical Industries and Associations and stakeholder groups.
The EDWP met every 2 months with the participation of the ED Secretariat to review and adjust the procedure considering the feedback received.
Output
From June 2017 to May 2021, 113 requests for pharmaceutical EDs were received. Of the requests, 82 percent (93/113) were for PCs of which 32 were accepted based on the EDWP selection criteria. Twenty requests for Multi-HTA EDs were received, resulting in six EDs. In all, thirty-eight pharmaceutical EDs were conducted in JA3. Table 1 provides an overview of the requests received during EUnetHTA JA3 and the resulting number of EDs. Most EDs were requested by “big pharma” with SMEs representing only 15 percent of requests.
Pharmaceutical Early Dialogues in Joint Action 3: Overview of Requests Received and Early Dialogues Conducted
a The Early Dialogues Working Party agreed that an additional product met the selection criteria, but, due to resource constraints, it was refused.
The most frequent reason cited by the EDWP for refusing an ED request (73) was that the product did not meet the eligibility criteria. In these cases, the product did not represent a new mechanism of action in the indication, and/or the unmet need criterion was not met (i.e., other treatments available), and/or the severity of disease criterion was not met. There were three exceptional cases of requests being outside the remit of some EDWP members, and lastly several refusals based on lack of HTAb resources at the end of JA3 despite the products meeting the selection criteria.
Analyzing the EDs, a Successful HTAb Alignment and ED Impact
The number of HTAbs taking part in each ED procedure ranged from 4 to 9 with an average of 6.7. The mix of experienced organizations and those less experienced allowed for capacity building within EUnetHTA, and by the end of JA3, the newer members of the EDWP had also taken on the SC and R leadership roles. Enlarging participation, capacity, and competence building will likely be vital for the future development of these early interactions (not least by facilitating adoption and buy-in of outcomes/harmonization across Europe).
In analyzing the common HTAb recommendations, the frequency (percent) of different levels of agreement for each domain was calculated. The domain “health economics” was omitted from the analysis as not all EDWP members conduct health economic evaluations.
The comparisons of level of agreement between HTAbs were based on four categories:
- Full = Alignment (100 percent common HTA responses);
- Full* = Alignment* (100 percent common HTA responses) together with national specificities outlined by at least one HTAb;
- Partial = When >50 percent of HTAb agreed on a common response; and
- No common agreement = When <50 percent of HTAb agreed on a common response.
When HTAbs reached a clearly stated common recommendation with no sentences in the Final Recommendations stating that some HTAbs did not agree on the consolidated recommendations, full agreement (“Full”) was allocated. The allocation of “full alignment” did not prevent the supplementation of national specifications from HTAbs. In such cases, the category “Full*” was given for the analysis. National specificities, any additional specific request, that is, additional subgroup, additional outcomes, additional comparators, and so on by HTAbs were also collected and analyzed by subdomains.
In summary, the analysis in Table 2 shows:
• The population, intervention, comparator, and outcome (PICO) framework provides a standard format for specifying research questions. These criteria and study design were discussed in almost all the EDs, the topic of health economics was addressed in more than half, and PLEG in one third.
• A high percentage of alignment between HTAbs with more than 80 percent full alignment on all PICO items, for example:
o RCTs clearly remain the gold standard and are requested commonly by HTAbs to obtain high quality data for HTA.
o The choice of an appropriate comparator is crucial, especially when the study population has a considerable degree of heterogeneity. A well-defined study population is directly related to the recommendations for the comparator arm.
o Quality of Life data and meaningful, patient reported outcomes (PROs) are important for HTAb.
o Dealing with missing data is a central topic for HTAb.
• Although approaches to recommendations on PROs could differ between agencies performing health economics evaluation and those focusing on clinical evaluation, there was alignment on the need to have PRO data gathered during development.
• Modifications of development plans were proposed by applicants in twelve out of twenty-one (57 percent) EDs after receiving the EUnetHTA List of Issues including major changes, for example, design adaptation, primary end point choice, changes of comparator, and the addition of a study, but also population criteria, intervention, and other outcomes. Further clinical development adaptations can be expected after applicants have received the final EUnetHTA recommendations.
In addition to the qualitative analysis, PLEG was increasingly addressed in EDs with the participating HTAbs providing advice on PLEG topics in fourteen of the thirty-two PCs(Reference Moseley, Vamvakas and Berntgen3).
Results of Analysis on Alignment between the HTAbs
Abbreviations: ED, early dialogue; HRQoL, health-related quality of life; HTAbs, HTA bodies; PROs, patient reported outcomes.
Establishing Successful Patient Contribution
Eighty-five percent of the EDs in JA3 involved patients (see Table 3). A total of forty-five patients were involved in the thirty-eight EDs; 100 percent were satisfied with the three approaches proposed (interviews and participation in meetings) according to the analysis of twenty-five patient feedback questionnaires. The majority (38/45) of patients had access to the Briefing Book provided by the applicant and thus provided valuable input regarding the submitted development plan.
Summary of Patient Involvement in EUnetHTA Early Dialogues by Approach
Of the twenty-one EDs analyzed in the qualitative analysis (see Table 4), sixteen (seventy-six percent) were performed with patient involvement. The patient contribution was focused on the choice of population to be included (inclusion/exclusion criteria), comparator and outcomes for almost all EDs, but also on study duration and intervention.
Results of the Qualitative Analysis on Patient Input during EUnetHTA Early Dialogues
The percentage of patient input included for the domains/subdomains was high: 67–100 percent.
Overall, the high percentage of inclusion of patient input for the domains/subdomains in the recommendations highlights the relevance of patient contribution to the ED process.
Patient feedback on intervention, potential difficulties of future treatment usage (frequency of administration, acceptability of injection, convenience of oral treatment, etc.), and issues related to current treatment administration (need for hospitalization or specific test or issue of access to treatment) that could become future concomitant treatment were less often reflected in the HTAb recommendations. This could be explained by the lack of questions/issues initially raised on the topic, but also by the fact that the benefit of the way an intervention is administered comes only into effect if the clinical benefit is proved. Additionally, the importance of specific symptoms was often not taken into account or only done so at the country level (i.e., it was included only in the individual HTAb recommendations). The latter derives from different approaches of the HTAbs for providing feedback on outcomes; for example, some HTAbs proactively provide a list of patient relevant outcomes for which patient feedback was considered.
Involvement of External Experts
Although the involvement of patients and patient representatives in the EDs was successful, the identification and recruitment of European-level clinical experts (i.e., having knowledge of the clinical practices in the different European countries) was challenging. While most of the EDWP members regularly included experts in their work, this was considered national-level input that would not necessarily scale to the European level due to variations in care pathways, and so on, in each country. Although consensus was reached that an expert could be used at the EU level, multiple calls for participation by the ED Secretariat remained unanswered.
Collaboration with EMA
From drafting the guidance document to the creation of shared templates and processes for exchanging information, and the systematic joint chairing of PC meetings, close collaboration with EMA was essential for the conduct of PCs. The joint conduct of thirty-two EDs allowed EMA and HTAbss to better understand their respective remits and offer more efficient scientific recommendations to HTDs. The exchanges between EMA and EUnetHTA HTAbs at multiple stages of the procedure prior to the meeting with the applicant allowed for better preparation for the meeting; this represented added value compared to previous parallel scientific advice.
Process Improvement During JA3
The EUnetHTA ED process evolved during the project period based on feedback received from all stakeholders involved in EDs (HTAbs, EMA, HTDs, and patients). The feedback allowed for the simplification of EUnetHTA internal processes and acceleration of the ED process itself. The organization of the List of Issues document according to PICO and the incorporation of PROs as proposed by the EDWP are also examples of these modifications and common achievements to further improve the efficiency of the process. During the JA3 prolongation period, an ED without a face-to-face meeting was tested, thus allowing for a written-only procedure option targeting less-complex requests within a shorter timeframe.
Some Barriers Encountered During JA3, to be Considered in the Future
Human and Financial Resource Constraints
Projects and Joint Actions function on limited and strict budgets.
One of the hurdles during JA3 EDs was resource constraints for HTAbs and recruitment of SC and R; there was a need for a sustainable ED financial mechanism that could allow HTAbs to run more EDs as requested. Although an Early Dialogues Financing Mechanism (EDFM) was developed and a framework for a fee-for-service model established and agreed upon by all participating HTAb, it could not be piloted during JA3. Nevertheless, the EU HTA Regulation provides the possibility for the creation of a fee-paying mechanism (Article 31.1C) (12) based on the experience of the first 3 years. In this case, the information gathered for the EDFM may be useful in its development.
The Future of EU-Level EDs
Four key areas of recommendations were identified for a future system of European HTA EDs and highlighted in the EUnetHTA White Paper on a Future Model of EU HTA Collaboration (13), specifically the organizational framework, the conduct of EDs, IT needs, and the involvement of experts.
First, the organizational framework as setup during JA3 could be maintained, that is, the structure of the ED activity, particularly with a one-stop for all requests, the EDWP as a dedicated and experienced subgroup of HTAbs, the SC and R roles, and the selection criteria to assure that EDs are conducted for innovative products for which an ED presents an added value. The systematic involvement of patients should be preserved. As the central entry point, the ED Secretariat witnessed the high demand from HTDs and was also able to recruit patients/patient representatives to participate in EDs rather easily, thus demonstrating the high interest of EDs from these two key stakeholder groups. PCs with EMA were also an added benefit for HTDs, and therefore this collaboration should continue in the future.
Anticipation of those areas where advice can be made for several technologies in the same indication at the same time would be ideal and would allow for indication-specific advice that would be applicable to all requests in the same indication. Additionally, such an approach would be more efficient both financially and in terms of time spent.
To further enable for better planning, a rotating schedule of EDWP partners to take over the coordination functions has been proposed. The Open Call provided another approach to select all the EDs that would be carried out during a given period at the same time. This efficiently allowed the HTAb to plan and share the workload. However, capacity needs to be built for future HTA consultations to meet the high demand for early advice. Indeed, capacity is one of the major challenges for the future collaboration and will prove vital to the future success and adaptation. Broader HTAb involvement will also facilitate outcome adaption and mutual understanding (potentially some harmonization) of methodological approaches across the European Union.
Second, in terms of the recommendations resulting from an ED, the further development and use of PROs to increase evidence on patients’ preferred outcomes and patient-based outcomes should be requested in clinical trials. That information should be a criterion to be systematically discussed in the development plans.
Further developing the collaboration with EMA, for instance, the qualification of registries for PLEG should be explored since this advice is often provided by EMA at a much earlier time than the ED, and thus it is often difficult for HTAbs to provide any additional or conflicting input at the time of the ED.
Third, a fluid and efficient IT structure is needed allowing not only for the execution of the project management aspects of an ED, but for the secure and efficient sharing of documents with participants external to EUnetHTA.
Fourth, and lastly, the involvement of patients and clinical experts should be strengthened. Although the involvement of patients and patient representatives was successful, further effort is needed to integrate the patient position on outcomes (especially symptoms and side effects) into the final recommendation. Additionally, work is needed to incorporate clinical experts in the procedure. A transparent approach should be implemented for these experts much as there already is for patients.
Conclusions
On average, more than six HTAbs participated in each EUnetHTA ED and, in the majority of instances, were conducted in parallel with the EMA allowing for HTDs to receive timely and pertinent advice from both HTAbs and regulators on their proposed development plans. Incorporating the advice into the product development to meet the evidence requirements of both Regulators and HTAbs may contribute to gaining European approval and positive HTA assessment. The conduct of EDs was one of the successes of the JA3, in terms of collaboration, common procedures, scientific alignment, common general principles, processes gradually improved, independence, capacity building, and inclusiveness. They allowed HTAbs to exchange and better understand each other’s specificities and common points. This is further reflected in the volume of requests from industry and the positive feedback received from both industry and patients. Considering the EU Regulation on HTA and the Joint Scientific Consultations (JSCs), it seems particularly useful that EDWP members gained detailed knowledge of each other’s positions and frameworks: It will make the implementation of the legal requirements smooth in practice.
The well-established structure for EDs could, with a few adjustments, be reproduced in the future European HTA system. EDs will be continued as JSCs in the next 2 years through “EUnetHTA 21” (14), a service contract for the provision of joint work supporting the continuation of European cooperation on HTA before the HTA Regulation becomes applicable. During this period, an intensive exchange on EU HTA methods development and JSCs is foreseen to advise HTDs on the clinical development plan in preparation for future joint assessments. The resource constraints were serious and must be addressed in the future by creating a sustainable, reliable centrally organized funding system.
Funding
The contents of this paper arise from the project “724130/EUnetHTA JA3,” which has received funding from the European Union’s Health Program (2014-2020). Sole responsibility for its contents lies with the author(s), and neither the EUnetHTA Coordinator, the European Commission, nor any other body of the European Union is responsible for any use that may be made of the information contained therein.
Conflict of Interest
The authors declare that they have no conflicts of interest.