The implementation of scientific consultation or advice between pharmaceutical manufacturers and health technology assessment (HTA) bodies has become more widespread in practice (Reference Backhouse, Wonder, Hornby, Kilburg, Drummond and Mayer1–Reference Cuche, Beckerman, Chowdhury and van Weelden4). Thus, manufacturers participate optionally in scientific advice (SA) at the English NICE to obtain feedback on the design of clinical trials and generation of robust scientific data to develop evidence about the clinical effectiveness and value of their products (Reference Longson, Crabb and Osipenko5;Reference Maignen, Osipenko, Gajraj and Chivers6). A similar process is taking place at the Dutch Zorginstituut Nederland (ZIN) (Reference Cuche, Beckerman, Chowdhury and van Weelden4). The Italian AIFA offers HTA advice when actively involving the pharmaceutical industry together with other stakeholders to define the respective research question of interest (Reference Siviero, Montilla, Sammarco, Trotta, Tafuri and Pani7), and in Canada, Canadian Agency for Drugs and Technologies in Health (CADTH) recently introduced a voluntary, fee-for-service SA program offered to pharmaceutical companies (8). In Sweden, the pharmaceutical benefits board (TLV) organizes joint SA meetings for the manufacturers together with the national approval authority (MPA) (Reference Cuche, Beckerman, Chowdhury and van Weelden4). Many other European countries are starting to participate in HTA dialogues by means of EUnetHTA initiative (9).
In Germany, the HTA SA for pharmaceutical manufacturers is organized by the decision-making Federal Joint Committee (FJC) within the early benefit assessment (EBA) of pharmaceuticals (Reference Ruof, Schwartz, Schulenburg and Dintsios10). The FJC is the German self-administrative body of physicians, hospitals, and health insurance funds. It effectuates the framework provided by the legislation and ensures that legal instructions are implemented in the healthcare system. The regulations issued by the FJC represent binding sublegal norms, which apply to the statutory health insurance funds, the insured persons, physicians, and other service providers, covering prescription of medicines, national needs-planning for specialist practices, assessment of examination, and treatment methods in outpatient and inpatient care, services ordered by doctors, and disease management programs etc. (Reference Busse and Blumel11).
The FJC commissions the Institute for Quality and Efficiency in Health Care (IQWiG), which was established as a professionally independent, supporting scientific institute. IQWiG primarily prepares evidence reports on pharmaceuticals and nondrug interventions, and assesses the EBA dossiers of new pharmaceuticals. The SA is regulated by the Social Code Book V (Section 35a, Paragraph 7) and substantiated in the related legislative decree (Section 8, Paragraph 1, Legislative Degree on the benefit assessment of pharmaceuticals). In accordance with these, the FJC advises pharmaceutical manufacturers by request and on the basis of submitted relevant documents.
FJC rules for SA require the advice procedure to be laid out in the specific request form as well as the respective advice fees, which depend on the level of advice offered. This form represents a prestructured matrix which requests company's contact details, active substance description, product's market authorization status, list of annexes, and questions for discussion. The fees amount to €2,000 for general enquires on the rules of procedure, and to €7,000 for necessary trial data to be submitted within the EBA; €10,000 is the fee for enquires on the appropriate comparative therapy (ACT). An advance payment of 5,000 € is required to initiate the SA.
The complete process of the SA, together with the involved units of the FJC, in addition to the administrative office of the FJC, also includes the responsible subcommittee for pharmaceuticals, the specific working group for the EBA of pharmaceuticals according to Section 35a of the German Social Code Book V, and the specialist counseling group Medicine with respective methodical expertise in evidence-based medicine and is shown in chronological sequence in Figure 1. The national approval authorities, the Federal Institute for Drugs and Medical Devices (BfArM) and the Paul Ehrlich Institute (PEI), can be involved in the SA process by means of a written request from the manufacturer.
Process of scientific advice at the Federal Joint Committee (FJC). The participating stakeholders within the scientific advice at the FJC and the timelines of the process. Next to the FJC and the manufacturer representatives of the approval authorities are optionally and on request of the manufacturer involved as well. After the manufacturer submits the requirement form to the administrative office of the FJC the specialist counseling group of the FJC is performing literature searches to inform the working group for the early benefit assessment of pharmaceuticals (according to Section 35a of the German Social Code Book V) of the FJC's subcommittee for pharmaceuticals. This working group prepares the first version of the answers on the manufacturer's request which are finalized by the subcommittee for pharmaceuticals. Subsequently the scientific advice takes place at the FJC where the manufacturers meet the representatives of the administrative office for explanatory responses. Finally, a written protocol is prepared by the administrative office (if necessary the subcommittee for pharmaceuticals is also involved) and forwarded to the manufacturer.
Scientific advice can either be “early” or “late,” depending on the stage of the clinical trials. Whereas manufacturers can modify their product development plans and discuss the requirements with FJC prospectively within the “early SA” before pivotal trials have begun, the purpose of the “late SA” is for the FJC to offer the manufacturers precise information on the requirements for EBA. The latter was especially relevant during a transition period after the introduction of EBA when manufacturers were not able to anticipate EBA in their product development plans. It also remains important in cases where manufacturers cannot adapt their study designs due to the regulatory authority guidelines from the EMA and FDA.
Between 2011 and 2014, a total of 260 SA procedures with pharmaceutical manufacturers took place (Reference Hecken12). The FJC has not published anything with regard to the SA and the involved stakeholders, which is a consequence of the confidentiality of the procedure. The manufacturers’ perspective on the process has been captured only in an anecdotal manner (Reference Cassel and Heigl13;Reference Sattelmeier, Prenzler and Frank14). The only available literature on this topic is a discussion paper published previously by Dintsios and Schlenkrich in German (15). The manufacturers are, alongside the FJC, the main stakeholder involved in the SA procedure. Through their experiences they offer insights and allow identifying opportunities for improvement and increased efficiency with regard to the process and the content of these consultations.
METHODS
In 2012, and for the purpose of subsequent quantitative elicitation of the manufacturers’ experiences with the offered SA, four SA protocols were analyzed that were made available by four manufacturers, respectively. Afterward, based on these extracts, guided problem-centered face-to-face interviews (Reference Witzel16) were conducted with individual representatives of research-based pharmaceutical companies from departments involved in SA meetings at the FJC to capture their views on this topic. The results were reported following, where applicable, the standards of reporting qualitative research (SRQR) (Reference O'Brien, Harris, Beckman, Reed and Cook17). Thereafter, a specifically developed questionnaire with the purpose of broadening the information basis and exploring potential changes of the SA longitudinally was used at different points in time between April 2013 and March 2015. The questionnaire was sent electronically to the responsible departments of the (more than 40) members of the German Union of Research-based Pharmaceutical Companies (vfa e.V), accompanied by a reminder message sent out every 2 months.
The questionnaire comprised eight items concerning the type of advice, participants, topics of advice, and details on the ACT, population subgroups as well as other relevant issues (e.g., study design and duration) and the effort involved in the preparation. Free-text responses were also permitted (see Supplementary Figure 1). The participating manufacturers and their products were anonymized for the analysis.
Due to the small number of responses received and the relatively narrow time-span covered, we abstained from a trend analysis. Instead, we divided the observation period into two similar temporal parts, hypothesizing that initial frictions would diminish and learning curve effects would be realized over the course of time. Furthermore, in addition to using the Chi2-tests, we used Fisher's exact tests as a more meaningful test for small sample sizes for discrete variable sets derived from the questionnaire. As the null hypothesis, we set no statistically significant changes for the variables between the two phases. In case of more than two answer categories or levels, the statistical tests were performed for all of them as well as merging the positive categories (for instance “very good,” “good,” “sufficient”) and contrasting them with the negative category (for instance “poor”), leading to a dichotomization of them. Similarly, we examined the differences between early and late SA relating to evidence before commencing and after the start of pivotal trials or their completion, respectively.
Finally, between April and June 2017, we interviewed six participants from the first qualitative survey again and one more representative from a pharmaceutical company to gather their current views on the SA after 6 years since its establishment, following a less in depth approach this time.
RESULTS
Qualitative A
All but one manufacturer approached, replied to the qualitative part. The seven interviewed staff members of related departments had taken part in a total of 20 SA procedures (range: 1–3, one exception 10) between the beginning of 2011 and the end of 2012. The interviews lasted on average 47 min. (range: 30 min to 2.5 h). IQWiG participated only twice in the SA, the national approval authorities (BfArM and PEI) only once. In all the other procedures, only FJC representatives participated.
The structured and summarized content analysis of the interviews in the course of the preceding qualitative enquiry by the end of 2012 highlighted the following issues: (i) many representatives of the FJC showed a lack of knowledge about approval-relevant requirements for registration trials; (ii) inconsistent statements were made very often even during the same SA; (iii) the rationale offered by the FJC was often perceived as being rather dogmatic; (iv) compared with NICE, the experiences gained by the manufacturers differed in the sense that the latter offered more complete methodical advice (e.g., sample size, endpoints, trial duration, etc.), whereas the advice by the FJC was deemed to be more vague and thereby allowing less planning reliability. The fact that representatives from the global head-quarters were able to participate and that simultaneous interpretation was provided was viewed positively. The same holds true for the ability to realize learning curve effects when taking part in more than one SA meeting.
Some points were mentioned only once and, therefore, it was not possible to derive any heuristics. For example, one interviewee stated that even though the SA was nonbinding, the subsequent protocol turned out to be more specific and narrower than the consultation itself. It should be noted that even though the protocol is confidential, contents thereof may be partly disclosed in the subsequent hearings by the manufacturers and thereby become public. In one case, the argumentation of the manufacturer seemed to be accepted, but after feedback discussions with the subcommittee for pharmaceuticals (see Figure 1) the position of the administrative office of the FJC changed their stance again before finalizing the protocol.
Quantitative
With regard to the quantitative elicitation part, a total of sixty-one completed questionnaires from nineteen manufacturers were included leading up to the beginning of 2015 (corresponding to almost 25 percent of the overall SA issued by the FJC). Fourteen cases of sixty-one (30 percent) concerned early SA, forty-four (72 percent) cases concerned late SA, another two (3 percent) concerned repeated SA and finally, four (6 percent) referred to miscellaneous content (multiple answers possible).
In only thirteen cases (21 percent), and limited to the first part of the observation period, did other stakeholders (external statisticians, service providers, medical association representatives, IQWiG representatives, representatives of the national approval authorities) participated in the SA in addition to the manufacturer and the FJC. The contributions of the national approval authorities and external service providers were deemed to be particularly helpful by the manufacturers.
Referring to the topics of discussions, in the majority of the cases, it was stated that the questions asked were answered in full by the FJC (Figure 2). In addition, the completed questionnaires by the manufacturers contained further topics to be discussed as singular questions.
Most frequent discussion topics and completeness of their consultation. The figure presents the absolute frequencies of the completeness of the most frequent consultation subjects.
According to the rules of procedure of the FJC, the criteria for determining the ACT are: (i) If a medicinal product is considered as the comparator, it must be approved for the respective therapeutic indication. (ii) If a nonpharmaceutical treatment is considered as the comparator, this must be deliverable within the framework of the statutory health insurance. (iii) Pharmaceuticals or nonpharmaceutical treatments whose patient-relevant benefit has already been determined by the FJC are preferred as comparator. (iv) The comparator should belong to the appropriate therapy in the therapeutic indication according to the generally accepted state of medical knowledge. (v) If there are several alternatives, the cheaper therapy is selected, preferably a therapy, for which there is a reference price (this criterion applies to price negotiations only). Since 2013, the option for multiple comparators has been regulated by the law.
In 70 percent of the cases (72 percent for early and 69 percent for late SA) manufacturers agreed with the ACT set by the FJC within the SA, whereas vice versa the FJC accepted the proposed ACT by the manufacturers in only 57 percent (64 percent for early and 54 percent for late SA) of the respective cases. With regard to the supposed rationale of the FJC when choosing the ACT the responses revealed in 53 percent (36 percent for the early and 60 percent for the late SA) of the cases that economic reasons were decisive from the manufacturers’ perspective. Approval issues in the sense of overlapping indications, on the other hand, were relevant for the determination of ACT in 69 percent (64 percent for early and 70 percent for late SA) of the cases. In six cases (10 percent), respondents stated other reasons, including guidelines and the German healthcare context or already assessed pharmaceuticals in the respective indication.
When it came to splitting the target populations into subgroups, the manufacturers reported that, in 24 percent (11 percent for early and 31 percent for late SA) of the cases, the approach of the FJC lacked appropriateness either because there was no methodical rationale or because these subgroups were neither specified in the respective study protocols nor included in the (intended) label. This topic was summarized using the term “slicing,” including indication-specific subgroups as well as prespecified or post-hoc defined subgroups (the latter was especially an issue in late SA after the pivotal phase III trials had commenced).
In 70 percent (86 percent for early and 68 percent for late SA) of cases the SA was deemed to be unambiguous and in 75 percent (86 percent for early and 73 percent for late SA) it was deemed to be complete. Depending on the type of advice for early SA, the perception of manufacturers on the examined topics was numerically, but not statistically, more positive compared with late SA. This is mainly due to the chance to adapt parts of the study design before the study has started, even if changing the study protocol for phase III trials is considered to be rather challenging.
The quality of the SA was judged by the traceability of the FJC's rationale, the atmosphere of the consultation, and a correct rendering of the SA contents in the protocol (Figure 3).
Quality of consultation. The figure shows the relative frequencies of the quality levels (very good, good, sufficient, and poor) for the consultation criteria quality of protocol, atmosphere and traceability of the scientific advice.
In 4 percent of the SA procedures (53 of 61 questionnaires with an answer on this topic served as the reference basis), the administrative office of the FJC changed its position at the meeting directly, in 11 percent, it had to first consult with the respective subcommittee for pharmaceuticals, and finally, the FJC insisted on its own positions after the completion of the SA in the remaining cases.
The cost for the manufacturers to prepare for the SA could be specified either in full-time equivalents or monetarily. It ranged depending on the type of SA (i.e., early, late, or repeated) from €5,000 to €90,000 excluding the obligatory fees for SA or from 1.6 to 18 full-time equivalents on a monthly basis.
No statistically significant differences in the perception of the manufacturers were detectable between the two types of advice (early versus late SA) except for study endpoints (p = .04/.065) and study duration (p = .011/.033) based on Chi2-tests and Fisher's exact test of different degrees of freedom. The same may be observed, in addition to some weak numerical trends, for the two temporal parts of the observation period, except for the propositions on study design (p = .016/.050), agreement with the defined ACT (p = .005/.007), and adequate subgroup definitions (p = .035/.057). Detailed results are given in Table 1.
Differences between the Two Observation Periods
Note: The table shows the differences of the questionnaire's discrete variables sets between the two observation periods.
ACT, Appropriate comparative therapy; FJC: Federal Joint Committee;
1st, before 1st Quarter 2013; 2nd, after 1st Quarter 2013 (including 1st Quarter 2013); n. s., not statistically significant (Fisher's exact test).
In general, a trend for a positive change can be seen over the course of time, whereas the differences between early and late SA were less than expected. Especially concerning the answers on the study design, there is some evidence that FJC has developed a routine. The increasing level of acceptance amongst manufacturers of the proposed ACT by the FJC, among other reasons, is due to a legal reform introduced in June 2013 by the lawmaker, which allows a more flexible choice of the ACT. The results of Fisher's exact test on the change of agreement on the subgroup definitions (“slicing”) were no longer significant (p-value .035 versus .057). Thus, a conclusive interpretation of this topic is not yet possible.
Qualitative B
The final interviews were conducted with six staff members who had taken part in the first qualitative survey and one additional staff member from seven different research-based pharmaceutical companies. All the interviewees had experience with SA. In all cases, the manufacturers asked for the active participation of the national approval authorities with the exemptions of those cases, where the manufacturer was already involved in consultations with the national approval authorities or it was assumed that there was no need for their participation. The authorities always submitted written comments and in some cases they participated by telephone in SA.
With respect to the content of the responses provided by the FJC in the SA sessions, these were deemed by all interviewees to be increasingly satisfactory. The FJC showed a greater willingness to partake in discussions on controversial issues. Nevertheless, criticism of vague statements on methods and evasive answers persists in parts, especially with respect to concrete questions on endpoints. Six interviewees stated that they could obtain some new insights on issues which were not anticipated before or had been interpreted differently. The one interviewee who stated the opposite was representing an orphan drug manufacturer, with orphan drugs being granted an added benefit by law and hence marginalizing the impact of the SA on the outcome of the EBA. Finally, five interviewees expected that SA would have an impact on their study designs, except for international trials which have already begun. Depending on the timing of SA the impact of advice differed. Even for phase-III trials which have not yet started, changes to their design were considered to be hardly realizable.
DISCUSSION
Differences between the results of the first qualitative and quantitative section can be explained by the different elicitation periods. The interviews conducted during 2012 only reflect the experiences of the very early beginning of the SAs more in terms of a snapshot when the procedure was just introduced and provided guidance for the development of the questionnaire. In contrast, the positive change during the observation period of the quantitative survey, assuming a more or less established SA, stems from learning curve effects and legal reforms with a direct impact on the content of the consultations. This was also confirmed by the second qualitative section.
From the perspective of the German approval authorities, the condition of a European, largely harmonized pharmaceutical authorization process, where a mainly nationally regulated pharmaceutical reimbursement system causes inevitable frictions, which, although not preventable, could be reduced at least through joint advice discussions (Reference Beinlich, Müller-Berghaus, Sudhop, Vieths and Broich18). In 2013, BfArM and PEI provided 439 SA procedures, compared with ninety-eight advice meetings held at the FJC, for twelve of which they provided written advice. While the numbers of advice meetings held at the FJC are increasing, the national approval authorities are involved in only a fraction of these and only on request from the manufacturer. From their perspective, prompt and consistent involvement in the advice procedures, regarding EBA, would be useful and desirable (Reference Beinlich, Müller-Berghaus, Sudhop, Vieths and Broich18). Meanwhile, a structured collaboration among BfArM, PEI, and the FJC has recently (2016) been established. While a national joint SA is a positive development, it has already been clarified that this dialogue is not intended to harmonize study requirements (19). This will continue to make it difficult for manufacturers to generate evidence acceptable to both the approval authorities and the FJC in one study program.
In the recent written opinion (February 2015) on the EBA, the German Association of the Scientific Medical Societies (AWMF) emphasized the need for improvement of the procedural processes as well as several aspects of the early benefit assessment of pharmaceuticals, including the determination of the ACT and the prioritization of different endpoints (20).
According to the AWMF, the ACTs determined by the FJC differed from the comparators of the pivotal trials in more than just a few cases. Thus, the FJC suggested ACTs pharmaceuticals that have a market authorization for the indication of interest, but, due to limited evidence, are not recommended by the current national or international guidelines. To gain a complete understanding, additional clinical information beyond that included in available trials and the knowledge about the German healthcare structures is essential. Thus, obtaining an independent expert opinion for a sophisticated evaluation based on external evidence is considered by the AWMF to be reasonable and conducive (20). Up to now, the FJC has not planned to formally include the expert opinion of the medical societies in the SA.
In addition to the SA at the national level with its different HTA jurisdictions and decision makers, the practice of implementing parallel advice at a European level together with the European Medicines Agency (EMA), where at least a partial harmonization and an improved transparency and predictability can be achieved (Reference Enzmann21), has recently become more prominent following international pilots (Reference Backhouse, Wonder, Hornby, Kilburg, Drummond and Mayer1–Reference Cuche, Beckerman, Chowdhury and van Weelden4;Reference Fronsdal, Pichler and Mardhani-Bayne22;23), facilitated by Tapestry networks (24). Tafuri et al. analyzed the parallel SA meetings and showed that in cases where the EMA and HTA bodies did not agree on the comparator, the suggested solution was most frequently an indirect comparison (Reference Tafuri, Pagnini and Moseley25). However, given the stringent methodological requirements, it will be challenging to obtain a positive result within the EBA of pharmaceuticals in Germany on this basis (Reference Lebioda, Gasche, Dippel, Theobald and Plantor26).
In the Report on the public consultation on the modalities of stakeholder consultation in the future Health Technology Assessment Network, the Health and Consumers Directorate-General of the European Commission asked eight pharmaceutical companies and nine pharmaceutical associations for their experiences within the EUnetHTA, the European Network for HTA supported by the European Commission (27). The result of this survey was as follows: the relevance of “early advice” is ranked after mandatory “guidelines” in second place, even ahead of the “relative effectiveness assessments”. This relevance prioritization has found expression in the form of the SEED (Shaping European Early Dialogues for Health Technologies) Consortium following a call of the Executive Agency of Health and Consumers of the European Commission (EACH) (28).
SEED was composed of fourteen European HTA bodies including FJC and IQWiG and led by the French Haute Authorité de Santé (HAS) with EMA having mainly an observer status. Its aim was the realization of a pilot of early dialogues between HTA bodies and health technology companies in the development phase of their products. Under the SEED initiative, seven dialogues on drugs and three on medical devices were conducted in total and an evaluation of the SEED performance is still awaited. The SEED activity has moved under the early dialogues in EUnetHTA project Joint Action 3. It remains to be seen if and how such collaborative consortia may sustainably evolve.
In their discourse on European collaboration on relative effectiveness assessments, Kleijnen et al. (Reference Kleijnen, Toenders and de Groot29) point out that differences in methodological approaches (e.g., with regard to the choice of comparator) should be identified at an early stage and resolved by the manufacturers (for instance, by including multiple indirect comparisons in the assessment). They emphasized further, that a significant part of the scoping-process should be done long before the marketing authorization date, despite the risk that an application for regulatory registration could be rejected or withdrawn.
A possible explanation for the heterogeneity across the different procedures is provided by Cavazza and Jommi (Reference Cavazza and Jommi30), who conclude in their comparative work about stakeholders’ involvement in HTA, that, due to the importance of national administrative traditions and the characteristics of different healthcare systems, respective involvement models on each stage of the HTA process cannot easily be transferred from one country to another.
As for the changes in position of the administrative office of the FJC within SA after consultation with the EBA implementing and appraising subcommittee of pharmaceuticals, in contrast to the strictly independent NICE SA procedure (Reference Maignen, Osipenko, Pinilla-Dominguez and Crowe31), this observation indicates essential dependencies and raises serious governance issues.
There are some important limitations of the analysis. It captures only the responses of manufacturers organized in the German Union of Research-Based Pharmaceutical Companies including the period immediately after the introduction of SA, where the respective processes had to be established and the competencies of the FJC were just developed, representing only a part of the cases. Smaller companies that took part in SA in the analyzed period were not surveyed. The results reflect the opinion of the surveyed staff members of the participating manufacturers. It remains, therefore, unclear if these positions are aligned with their employers. Because the pharmaceutical industry is very reluctant with regard to sharing information which might be used by competitors, participating manufacturers and their products had to be anonymized.
As a consequence, some crucial information, such as substance or substance class, intended label, orphan designation, target population size etc., which would have offered even deeper insights was not made available to the readership. On the other hand, we used qualitative and quantitative approaches in our study to gather as much data as possible and to present the experiences of the industrial stakeholders with SA at a national level for the third biggest pharmaceutical market in the world. Because HTA is about assessing health technologies, the perspective of the developers and producers of these technologies is assumed to be at least interesting for all involved stakeholders.
The present study discussed the survey of the pharmaceutical industry's experiences with the SA offered by the FJC within the early benefit assessment of pharmaceuticals in Germany based on approximately one quarter of the given advice procedures, albeit within a rather narrow time period from the beginning of 2011 until March 2015. Hence, surveys on the SA with an adequate response rate should be continued to gather a broader basis of more robust findings, because the number of advice procedures is continuing to increase.
In conclusion, interestingly and unexpectedly, no statistically significant differences in the perception of the manufacturers were detected between early versus late SA except for study endpoints and study duration. In general, a positive trend in the pharmaceutical industry's experiences with SA can be seen over time with propositions on study design, agreement with the defined ACT, and adequate subgroup definitions showing statistically significant differences in favor of the second period. A more active involvement of additional stakeholders, such as national approval authorities and medical societies and the incorporation of procedural elements from other healthcare systems with well-established consultations and longer experience, shared by the internationally operating pharmaceutical industry, could at least improve the quality of the SA offered by the decision maker in Germany.
SUPPLEMENTARY MATERIAL
Supplementary Figure 1: https://doi.org/10.1017/S0266462317004536
CONFLICTS OF INTEREST
In addition to his academic affiliation, C.M.D. is employed by Bayer Vital GmbH. During the preparation of the manuscript, S.Sch. was a trainee at Bayer Vital GmbH. C.M.D. conducted the interviews, developed the questionnaire and prepared the manuscript. S.Sch. analyzed the data together with C.M.D. C.M.D. acts as the overall guarantor.
