4.1 The role of the HTA Department

Our interviewees pointed to a number of areas in which the HTA Department successfully fulfilled its formal role of providing evidence for reimbursement decisions, as set out in the legislation. Appraisals undertaken by the HTA department were widely viewed as independent, and conducted according to predefined criteria. A high-ranking NHIF official stressed, “The HTA Department (…) is more like an independent group of experts to support the NHIF”[12]. According to a former HTA Department senior official, “These [appraisal documents] have fifty-sixty pages which is too much but it is a comprehensive description on efficiency and safety, health gain, and review [on] the health economic analysis, which includes the price of the product and the comparator. And at the end there is an opinion” [20]. A further strength of the HTA Department was its timeliness in conducting appraisals. Interviewees typically reported that the Department normally achieves the preparation of an evaluation within the 43-day time window. A consultancy company manager highlighted, “There is no such thing as having the submission on hold [at the HTA Department]” [10].

Nevertheless, some interviewees still saw the HTA Department’s role in supporting evidence-based decision-making as sub-optimal. Most notably, the assessments provided one-off snapshots in time – the HTA Department only contributed to the earliest phases of the reimbursement process, with unrealised potential for subsequent contributions. According to a former HTA Department employee, “[HTA Department] has only 43 days to assess the technology. So if something changes, for example new evidence comes out or the price is changed, the HTA Department is not involved anymore” [15]. A drug company representative also noted the absence of a process to revise previous decisions: “the follow-up process is missing. This is an important limitation of the [reimbursement] system” [4].

The interviewees conceded that there was insufficient commitment of decision-makers to publicly disclose information on technology assessment documents that would justify decisions on public reimbursement. An academic explained that “if there is no need for justification, implicit judgment becomes the most important part [in decision-making]. And those consultants who got the insights on how the implicit judgments are made become successful” [14]. Although HTA experts – and even, occasionally, policy-makers – publicly acknowledged the importance of transparency, concrete remedies were not put into practice. A senior HTA consultant explained that “actually in public no one is against it. The answer that people usually say is that they want do it [put HTA documents in the public domain] in the near future (…) but it just never happens” [15]. An interviewed academic was emphatic on this point, “Unfortunately we were not brave enough to move into a direction where we can operate HTA in a transparent way” [14].

This institutional shortcoming could be addressed in the future, as is being suggested in on-going consultations about possible publication of executive summaries from appraisals including major questions on elements from the submission (e.g. comparator, cost-effectiveness ratio etc.). A member of the HTA Department emphasised, “I don’t want to be a part of a black box” [17]. Yet the proposed improvements to transparency may face opposition from some stakeholders, including the manufacturers. A pharmaceutical company representative explained that more transparency could even be risky for manufacturers: “If an evaluation is negative by the HTA Department and then the NHIF still makes a positive recommendation, it would be immediately attacked by competitor companies. They would use these information against each other” [9].

A related area of unrealised potential arose from the limited extent of consultations with key stakeholders, especially clinicians, manufacturers and patient organisations. According to a current HTA Department employee, “There is a need [for consultation with clinicians] and the HTA Department would like to have more support from external experts especially in the clinical part (…) There were no attempts [to formalise consultations] and also it is hard to motivate the busy physicians. This is not in the everyday practice” [24]. This is the situation even though representatives from the Colleges of Medical Professionals also give their opinion at the HTAC meetings. Moreover, former HTA Department employees explained that “This [consultation with clinicians] is sub-optimal. There should be a well-regulated process of doing this” [15] and “Key opinion leaders are not really interested in HTA. They have a perception of HTA that it is a bureaucratic thing, shifting papers back and forth” [3]. Similarly, manufacturers complained about the limited feedback on their submissions: “A basic problem is that this process is not a consultation type process. There are no consultation opportunities to conduct a proper and objective evaluation of the submission” [9]. This issue was recognised by those conducting assessments; negotiations are underway to create a platform for sharing health economic models to facilitate two-way discussion between the HTA Department and manufacturers. Unlike the situation in some other countries, patient organisations have no formal contact with the HTA Department. A representative of a patient organisation explained: “In general a Hungarian patient or a Hungarian patient organisation never hears about HTA. They are not aware of the process in which the HTA Department is involved. (…) On the other hand, (…) it is not facilitated to invite [patient organisations] to collaborate. This is not even on the agenda” [21].

4.2 Influence of the HTA Department

The influence of the HTA Department on reimbursement processes was widely viewed as limited. A former key governmental official emphasised, “The influence of HTA is smaller than it would be required. This is changing and there are steps forward but still it is smaller than required” [25]. This point was summarised, perhaps most vividly, by a high-ranking government official: “The HTA Department works as an ante-room, where people wipe off their shoes” [19].

Many of our interviewees stressed that the HTA Department had limited influence on the NHIF. A senior consultant stressed, “there is an ambivalent relationship between the NHIF and the HTA Department. (…) for the NHIF the technology assessment is too academic and requires that everything should be perfect” [10]. A key issue was the narrow scope of appraisals, including only evaluations on the validity of data included in manufacturer submissions. According to a high-level governmental official, “they only review that an analysis is appropriate or not, in the context of the given methodology. This is a too narrow perspective” [19]. Contrastingly, some interviewees felt that the HTA Department should be providing more direct advice to decision-makers. An academic explained that “The result of this kind of restriction, [is] that the opinion is usually quite neutral and you can interpret it in different ways. (…) But they never made a strong statement like ‘the quality of the submission is extremely poor, extremely biased and there is no evidence on some point’” [14]. Some of our interviewees also mentioned opaque phrasing of recommendations provided by the Department that were ill-suited to real-life decision-making purposes. An NHIF employee noted, “It is not helpful for instance when they say that the cost-effectiveness is uncertain” [12].

Even more important was the lack of inclusion of a NHIF institutional perspective in appraisals undertaken by the HTA Department. A former NHIF employee added, “The main weakness is that [HTA] does not serve the interest of the payer. Because it does not lead to better decisions” [3]. Specifically, the HTA Department does not prepare recommendations on whether a new technology should be reimbursed or not. This disjuncture of organisational perspectives could, in certain circumstances, lead to the by-passing of the HTA Department by the NHIF. This was particularly the case when the NHIF had enough knowledge about the technology, and the agreement with the manufacturer on price was seen as acceptable. In this case, the NHIF saw the contribution from HTA as redundant. “Instead of a normal procedure it could be acceptable to have a simplified procedure” [12], was an assertion made by a high-ranking NHIF employee. Consequently, it appeared that in some cases emerging recommendations from the HTA Department were disregarded a priori. A former HTA Department employee stressed, “In some cases the final decision is made before the opinion [of the HTA Department] is discussed. (…) NHIF already has information whether the drug has a place in the therapeutic practice in Hungary or not” [20].

Yet another factor limiting the HTA Department’s influence on the NHIF was its reliance on list prices as opposed to the actual, often discounted, prices negotiated with drug manufacturers. While the Hungarian system relies heavily on price–volume agreements between the manufacturer and the NHIF, the HTA Department is not provided access to the details of any negotiations taking place in parallel to its appraisal process. An NHIF employee explained, “There is an information asymmetry between the HTA Department and the national payer [NHIF]. This is the result of the confidential prices which are set through national tendering or negotiation process in case of innovative products” [12]. The HTA Department has no information regarding the state of negotiations on actual prices. According to a representative of a drug company, “From this point on the whole HTA system worth nothing. Manufacturers submit analyses which have completely wrong input data as price” [6]. Similarly, a former HTA Department employee reacted to this situation by saying, “Basically sometimes you are just comparing two numbers that have nothing to do with reality. Of course nobody cares what the result of the comparison is, because it is nonsense” [15].

The HTA Department had even less influence on decision-makers in ministries. A former NHIF employee saw the HTA Department’s impact as negligible. “Generally, the government decisions are so centralised and so hierarchical that the information [from the HTA Department] is not valuable in terms of [evidence-based] decision-making” [3]. Other interviewees pointed out that the influence of the HTA Department was constrained by political considerations that top-level decision-makers were sometimes inclined to prioritise over HTA. A former NHIF official elaborated that “the final decision is made by politically influenced stakeholders who absolutely do not understand this way of analysing drugs. There is a gap and this gap is quite huge. (…) There is a need for a stronger relationship between HTA and decisions” [16]. Reimbursement procedures and the related timeline of decisions are only transparent until a certain point. Once ministerial bodies (e.g. state secretary of health or Ministry of Finance) got involved very frequently their opinions drove decisions and delayed announcements on reimbursement status. “Timelines are not transparent in the governmental sector. The fact that the Ministry of Finance has such a significant influence on the reimbursement of health technologies blocks constantly the planned timelines and the deadlines that should be maintained throughout the entire reimbursement decision-making process” [8].

In addition to factors constraining the HTA’s influence in relation to the NHIF and the Ministry of Health, the influence of the HTA department was undermined by problems pertaining to its organisational capacity. A NHIF official stressed, “The HTA Department currently does not have the capacity to become a prominent body in the technology assessment” [13]. There are approximately 80–100 submissions on pharmaceuticals and a similar number on medical devices per year. There are 13–15 permanent employees who work on evaluating the submissions and representing the HTA body in committees. The impact of these organisational constraints was exacerbated by the tight timelines imposed by the reimbursement legislation. A former HTA Department employee stated that “The 43 days for evaluation (…) does not make sense” [23]. A former senior officer at HTA Department explained that the time window also includes holidays, and therefore it “actually becomes 30 days of evaluation that is done by one health economist and one medical expert, who have more submissions in parallel” [20].

Human resource considerations were even more critical because of revolving door between the HTA Department and manufacturers and consultancies. A former HTA Department employee explained, “the biggest barrier of the development [of the Department] is the high fluctuation. (…) The entire staff (…) changed in the last 3.5 years. There is no continuity at the institution” [23]. A key factor in this brain drain from the HTA Department was the low level of remuneration given the high level of skills and expertise of its employees. A consultancy employee pointed out the usual career path of young health economists: “people start their career at the public sector, work there for four-five years and then go to manufacturers or consultancies” [15]. An NHIF employee described this process as “unfortunately the HTA Department works as a spring board for young people” [13].

Yet low salary was not the only reason for the outflow of highly qualified personnel. Many interviewees stressed that the limited influence on the reimbursement system can also contribute to decisions to move to the private sector. As a former NHIF employee pointed out, “they [HTA Department employees] also understand that it is not only about money, but what they do is essentially without impact on the final decision. It results a huge frustration” [3]. A former HTA Department employee criticised the system as “You are in a job where you compare two numbers and nobody cares about it” [15].

Finally, there was also evidence of a vicious cycle whereby the HTA Department’s limited impact on the reimbursement process encouraged actions by other stakeholders that further diminished its influence. This was perhaps best illustrated by the persistent low quality of manufacturers’ HTA submissions. A HTA department employee explained that “people who worked in the HTA Department now work for pharma. They saw no impact, so they don’t want to create a normal dossier” [17]. Similarly, a former NHIF employee noted that “If someone would take the real decisions based on HTA submissions [by manufacturers], then the HTA submissions would have been revised and rethought years before. At the level of real decisions, [the decision-makers] are not interested in the HTA submission. That is why the HTA submissions are not for decision-making but for themselves” [16]. However, there were other reasons behind the low quality of submissions, including limited data availability. As a representative of academia emphasised, “Source of valid information is an important challenge for the HTA submission. Manufacturers, consultants and the governmental sector including the HTA Department do not necessarily use the same source for data” [8].

4.3 Perceived costs and benefits of the HTA system

Although the HTA Department has limited perceived influence on the reimbursement process, it still provided many important benefits. Given the lack of available data from HTA dossiers, appraisal documents by the HTA Department or price negotiation contracts between the NHIF and manufacturers, it is not currently possible to express these benefits in monetary terms. Nevertheless, it was clear that the HTA Department was seen as creating many important intangible benefits for the Hungarian health policy in the long run. These spillover benefits were most commonly associated with developing experience in applying HTA principles in policy-making, including awareness of their practical advantages and disadvantages. A senior manufacturer employee maintained, “There was ‘hurrah-optimism’ at the beginning [HTA implementation in Hungary], but this is over. We realised how and what can be used from an HTA system. And we also know what the HTA system is not useable for” [6]. Consequently, even senior NHIF officials ignored some of the recommendations coming from the HTA Department, but they still relied on the principles of HTA in developing health policy. Ministerial policy-makers tended to be even more strategic in this regard, showcasing HTA as a key achievement in building an evidence-based policy-making infrastructure. A former high-ranking NHIF employee emphasised, “We can tell the world that we have a good HTA system” [16].

A key specific benefit achieved by the HTA Department was the institutionalisation of explicit criteria for evaluating health technologies. An HTA consultant explained, “It is good that there is a platform to consider cost-effectiveness” [10]. A former key HTA Department official stressed, “at least someone sits down at some point, thinks about, reads and evaluates the report about the technology. And this is forwarded to the decision-makers” [20]. This was associated with curtailing informal influence wielded by economically powerful players such as drug companies. A former high-level governmental official recalled, “Before the establishment of HTA system there was a ‘free-robbery’ by companies. Before [the introduction of HTA in] 2004, pharma companies did whatever they wanted” [1]. This has changed dramatically thanks to the operation of the HTA Department and the requirement to submit HTA dossiers. An academic reported that “It is very rare that someone wants to receive reimbursement based only on lobbying” [8].

Last but not least, the HTA Department underpinned a broader HTA system, including a range of educational activities unparalleled in the CEE region. As an academic expounded, “We have a HTA agency and many trained HTA experts in Hungary, which is unique in the region. There is an opportunity to study [HTA] in a post-graduate program without leaving the country” [14]. In particular, the HTA Department was a key element of the infrastructure allowing for participation in international collaborations. A former HTA Department official stressed, “the Department was successful in joining to international programs and teams. This results in more experience” [20].

Notwithstanding the benefits of the HTA system in its current form, our interviewees also pointed out its costs. Overall, public resources spent on the Hungarian HTA system are considered to be negligible and the direct costs of the HTA system are mainly borne by manufacturers. From the manufacturer’s perspective, a key cost associated with the current form of the appraisal process led by the HTA Department is the administrative fee for each reimbursement submission, amounting to 1,500,000 HUF (~€5000) per reimbursement application for pharmaceuticals and 700,000 HUF (~€2300) for simple medical devices.

Estimating the public costs associated with running the HTA Department proves more difficult. First, administrative fees levied on drug companies are shared between the NHIF and the HTA Department, based on an agreement whose details are not publicly disclosed. According to a senior NHIF employee, “[the share from the fee] is based on a contract between the organisations. So there is no need for case-by-case negotiation” [12]. It is estimated that the HTA Department receives 80% of the fee for pharmaceuticals and 30% of the fee for simple medical devices. The HTA Department also has some external funding – for instance, grants from the European network for Health Technology Assessment (EUnetHTA). The HTA Department’s annual budget then according to these estimates should be between €400,000 and €450,000. However, since the operational costs of the Department are not separated from the cost of the NIPN, to which the HTA Department belongs, it is not clear whether the revenue generated from administrative fees plus income from externally funded projects is sufficient to cover its running costs.

An additional cost, from the manufacturers’ perspective, is incurred by having to develop HTA submissions, a task commonly outsourced to external consultancy companies. While the prices of developing submissions are not publicly available, the average price mentioned by most interviewees was around €10,000, with the lowest limit of about €4000. According to a manufacturing company’s representative, referring to a general submission, “For this cost they [consultants] collect the input data for the [health economic] model. (…) Then they evaluate the results, make the sensitivity analysis and prepare all the required appendix or attachments” [9]. New health economic models for economic evaluation are rarely developed for Hungarian submissions; in most cases, global models are adopted to the Hungarian context. Prices of consultancies may increase if a new model needs to be developed, local data collection from primary sources is required or if consultations with experts or additional studies such as systematic reviews are needed. In such cases, a very complex submission is typically priced at around €35,000–€45,000.

Estimating the total annual cost of developing HTA submissions by the pharmaceutical industry is challenging. A reasonably reliable method considers official yearly revenues of three to four major consultancies that focus on developing HTA submissions. There are also a handful of smaller companies with negligible annual revenue compared to the size of the consultancy market. Calculated this way, the consultancy market can be estimated to be around €3,000,000–€3,500,000. While this figure is likely to overestimate the market size (the consultancy firms may still provide other services not related to HTA), it was recognised as a reasonable estimate by a number of interviewees familiar with the sector.

As the perceived benefits of the HTA system were intangible and its costs were difficult to estimate, some interviewees maintained that public and private money spent on the system did not represent good value. This was evident given the low adherence of decision-makers to HTA recommendations. A high-level government official explained, “Currently the importance and the role of the technology assessment disappears quickly from the system. So we could say that funds spent on this process are wasted money” [19]. A former NHIF employee criticised the system by saying that early developments should not be the only perceived values anymore: “HTA moved the system from pre-historic stage to (…) an Anno Domini stage. But that it is. And how long can you be grateful to this?” [3].

Some interviewees noted that, even if manufacturers have to bear administrative costs and the costs of consultancies, the most important beneficiary is still the private rather than the public sector. Other interviewees pointed out that this system has only one clear winner, the consultancy sector. A representative of a consultant company pointed out that “benefits are there for all the consultants who are doing the [HTA] dossier[s]” [16].

Yet the perceived balance of costs and benefits could be altered by very recent policy changes, including a formal review of the methodological guideline for economic evaluation to better reflect the needs of the decision-maker, improve the skills of HTA Department workers in the validation of health economic models, build an international network and initiate direct communication to manufacturers. A former HTA Department employee explained, “It is not the worst system in the world. But it would be so easy to improve it. (…) We just need to write a few additional legal texts here and there, and we would have a much more efficient and transparent system” [15].