Healthcare expenditures are increasing in all Western countries; one of the major drivers is the introduction of many new and expensive technologies and their rapid diffusion into clinical practice. Payers, therefore, have started to take a closer look not only at the costs, but also at underlying evidence for the effectiveness or comparative effectiveness of procedures that have not been in use long enough for adequate evaluation. To regulate coverage and reimbursement of (new) medical interventions, several countries have defined benefit catalogues. Benefit catalogues can include either explicit listings of health services that are reimbursed, serving as positive lists (e.g., France, Poland, Spain, Austria), or the catalogues can be implicitly defined and include negative lists of services that will not be reimbursed (e.g., hospital care in Germany, United Kingdom). The EU-Health Benefit BASKET Project observed the trend toward more explicit definitions of benefit catalogues (Reference Schreyogg, Stargardt, Velasco-Garrido and Busse19). The BASKET authors conclude that ideally the decision on whether to reimburse medical services should be based on objective decision criteria such as evidence for effectiveness, safety, and cost-effectiveness. However, despite increasing availability of evidence-based analyses, in reality the actual decision-making process often still lacks transparency. Some countries, such as Canada, have, therefore, started to develop a comprehensive continuum of steps within the decision-making process as a framework for more consistency in decision making and for transparent and evidence-based recommendations and decisions on the use of new technologies (Reference Johnson, Sikich and Evans13;Reference Levin, Goeree and Sikich15).
Austria has followed the international trend to define benefit catalogues for in- and outpatient care more explicitly and to fit evidence-based technology assessments into the general policy-making process. Within the Austrian diagnosis-related group (DRG) system for hospital services (in Austria it is called “leistungsorientierte Krankenanstaltenfinanzierung” (LKF) (1)), interventions that are resource-intense, such as costly surgical procedures or high-tech based interventions, are financially compensated as “Extra Medical Services” (EMS) (German: Medizinische Einzel-Leistung, MEL) additional to the main DRG calculation. A positive list of EMS that are reimbursed is provided and maintained on an annual basis by the Austrian Ministry of Health MoH (2;3). Hospital trusts can apply for reimbursement of new EMS annually and after a decision process the MoH decides whether or not to include these new medical services on the positive list.
In 2008, the decision process was revised, and health technology assessments were introduced as a supporting tool to provide evidence analyses on new proposed EMS. To begin, a search was carried out for similar evidence-based appraisals of hospital interventions. Using the GRADE working group Web site (10), a hand-search was carried out looking for single technology appraisal processes, formats, and the application of GRADE as a tool for making recommendations for reimbursement decisions in were available in the English language and on the Internet. The following publications/protocols were selected as good practice prototypes: The STA/Single Technology Appraisal evidence reviews developed by NICE (17) for the transparent process and template and the OHTAC/Ontario Health Technology Advisory Committee (18) for the transparent decision-making framework using GRADE for recommendations.
This study has multiple aims: first, to describe the newly established decision-making process in Austria; second, to demonstrate how HTA contributes to hospital technology decisions; and lastly to analyze the impact of the first 3 years of HTA-supported decision making on reimbursement of new EMS in Austria.
METHODS
Process and Project Management
The LBI-HTA supports the annual process of decision making regarding coverage of new interventions by providing evidence syntheses on the effectiveness and safety of the pre-selected individual medical services in question. For Re-evaluations/“updates” of earlier-performed assessments (such as repeated proposals for EMS rejected in earlier years, or technologies with preliminary status), an update of newly published literature is performed by using the same search strategy as in the primary assessment.
The evaluation process for approximately ten newly proposed EMS starts simultaneously in the 3rd week of January and must be completed by end of March. The assessment process involves one main researcher/first author, one co-researcher/second author and one external reviewer per assessment, one information specialist, one person for internal review and methodological advice, as well as co-workers who perform hand-searches, literature retrieval, spell checking, and layout for each assessment. Therefore, precise project management is necessary to coordinate all participants to fulfill all quality standards of the assessment procedure within the short timeframe. The assessment procedure consists of predefined work steps, shown in Figure 1.
Annual decision process for inclusion or exclusion of new proposed “Extra Medical Services” (EMS) to the Austrian DRG system.
Scoping and Definition of Research Questions
During the scoping process (7), the specific questions to be addressed for each technology assessment are determined. Issues of interest such as patient population, comparators, or outcome parameters are defined as clearly as possible. Clinical experts and the product manufacturer are contacted during the scoping process to collect additional information. The scoping process produces the study protocol for the technology assessment (Reference Gartlehner, Wild and Felder-Puig9). Clinical experts give additional support by helping to define relevant clinical endpoints for patients.
Systematic Literature Search and Literature Retrieval
Systematic literature searches are coordinated by one information specialist for all assessments. Several medical databases (MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials and NHS-CRD-HTA/INAHTA, EuroScan) are searched for primary (clinical studies) as well as secondary (systematic reviews, meta-analyses, and other HTAs) via a predefined search term. Additionally, a hand search with Scopus is performed. Two researchers per assessment decide independently on the inclusion of relevant literature. The selection process for study inclusion is described by the PRISMA (Reference Moher, Liberati, Tetzlaff and Altman16) (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow chart. After the inclusion process, full text literature is retrieved for all assessments simultaneously by the information specialist and co-workers.
Assessment, Critical Appraisal, and Recommendations
Two researchers for each assessment analyze all available evidence by evaluating the technology systematically and independently. The assessment process aims to produce an estimate (including uncertainty) of the effectiveness and safety of the medical service being investigated for a specific indication. For quality reasons, data extraction from literature is always double-checked by a second person. After the critical appraisal of the available evidence, the grading of the evidence is then performed using the GRADE (Grading of Recommendations Assessment, Development and Evaluation) methodology (Reference Guyatt, Oxman and Vist11). The GRADE system is based on a sequential assessment of the quality of evidence, consistency of effects, directness, and magnitude of the effects for all patient-relevant endpoints followed by the assessment of the balance between patient-relevant benefits versus drawbacks and subsequent judgment about the strength of recommendations (Reference Kunz, Lelgemann, Gayatt and Von der Evidenz zur14). Subsequently a report is formulated summarizing all available evidence on the effectiveness and safety of the medical service under investigation, resulting in one of four possible levels of recommendation (LOR), as seen in Table 1.
Levels of Recommendation for Decision Makers
Internal and External Review
All assessments are appraised by an internal reviewer for methodological issues. Since 2009, an external expert for subject-specific and medical issues has also reviewed the preliminary report before the final document is produced.
RESULTS
The New Decision Process
The Federal Health Commission (FHC) introduced two new policy tools into the decision process in 2008: First, evidence analyses by an independent academic institution (Ludwig Boltzmann Institute for Health Technology Assessment/LBI-HTA, the authors) were introduced as decision support for/against the coverage of new EMS in contrast to the former solely expert-based recommendations. Second, “preliminary coverage under conditions” (only in university hospitals; no separate tariffs, but reimbursement within an existing DRG group) was introduced as a new regulatory category; EMS with “preliminary” status must be re-evaluated every other year.
Before December of each year, hospital trusts are invited to send proposals for new EMS to the Ministry of Health (MoH) (see Figure 1). A list is compiled and handed over to the committee of regional representatives, the main payers, and administrators. This working-group meets in early January to discuss the list of technologies. They “unofficially” approve the list and recommend the technologies that qualifying for evidence syntheses. This is not a priority setting process but an additional analysis and validation for those technologies not already reimbursed within the DRG system. Upon “official” approval of the list and the recommendations by the Federal Health Commission, the LBI-HTA is commissioned and receives the topics for assessment by the end of January. The completed assessments, including recommendations based on GRADE, are returned by the end of March to the secretariat of the FHC, which distributes the assessments to the regional decision makers. Between mid-April and mid-May, the committee of regional representatives prepares the proposals for the final decisions by considering the results of our assessments. Cost-analysis based on number of patients, unit-costs, budget impact, and regional access are considered to be issues of the political appraisal and decision-making process and are not prepared by LBI-HTA. In June, final decisions regarding coverage for the upcoming year is made by the political body of FHC. The whole process, from preparation of the application to submission requires approximately 6 months. Final decision making takes another 6 months and coverage begins a half year later. The synthesized evidence is published online (free access) on the 15th of July each year.
Decisions of the First 3 Years
All assessments that were performed covered surgical or diagnostic interventions; none covered medications: The majority of the interventions under investigation were new surgical procedures in orthopedics, cardiology, and oncology; other non-surgical interventions were of diagnostic (Optical Coherence Tomography) or therapeutic (radionuclide therapy) nature.
Some (eight) of the assessments were also on “established” (old, covered) interventions.
In the past 3 years, a total of twenty-five EMS-assessments on thirty-three different interventions were performed by the LBI-HTA. Additionally nine updates synthesized a follow-up of the previous year(s) assessments. Some of the assessments covered more than one technology. For example, under the umbrella of the EMS “minimally invasive methods for the treatment of stress urinary incontinence”, five different products and interventions were covered; under the umbrella of the EMS injection therapies for chronic back pain, three different interventions were covered. On the other hand, one specific product (an artificial disc) might be indicated for different indications such as in the “cervical or lumbar spine.”
The majority of the included studies within the assessments were of very low to low quality according to GRADE. Moderate to high quality of literature was found in some established orthopedic interventions (kyphoplasty and vertebroplasty, chemonucleolysis and intradiscal electrotherapy, artificial spinal discs) and one cardiological intervention (drug coated balloon catheter). The main reason for GRADEing low quality was due to study design; most of the trials on the new interventions were case series or uncontrolled cohort studies; only a few assessments, such as on artificial spinal discs, contained randomized controlled trials.
Recommendations
The recommendation for in- or exclusion of EMS to the DRG system based on the underlying evidence was determined according to a predefined recommendation key consisting of four different levels of recommendation (LOR), see Table 1. In total, thirty-three recommendations were given: No assessment resulted in LOR 1 (Recommendation, acceptance; clear evidence for a net benefit of the intervention), because there was no intervention under investigation for which striking evidence of net benefit was found.
Eleven assessments (33 percent) resulted in LOR 2 (Recommendation with limitations; net benefit of the intervention limited by weak evidence), 20 assessments (60 percent) resulted in LOR 3 (Preliminary rejection; weak evidence is showing no net benefit), and two assessments (6 percent) resulted in LOR 4 (rejection; clear evidence for no benefit).
Decisions
In total, of thirty-three interventions in the twenty-five assessments, the decisions on seven interventions were positive: two interventions (6 percent, both LOR 3) were accepted without re-evaluation (percutaneous pulmonary valve implantation for right ventricular outflow tract dysfunction in patients with congenital heart defects and combination radionuclide therapy or single therapy with Y90 and Lu177 in inoperable tumors); five interventions (15 percent, all LOR 4) were temporarily accepted with conditional requirements such as re-evaluation within a certain time period.
The decisions on twenty-six interventions (eighteen assessments) were negative: eighteen interventions (55 percent, one with LOR 2, five with LOR 3, twelve with LOR 4) were rejected by the FHC commission based on the proposals of the regional committee for decisions. For 8 interventions (24 percent, one with LOR 2, four with LOR 3, and three with LOR 4), former decisions on general acceptance were withdrawn and exchanged with “subject to approval.” Those “delisted” (see Table 2b) were all orthopedic interventions for acute or chronic back pain: kypho- and vertebroplasty, chemonucleolysis, intradiscal electrotherapy, percutaneous nucleotomy and percutaneous laser disc decompression, injection therapies (facet- or sacroiliac joint near injections, epidural injections), radiofrequency, and finally the artificial spinal disc for cervical spine.
Characteristics of the Health Technology Assessments With Positive Decision: GRADE Quality of Evidence, Recommendation, and Final Decision
EMS, Extra Medical Services, in German MEL: Medizinische Einzelleistung; GRADE, Grading of Recommendations Assessment, Development, and Evaluation; MoH, Ministry of Health
DISCUSSION
After 3 years of experience implementing new elements for Austrian decision making on new hospital interventions, we want to present a first résumé on the successes and challenges within this process. The key issues are as follows.
Introduction of Two New Elements in the Policy Process
Implementing evidence syntheses in the decision-making process and introducing the term of preliminary or limited coverage as new regulatory instruments appears to have been a wise step. It not only provided additional time to gather good evidence, thus allowing a final decision based on firm rather than sandy grounds, but also it should lead to greater acceptance by clinicians, especially those in university hospitals, by convincing them that evidence-based decision making is not keeping “real” innovations away from patients.
Characteristics of the Health Technology Assessments With Negative Decision: GRADE Quality of Evidence, Recommendation, and Final Decision
Note. All MEL assessments are available on the LBI-HTA Web page: http://eprints.hta.lbg.ac.at.
a Approval based on regional conditions (volume-quality, interdisciplinarity and back-up, and post-operative care)
EMS, Extra Medical Services, in German MEL, Medizinische Einzelleistung; GRADE, Grading of Recommendations Assessment, Development, and Evaluation; MoH, Ministry of Health; LDL, low density lipoprotein.
Handling Time Constraints and Quality Assurance
Carrying out many assessments within a period of only 8–9 weeks and working with the common standards of scientific rigidity and quality assurance (double checking of literature selection and extraction tables, peer review) requires that logistics be planned well. Key components include the following: (i) Standardization of certain work-steps: a given editorial format for writing the synthesized evidence; project management by one researcher and pre-defined roles of contributors (first author, second author, information specialist, methodologist); pre-formulated letters to industry, etc. (ii) Good work division between coordination of rapid communication with the FHC as the interface to policy, exact planning of the many searches by the information specialist, (only) one main responsibility for one assessment for each LBI researcher (first author), and one co-authorship for a second assessment, methodological supervision, exact planning of 1 week for peer-review by an external expert beforehand.
In 2008, the first and pilot year, we decided not to peer-review the evidence syntheses because of time-constraints We rapidly revised this decision in 2009 after discovering that two assessments were used not only for decisions making on coverage, but also in lawsuits, for example, of private patients having paid large sums for assumed innovative, but in reality experimental new interventions in private practice. We also felt more secure involving clinical specialists and experts via formal peer-review. In 2010, some of the peer-reviewers came under pressure from their own peers and, therefore, the next step will be to blind the peer-reviewers. A public consultation is not under debate—yet.
Educational Effects, Recommendations, and Decisions
The process of proposing, assessing, deciding upon coverage, and reimbursing of new interventions, requires approximately 1.5 years. Only a small percentage of interventions are finally accepted for coverage within the public healthcare system. For those who make the effort to propose new interventions for coverage, this is de-motivating. Although for many years technologies and interventions, most of them based on evidence of singular case-studies, have been pre-maturely proposed for coverage, the new transparent processes and protocols consequently lead to fewer pre-mature or experimental technologies being proposed now.
In most cases, the FHC adheres to the recommendations, especially when there is a clear “no evidence,” but in some cases the commission decides differently, especially when there are only a few (and sometimes young) patients (e.g., percutaneous pulmonary valve implantation for right ventricular outflow tract dysfunction in patients with congenital heart defects) or when a “market” assumed to expand very fast without clear-cut indication (percutaneous aortic valve replacement in inoperable patients) has to be kept under control with the instrument of limited approval. A total of 60 percent of proposals are rejected (as much by the recommendations based on GRADE as by the policy decision). This percentage parallels experiences in the neighboring country of Germany, where 56 percent of all new proposed interventions are rejected. Another 21 percent have been accepted for coverage in the past 3 years in Austria without condition or with certain requirements to fulfill. Again, this percentage corresponds with the German data, in which approximately 22 percent of new applications are accepted after negotiations between payers and appliers (4).
Evidence Synthesis, National and International Accountability, and GRADE
Because technologies are often entering the healthcare market at the same time, they are being assessed by other HTA institutions, either at the same time, a year before or a year later. The use of each other assessments is of utmost importance: while the methodology of evidence synthesis is well acknowledged and reliably used, the use of the GRADE system for developing recommendations shows certain variability (Reference Ballini, Minozzi, Negro, Pirini and Grilli8;Reference Ibargoyen-Roteta, Gutierrez-Ibarluzea and Rico-Iturrioz12), for example, concerning the involvement of clinicians and patients when defining the patient-relevant endpoints. The consequence of the harmonization of methodology is increased national and international accountability, which provides decision makers with strong arguments about the effects/effectiveness/advisability of new health technologies for patients, clinicians, and society.
Open Publication of Evidence Syntheses
For 3 years, assessments have been publicly available only after the political process, when the negotiations and decisions were completed. This late release policy is now being discussed. Proponents of late (3 months after actual completion of assessments) publication argue that earlier publication would open the doors for lobbyists and that a delay provides a comfortable margin for decision makers. Opponents state that evidence syntheses should be/are “bomb-proof” and transparent in showing evidence (or often lack thereof). They argue the analyses speak for themselves, so, therefore, early publication is a key part of transparent evidence-based decision making.
Established Interventions Under Question/Disinvestment
Several interventions that were previously included unconditioned and unlimited in the service benefit catalogue have undergone reassessment within the past 3 years. These include several minimal-invasive interventions for chronic back pain as well as kypho- and vertebroplasty. The reasons for re-assessment of those well-established technologies was their generally low frequency of use in peripheral clinical settings, and the according need to regulate quality assurance (minimal frequencies, infrastructural, and back-up medical services). The policy decision changed to cancellation of general coverage for those interventions. The coverage is now subject to approval by regional competence centers.
Nevertheless, even if the reasons for withdrawal of general coverage are evident and plausible, for the sake of acceptance, there is a strong need within the policy process to define criteria for reconsidering the status of those interventions and for disinvestment decisions.
Exclusion of Expensive Medications
At this time, new drugs such as oncologic medications are not included in the assessment process. The reason is—as in most countries—that the regulatory and assessments processes for medical devices, surgical interventions, and drugs are performed in separate procedures and by separate decision makers. While Austria has a positive list for outpatient drugs, inpatient drugs are decided upon by an in-hospital drug commission via a negative list (explicit exclusion). This separation is currently under debate and might change in the near future.
Academic Publications and Careers
The format for evidence syntheses was created based on three assumptions: the assessments are supposed to be easily readable by politicians and administrators, scientifically accepted by the clinicians and publishable by the researchers without too much additional effort. Much of the evidence synthesized has been published easily in either German or English language peer-reviewed journals because of its scientific validity and its timeliness to decision making. Nevertheless, it is a problem for some journal editors that the assessments have already been published online in a similar format. A solution might be—as NETSCC former NCCHTA—did, to edit the full evidence synthesis as a journal article.
Conclusions
One lesson that other countries can learn from the Austrian experiences is that the introduction of transparent evidence syntheses for decision support must be accompanied by transparent processes for pre-selection of topics and for political appraisal and decision making after the assessment phase. Second, the introduction of a new regulatory instrument for reimbursement, namely “preliminary coverage under conditions” was easily implemented in the re-organization of the assessments and appraisal process.
For Austria, these first 3 years have proven to support decision makers with strong arguments derived from underlying evidence to decide for or against the coverage of new interventions. In times of financial constraints these strong arguments weigh increasingly heavily and thus HTA is becoming ever more important. The future of pre-coverage assessments of new hospital interventions lies in international cooperation. At least in Europe, but also overseas, we live in very similar medico-technological healthcare systems. New interventions reach the markets at the same time (European Medicines Agency [EMA] and Food and Drug Administration [FDA] approvals) or almost simultaneously (CE- or FDA-marked devices, surgical interventions).
Public resources for the assessment of new interventions could be spent more efficiently with collaborations and reduction of redundancies. Preliminary co-operative schemes with institutions carrying out similar pre-coverage assessments have already been piloted (2009) and conducted on a regular basis (2010+) with the German “Medizinischer Dienst des Spitzenverbandes Bund der Krankenkasen e.V. MDS” (6). Other co-operative ventures are under way (5) within EUnetHTA Joint Action 2010–2012. It is the remit of EUnetHTA to support and facilitate structured collaborations on such effectiveness analyses, but the actual recommendations for national reimbursement will remain national tasks.
CONTACT INFORMATION
Philipp Mad, Dr, Ludwig Boltzmann Institute for Health Technology Assessment, Vienna, Austria
Sabine Geiger-Gritsch, Dr, Ludwig Boltzmann Institute for Health Technology Assessment, Vienna, Austria; Institute of Public Health, Medical Decision Making and Health, Technology Assessment, Department of Public Health, Information Systems and Health Technology Assessment, UMIT–University for Health Sciences, Medical Informatics and Technology, Hall, Viena
Gerda Hinterreiter, Mag, Stefan Mathis-Edenhofer, Dr, Claudia Wild, Priv-Doz, Director of LBI-HTA, Ludwig Boltzmann Institute for Health Technology Assessment, Vienna, Austria
CONFLICTS OF INTEREST
All authors report they have no potential conflicts of interest.
