2.1 Organisation and governance

The organisation and governance of HTA in the various countries has witnessed significant changes over the last 10 years or so. In general, there has been an expansion in the number of HTA bodies and breadth of activities in each country, with one or more entities assuming a key regulatory or advisory roleFootnote 2 in policy making, typically with regards to coverage and reimbursement and, sometimes, pricing decisions. In some countries, such as France and England, there has been some degree of reorganisation to consolidate and rationalise assessment functions.

In Germany, HTA came to the fore principally over the last 10 years. The 2000 health-care reform law (Statutory Health Insurance Reform Act of 2000) first established a Coordinating Committee responsible for technology assessment in the hospital sector (IQWiG, 2010a). The law also charged the German Institute for Medical Documentation and Information (DIMDI) with the administration of a federally funded HTA programme to provide evidence in support of decision-making processes in health care (Perleth et al., Reference Perleth, Gibis and Gohlen2009). The German Agency of Health Technology Assessment (DAHTA) was later created within DIMDI. Another round of reforms in 2004 (SHI Modernization Act) established the Federal Joint Committee (G-BA) to serve as the supreme decision-making body of the ‘self-governed’ health-care system; its directives define the provision and reimbursement of a wide range of health technologies as well as non-medical treatment within the benefit package (Gerhardus, Reference Gerhardus2006). In the same year, IQWiG was founded to serve as an independent advisory body that conducts assessments on the benefits and harms of medical interventions (IQWiG, 2010a). One reason for IQWiG's creation was to strengthen the use of evidence-based medicine and to aid the G-BA's appraisals based on ‘the principles of effectiveness, necessity and cost-effectiveness’ (Perleth et al., Reference Perleth, Gibis and Gohlen2009).

Similarly, in Sweden, the Pharmaceutical Benefits Board (LFN, later the TLV after expanding its remit to include dental care) was introduced in 2002 following reforms aimed at enhancing the cost-effective use of pharmaceuticals and ensuring equal benefits across the country. The TLV significantly changed the introduction of drugs in Sweden; rather than automatic reimbursement within the benefit scheme, reimbursement and pricing decisions were based on an assessment of available evidence, with resultant appraisal underpinned by the principles outlined in the 1982 Health and Medical Service Act – health dignity, need and solidarity, and cost-effectiveness (Sorenson et al., Reference Sorenson, Drummond and Kanavos2008a).

While HTA has always been highly centralised in France, in 2004 the French National Health Authority (Haute Autorité de Santé, HAS), established by the Health Insurance Reform Act, replaced the previous National Agency for Accreditation and Evaluation and assumed the directorate responsible for HTA at the French Medicine Agency, in attempts to place all activities focused on improving quality of care under the remit of one body. Besides ensuring greater efficiency, the reorganisation was driven by a desire to facilitate equity within the health-care system. HAS’ responsibilities are diverse, including technology assessments, best care standards, guideline publication and health organisation and professional accreditation.

The breadth of organisations involved in HTA in England has always been diverse. During the 1970s and 1980s, a growing number of bodies, from the Medical Research Council to the Department of Health and universities, became involved in HTA. Later, in the early 1990s, a national HTA Programme was established to set research priorities, commission HTA studies, and disseminate reports. During this time, there was increasing concern over national variations in care (‘post-code prescribing’) and a desire to ‘de-politicise’ decisions about which technologies to cover in the NHS. In response to these issues, NICE was established in 1999 to serve as an arms-length organisation that employs the best available methods to provide national guidance on health technologies, interventional and diagnostic procedures and clinical guidelines. In 2004, NICE assumed the responsibilities of the Health Development Agency, expanding its remit to include the evaluation of public health interventions (e.g. preventive screenings, smoking cessation). Following recent transitions in government, NICE's role in the NHS is set to expand further in the next year or so, with a role in social care and responsibility for setting quality standards (e.g. stroke, dementia), which will feed into various commissioning activities, regulation and pay-for-performance schemes. At the same time, other functions of NICE's work may well evolve, with the announcement of the move towards value-based pricing (Department of Health, 2010).

There are some important differences in the role these organisations play in HTA. In England and Sweden, NICE and the TLV, respectively, are regulatory bodies with direct decision-making authority and are involved in both the assessment and appraisal process. In contrast, IQWiG in Germany is positioned further downstream in the decision-making process, where it assumes an advisory role, as does the French HAS, an arrangement arguably influenced by their social insurance-based health-care systems. IQWiG and HAS conduct assessments and make recommendations on coverage and reimbursement, generally regarding pharmaceuticals, to the G-BA and Ministry of Health, respectively, who appraise the evidence and render a final determination. HAS also advises individual insurance plans responsible for decisions regarding other types of health services. As a result of their regulatory function, the workings of NICE and the TLV are more closely linked to, and therefore influential, in the policy-making process, which we discuss further in the following sections.

2.2 Objectives and scope

The objectives guiding assessment requirements and methods have changed over time. All four agencies were established to improve the effectiveness and quality of health care through their activities, but only NICE and the TLV also focused on efficiency or value for money, based on cost-effectiveness analysis, as an explicit aim of HTA. Recent reforms in France and Germany, however, have introduced such objectives to HAS and, to some extent, IQWiG (Chalkidou et al., Reference Chalkidou, Tunis, Lopert, Rochaix, Sawicki, Nasser and Xerri2009; Chevreul and Durand-Zaleski, Reference Chevreul and Durand-Zaleski2009).

In 2008, the French Social Security Finance Act introduced the consideration of value for money in HAS's reviews and recommendations. A HAS Commission for Economic Evaluation and Public Health (Commission évaluation économique etde santé publique, CEESP) was established to oversee the integration of cost-effectiveness into public decision making as well as clinical practice. The CEESP will issue a recommendation on a drug's cost-effectiveness, which will be considered alongside advice regarding reimbursement and pricing. However, cost-effectiveness will only be considered when reassessing classes of drugs, medical devices, or organisational aspects of health delivery already in use; new technologies will continue to be evaluated primarily on therapeutic benefit. To date, the use and implementation of this new approach is uncertain and currently under discussion (Chevreul and Durand-Zaleski, Reference Chevreul and Durand-Zaleski2009).

A new German law (Neuordnung des Arzneimittelmarktes, AMNOG) enacted in January 2011 now requires an early assessment of the benefit of medical products. Following regulatory approval, the G-BA will assess whether the new medical product demonstrates additional benefit in comparison to a corresponding established therapy. Assessment of added benefit will be based largely on manufacturer data (e.g. clinical studies) and principally relate to relative effectiveness, although costs of the therapy will also be considered. Evidence of cost-effectiveness will only be sought if a price cannot be negotiated between the manufacturer and the sickness funds based on the G-BA relative effectiveness and cost–benefit assessment. Chalkidou et al. (Reference Chalkidou, Tunis, Lopert, Rochaix, Sawicki, Nasser and Xerri2009) suggest that such changes in both countries were driven by the recognition that the lack of early and comparative evaluation was limiting comprehensive assessments, and by the growing need to prioritise expenditure across different types of health technologies.

The scope of HTA activities undertaken by these agencies has also evolved. While the focus on assessments has been principally on pharmaceuticals, this has expanded over recent years to include other technologies and interventions. For example, NICE assesses not only drugs, but also public health interventions, surgical procedures, and has recently established a separate technology appraisal programme for medical devices (Sorenson et al., Reference Sorenson, Drummond, Kanavos and McGuire2008b). HAS also evaluates a wide range of health-care interventions. Expanding HTA to non-pharmaceuticals, however, has introduced new procedural and methodological challenges (Sorenson et al., Reference Sorenson, Tarricone, Siebert and Drummond2011). In the case of medical devices, for example, application of existing evidence standards – developed with pharmaceuticals in mind – can be problematic, as randomized controlled trials are often unavailable or too small to detect meaningful differences in clinical and economic outcomes between technologies. Moreover, devices are often developed iteratively, so there is unlikely to be a substantial ‘steady-state’ period when all relevant evidence can be collected and evaluated.

2.3 Processes and procedures

Increasingly over the last decade, it has been debated that good process in HTA is important, at least from an instrumental perspective (Culyer and Lomas, Reference Culyer and Lomas2006). While the notion of ‘best practices’ for conducting assessments continues to evolve, consensus is coalescing around several dimensions (Drummond et al., Reference Drummond, Schwartz, Jonsson, Luce, Neumann, Siebert and Sullivan2008; Chalkidou et al., Reference Chalkidou, Tunis, Lopert, Rochaix, Sawicki, Nasser and Xerri2009). Good process entails independence for those conducting assessments; transparency; openness about those involved in the assessment process; explicit timelines for completing assessments; opportunities for stakeholder input; and clear rules for appealing decisions.

While it is beyond the scope of this paper to evaluate each country on these principles, it is possible to note several developments in this area. As discussed earlier, most HTA bodies operate with a degree of independence from government and recognise and strive to protect against any conflict of interest in their processes and policies (Chalkidou et al., Reference Chalkidou, Tunis, Lopert, Rochaix, Sawicki, Nasser and Xerri2009). In England, NICE has increasingly made its processes and guidance publicly available to improve transparency. With the same aims in mind, IQWiG publishes previously unpublished (manufacturer) data considered in benefit assessments (IQWiG, 2008) and the TLV publishes its decisions on its website. Both England and Sweden have also instituted formal appeals processes. Finally, all four agencies afford opportunities to stakeholders to contribute input on various aspects of the assessment process, which we discuss subsequently in further detail, with some countries, such as Germany, making it a legal requirement.

Although steps have been taken to improve the HTA process, there are potential trade-offs involved. For example, opportunities for stakeholder input and appeals can delay assessments and resulting decisions (Haycox, Reference Haycox2008). Greater transparency can leave the HTA process open to challenge by stakeholders. Additionally, it can be argued that procedural aims or principles are often context-specific and therefore adhering to one particular set of expectations and practices across systems or even decisions may not desirable or appropriate.

2.4 Stakeholder involvement

As highlighted above, countries have sought to increase stakeholder involvement in various aspects of HTA in recent years as a way to enhance the relevance, acceptance, transparency and legitimacy of the process (Syrett, Reference Syrett2003; Milewa and Barry, Reference Milewa and Barry2005; Milewa, Reference Milewa2006; Abelson et al., Reference Abelson, Giacomini, Lehoux and Gauvin2007). In addition, stakeholder involvement was seen as a way to acquire important information on new treatments and people's medical preferences and values. Today, various stakeholder groups, including industry representatives, health professionals and patients and patient groups are participants in HTAs.

Manufacturers have assumed an increasingly integral role in HTA processes. In all countries, they submit evidence to be used or considered in assessments. In England, France and Germany, industry representatives also serve on assessment or appraisal committees that review the available evidence and develop recommendations. Given the growing number and diversity of new technologies, which often add more complex evidence and assessment requirements, HTA agencies have looked to work with industry more closely to outline evidence requirements and scope assessments. For example, in 2008, IQWiG launched its programme ‘IQWiG in dialogue’, which aims to offer representatives from the scientific community, industry and the Institute the opportunity for scientific and technical discussions on various topics related to the work of the Institute (IQWiG, 2010b).

Similarly, health professionals, medical associations and payers are often involved in assessment or appraisal committees. The G-BA, for example, includes members from associations of physicians, hospitals and sickness funds, along with patient representatives, as does many of IQWiG's governing boards and advisory panels (Nasser and Sawicki, Reference Nasser and Sawicki2009). NICE recently established a comparable body, a Partners Council, to engage with industry, NHS managers, clinicians and academics to discuss strategic challenges (Chalkidou et al., Reference Chalkidou, Tunis, Lopert, Rochaix, Sawicki, Nasser and Xerri2009). All of these stakeholder groups, in addition to manufacturers, patient groups and the general public, are also offered opportunities to identify topics for assessment in most countries.

Historically, patients, consumer groups and the public have been the least involved or represented in the HTA process, but this has changed over time. In England, for example, NICE established a Citizens Council in 2002 to gather public perspectives on key issues that inform the development of the Institute's guidance. The Council was partly a response to criticism that the ethical and social aspects of decisions were not being sufficiently articulated and that NICE was recommending too many new drugs, sometimes heavily influenced by industry and interest groups, leading to aggravated forms of implicit rationing elsewhere (Cookson et al., Reference Cookson, McDaid and Maynard2001; Devlin et al., Reference Devlin, Parkin and Gold2003). The Council is comprised of lay members reflective of the broader population; health professionals, industry and interest group representatives are precluded from participation. To date, the Council has discussed such issues as clinical need, age and rare conditions. In recent years, HAS and IQWiG have established similar opportunities for these stakeholder groups. HAS, for example, holds patient and consumer focus groups and public debates to explore sensitive and controversial topics and these groups may be members of document revision groups, which aim at improving the comprehensibility of guidance.

However, stakeholder involvement is not without challenge. For example, it has been argued that the opinions of clinicians and other professional actors often take precedence over more ‘lay’ representatives in assessment committee or working group meetings, especially given the latter almost never argue against health technologies and more frequently make more emotion-driven arguments (Milewa and Barry, Reference Milewa and Barry2005; Milewa, Reference Milewa2006). This is due, in part, to the fact that patient group representatives, for example, are often personally invested in ensuring access to a particular technology or treatment, but also because they are sometimes affiliated with or funded by pharmaceutical companies.Footnote 3 Milewa (Reference Milewa2006) suggests that the perspectives of ‘non-professionals’ are thus more likely to have their credibility or legitimacy questioned.

In relation, a growing role for industry in the HTA process raises concerns about the potential for ‘regulatory capture’, where manufacturers influence regulators through lobbying, superior information (i.e. about the products under assessment), and potentially other means to guard against unfavourable regulation. Other invested stakeholders (e.g. patients, general public), however, face major obstacles, such as limited resources and a lack of information, to exercise influence (Olson, Reference Olson1965). Consequently, decisions may ensue against the interests of a broader range of stakeholders and biased in favour of industry, a highly resourced group with significant stakes in resulting policy (Schlozman and Tierney, Reference Schlozman and Tierney1986).

A broader concern is that, overall, increased stakeholder involvement has not substantially resulted in greater consideration of a broad range of views and values, but rather serves to merely justify or legitimise difficult decisions or those that would be taken anyway. Indeed, explicit resource allocation can heighten the politicisation of decisions (Oberlander et al., Reference Oberlander, Marmor and Jacobs1994); therefore, stakeholder involvement may be a mechanism to minimise potential challenges regarding their rationality and legitimacy and the assessment process in general.

2.5 Assessment methods

There have also been several key developments in assessment methods over the last 10 years. Generally, assessments can involve different methodological components, including a clinical effectiveness assessment (i.e. therapeutic benefit, safety) and an economic evaluation, typically involving cost-effectiveness analysis. They can also evaluate broader social and ethical impacts.

In England, NICE has been recognised for its methodological rigour and, in particular, its assessment guidelines, which set standards for consistency and compatibility of the studies submitted to NICE. One of the more contentious aspects of the guidelines is NICE's requirement for the use of quality-adjusted life years (QALYs) to measure health benefit (NICE, 2008a). Although the QALY is broadly accepted, there is some debate regarding its limitations, namely that there are methodological problems with its use and that it does not fully capture the social value of interventions (Johnson, Reference Johnson2009; Oliver and Sorenson, Reference Oliver and Sorenson2009). Drummond (Reference Drummond2009) also notes that NICE has been influential on other methodological approaches, most notably in the use of mixed treatment comparisons in the use of probabilistic sensitivity analysis to account for uncertainty surrounding cost-effectiveness estimates. The TLV in Sweden also adopted and uses the QALY approach.

In contrast, HAS instituted a service medical rendu (SMR) and amelioration du service medical rendu (ASMR) rating to measure or represent therapeutic value and relative medical benefit for new drugs, respectively.Footnote 4 Drugs are evaluated alone and then against comparator products, primarily based on clinical studies, where they receive a rank from I to V, with I indicating a breakthrough treatment and V denoting no therapeutic benefit or added benefit compared to existing alternatives. However, HAS is currently developing a ‘social benefit measure’ (service renduà la collectivité) to allow for assessments to not only evaluate the therapeutic benefits of interventions, but also economic endpoints and important ethical, social and legal considerations (National Health Authority, 2007). While its use to date has been limited to screening programmes, there are plans to apply it to pharmaceuticals and other interventions.

Germany has also taken a different approach. In 2009, IQWiG, in line with its expanded remit to consider costs, adopted an alternative approach, ‘efficiency frontier analysis’. This method, which has generated considerable debate within the scientific community in and outside of Germany,Footnote 5 uses prior funding decisions for similar products in the same disease area to determine the maximum ceiling reimbursement level for a drug. However, no decisions using the efficiency frontier have been made to date and IQWIG's recommendations continue to rely almost entirely on cost-minimisation analysis, which does not assess efficiency.

In response to criticisms that the time taken to assess technologiesFootnote 6 poses barriers to patient access to new treatments, HTA bodies in England, France and Germany have established new expedited processes and methods for evaluating certain categories of drugs, typically those that are deemed highly innovative or for those treating life-threatening illnesses. England, for example, is increasingly conducting more single technology appraisals (STAs), which consider only a single technology in a single indication and rely principally on manufacturer submitted data, rather than a de novo analysis. Such approaches have allowed some drugs to be available a few months after launch. In Germany, the new reforms will require manufacturers to submit their evidence dossiers to the G-BA and IQWiG and agree on a price within one year. However, it is important to note that greater timeliness may come at a price, in the form of less robust assessments and reduced stakeholder input.

2.6 Applying evidence to decision making

Evidence generated by assessments are subsequently used to support decisions (appraisal) on the value of new technologies. Countries differ, however, in terms of what criteria are considered in the appraisal process. All countries, for example, prioritise evidence on therapeutic benefit (Sorenson et al., Reference Sorenson, Drummond and Kanavos2008a), but to date, only NICE and the TLV explicitly consider cost-effectiveness. Cost-effectiveness data is appraised using a decision threshold, measured in cost per QALY, to support determinations of value for money. The threshold differs between countries and is often implicit and case dependent. It has been estimated that NICE's and TLV's threshold ranges from £20,000–£30,000 to £45,000–£50,000, respectively (Persson and Hjelmgren, Reference Persson and Hjelmgren2003; Devlin and Parkin, Reference Devlin and Parkin2004; Rawlins and Culyer, Reference Rawlins and Culyer2004; NICE, 2008a).

However, there have been recent debates around whether the threshold should be raised (or lowered) and if a broader range of decision criteria, reflecting political, social and ethical considerations, should be more formally integrated into decision making (NICE, 2008b; Kennedy, Reference Kennedy2009; Raftery, Reference Raftery2010a; Towse, Reference Towse2009). Discussions around these issues stemmed partly from concerns that certain patient populations (e.g. those with rare or orphan conditions or terminal disease, elderly and disabled) were disadvantaged by existing decision rules and that the value of innovation was not accurately captured by current approaches. In recent years, both NICE and the TLV have considered adopting or have applied a revised approach to the threshold in certain circumstances. Sweden, for example, has discussed adjusting the threshold to better account for need or equity considerations, decision criteria outlined in the Board's founding principles, especially for drugs that are potentially expensive or address unmet medical needs. NICE recently extended its threshold for drugs aimed at end-of-life care under some circumstances to facilitate cancer drug access in the NHS, albeit under intense media, ministerial and public pressure to do so. While decision makers do consider other criteria (e.g. disease burden, equity, innovation), at both the assessment and appraisal stage, it is often unclear (and lacking transparency) what aspects are indeed considered and with what weight (Kennedy, Reference Kennedy2009). This is likely in part reflective of the need and desire of decision makers to maintain a degree of flexibility to consider political influences or other context-specific factors of importance.

The different national approaches to assessments and the appraisal process begs the question whether HTA bodies arrive at the same or different decisions. A recent study by Kanavos et al. (2010) examining all decisions made by six HTA agencies between 2007 and 2009, including NICE, HAS and the TLV, found a significant degree of heterogeneity across the coverage decisions made by the different agencies, with outcomes differing in more than half of the cases. A closer examination of agency decisions across similar cancer, central nervous system and orphan drugs showed a number of factors driving the differences in decisions, including divergent clinical and economic evidence requirements, preferred clinical endpoints, data interpretation, choice of comparator and use of cost-effectiveness thresholds. For example, compared to NICE, the TLV was more willing to consider need over cost, as intimated earlier, approving drugs beyond the threshold, up to £75,000, when there is high clinical need in certain sub-populations. Other research (Bending, Hutton and McGrath, 2010; Patel et al., 2010) on cancer drugs substantiates the influence of evidence requirements, where agencies (i.e. NICE) focused on cost-effectiveness less frequently gave positive recommendations (about 50%) compared with those only requiring evidence of clinical effectiveness (i.e. HAS).

Data availability and quality may also play a role in differential decisions across countries, introducing uncertainty into the appraisal process and opportunities for misinterpretations of existing evidence. In response, all four HTA agencies are experimenting with new approaches to generate better pre- and post-market data. For example, all countries have recently employed some form of coverage with evidence development (CED), risk-sharing agreements, or patient access schemes, where coverage of a technology is made conditional based on arrangements for additional post-market evidence collection or meeting certain health or financial outcome targets. Concurrently, agencies are assuming greater involvement in prospective data generation (Chalkidou et al., Reference Chalkidou, Tunis, Lopert, Rochaix, Sawicki, Nasser and Xerri2009). NICE, for example, does so through its ‘only in research’ option of conditional reimbursement (Chalkidou et al., Reference Chalkidou, Lord, Fisher and Littlejohns2008), with a similar programme in use in Germany and under consideration by HAS in France. Such programmes condition the use of an intervention to those patients receiving it as part of a well-designed programme of research, and are typically applied when there is insufficient evidence to make a conclusive coverage or reimbursement decision. These approaches, however, often require more sophisticated data collection and assessment methods and additional time and resources, thereby potentially adding further complexity to the HTA process. Increased analytical sophistication could result in making assessment findings less understandable and transparent and therefore more exposed to resistance by decisions makers and other stakeholders. A recent commentary (Raftery, Reference Raftery2010b) on the English NHS multiple sclerosis risk-sharing scheme highlights potential issues of governance, methodological requirements and ethics raised by these arrangements.

2.7 Implementation

The implementation of HTA-based decisions has assumed greater importance over the last decade. Although implementation was originally beyond the initial remit of almost all four bodies (Chalkidou et al., Reference Chalkidou, Tunis, Lopert, Rochaix, Sawicki, Nasser and Xerri2009), most have subsequently strengthened their focus on this issue, with NICE taking the most comprehensive approach.Footnote 7 Since 2004, NICE has operated an implementation programme to support guidance adoption and to evaluate the uptake of guidance. Additionally, financial and regulatory levers are employed in England to promote adoption (e.g. funding mandate that creates an entitlement for patients to access technologies receiving a positive NICE recommendation) and strengthen links between guidance and performance standards and payment systems; financial incentives are also used in Sweden.

3.1 Influence on coverage and reimbursement decisions

The impact of HTA on national policy varies across and within countries, but is arguably a function of the regulatory and legislative authority and instruments available to each HTA body. As intimated earlier, HTA has been most directly influential on national policy in England and Sweden, particularly with regards to pharmaceutical coverage and reimbursement. For example, as previously mentioned, in the case of NICE, the NHS Constitution makes all positive NICE decisions an entitlement, and in Sweden, a positive approval decision by the TLV is required for a pharmaceutical to be made available on the health system. In Germany, the G-BA is not mandated to follow IQWiG's recommendations, although it is required to provide justification if its policies deviate from the Institute's advisement. This is also true in the French context, where HAS only gives an opinion to the Ministry of Health. However, available evidence suggest more than 95% of HAS recommendations are followed (Rochaix and Xerri, Reference Rochaix and Xerri2009). Moreover, in the German case, under law, new drugs and inpatient medical services are covered by default and are assessed for possible exclusion only if the G-BA requests an evaluation by IQWiG. Consequently, Germany pays higher prices and covers more new drugs than other European countries (Nasser and Sawicki, Reference Nasser and Sawicki2009). However, the 2011 reform strives to address this issue by requiring early comparative benefit assessment to prevent additional cost and harm of medical services without a proof of benefit.

For countries with some level of decentralised decision making regarding access to or funding of treatments, there is concern that HTA has had less of an impact, or an inconsistent impact, on policy making at the local level. For example, as NICE only examines a fraction of all services provided across the NHS, most local decisions are not supported by HTA. Moreover, given funding mandates for local implementation of NICE technology appraisal guidance, concerns have been raised that implementation requirements potentially shift resources away from other, more cost-effective services (Devlin et al., Reference Devlin, Parkin and Gold2003; Sheldon et al., Reference Sheldon, Cullum, Dawson, Lankshear, Lawson, Watt, West, Wright and Wright2004), although it is not clear what these services may be, as the opportunity costs of adopting particular technologies are not considered in assessments and local authorities rarely have formal mechanisms in place or the resources to make such judgements (Audit Commission, 2005; Appleby et al., Reference Appleby, Devlin, Parkin, Buxton and Chalkidou2009; Chisholm et al., Reference Chisholm, Briggs and Askham2009). In the Swedish context, Anell and Persson (Reference Anell and Persson2005) indicate that is not clear the extent to which formulary committees, organised by the county councils, consider available HTA-generated recommendations, particularly economic evidence, to support decision making, and that the recommendations of the committees and the TLV often differ. Furthermore, HTAs may be limited in accounting for important geographic specificities in local policy making, in that use of a technology may be cost-effective in one region, but not in another due to different clinical practice patterns or patient population characteristics and needs.

The impact of HTA on policy is also dependent on the particular circumstances surrounding an assessment or decision. For instance, decision makers may not fully understand or accept an assessment, resulting evidence, or final recommendation, and even if they do, the consequences (e.g. a negative recommendation to adopt a given technology) may not always be accepted, especially if it results in challenge from industry, clinicians, patients and the general public. In Germany, for example, the Federal Constitutional Court challenged an earlier decision that restricted reimbursement for bioresonance therapy following request for treatment from a 19-year-old patient suffering from Duchenne muscular dystrophy. Based on the life-threatening nature of the illness, sickness funds were required to cover the treatment despite a lack of solid scientific basis demonstrating benefit (Schmidt and Kreis, Reference Schmidt and Kreis2009). Similarly in England, various stakeholder groups have pressured policy makers to circumvent negative NICE decisions on cancer drugs and multiple sclerosis treatments, and as a potential by-product, the new coalition government announced intentions to create a ‘cancer fund’ to pay for oncology therapies rejected by NICE. Williams and Bryan (Reference Williams and Bryan2007) suggest the more high profile the decision body or the decision being taken, the more likely external stakeholder groups and the media are likely to intervene if decisions are viewed as negative. For example, HTA decisions can significantly impact potential financial gains (or losses) for manufacturers, so they understandably exercise voice to ensure such interests are protected, especially if the decision of a particular HTA body has some degree of international influence, as in the case of NICE. Besides political considerations, assessments may or may not be influential in decisions depending on administrative capacities, equity concerns, broader societal preferences, and decision makers own values and experiences (Goddard et al., Reference Goddard, Hauk, Preker and Smith2006; Owen-Smith et al., Reference Owen-Smith, Coast and Donovan2010). Furthermore, the evidence generated by HTAs may not sufficiently meet the information needs of decision makers or may not be made available early enough to benefit decisions (Hutton et al., Reference Hutton, McGrath, Frybourg, Tremblay, Bramley-Harker and Henshall2006; Williams and Bryan, Reference Williams and Bryan2007).

3.2 Clinical practice

The available evidence evaluating the influence of HTA on clinical practice is sparse and predominately relates to NICE. Several studies concluded that practice generally reflected the recommendations of the NICE technology appraisal(s) evaluated (National Cancer Director, 2006; NICE, 2006a, 2006b), but generally more so in the case of pharmaceuticals than procedures and devices (Sheldon et al., Reference Sheldon, Cullum, Dawson, Lankshear, Lawson, Watt, West, Wright and Wright2004). Other researchers, however, found limited impact on practice patterns and evidence of a high degree of regional variation in guidance adoption (Brickwood, Reference Brickwood2004; Hitchen, Reference Hitchen2008; Owen-Smith et al., Reference Owen-Smith, Coast and Donovan2010).Footnote 8 Although, for some interventions, particularly cancer and obesity drugs, there is evidence of improved uptake of NICE-approved therapies and reduced regional variation in care (Sheldon et al., Reference Sheldon, Cullum, Dawson, Lankshear, Lawson, Watt, West, Wright and Wright2004; Department of Health, 2009).

In a qualitative analysis of stakeholder perspectives on the usefulness of NICE guidance in practice (Owen-Smith et al., Reference Owen-Smith, Coast and Donovan2010), clinicians generally agreed with the importance of evidence-based medicine and found guidance useful, but its utility declined in certain contexts: the secondary care setting, when requisite funding was insufficient, and when they disagreed with the recommendations. Interestingly, several practitioners noted that the existence of NICE guidance can make it more difficult to resist patient demands, resulting in overtreatment, but that when the Institute said ‘no’ or placed restrictions on access, it offered a ‘good defence’ against blame for rationing treatment.

Similar conclusions regarding impacts on practice can be drawn in the French context. Criticisms have been raised that the process of disseminating guidelines is not well-structured and that adherence amongst physicians is poor, even though they are legally required to follow them in most cases. Moreover, guidelines are frequently produced on clinical issues that are not well-defined (i.e. little clinical certainty exists) and do not reflect areas where the most significant practice variations exist. Commentators have argued for better tools to prioritise and update or revise guidelines in attempts to be more responsive to emerging topics and stakeholder needs (Caniard, Reference Caniard2002).

The influence of HTA on clinical practice depends partly on available incentives for guidance adoption, in addition to strong professional support and a clear and robust evidence base demonstrating a given treatment provides value (Sheldon et al., Reference Sheldon, Cullum, Dawson, Lankshear, Lawson, Watt, West, Wright and Wright2004). Several countries have therefore introduced a range of mechanisms to support practitioner uptake, including the use of financial incentives (e.g. provider performance standards) and through educational and training programmes to enhance understanding of HTAs and how to access and interpret associated evidence. For example, the newly proposed Best Practice Tariff in England aims to align provider payment with NICE clinical guidelines by providing additional payment for delivering services that meet quality standards. To enhance professional support, a network of local experts was developed in Sweden help to ensure HTAs are applied in clinical practice (Sorenson et al., Reference Sorenson, Drummond and Kanavos2008a).

3.3 Health and economic outcomes

There is limited evidence demonstrating the health and/or budget impact attributable to changes in policy and practices associated with HTA.

Some proponents of HTA promulgate its use based on conclusions that it reduces health-care costs. However, cost containment or cost-savings was never an objective of any of the four national bodies (Chalkidou et al., Reference Chalkidou, Tunis, Lopert, Rochaix, Sawicki, Nasser and Xerri2009), with aims of better quality of care, equitable access to care, and value for money taking precedence. In the case of NICE, its guidance has most likely been cost-increasing, in the order of £1.65 billion per year in additional NHS investment (NICE, 2009). This is not surprising since most interventions that are deemed cost-effective are more expensive than their comparator interventions. The French HAS also claims that any adoption of cost-effectiveness analysis would not be used to save money (by reducing services), but attain more efficient use of resources (Chalkidou et al., Reference Chalkidou, Tunis, Lopert, Rochaix, Sawicki, Nasser and Xerri2009).

Whether overall costs increase or decrease as a result of HTA and how this should be evaluated depends on a number of factors. One issue relates to the baseline or starting point from which to derive conclusions about cost impact. For example, in France, where the use of drugs is significantly higher per capita than in England or Sweden, HTA may result in cost-savings without harming health outcomes. Another consideration is a particular system's structure, including its funding model. In England where budgets are set by government rather than supply and demand, 2002–2010 saw a significant increase in NHS spending, which NICE took advantage of, rather than caused, by then allocating some of the additional funding toward new technologies; a situation that will change in the future with restricted spending. In addition, the effect of HTA decisions on total costs is rarely measured or taken into account, therefore it is difficult to accurately assess impact on this dimension. NICE has tried to address this issue recently by costing the national impact of implementing its guidance for certain technologies or conditions. Finally, it is not entirely possible to assess the cost impact of the counterfactual; that is, if a given HTA decision had not been implemented.

3.4 Innovation

Critics of HTA often cite potential negative impacts on innovation by creating yet more ‘hurdles’ for industry. However, there is no evidence coming from any of these countries (or elsewhere) to substantiate such concerns. In fact, as Chalkidou et al. (Reference Chalkidou, Tunis, Lopert, Rochaix, Sawicki, Nasser and Xerri2009) point out, HTA systems can serve to create a more predictable and consistent way for industry to get their products to patients and for payers to make purchasing decisions. For products of true value, the HTA process should be of no real concern and even welcomed. In particular, for countries applying value-based approaches, manufacturers of truly innovative technologies will be rewarded with high or higher prices and from the onset will be armed with some indication of the type of innovation that is valued by decision makers (Hughes, Reference Hughes2008; Claxton, Reference Claxton2009; Kennedy, Reference Kennedy2009).