Health technology assessment (HTA) is a multidisciplinary form of policy analysis that provides decision makers with information on the clinical effectiveness, safety, cost, organizational and other implications of health technologies (1).
In Canada, healthcare decision making and priority setting are decentralized and occur at the provincial or regional level, whereby funding is allocated from the provincial or regional governing bodies to local healthcare institutions (2). While hospitals usually operate within an annual budget, based on historical allocation of funding from the regions, increasingly the annual budget model is complemented by activity-based funding, or other funding innovations to spur evidence-based care within the constraints of limited dollars. Nevertheless, most decisions regarding health technologies are made at the individual hospital level, within the constraints of their annual budget.
Canadian HTA bodies exist at various levels to reflect this provincial, regional, and individual hospital decision-making structure. The Canadian Agency for Drugs and Health Technologies (CADTH) is a national HTA agency, and provincial entities are present in Alberta, Ontario, and Quebec. In the past 25 years, the HTA infrastructure in Canada has expanded to other levels of jurisdiction to facilitate greater alignment of HTA products to decision-makers’ and end-users’ needs (2). Despite the existence of HTA agencies at the provincial level in Quebec and Ontario, local HTA entities (most of them hospital-based units) are emerging in both jurisdictions to assist decision makers in the acquisition, continuation, and disinvestment of healthcare technologies and, ultimately, impact on regional and local hospital policies and clinical practice.
There is an increased interest in local HTA as a means to improve the efficient use of finite budgets to deliver healthcare services and to complement the activities at the national level by ensuring buy-in and contextualization at the local level (2;3). In Canada, local HTA refers to HTA activities in an institutional hospital setting or a regional setting (regions may also denote clusters of hospitals which share the same governing body across acute and ambulatory care services). Although the principle methods to assess the clinical evidence generally remain similar for all levels of HTA, local HTAs can be further tailored to meet the needs and timelines of the local decision maker, and to include local data on costs, clinical practices, and usage. In addition to contextual considerations, local HTAs can help to foster a culture in the use of evidence to support informed decision making among hospital administrators and frontline clinicians (3).
On 18–19 November 2013, the pan-Canadian HTA Collaborative hosted a symposium in Ottawa, Canada, Hospital/Regional HTA: Local Evidence-based Decisions for Health Care Sustainability, in partnership with the Centre for Medical Evidence, Decision Integrity & Clinical Impact (MEDICI) and the Ottawa Hospital Research Institute with support from CADTH. The Collaborative brings together several HTA agencies in Canada to identify and collectively address common issues in HTA, such as processes, methods and information sharing. The goal of the symposium was to showcase the diversity of local HTA across Canada, while identifying common concerns and gaps expressed by local HTA producers and discuss opportunities to address them cooperatively. Over sixty participants interested in the promotion and expansion of local HTA attended the symposium, including HTA producers, healthcare decision makers, researchers, and industry.
Our study reports the state of local HTA in Canada and its benefits and challenges in influencing hospital policies and clinical practice derived from the presentations and discussions at the Symposium.
METHODS
The scientific program was comprised of an initial 2-hour workshop, HTA 101 for Decision Makers and Policy Makers at the Regional and Hospital Levels, followed by twelve presentations from Canadian speakers, who represented the local HTA producers and users. One international speaker was also invited to present the Adopting Hospital Based Health Technology Assessment (AdHopHTA) initiative in Europe on adopting hospital HTA. Each session lasted between 30 and 45 minutes. Presentations centered on the various models of local HTA, perspectives and practices of local HTA across Canada, the contextualization of HTA reports to meet local needs, strategies for knowledge mobilization to inform local health technology use, and successes and challenges of local HTA. Other sessions introduced and proposed a pan-Canadian network for local HTA to facilitate information sharing and collaborations among its producers.
Discussions occurred after each session and were open to all attendees and stakeholders to contribute. They were also audio-recorded and transcribed. The transcription was reviewed by one reviewer (L.A.T.) to identify, define, and organize unique themes. Another individual (J.P.) reviewed the coding framework to ensure consistency in the interpretation of the content in the transcript.
As this study reported on presentations and discussions from a symposium on hospital and regional health technology assessment, a formal literature search strategy was not developed. Instead, we searched PubMed for publications by the symposium speakers as well as papers on health technology assessment in local settings, hospitals, and regional agencies in Canada and internationally. The search was also supplemented by reviewing the bibliographies of relevant papers. For instance, a 2014 Canadian systematic review on hospital-based HTA provided further coverage of the published literature in this area (4). Language restrictions and a search timeframe were not imposed.
RESULTS
Discussions at the Symposium generated five unique themes, including: assessment of evidence for local HTA, contextualization, stakeholder engagement in local HTA, knowledge translation and impact of recommendations, and challenges and opportunities for HTA. Presentations and supporting documents from the symposium are available online at https://www.cadth.ca/hospitalregional-hta-symposium-0.
Assessment of Evidence for Local HTA
Some local HTA producers have made an effort to assess organizational impact and actual costs, and to weigh these against opportunity costs for the hospital budget. At the MEDICI hospital-based HTA and knowledge translation program within the London Health Sciences Centre and St Joseph's Health Care, a process of contextualized evidence-based assessment and context analysis (including costs and organizational issues) is used to create trade-off table (called Know4go), which makes the local tradeoffs and opportunity costs transparent when choosing among several opportunities for health technology investment or disinvestment. After decisions are made, the tradeoff table is also revised to reflect changes in evidence, costs and resources over time, and allowing reassessment of the technology and evolving competing priorities.
Historically, when assessing opportunity costs, local HTA bodies have generally been constrained to work primarily within the perspective and budget of the hospital or region concerned (5), rather than primarily taking a healthcare system perspective to optimize opportunities across the system. Nevertheless, as innovative funding models change, and budgets flow across silos and become interdependent across in-hospital and ambulatory settings, the range of perspectives for local HTA is broadening to become one of local institutional assessment combined with broader system assessment (depending on the budget flows, and range of accountabilities across inpatient and ambulatory settings).
Technology assessment is based on a synthesis of scientific evidence, but HTA in itself is a value-based process (6;7). In general, value is assessed by estimates of clinical benefit to local patients, together with other aspects of value, such as institutional efficiencies gained, safety improvements, or better alignment with values of the institution or its patients. In the era of declining hospital budgets, value may also be defined, in part, by cost savings.
In the real world of local decisions, research evidence is one of many elements in the process. Decisions also depend on local politics, and evidence from clinical trials competes with other factors, including institutional constraints, stakeholder pressures, values and preferences, and field experience. Without appropriate HTA processes in place, decisions may be driven more by political motivations than by an objective assessment. This phenomenon is especially true for technologies that require a substantial financial investment, such as MRI machines, or for innovative technologies that bring the hospital high visibility and positive media attention within the community. Innovative technologies are often presumed to have “proof” of benefit beyond existing alternatives by the proponents of the technology. However, a rigorous process of assessment of the evidence of incremental benefit, risk, cost, and institutional impacts is necessary to verify to what degree the new technology would be expected to deliver improved outcomes over existing alternatives, and to whether the net costs to implement and manage the technology are commensurate with the net benefits.
Many decisions are still made through committees or advocacy groups at the local level, often without explicit accountability beyond the local proponents of the new technology. In the absence of an appropriate mechanism for HTA, decisions may be based mainly on information provided by those with vested interests, such as the manufacturer, or researchers and clinicians with professional interests in the technology. In this context, it is easy to confuse technology acquisition committees with technology assessment committees.
Contextualization
Evidence synthesis and economic analysis are essential parts of local HTA, but they may not be sufficient for local decision-makers’ needs. With regard to local policy decisions, decision makers also need to evaluate the organizational, ethical, and legal aspects, as well as consider the patient safety of the health technology in question (5). Hospitals can prepare their own HTAs or contextualize HTAs produced by other organizations. Contextualization is often the critical missing ingredient in HTAs from agencies distant from an institutional or a regional setting.
In Newfoundland and Labrador, many projects at the Contextualized Health Research Synthesis Program (CHRSP) are based on existing evidence syntheses. Part of the assessment process involves the contextualization of the evidence to reflect current practices and the local context. Factors examined include patient-client populations, design or site-of-service, human resources, organizations and systems, and economic and political considerations (8).
In Alberta, HTAs prepared by CADTH at the request of the provincial regional health authority, Alberta Health Services, are put into the local context for decision making by the Health Technology Assessment & Innovation (HTAI) unit (9).
At the MEDICI Centre in London, Ontario, contextual factors are incorporated into the HTA process through a deliberative process of defining the “SLEEPERS” (i.e., the Social, Legal, Ethical, Environmental, Political factors; Entrepreneurial/innovation value, further Research needs, and Stickiness/sustainability factors) and their perceived importance to the decision at hand. The “SLEEPERS” framework ensures the local decision makers consider the importance of contextual issues on the decision, which are often not addressed by the traditional evidence base, but which are potentially of great import when assessing the range of impacts and considerations inherent in deciding between different technology investment or disinvestment decisions.
Stakeholder Engagement in Local HTA
It is crucial that local HTA is methodologically sound and based on a systematic, transparent and participatory process. To achieve this, numerous Canadian local HTA programs engage stakeholders throughout the HTA process. Integrating local stakeholders and end-users ensures that the recommendations reflect the multidisciplinary perspectives and contextual considerations of those who will be using the technology (5). Moreover, their expertise strengthens and validates the HTA, and their involvement improves the likelihood that they will assist with knowledge transfer, uptake and implementation of the recommendations. For instance, the CHRSP works closely with the provincial ministry of health and the four regional health authorities. Together, they prioritize the assessments and come to a consensus on the research questions and outline the relevant policies and issues in their region. An external clinical expert participates in the assessment from refining the research questions to providing peer review of the draft report. Key stakeholders, including the Chief Executive Officers of the four regional health authorities and the deputy minister of health, also are involved throughout each assessment, along with the clinical expert and a health economist.
Local HTA recommendations can be judged by a “jury” of representative, credible, and respected stakeholders in the institution to insure an unbiased and transparent deliberative process. At the McGill University Health Centre and the Centre hospitalier universitaire de Québec-Université Laval (CHUQ), HTA stakeholders include nurses, allied health professionals, patients, administrators, and clinicians chosen by their peers. Additional representatives are appointed from the clinical disciplines most affected by that particular HTA. The stakeholders provide expertise upfront, and through their continued involvement, they may ultimately influence acceptance of the recommendations and translation into practice. Involvement of patients and community representatives within the HTA process remains an innovative approach, with the Quebec hospital-based HTA units demonstrating leadership in this area. Most local HTA units in other areas of Canada have not yet formally integrated patients and community stakeholders into the process of HTA.
Knowledge Translation and Impact of Recommendations
At the Alberta Health Services, the HTAI unit combines HTA with knowledge translation, both of which are essential in closing the “know-do gap.” Knowledge translation involves the translation of research to knowledge that is relevant to and understood by decision makers (9). As well as contextualizing and disseminating HTA information, the Unit promotes organizational awareness of HTA and the ability to use this evidence. In addition, the HTAI unit creates linkages between relevant groups within and external to their organization, identify when HTAs, reassessments and access with evidence field evaluations could be useful, and evaluate innovative and best practices in HTA. Most local HTA units have a similar role in educating clinical staff, administrators, and medical students about HTA.
HTAs that fail to inform health policy decisions are a “waste of time and effort” (10). To determine their influence the impact of their recommendations must be evaluated. An impact evaluation from the McGill Technology Assessment Unit (TAU) covered 63 assessments produced since the Unit was established (11). Impact was measured by the number of recommendations that were accepted, and the dollars spent on new technologies, versus those saved through the adoption of cost saving recommendations. Their impact assessment concluded that each HTA should clearly identify the authority that requested the assessment, the administrators and clinicians responsible for acceptance of the assessment, and those responsible for implementing the recommendations (11). It estimated the TAU has saved the hospital an average of CAD$1.14 million annually.
When the MEDICI Centre assessed the impact of its hospital-based HTA program in London, they found that there was a twofold return on investment, meaning that for every $1 invested in the MEDICI hospital-HTA program, there was an average of more than $2 in value returned to the hospital (12).
A 2014 systematic review assessed the evidence on the impact of and barriers and facilitators to the implementation of local HTA recommendations. The findings suggest that local HTA can impact decision making with respect to technology acquisition (4). Other local HTA units also measure impact through cost savings or by monitoring the outcomes that result from introducing a technology (i.e., Did it achieve what it was supposed to in terms of patient benefits or cost savings for the system?). For example, HTAI in Alberta also is involved in a postpolicy implementation review to assess the consequences of recommendations.
Challenges and Opportunities for Local HTA
One of the main challenges of local HTA is getting timely access to the healthcare professionals, who need to be involved for each assessment. In the early stages of establishing HTA, it is paramount to engage hospital administrators to support and participate in the process. An HTA needs the support of the relevant administrator. Advice that has not been requested is seldom taken. Close proximity to decision makers and clinicians is a distinct advantage of providing HTA within an institution, but at the same time, scientific independence must be maintained to avoid influence bias.
Potential pitfalls of local HTA include implementing technologies recommended by external HTAs as clinically and cost-effective, whereas at the local level, the lack of a training plan to address the learning curve and support introduction of the new technology results in poor outcomes and subsequent abandonment of the technology. On the other hand, the transferability of a local HTA to another organization can be an issue because the local context can differ, but information sharing, such as best practices or methodologies, across institutions can still be of value (3). Standardization of HTA methods at the local, regional and national levels should be promoted to reach this goal.
At the Ottawa Hospital Health Technology Assessment Program, the challenges include: sustainability, acceptance by senior hospital leadership, determining the best hospital-based HTA model to use, the hospital fiscal climate, and sustainable financial support. With cutbacks of key staff, hospital administrators may be reluctant to invest in other initiatives, including HTA.
The role of HTA in enabling disinvestment was another relevant responsibility emphasized by most local HTA agencies. The CHUQ, a regional network of five hospitals, is participating in a project, funded by the Canadian Foundation for Healthcare Improvement, to determine how the funds saved through identifying and disinvesting in obsolete technologies can best be reallocated for introducing new technologies (12).
DISCUSSION
Many jurisdictions face continual hospital budget cuts that force them to look at innovative ways to address these shortfalls, while still managing to adopt healthcare technologies that optimize patient care. HTA at the hospital and regional levels is gaining recognition and importance in many parts of Canada given its potential for greater impact on hospital policies and clinical practice by involving the end-users and other stakeholders in the assessment and decision making. More could be achieved through effective collaboration or networking to combine efforts and experiences across the nation. A more effective approach to share local HTA information and avoid wasteful duplication of effort and to provide this information to decision makers is necessary.
Similar work is under way in Europe with the AdHopHTA initiative funded by the European Union's 7th Framework Programme (13). Even a simple registry of those active in hospital-based HTA would be a start, together with ongoing or completed HTAs, and with accessible decision-relevant summaries to meet the needs of busy hospital and regional decision makers. Because most hospitals across Canada face similar decisions regarding which health innovations to take up versus which to abandon, creating a national network could reduce duplication in efforts to assess global evidence, while simultaneously expediting local progression to the contextualization and decision-making stage.
One of the most common sentiments expressed by producers and users of hospital-based health technology assessments is the constant challenge of providing assessments that meet standards for evidence-based evaluation, while also meeting demanding hospital decision-maker timelines. If efforts to synthesize the global evidence and to create templates for contextualization of economic impacts and other “SLEEPERS” impacts could be distributed across existing local and provincial or national HTA units, then opportunities for effective evidence contextualization and knowledge translation could theoretically be achieved more efficiently.
Furthermore, because hospitals around the world have overlapping agendas which have recently been buoyed by the recognized need to be part of the global progression toward universally effective and efficient health care (13), a network that moves beyond national borders to international collaboration could provide a compelling future of collaboration that improves the speed of identification and assessment of technologies with promise.
Furthermore, this network would contribute to a collective knowledge sharing about the impact of local HTA on real-world outcomes. Given the interest expressed at this symposium, and the rising global experience in local HTA units existing to complement national HTA agencies, further opportunities to develop a formal global network of hospital/regional-based health technology assessment units to improve efficiency and impact are now being actively explored (12).
CONCLUSIONS
In Canada, there is expanding interest in local HTA as a means to complement macro level HTA to improve the efficient and effective delivery of health services in hospitals or regions faced with demands for technology but constrained by a limited budget. Initial experience suggests that hospital-based and regional-based HTA efforts are making a measurable impact on local decisions, with improved opportunity incorporate relevant factors and to facilitate stakeholder and decision-maker buy-in through direct engagement. Some important challenges faced by local HTA producers include the efforts required to transparently engage all relevant stakeholders in the HTA process, and potential lack of training and support with the introduction of a new technology that can lead to the misuse or premature abandonment of the technology if not adequately addressed during assessment and implementation. Opportunities to share information and increase collaborations among local HTA producers, and sharing templates and methods for contextualization will be necessary to building a future where decision priorities are addressed within timelines required for optimal impact on decision making.
CONFLICTS OF INTEREST
The authors have no conflicts of interest to declare.
