There is a growing movement internationally to re-examine the safety, effectiveness, and cost-effectiveness of existing health technologies (e.g., devices, drugs, diagnostics, and/or procedures). The rationale for this is simple: technologies previously adopted may eventually become obsolete, may have never been formally assessed through traditional research that includes randomized clinical trials, or may be used beyond their originally intended scope (i.e., overused or misused). All such instances are categorized as sub-optimal technology use and can compromise quality and efficiency within the healthcare system.
While health technology assessment (HTA) efforts have primarily focused on informing policy-making decisions concerning the adoption of new technologies (Reference Haas, Hall, Viney and Gallego1;Reference Henshall, Schuller and Mardhani-Bayne2), the utility of such programs for evaluating existing technologies has been recognized for over 3 decades (3–5). With increasing attention to the prevalence of sub-optimal technology use, through initiatives such as the international Choosing Wisely campaign (6), HTA programs globally are refocusing efforts toward holistic approaches to ongoing evidence-based management (i.e., evidence drives understanding of optimal use) of technologies throughout their lifecycle (Reference Bryan, Mitton and Donaldson7–Reference Noseworthy and Clement9).
Health technology reassessment (HTR) is an evidence-based policy approach to assess the clinical, economic, social, and ethical impacts of an existing technology to inform its optimal use (Reference Noseworthy and Clement9). To ensure optimal use, HTR can result in several outcomes, including no change, decreased use, increased use, or complete exit of the technology from the system (Reference Soril, MacKean, Noseworthy, Leggett and Clement10). Conceptually, HTR is directly informed by the historical approaches to multiple technology assessment (Reference Chambers, Salem, D'Cruz, Subedi, Kamal-Bahl and Neumann11), as well as the assumptions and underlying theories in the collective international literature regarding optimal use (Reference Leggett, Noseworthy, Zarrabi, Lorenzetti, Sutherland and Clement12;Reference Leggett, Mackean, Noseworthy, Sutherland and Clement13). As such, the HTR process purposefully merges components of established methods and analytical tools, such as those from HTA, decision analysis, comparative effectiveness and appropriateness research, and implementation science, rendering it broadly applicable within research, practice, and policy domains (Reference Soril, MacKean, Noseworthy, Leggett and Clement10;Reference Leggett, Noseworthy, Zarrabi, Lorenzetti, Sutherland and Clement12;Reference MacKean, Noseworthy and Elshaug14).
However, HTR is one of many concepts within the health technology management literature. A recent scoping review identified forty-three unique terms applied internationally to identify and eliminate sub-optimal technology use (Reference Niven, Mrklas and Holodinsky15). In addition to “reassessment,” such terms included disinvestment, de-listing, or de-adoption, rational priority setting, and HTA. Each of these portend differing methodologies, nomenclature, and motivations (e.g., improving effectiveness, efficiency, or both) (Reference Niven, Mrklas and Holodinsky15). Despite this growing scholarship, there is relatively poor understanding of how these concepts and processes might relate to or be represented within HTR (Reference Gnjidic and Elshaug16). Without such clarity, there is likely to be confusion among healthcare system stakeholders wishing to implement an HTR program in their respective healthcare settings.
This study aims to describe the concepts and common themes underlying the various technology management processes and to reconcile their relationships directly within the HTR process. The intent of this discussion is to provide a clearer vision of the “bigger picture” with regard to managing existing technologies in the system, in that, they will understand what needs to be achieved, why it needs to be achieved, and how it can be achieved.
METHODS
Review of Evidence
Several high-quality reviews of studies describing experiences and approaches to HTR and related initiatives have already been published (Reference Leggett, Noseworthy, Zarrabi, Lorenzetti, Sutherland and Clement12;Reference Niven, Mrklas and Holodinsky15;Reference Mayer and Nachtnebel17;Reference Seo, Park and Lee18). To leverage this established work, an electronic database search for relevant systematic reviews was performed. Identified citations were examined in duplicate using the inclusion and exclusion criteria presented in online Appendix 1. The reference lists of included systematic reviews were then reviewed to identify the original articles for final analysis. The primary criterion considered for inclusion of original articles was the description of experiences and approaches (e.g., frameworks or models) to evaluate optimal use of an existing technology in practice.
Evidence Synthesis and Visual Mapping
Synthesis of evidence followed a deductive approach to identify and document information such as the problem(s) being addressed, the needs of the policy context, the desired results, the influential factors, and the strategies used. A catalogue of all extracted information and original sources were then captured in a draft schematic to ensure transparency and comprehensiveness of information. The draft schematic served as a visual representation of common conceptual groupings as well as the sequential nature of decision-making concepts within an HTR process.
Expert Stakeholder Consultation and Consensus
Expert stakeholders were consulted to review the sequence, content, and overall coherence of the evidence synthesized in the draft schematic; such consultation is critical as it calls upon the experiential knowledge of the expert stakeholders. Expert stakeholders were also directly embedded within the research team, which was comprised of clinicians, researchers (academic and health system settings), and a health system administrator. In-person and online deliberations with the research team were held over a 1-month period to achieve consensus on the final schematic.
RESULTS
The final schematic resulting from the expert stakeholder consensus process is depicted in Figure 1. The schematic is intended to serve as a practical guide; the information is organized to highlight the breadth of possible issues that relevant users, such as a health system decision maker, might face when considering implementation of an HTR program, as well as the possible scenarios or options that might best respond to their needs and policy context.
A structured approach to frame optimal technology use. Six questions (in bold) are posed to guide a user through the health technology reassessment (HTR) process. Potential responses to the first five questions are listed in the text boxes, with selected colors and orientations, directly below each question. Text boxes of identical gray coloring represent concepts that are related across the domains. An overlap of textboxes represents concepts that are not well distinguished from one another or commonly interchanged. The text boxes in white represent concepts that can be applied for any scenario. Lastly, text boxes in black depict concepts and sequence recommended exclusively when the value of an existing technology is unknown. The proposed foundational actors are outlined along the bottom of the diagram to depict the importance of their engaged throughout the HTR process.
To reconcile the relationships within and among common conceptual groupings, the stakeholders framed the schematic around six key questions that health system decision makers should reflect upon before initiating an HTR, including: (i) What is the value of the existing technology in use?; (ii) What is the current utilization gap?; (iii) What are the available tools and resources?; (iv) What are the levers for change?; (v) What is the desired outcome?; and (vi) Who are the foundational actors? A description of each of the questions is provided below.
1. What Is the Value of the Existing Technology?
Value of a technology is broadly considered as the impact and outcomes it achieves in relation to its cost (Reference Porter19). Thus, value can be measured in terms of clinical endpoints, such as patient safety, quality of life or satisfaction, and system or process-related outcomes of care, such as reduced hospitalization or length of stay. The spectrum of technology value commonly described in the literature includes: high value (known to be highly beneficial, acceptable costs); low value (known to have minimal benefit, cost irrespective); no demonstrable value (known to be ineffective); or harmful (Reference Elshaug, Rosenthal and Lavis20). Many processes documented in the literature have focused primarily on identifying and reducing the use of technologies of low value (often referred to as low value care) (Reference Elshaug, Moss, Littlejohns, Karnon, Merlin and Hiller8;Reference Niven, Mrklas and Holodinsky15;Reference Bhatia, Levinson and Shortt21–Reference Elshaug, Hiller, Tunis and Moss23). However, of equal importance is improving the use of high-value care (Reference Elshaug, Rosenthal and Lavis20). This broader view of value is an important distinction that must be considered when implementing an HTR program geared toward optimal technology use.
Further in some cases, the value of a given technology is known based on standard evidentiary criteria. However, we do not have robust measures of the value of all existing technologies (Reference Elshaug, Rosenthal and Lavis20). Many technologies were adopted before undergoing rigorous evaluation of clinical- and cost-effectiveness; thus due to this lack of supporting literature, the value of many technologies may be uncertain (Reference Scotland and Bryan24). Such uncertainty is noteworthy, as it suggests that additional research may be required before any changes in technology use can be recommended.
2. What Is the Current Utilization Gap?
The utilization gap is the difference between the optimal and the observed technology use. This could include overuse of a technology that is ineffective or harmful, overuse or misuse of an effective technology above or outside of its intended scope of use (i.e., wrong patient, wrong indication, wrong time), or underuse of a technology of proven clinical- and/or cost-effectiveness (Reference Bryan, Mitton and Donaldson7;Reference Elshaug, Rosenthal and Lavis20;Reference Morgan, Brownlee and Leppin22). Overuse and/or misuse are the most commonly described utilization gaps in the literature (Reference Saini, Brownlee, Elshaug, Glasziou and Heath25); this is likely consequent to the rise in efforts addressing low-value care (6;Reference Niven, Mrklas and Holodinsky15;Reference Elshaug, Rosenthal and Lavis20;26;Reference Elshaug, Watt, Mundy and Willis27). Identification of these utilization gaps can trigger the need for an HTR, particularly when supportive use and/or cost data are readily available. For example, geographic, provider, and/or temporal variations in technology use identified through local clinical and/or administrative data sources can provide supportive evidence to demonstrate the advent and extent of the utilization gap (Reference Elshaug, Moss, Littlejohns, Karnon, Merlin and Hiller8;Reference Paprica, Culyer, Elshaug, Peffer and Sandoval28). For many existing technologies, whose use in the healthcare system is prolonged and often reflexive, their optimal use may have never been established; hence, the utilization gap may be unknown (Reference Scotland and Bryan24).
3. What Are the Available Tools and Resources?
Assessment of the tools and resources available within a local decision-making context is necessary to establishing feasibility and success of an HTR program. At minimum, access to longitudinal use and/or cost data, skilled health services and policy research experts and staff to conduct the HTR, dedicated funding for HTR initiatives, and strong leadership from both clinical and funder/government stakeholders are recommended. The necessity of the aforementioned is drawn from the documented challenges in jurisdictions across Canada, in Australia, and in the United Kingdom (Reference Elshaug, Hiller, Tunis and Moss23;Reference Paprica, Culyer, Elshaug, Peffer and Sandoval28–Reference Rooshenas, Owen-Smith, Hollingworth, Badrinath, Beynon and Donovan30). In the event that a technology's value and/or utilization gap is unknown, these tools and resources may also be used to address any knowledge gaps by generating new information or evidence. For example, prospective monitoring of a technology's use to inform its current value can be achieved through the conduct of comparative effectiveness, appropriateness, and/or implementation studies, as well as through the collection of real-world evidence (e.g., postmarketing surveillance).
4. What Are the Levers for Change?
Seven potential levers to close the utilization gap are proposed. These levers are drawn from the broader knowledge translation literature and include: clinical and/or decision-maker champions, clinical guidelines, educational initiatives, clinician reminders, audit and feedback mechanisms, incentives/disincentives, and meso- or macro-level policy change (Reference Niven, Mrklas and Holodinsky15;Reference Elshaug, Hiller, Tunis and Moss23;Reference Daniels, Williams, Robinson and Spence29). Previous work evaluating the effectiveness of the aforementioned levers suggests that all modalities are effective to varying degrees, albeit there is little understanding of their relative effectiveness, mechanisms of action, or rationale for selection (Reference Grimshaw, Thomas and MacLennan31). Some argue that variations in effectiveness are not necessarily attributable to the tools and levers themselves, but rather to their alignment with the barriers and/or facilitators to change that are confronted in the decision-making context (Reference Michie, Johnston, Francis, Hardeman and Eccles32). Thus, thoughtful selection, design, and application of an appropriate lever(s) must be driven not only by the identified utilization gap, but also by the available tools and resources as well as culture within the given decision-making context (Reference Noseworthy and Clement9).
5. What Is the Desired Outcome?
With overuse of low value technologies, the most commonly sought outcome described in the literature is that of disinvestment, defined as “the processes of partially or completely withdrawing healthcare resources from existing healthcare investments that are considered to deliver little or no health gain for their cost and as such, are no longer considered as efficient use of current healthcare resource” (Reference Leggett, Noseworthy, Zarrabi, Lorenzetti, Sutherland and Clement12;Reference Leggett, Mackean, Noseworthy, Sutherland and Clement13;Reference Niven, Mrklas and Holodinsky15;Reference Polisena, Clifford, Elshaug, Mitton, Russell and Skidmore33). Recently, other emergent outcomes have included de-adoption (Reference Niven, Mrklas and Holodinsky15) and de-implementation (Reference Prasad and Ioannidis34), which are primarily concerned with changes at the clinical practice level to decrease or remove low-value care.
Interestingly, these terms have been documented as eliciting fears of rationing and/or budgetary cuts (Reference Daniels, Williams, Robinson and Spence29;Reference Rooshenas, Owen-Smith, Hollingworth, Badrinath, Beynon and Donovan30), as well as perceived encroachment on clinician autonomy and patient choice (Reference Elshaug, Hiller, Tunis and Moss23). Such outcomes also do not relate to the underuse of high-value care. Thus, we argue that the outcomes of the HTR process be framed around the desired change in optimal technology use. Hence, increased use, unchanged use, decreased use, or complete exit of the existing technology (from the healthcare system) are all potential outcomes of an HTR. In addition, the HTR process may identify that additional research and new evidence are required steps before any recommendation regarding technology use can be made.
6. Who Are the Foundational Actors?
Underscoring the entire HTR process is the concerted engagement of foundational actors, including healthcare providers (i.e., physicians, nurses, allied healthcare professionals), researchers, health system administrators and decision makers, and policy makers, and patient and public representatives. Both top-down initiatives, for example those led by health system administrators or decision makers (Reference Elshaug, Hiller, Tunis and Moss23;Reference Daniels, Williams, Robinson and Spence29), and bottom-up approaches, such as the physician-led Choosing Wisely campaign (6), have been described in the technology management literature. However, limited success of such initiatives has been attributed to insufficient engagement and/or consultation across stakeholders groups (Reference Daniels, Williams, Robinson and Spence29;Reference Rooshenas, Owen-Smith, Hollingworth, Badrinath, Beynon and Donovan30;Reference Sevick, Soril, MacKean, Noseworthy and Clement35). For example, top-down initiatives without healthcare provider support can inundate and lead to disengagement among frontline stakeholders tasked with implementation (Reference Sevick, Soril, MacKean, Noseworthy and Clement35).
It is recommended, therefore, that HTR processes undertake intentional and broad engagement. This legitimatizes the process and may increase the likelihood of success (Reference Elshaug, Rosenthal and Lavis20). Furthermore, meaningful engagement may be encouraged with continuous stakeholder participation throughout the HTR process, transparency in methods and processes, ongoing knowledge exchange and use, and authenticity in these interactions (Reference MacKean, Noseworthy and Elshaug14).
DISCUSSION
With the increasing international focus toward the lifecycle management of technologies comes a need for clarity around mechanisms to support such a process. In this study, we attempt to address this knowledge gap by identifying and documenting the relationships between the various concepts and processes in the technology management literature using a practical, visual guide. The growing body of literature in this field, concomitant to the abundance of documented nomenclature, not only call for such a schematic, but have also enabled its development.
We sought to describe and connect the various technology management concepts within the HTR process because, in contrast to other suggested and documented processes for reducing use of low-value care and optimizing technology use (Reference Niven, Mrklas and Holodinsky15), it enables a potentially more holistic approach to promoting optimal technology use. This has potential to be more accepted by stakeholders, particularly front-line healthcare providers who are responsible for the consumption of these technologies, and ultimately ensuring their optimal use.
The six discussion questions in the schematic enable a user to not only identify the steps to implementing an HTR initiative, but also understand the potential practical challenges. Based on previous documented experiences (Reference Elshaug, Hiller, Tunis and Moss23;Reference Daniels, Williams, Robinson and Spence29;Reference Rooshenas, Owen-Smith, Hollingworth, Badrinath, Beynon and Donovan30;Reference Sevick, Soril, MacKean, Noseworthy and Clement35), HTR initiatives are likely to face uncertainty with regards to the value and use of existing technologies, particularly due to limited or no available evidence to inform the clinical or cost-effectiveness of those long entrenched in the system (Reference MacKean, Noseworthy and Elshaug14). Uncertainty in the appropriate threshold for the quantity and quality of evidence (i.e., randomized controlled trials or more pragmatic or nonrandomized studies) to inform HTR outcomes, may also prove challenging (Reference Soril, MacKean, Noseworthy, Leggett and Clement10). While it is recommended for users to generate knowledge, through de novo studies or collection of real-world evidence, to address these limitations, such efforts require substantial resources and methodological expertise. Therefore, established HTA programs, with already high demands for assessing new technologies, may face practical issues related to insufficient resources, appropriate training and skills, and priority fatigue with the adoption of HTR initiatives (Reference Elshaug, Hiller, Tunis and Moss23;Reference Sevick, Soril, MacKean, Noseworthy and Clement35).
As the field of technology management and optimal use advances, thoughtful framing of the HTR process is critical. Ultimately, the clarity provided in the present discussion will help establish the identity of HTR among relevant healthcare system decision makers and, hopefully, increase understanding, acceptability, and uptake of HTR practices.
CONFLICTS OF INTEREST
L.J.J.S. is supported by an Alberta Innovates – Health Solutions (AIHS) Graduate Studentship Award. There are no other funding details to disclose. The other authors have nothing to disclose.
SUPPLEMENTARY MATERIAL
Supplementary Material 1: https://doi.org/10.1017/S0266462318000120
