Acceptance of new health technologies differs from person to person. A recent United States (U.S.) Institute of Medicine (IOM) report stated that patients preferred technologies that helped them better meet their personal goals, helped them live nearly normal lives where the value of time was concerned, and ensured that out-of-pocket expenses would be covered. Clinicians, meanwhile, attached importance to the level of confidence in the effectiveness of services offered, while health insurers considered effectiveness and efficiency and sought high levels of evidence (1).
There are two stages in the evidence-based decision-making process: assessment and appraisal. Assessment involves searching for evidence of synthesis through a critical appraisal of the quality of existing studies and economic evaluations. Appraisal is the process of determining the quality of value judgments. Goetghebeur et al. suggested mathematical models, such as multi-criteria decision analysis (MCDA), which might be helpful in forming a value estimate (Reference Goetghebeur, Wagner and Khoury2). The group discussed a value matrix using the limitations of current interventions, improvement of efficacy/effectiveness, improvement of safety and tolerability, improvement of patient-reported outcomes, convenience, and adherence with regard to the type of medical service for a healthcare intervention.
Medical necessity has been an important principle in decision making for healthcare reimbursement in the United States, and the definition of medical necessity is important to consumers, policy makers, and stakeholders. However, the definitions applied to the term have varied so widely that efforts have been made to reduce the variability of interpretations. Singer et al. mentioned that judgment of the effectiveness of a new health technology is determined by scientific evidence, demonstrating a causal relationship between the intervention and health outcomes criteria (Reference Singer, Bergthold and Vorhaus3).
The Blue Cross and Blue Shield Association (BCBSA) in the United States has five criteria for determining medical necessity, including conclusive scientific evidence of the health effects of the technology, evidence that the technology's benefits are comparable to any established alternatives, and generalizability regarding the technology as a medical necessity (4).
Since July 2007, an evidence-based decision-making system has been used in Korea for the adoption of new nonpharmacologic health technology in terms of safety and efficacy/effectiveness (Reference Park and Lee5). The Committee for New Health Technology Assessment (CnHTA) has reviewed new nonpharmacologic health technology using a systematic review process, as has been done by the National Evidence-based Healthcare Collaborating Agency (NECA). It was the first formal health technology assessment (HTA) activity in South Korea. However, no criteria for decision making concerning the social adoption of technology exist. We wished to explore the factors that influenced decision making in the assessment of new health technology, specifically in terms of safety and effectiveness, and to analyze which factors had positive influences and which had negative influences. These results will be helpful in developing guidelines and promoting greater objectivity in the decision-making process in Korea.
The CnHTA has reviewed new nonpharmacologic health technology using the HTA reports undertaken by the Center for New Health Technology Assessment at the NECA, which addresses safety and efficacy/effectiveness, factors considered by the Korean Food and Drug Administration (KFDA) in drug approval. The committee was composed of healthcare professionals, policy makers, lawyers, and representatives from nongovernmental organizations. NECA was established in 2008 to provide high-quality, evidence-based information to policy makers. The new health technologies recommended by CnHTA were accepted as submissions to the Expert Committee for decisions regarding coverage in the Health Insurance Review and Assessment Service (HIRA), which considers cost and cost-effectiveness. HIRA is nonprofit organization that conducts medical fees reviews, evaluates the appropriateness of medical benefits, and supports the government in making coverage decisions.
METHODS
We analyzed the decision-making results of CnHTA for fifty-three new nondrug health technologies in Korea from July 2007 to December 2010 (Table 1). Every decision made was based on a systematic review of the literature. Decisions made on therapeutic interventions were included, while diagnostic procedures were excluded from this analysis.
Characteristics and Results of Decisions for 53 Health Technologies
First, two researchers (Lee and Kim) located the sentences in each assessment report that addressed the reasons that recommendations were or were not adopted; these reasons were regarded as factors influencing decision making. Next, we categorized the sentences until there was nothing further to categorize. In the event of disagreement between the two researchers, a third would mediate to reach a consensus. We then classified each study according to these factors, allowing a double count if two or more factors influenced a decision. The factors were analyzed separately according to whether the recommendation direction was positive or negative. We also analyzed the factors according to the recommendation grade. The committee used the evidence levels and recommendation grades of the Scottish Intercollegiate Guidelines Network (SIGN) (6).
RESULTS
The factors that influenced decisions positively are listed in Table 2. From the perspective of effectiveness, the most common factor found to support a recommendation of adoption was comparable effectiveness to existing technology, followed by the existence of no alternative technology. In other words, effectiveness and the existence of an unmet need are the most influential factors in the positive direction.
Positive and Negative Factors Influencing Decision Making
Low level of evidence, lack of an alternative intervention, or a finding that the new technology overcame the limitations of existing technology were also regarded as having a remarkable influence on the adoption of a new technology (Figure 1). The characteristics of the fifty-three health technologies and the decision results are shown in Table 2.
Positive factors influencing decisions according to the recommendation grade.
The factors that influenced decisions negatively were the following: a higher complication rate than existing technology, lower effectiveness than comparable technology, lower levels of evidence, unintelligibility of the mechanism of intervention, inconsistency, absence of long-term outcomes, no comparative data, nonstandardized technology, heterogeneity between control and treatment, excessively diverse indications, and nongeneralizability (Table 2).
Implantation of an intrathecal drug infusion pump was recommended for reducing drug amount due to its lower complication rate. In the case of continuous femoral nerve blockage for patients with total knee replacement or total knee arthroplasty, the committee agreed that this intervention is more effective than alternatives, including intravenous pain killers or epidural patient-controlled analgesia (PCA) (Reference Barrington, Olive and Low7–Reference Ilfeld, Le and Meyer9). Eye brachytherapy was recommended owing to the potential for saving eyes and for limiting existing external radiotherapy, which has high complication rates. Gastric banding for obesity was favored for its lesser invasiveness and lower complication rate. Although the outcomes for weight reduction were not superior to gastric bypass surgery, the committee decided to adopt the technology for patients who preferred less invasiveness. Even though the level of supporting evidence was low, thrombectomy using an aspiration device in an intracranial vessel was adopted because potential benefits were observed and no alternative intervention was available from 3 to 8 hours after the onset of symptoms of ischemic stroke. Femtosecond laser keratectomy for corneal transplantation was adopted as a mechanism similar to existing technology for penetrating keratoplasty, despite little direct evidence regarding its effects.
The factors considered by CnHTA in determining whether new technologies were investigational were as follows: higher complication rate than existing technology, lesser effectiveness than comparable technology, lower levels of evidence, unintelligibility of the mechanism of intervention, inconsistency, absence of long-term outcomes, no comparative data, nonstandardized technology, heterogeneity between control and treatment, excessively diverse indications, and nongeneralizability.
Posterior lumbar dynamic stabilization using the Device for Intervertebral Assisted Motion (DIAM), Wallis system, X-stop, and interspinous U/Coflex was determined to be an investigational technology owing to the lack of explainable mechanisms of intervention, inconsistent results, and low levels of evidence. Autologous noncultured epidermal cellular transplantation was also regarded as an investigational technology because of inconsistency in the results and the low quality of evidence. Regarding the therapeutic use of autologous bone marrow cells in patients with peripheral arterial disease, there were limitations in generalizability given that studies had shown heterogeneity between control and treatment (Reference Akar, Durdu and Arat10–Reference Tateishi-Yuyama, Matsubara and Murohara12). Descemet's stripping automated endothelial keratoplasty had a higher complication rate than the alternative of penetrating keratoplasty, was and this factor counted negatively; however, the intervention was recommended for adoption owing to its transient nature and controllability (Reference Bahar, Kaiserman and Levinger13–Reference Hjortdal and Ehlers15).
DISCUSSION
Scientifically, a new technology may be adopted if evidence shows that its benefits outweigh its harms, especially when the technology has effectiveness equal to or exceeding that of an existing technology. CnHTA did not consider cost or cost-effectiveness in the decision-making process, but rather assessed whether society should adopt a new technology based on safety and efficacy/effectiveness. These principles are similar to those involved in medical necessity, which is the general principle behind reimbursement policy in the United States.
Even though the principle of medical necessity has long been used and been widely adopted by policy makers in the United States, there has been no explicit definition of the term at the level of federal government. As a result of several conflicts surrounding the interpretation of the principle, several efforts were made to decrease variation in the interpretation of “medical necessity.” The results of the report by Singer et al. exemplified such efforts (Reference Singer, Bergthold and Vorhaus3), and the concepts they proposed were adopted by the state of Hawaii in its definition of “medical necessity” (Reference Ilfeld, Le and Meyer9). This definition emphasizes that scientific evidence should demonstrate a causal relationship between the intervention and health outcomes for new health technologies. Singer et al. argue that low-quality clinical studies are not adequate for explaining causal relationships. If new interventions are not feasible for high-quality clinical trials owing to their use in rare or new diseases, decisions can be made on the basis of professional standards of care or expert opinions.
The Technology Evaluation Center of BCBSA uses five evaluation criteria to determine medical necessity, including quality of the body of studies, consistency of the results, benefits as strong as any established alternatives, and generalizability (Reference Park and Lee5).
The factors influencing decision making by CnHTA in whether to adopt a new health technology are similar to the factors that define medical necessity in the United States. In addition to comparable effectiveness and safety of alternative interventions, consistency, generalizability, and quality of the evidence, we were able to identify additional factors, including saving critical organs, the existence of no alternative interventions, deficiencies in existing technology that represent an unmet need, decreased invasiveness, and opportunities for diverse patient choices.
The safety-related factors considered by CnHTA in determining whether a new technology may be adopted were severity of complications from an intervention and whether it was transient and controllable. In terms of efficacy/effectiveness, the factors taken into account included not only whether the benefits were the same as or greater than those of existing comparable technologies, the consistency of the results, and generalizability, but also lower invasiveness, deficiencies of existing technology, and room for patients’ choice; these additional factors may outweigh the uncertainty of effectiveness when the supporting evidence is of a low quality. Development of an explicit set of guidelines will be required to incorporate such factors as unmet need, less invasiveness, and patients’ preferences into decision-making practices; such guidelines are especially needed in cases of existing uncertainty due to low levels of evidence.
CONCLUSIONS
Four years ago, an evidence-based decision-making system was introduced in Korea to the process of deciding whether to adopt new nonpharmacologic health technologies on the basis of safety and efficacy/effectiveness. However, no distinct criteria exist for decision making with regard to the social adoption of technology. This qualitative analysis of past decision-making results provided us with insight regarding which values decision makers on the Korean CnHTA considered in terms of safety and effectiveness. Thus far, no explicit guidelines have existed to assist decision making on the kinds of new technologies to be adopted for daily practice in Korea. To make the process more transparent and the decisions more consistent, it is necessary to document and clearly define guidelines for the entire process. These findings will help in the development such guidelines and enhance the objectivity of the decision-making process in Korea.
CONTACT INFORMATION
Sang Moo Lee, Gaeun Kim, Jeonghoon Ahn, National Evidence-based Healthcare Collaboration Agency, Seoul, Republic of Korea
Hae Sun Suh, National Evidence-based Healthcare Collaboration Agency, College of Pharmacy, Yonsei Institute of Pharmaceutical Sciences, Yonsei University, Seoul, Republic of Korea
Dae Seog Heo (heo1013@neca.re.kr), National Evidence-based Healthcare Collaboration Agency, College of Medicine, Seoul National University, Seoul, Republic of Korea
CONFLICTS OF INTEREST
All authors report they have no potential conflicts of interest.
