Modern health care is cost and technology intensive and expects value for money, creating demand for evidence-based practice and health technology assessment (HTA). In high-income countries, HTA is undertaken by trained professionals in specialized HTA institutions. However, in developing countries or resource-limited settings, HTA is often lacking, despite perceived need. Therefore, healthcare decisions are often subjective, sometimes arbitrary. Improved understanding of the practice of evidence-based medicine (EBM) in many developing countries, and organizations such as the Cochrane Collaboration, now facilitate evidence-informed decisions. Because creating HTA institutions is time and cost-intensive for most developing countries, alternate tools are needed. This study describes the development of “KNOW ESSENTIALS,” a tool for evaluating health technologies to facilitate informed decision making by stakeholders. The tool includes the components of conventional HTA and additional elements relevant to developing countries. It does not require high level of expertise or expense. It factors in locally relevant issues and can be used to generate evidence-based decisions rapidly. A practical example of using the tool along with potential strengths and limitations is described.
BACKGROUND
HTA is the scientific process of evaluating health technologies (pharmaceuticals, vaccines, surgical procedures, medical equipment and devices, etc.) to facilitate informed decisions by stakeholders: healthcare providers, payers, consumers, regulators, policy makers, etc. (1). In high-income countries, HTA is a formal discipline undertaken by trained professionals to guide stakeholders, including governments, to make decisions based on sound scientific principles.
Most resource-poor settings lack formal HTA mechanisms; in such settings, healthcare decisions are often based on less rigorous methods. Even in the few developing countries where HTA is being used, a recent survey (Reference Pichon-Riviere, Augustovski and Rubinstein34) evaluating the use of key HTA principles (Reference Drummond, Schwartz and Jönsson8) reported that, although the principles were considered relevant by HTA producers and users, the level of application was uniformly low. One of the obvious reasons identified is the chink between methodological rigor and timeliness of reports (Reference Hailey15). Perhaps an even more important reason is the existing, entrenched but less robust, system of decision making in such settings, and the challenge of bringing change; hence, existing decision-making drivers and processes in developing countries are briefly examined.
Inadequate Policy Processes
Although resource allocation for health as well as demand for new health technologies have increased in many low income countries, robust decision-making mechanisms have not developed in parallel. Decisions are often driven by past experience, thrust of donor agencies, and lobbying pressure (Reference Chalkidou, Levine and Dillon5). For example, a report from Peru noted that decisions on the HPV vaccine at local level were mainly driven by local political pressure, rather than scientific evidence (Reference Piñeros, Wiesner, Cortés and Trujillo35). In Rwanda, the government had a disproportionately large amount of funds for HIV/AIDS compared with malaria and other greater perceived needs, because donor grants were specifically allocated for HIV/AIDS (36). Likewise in India, sustained single-point focus on poliomyelitis eradication using supplementary immunization (owing to World Health Organization [WHO] and global pressure), has critically weakened the routine immunization program with other childhood vaccines (Reference Mittal and Mathew29). Another example is a publication exhorting researchers to support introduction of new vaccines (specifically rotavirus) in developing countries (Reference Wecker, Phillips and Hayes46). The benefits or cost-effectiveness of these vaccines in developing countries have not been considered.
Commercial pressure is also a major force skewing the decision-making process in developing countries; this is especially relevant for newer vaccines, expensive drugs, devices, and equipment (Reference Mathew24). For example, current immunization recommendations of the Indian Academy of Pediatrics (IAP) were produced by expert consensus at a meeting sponsored by a multi-national company. Not surprisingly, the stated objective of the recommendations was to produce guidance for three products recently marketed by the company in India (19).
Although poorer countries should be more careful in spending money, the reverse often happens. Healthcare systems sometimes successfully negotiate lower pricing for pharmaceuticals, but public health programs end up paying more than the negotiated prices (Reference Seoane-Vazquez and Rodriguez-Monguio38). Such observations corroborate the argument of Chalkidou et al. (Reference Chalkidou, Levine and Dillon5) that in many developing countries, “health services and technologies purchased with public funds are selected through idiosyncratic processes that often have little to do with systematic analysis of their potential health benefit or value for money.”
In many developing countries, “expert-based” guidance is used as a surrogate for robust methods, perhaps in good faith (Reference Kahveci and Meads22). A group of “experts” prepares a “consensus” statement on a given health technology. The basis for selecting the experts and the processes used to reach consensus are seldom described. Even more alarming is that some consensus statements selectively quote literature supporting a particular viewpoint, and are garbed as evidence-based guidelines (19).
For the individual patient, the choice of particular health technologies is often based on physicians’ advice. In developing countries, physicians often base their “advice” on non-scientific considerations, particularly the influence of the pharmaceutical industry (Reference Vancelik, Beyhun, Acemoglu and Calikoglu42). Material provided by pharmaceutical manufacturers is reported as the most frequently used resource by many physicians, with prescribing decisions influenced by training activities sponsored by pharmaceutical companies and visits by sales representatives (Reference Vancelik, Beyhun, Acemoglu and Calikoglu42).
Local Context Forgotten
In several developing countries, people access both traditional and Western medical services at the same time; many countries are even trying to develop integrated systems for healthcare delivery. Decision-making algorithms for alternative medicine (singly or combined with “Western” medicine) are largely undeveloped, posing a unique challenge to scientific decision making. Many developing countries also have multiple tiers (primary, secondary, tertiary) and sectors (public and private) of healthcare delivery; with little uniformity or standardization among institutions in the same sector or tier.
Clearly, healthcare decisions by all stakeholders in developing countries are often highly subjective. There is urgent need to bring in objectivity, reproducibility, and transparency. The emergence and spread of evidence-based medicine (EBM) was expected to address this need, through building local capacity for using systematic reviews, and influencing policy makers and other stakeholders to make evidence-based decisions. However, EBM has two major limitations. First, it focuses on generating evidence of efficacy and sometimes safety of health technologies, mostly interventions such as treatment modalities, preventive strategies, health education, etc. Second, it fails to factor in local needs and contexts for transferability and applicability of evidence generated in a different healthcare setting (Reference Wang, Moss and Hiller43). Therefore, in developing countries, evidence-based practice often merely refers to accessing current evidence and applying it without considering the context. For these reasons, reliance on EBM alone is insufficient for informed decision making, relevant to the local context. Of course, exceptions exist. For example, Thailand decided not to include the HPV vaccine in its universal coverage program (despite pressure from multinational drug industry, international agencies, and professional groups citing global experience), because vaccination at current prices was found to be cost-ineffective in the local context (Reference Teerawattananon, Tantivess, Yothasamut, Kingkaew and Chaisiri40). Very few healthcare settings have similar capability to arrive at evidence-informed decisions based on local needs, juxtaposed with global evidence.
Lacking HTA Capacity
HTA systems and institutions are urgently needed in developing country settings to facilitate locally appropriate decisions. The EUnetHTA Handbook on Health Technology Assessment Capacity Building (10) clearly shows that the process toward this desirable goal is time consuming and expensive. It is unlikely that resource-poor health systems would immediately invest in the development of locally relevant HTA. On the other hand, access to HTA documents on potentially important health technologies could induce undesirable attempts to apply the results directly to a local setting, without applying local information. Even worse, pharmaceutical manufacturers or other stakeholders could disseminate HTA reports from other settings to market specific health technologies in developing countries.
The long-term goal is creating HTA capacity in developing countries to facilitate evidence-informed decision making. While waiting for this development, alternate models of robust decision making are required. They should include current best evidence on health technologies, incorporate relevant contextual issues, and be objective, reproducible, transparent, and affordable. This study describes a tool for such purpose, “KNOW ESSENTIALS,” and explores its strengths and limitations. The mnemonic helps to remember the criteria required for informed decision making. The tool is based on collation and critical appraisal of current best evidence for decisions. The study also highlights additional work needed to make the tool globally applicable.
METHODS
Literature Search
The following search string in Medline was used: (“Decision Support Systems, Management”[Mesh] OR (“Decision Support Systems, Management/instrumentation”[Mesh] OR “Decision Support Systems, Management/man-power”[Mesh] OR “Decision Support Systems, Management/organization and administration”[Mesh] OR “Decision Support Systems, Management/standards”[Mesh] OR “Decision Support Systems, Management/utilization”[Mesh])). This identified 836 citations, of which 42 were judged relevant.
Development of KNOW ESSENTIALS Tool
The principle of HTA has been summarized as follows: (i) Does it work? (ii) Can it work here? (iii) Is it worth it? (Reference Drummond, Schwartz and Jönsson8). This principle guided detailed evaluation of research evidence on efficacy, contextual issues of applicability and effectiveness, and cost issues. Challenges for healthcare stakeholders in developing countries were identified through discussions with clinicians, health administrators, and policy makers. These additional considerations were incorporated in the tool to evaluate health technologies properly. Thirteen elements were combined into the mnemonic KNOW ESSENTIALS.
The first three elements, or “KNOW,” focus on healthcare problems, because a major potential limitation with HTA is considering “health technology” rather than “health problem” as the starting point for decision making. In developing countries, the health problem often presents unique contextual issues that need to be addressed before trying to evaluate solutions (health technologies). The other ten considerations, or “ESSENTIALS,” deal with the health technology and organizational issues relevant to the local healthcare system. The first four of these are the most important.
RESULTS
Components of KNOW ESSENTIALS Tool
1. Knowledge of Need (KN). Stakeholders evaluating a health technology should know whether there is justifiable need for this technology. It is intuitively assumed that all marketed health technologies are necessary, but many developing countries are faced with technologies that either do not address local primary needs (e.g., serological tests for tuberculosis in high prevalence countries), or are marketed for situations or indications for which they were not originally designed (e.g., computed tomography, magnetic resonance imaging, or positron emission tomography scanning without clear indications). This is particularly true in growing economies, where demand for “state-of-the-art” technologies routinely used in developed countries is also increasing. In developing countries, “need” is usually “determined” by limited local data or extrapolation from entirely different settings. For example, in many developing countries, the burden of invasive pneumococcal disease is not known and reliable estimates are missing, yet the pneumococcal conjugate vaccine is promoted (Reference Mathew24) based on the assumption that it is needed (2).
Knowledge of need (KN) is difficult to quantify. Consideration of the healthcare problem in terms of epidemiology (burden of disease to individual and community), relative importance vis-à-vis other clinical conditions, and priority for management within the healthcare system with limited resources provide a good basis. For example, based on knowledge of need, a public sector institution in a developing country may find it justifiable to concentrate limited resources on care for babies with low birth weight rather than on screening for genetic disorders. This choice may be quite different for another healthcare institution in the same or another country.
2. Outcome of Interest (O). There can be multiple desired outcomes in the management of a single healthcare problem. Stakeholders are interested in hard outcomes such as mortality, morbidity, or quality of life. Research data instead are often limited to soft or surrogate outcomes that may not reflect the outcomes of interest. Decision makers need to identify hard outcomes of interest and surrogate outcomes that best represent these to evaluate research evidence properly.
Most health technologies deliver some degree of benefit on one or more outcomes, but cannot always guarantee the outcomes that stakeholders are interested in. Ideally a technology is selected only when the desired outcomes are achievable by using it.
The outcome of interest can also vary with the focus of the stakeholder. Different stakeholders may have distinct perspectives and priorities for outcomes in the same health condition. Individual patients and their physicians are interested in curing illnesses safely, so the desired outcomes revolve around efficacy, safety, and affordability. Policy makers could be additionally interested in effectiveness, cost-benefit, and sustainability; healthcare providers also in logistics and feasibility. Therefore, clearly defining outcomes of interest for specific stakeholders is an essential step toward health technology assessment. This distinction is not always intrinsic in current HTA methodology.
3. Who is the target? (W). Evidence on most technologies is generated on selected cohorts of patients. Sometimes the filtration process and meticulous research procedures—careful monitoring, tight follow-up, frequent contact with professionals—make the characteristics of research participants and their experience of the disease quite different from real-world people with the same diagnosis. Evidence of benefit of a given health technology in the former does not necessarily guarantee same results in the latter. Therefore, it is important to analyze how closely research participants represent real-life patients. For example, based on research data, WHO and UNICEF recommend that breastfeeding be avoided to protect uninfected infants of HIV positive mothers, if replacement feeding is acceptable, feasible, affordable, sustainable, and safe (Reference Doherty, Chopra and Jackson6). Local research shows that both methods are associated with comparable mortality rates during infancy (Reference Rollins, Becquet and Bland37). However, careful analysis confirms that this holds true only for families that have support to provide appropriate replacement feeding. Because this may not be available to the large majority of people, application of this research evidence may not result in similar outcomes in real-life situations.
4. Evidence of effectiveness (E). Effectiveness of health technologies is often confused with efficacy; the latter is based on research data and estimates whether a health technology could work at all. Effectiveness is based on real-world experience and gives a better idea of whether the technology would work or has worked. While effectiveness is more important than efficacy, evidence on the latter is more readily available through systematic reviews, randomized trials, etc. Evidence of efficacy ought to be used only if evidence of effectiveness in the local population is not readily available.
Sometimes evidence of effectiveness or even efficacy may not be available for a specific population in a given setting. However, decisions still need to be made; for such cases KNOW ESSENTIALS includes consideration of evidence in similar cohorts also from quite different healthcare settings. For example, it could be justifiable to consider evidence of effectiveness on full term babies with low birth weight from one developing country in treating those in another country; however, data on term newborns may not be applicable for preterm newborns even if the research was done in the same institution. Likewise for primary care settings, research data from similar care settings in other countries would be more applicable than data from tertiary care institutions within the same country. Judgments on which pieces of evidence can be considered in a given situation are made daily in all healthcare systems that apply research evidence. KNOW ESSENTIALS ensures that this judgment is as objective as possible.
5. Safety (S). Safety of health technologies is established through research and user experience from healthcare providers and consumers. Absence of immediate and long-term, serious, and mild adverse events is confirmed by assessment of data from clinical trials, post-marketing research, and surveillance systems, etc.
Application of safety data from research to individual patients requires that study patients are comparable to real-life patients. Safety data on a new drug may be available from otherwise healthy trial participants and may not be applicable to patients with severe disease, co-morbidities, etc.
Although many technologies are associated with adverse events, their frequency or severity may be outweighed by the benefits, necessitating a value-based judgment. When public health interventions (screening programs, vaccination, etc) are considered, the risk of adverse events may be greater than desirable, but still justifiable in the interest of the “common good.” Trade-off between benefits and risks usually has to be worked out for the local setting.
In practice, proving safety beyond doubt can be much more difficult than finding evidence documenting risks of health technologies.
6. Social Issues (S). This term captures complex concepts such as acceptability (personal and societal), ethical aspects, and legal issues around a technology, and the likely response of consumers to these issues within the local setting. This is seldom available from scientific literature, and may require inputs from social scientists, legal experts, and the lay press. For most technologies, these issues may not prove restrictive; however, they can be important in special cases involving new technologies, indications, or user groups.
For example, using condoms decreases the risk of HIV transmission. However, in Uganda where HIV burden is high, condom use among married couples remains low. The reason is that insistence on condoms in a long-standing relationship is socially unacceptable. Even among women who managed to convince their partners to use condoms, the motivating factors were the woman's persistence, refusal to have sex, and suggestion that condoms were used for contraception rather than preventing HIV transmission (Reference Williamson, Liku, McLoughlin, Nyamongo and Nakayima45).
Similarly, potential ethical challenges associated with particular technologies are important to consider, especially issues concerning autonomy, beneficence, nonmaleficence, and justice (18). Identifying and disclosing interests of stakeholders for specific technologies can also be helpful. Rarely also moral challenges are posed. Several ethics databases have been developed to assist with decision making (Reference Droste, Dintsios and Gerber7).
7. Economic issues (E). This refers to three distinct concepts: (i) cost of the health technology (example cost of one dose of antibiotic), (ii) cost of providing the technology (cost of antibiotic dose, consumables, manpower to administer, etc.), and (iii) cost-effectiveness (cost to benefit ratio).
Ideally, information on all three should be available for the healthcare system under consideration. If one of these is not within the paying capacity of the decision-making stakeholder (patient, health provider, or payer), the technology cannot be considered.
A variety of databases help identify economic evaluations (Reference Glanville and Paisley14). However, lack of such data from developing country settings can be a major limiting step. Some assistance is available through the WHO's Global Burden of Disease initiative and the Disease Control Priorities Project, which provide data on both disease burden as well as relative cost-effectiveness of health technologies (47). Despite this, it may be impossible to determine local unit costs, and factors such as baseline risk to determine cost-effectiveness. KNOW ESSENTIALS recognizes this limitation and facilitates decision making even if cost-effectiveness cannot be unequivocally proven.
8. Novelty (N). Many novel health technologies tend to be used more readily because of faster publication of “positive” research results than negative results (publication bias) (Reference Begg and Berlin3), marketing pressure, and the general belief that “newer is better” in health care (Reference Kahveci and Meads22). In some instances, the technology itself is old but the context of its application novel: new indications, user-groups, or administration modalities. Novelty must be examined carefully and factored into decision making. Objective judgment is difficult when data are limited; in such cases, KNOW ESSENTIALS permits favorable consideration if the technology is life-saving; it dissuades from using technologies when safety data are inadequate or evidence of efficacy is too limited for value-based judgments.
9. Time to Outcome of Interest (T). This is particularly important for public health interventions and conditions with long-term consequences or requiring long-term management. In general, technologies requiring shorter time to achieve the desired outcome would be preferred. This is why, for example, interventions for obesity promoting health education or changes in life-style receive less attention than medicines. In some situations, the burden of disease can be so significant that a rapidly effective technology (short duration to outcome of interest) is required; this is particularly true for primary prevention of infectious diseases through vaccination.
10. Integration with Existing Services (I). This is an important issue for healthcare providers as well as consumers. Integrating a new technology with existing services without additional visits or cost is likely to be more acceptable to all stakeholders than technologies necessitating development of new or expensive delivery systems.
11. Alternatives (A). This issue again emphasizes the healthcare problem rather than the technology alone. Considering the problem rather than one potential solution, the decision needs to be balanced against alternate options. If other options are efficacious, safe, and affordable, they could be preferable to newer technologies unless these provide clear additional benefits. An ideal decision-making model would be to evaluate each technology using the KNOW ESSENTIALS tool and make the final decision based on assessment of each technology in the context of the healthcare problem. For example, to prevent consequences of chronic hepatitis B infection, conventional health technology assessment asks if a hepatitis B vaccination program should be started, rather than considering various options including universal or selective vaccination, interventions to prevent high risk behavior, health education, and combinations of these.
12. Long-Term Impact of Using or Rejecting the Health Technology (L). This complex issue balances epidemiological, economic and ethical consequences of using versus rejecting a health technology over the short- and long-term. This gains importance when a potentially useful technology can have adverse consequences in the long-run; for example using more potent antibiotics can treat infections more effectively (short-term gain), but could result in greater antimicrobial resistance (long-term adverse consequence).
The other issue to consider is what would happen if the technology is rejected, in terms of clinical, societal and epidemiologic contexts. These could range from none (in which case the technology has low priority) to serious adverse impact on morbidity or mortality.
13. Sustainability (S). From the perspective of individual users, technologies that are sustainable throughout their administration are preferred. For community interventions, technologies should be sustainable throughout the time it takes to control the health problem. From the perspective of policy makers, sustainability refers to whether the technology can be provided to all eligible people, on all occasions, for long periods of time.
Application of KNOW ESSENTIALS Tool
The thirteen elements given above are grouped into Background issues, Essential criteria, and Other criteria (Table 1). The tool is applied by collating and critically appraising information on each element. This can be done through a systematic review process with meta-analysis, or using other clearly defined search strategies with justification, for example, restricting search to published systematic reviews, locally prepared practice guidelines, or critically appraised evidence summaries. Ideally the identified literature should be critically appraised to ensure methodological robustness (low risk of bias). Each element is given a color code: Green, when available information favors the health technology under consideration; Red if available information does not favor the health technology, or Yellow if available information is insufficient to categorize as either Green or Red. An additional White (not applicable) code is available for the last six criteria. Table 1 presents the detailed criteria for coding each element.
Criteria and Definitions in KNOW ESSENTIALS Tool for Evaluation of Health Technologies
Note. When not applicable, a criterion may be marked as white.
HT, health technology.
The first step in using the tool is to examine the health technology from the perspective of the healthcare problem (Background issues). If any element is coded Red, the health technology can be rejected at this stage without proceeding further. If all are coded Green, proceed to the Essential criteria. If any Essential criterion is coded Red, the technology can be rejected; any Yellow code postpones the decision on the technology until more data is available. If all are coded Green, proceed to Other criteria. The technology can be considered favorably if the majority of Other Criteria are Green, rejected if majority are red, and deferred if majority are Yellow. Different stakeholders may code these six elements differently, resulting in stakeholder-specific decisions.
The coding of each element is entered into a decision-making table (Supplementary Table 1, which can be viewed online at www.journals.cambridge.org/thc2011008). This specifies the stakeholder using the tool, health technology and setting under consideration, and the context where it is being considered. The table has two additional columns for noting remarks and listing references. The tool can be applied independently by different stakeholders (health provider, payer, policy maker, patient, advocacy group, etc).
KNOW ESSENTIALS has been pilot-tested for a set of health technologies among a small cohort of stakeholders comprising health policy makers, clinicians, and consumers. The preliminary results were presented at the HTAi VII Annual Meeting (Reference Mathew25).
Example: Malaria Program
Consider that a Government health policy maker has been requested by the professional society of pediatricians in India to consider changing the national policy for treatment of complicated malaria in children from quinine (used currently), to artemisinin-based treatment. In 2006, India's National Malaria Program recommended quinine as the first-line choice, suggesting artesunate or artemether as less preferred alternatives (Reference Kundu, Ganguly, Ghosh, Choudhury and Shah23). In 2008, it was modified to quinine with tetracycline, doxycycline, or clindamycin (20) in line with the WHO's 2006 recommendations. Because complicated malaria continues to be fairly common with mortality exceeding 20–30 percent, despite appropriate therapy with parenteral quinine, and quinine administration requires carefully monitored infusion, artemisinin derivatives could be a better option.
A search for current best evidence shows that two Cochrane reviews confirm benefit in adults (Reference Jones, Donegan and Lalloo21;Reference McIntosh and Olliaro28), but lack data for children. The WHO's 2010 Guideline (48) also cites lack of evidence to recommend artemisinin based treatment as the first-line choice in children. A recent special theme section of this Journal (Reference Oxman, Yohannes and Røttingen32) published six papers presenting policy briefs on improving access to artemisinin-based combination therapies for malaria in Africa, but no guidance on artemisinin first line therapy for children with severe malaria. Likewise, the EVIPNet Web site (13) does not offer guidance on this specific issue. Table 2 shows how the KNOW ESSENTIALS could be applied to enable an informed decision.
Application of KNOW ESSENTIALS for Informed Decisions Using the Decision-Making Table
DISCUSSION
From the perspective of high-income healthcare settings, the application of the KNOW ESSENTIALS tool may appear unjustified. This is understandable, because they apply robust decision-making mechanisms. Two considerations prompted the development of this tool. The practice of decision making in developing countries and other similar settings poses unique challenges. Second, the utility of the tool has to be considered against current decision-making systems in these settings that lack robust HTA-based systems. This tool is designed to enhance decision-making processes in settings where “idiosyncratic processes” for selecting technologies as described by Chalkidou et al. (Reference Chalkidou, Levine and Dillon5) are the norm.
It can be argued that because the tool includes elements of current HTA methodology, it does not add a new dimension to decision making. However, it is designed for settings where conventional HTA does not exist, or external HTA cannot be directly applied. In such a milieu, the capability to make contextually relevant and locally appropriate rational decisions using existing resources is an attractive option. That the tool includes all the elements of conventional HTA is actually a favorable point suggesting robustness.
How does the KNOW ESSENTIALS tool compare with current decision-making models in industrialized countries? Literature search suggests that this tool encompasses the EUnetHTA HTA Core Model's nine domains (11;12). However, KNOW ESSENTIALS is not inspired by this model. The online pilot version of the Core Model is currently available only to EUnetHTA member agencies, and inaccessible to developing countries. KNOW ESSENTIALS is simpler to use, does not require training in HTA methods, is built on the expertise in literature searching and critical appraisal available in many developing countries, and can facilitate decisions rapidly.
Comparison to other rapid decision tools, such as mini-HTA and rapid reviews, is also needed. Mini-HTA was developed in Denmark to support approval of new treatments (Reference Ehlers, Vestergaard and Kidholm9). Examination of the criteria used clearly shows that mini-HTA was designed to bypass the resource intensive process of detailed evaluation of a new health technology in a developed country setting. KNOW ESSENTIALS is quite different in that it is not designed to fast-track the introduction of largely efficacious and safe technologies in the context of a new healthcare system or population or indication, but intended to address issues much more upstream in the decision-making process. Because mini-HTA also addresses the critical components of evaluating efficacy, safety, and economics of health technologies (besides organizational and other considerations); and is oriented toward rapid decision making relevant to local healthcare settings, some parallels with KNOW ESSENTIALS are expected. It can be argued that KNOW ESSENTIALS, being more comprehensive and compact, can be used in place of mini-HTA even in high-income countries. However, this requires formal comparative evaluation in different healthcare settings across multiple health technologies.
The rapid systematic review is emerging as another short-cut to address the need of providing robust guidance to end-users quickly (Reference Hailey, Corabian, Harstall and Schneider16). Its “rapidity” hinges on bypassing one or more elements of systematic review methodology, by introducing restricted literature searching (limited databases, absence of additional searches, etc.); or limiting questions (usually to efficacy and safety); or removing appraisal for risk of bias; or omitting peer review (Reference Watt, Cameron and Sturm44). Although it may be possible to produce a final report much faster, the critical component of robust methodology could be seriously compromised. It is difficult to address rapidity and scientific rigor in the same breath. The processes used to make the review rapid actually go against the principles of the systematic review process; therefore, they should be considered in rare circumstances where the pressure for a rapid decision is high enough to warrant bypassing scientifically robust systematic literature review. At present, it may be prudent to await the development of a formal methodological process for conducting rapid reviews based on confirmation that omission of certain components of full systematic reviews does not impact the robustness of its conclusions. During this period, the KNOW ESSENTIALS tool could fill the gap.
On the other hand, because there is no consensus on how much literature searching in terms of sources, databases, types of research, appraisal criteria, updating methods, reporting formats, etc., is adequate (Reference Booth4), rapid review methods could be a useful alternative. Additional arguments favoring “rapid reviews” are that they provide opportunities for preparing context-specific guidance (Reference Hailey, Corabian, Harstall and Schneider16;Reference McGregor and Brophy27) and are primarily designed for “policy makers in a hurry” (Reference Watt, Cameron and Sturm44). KNOW ESSENTIALS tool meets these requirements.
Strengths and Limitations
This decision-making tool has several practical merits. It incorporates all the elements of traditional HTA, without requiring HTA methods expertise; focuses also on the healthcare problem, rather than technology alone; and incorporates elements relevant to local healthcare systems. Moreover, it can be applied rapidly and used by multiple stakeholders to provide stakeholder-specific decisions. KNOW ESSENTIALS is built on technical skills available in many developing countries and does not require elaborate training, so it would be much less expensive than investing in HTA institutions. Of course, the obvious pre-requisites for success are: literature searching and retrieval skills and facilities, critical appraisal skills, and knowledge of locally relevant contextual issues.
The tool also has some limitations. The consistency and reliability of coding need to be determined among stakeholders, and solutions must be worked out if people within the same group of stakeholders arrive at different conclusions. Thresholds at which a technology can be definitely accepted or rejected in a given setting should be determined. Decision making in the gray zone where data on most elements is unknown and for technologies where evidence is emerging must be examined. A practical challenge would be to encourage stakeholders comfortable with subjective decision making to give up existing practices.
Further steps planned to refine KNOW ESSENTIALS include seeking feedback from experts in HTA methodology and decision makers in high- and low-income country settings. KNOW ESSENTIALS could be compared with other rapid decision-making tools, such as mini-HTA and rapid reviews, through a randomized trial; and challenges when used in different healthcare systems and settings could be explored.
CONCLUSION
KNOW ESSENTIALS is likely to facilitate informed decisions on health technologies in healthcare systems and settings lacking formal HTA. It incorporates all elements of conventional HTA and includes additional ones. The tool particularly emphasizes the importance of considering health technologies with the health problem and locally relevant issues in perspective. It is relatively simple to apply and can be used by various decision makers to arrive at stakeholder-specific evidence-based decisions.
CONTACT INFORMATION
Joseph L. Mathew, MD (Paediatrics) (joseph.l.mathew@gmail.com), Assistant Professor, Advanced Pediatrics Centre, Postgraduate Institute of Medical Education and Research, Sector 12, Chandigarh, India 160012
CONFLICT OF INTEREST
The author does not report any potential conflicts of interest.
