Health technology assessment (HTA), examines the consequences of the application of health technologies and is aimed at better informing decision makers. As such, HTA has become a topic of great interest, although it has also attracted controversy. Advocates of HTA argue that it helps to promote efficiency of resource allocation, whilst critics state HTA is simply a means to restrict access to new and costly technologies (Reference O'Donnell, Sissi, Pashos, Miller and Smith1).
Over the past decades different countries have established specialized HTA organizations aimed at better informing healthcare policies and clinical practice. Early models of HTA organizations or agencies were instituted in Sweden, Canada, and Australia. In 1999, the United Kingdom introduced the National Institute for Clinical Excellence, renamed more recently as the National Institute for Health and Care Excellence (NICE). NICE perhaps remains the most well-known HTA organization worldwide (Reference Morrison and Batty2). In fact, most high income countries HICs use some form of HTA process or agency to facilitate decision making and priority setting within their health systems (Reference Bulfone, Younie and Carter3–Reference Castro5).
In 2011, Latin America established a regional HTA network (RedETSA) to advance the use and institutionalization of HTA within the region. Colombia, a 48 million inhabitant's country, established by law 1438 in 2011 its Health Technology Assessment Institute (IETS) to support coverage decision making within its mixed (wage-based and subsidiary) mandatory health insurance system.
Increasing interest from different countries in improving or developing more systematic and transparent processes to set priorities and allocate resources has created a propitious trend for HTA to emerge and grow. However, according to Drummond and Weatherly (Reference Drummond and Weatherly6) country-specific variables such as history, culture, politics, healthcare systems organization, and rationale for undertaking HTA should be considered before implementing its use. HTA also seems to be surrounded by political and social values that need to be borne in mind alongside good methodological practice. Colombia was no exception; hence, there was an opportunity through this research to identify those aspects that may help or hinder HTA development and use in this setting. Previous authors like Rajan et al. (Reference Rajan, Gutierrez-Ibarluzea and Moharra7), have also analyzed the motives, enablers, and barriers to promote or initiate HTA in different contexts.
It is often assumed that policies and decisions are made and implemented in a rational way (Reference Simon8). However, the real world is clearly more complex than this linear process, meaning that designing and implementing a successful policy change is unlikely to be simple (Reference Buse, Mays and Walt9). Indeed, authors like Lindblom (Reference Lindblom10) have proposed that decision makers take incremental steps for policy solutions by comparing only a small number of possible alternatives which are not substantially different from the status quo. Accordingly, decision makers will test the political waters in deciding whether or not to pursue a given course of action (Reference Buse, Mays and Walt9), these representing a contested and dynamic policy process.
Although country-specific HTA health organizations and processes vary according to setting, some procedural principles (such as transparency, robust and appropriate methods for combining costs and benefits, explicit characterization of uncertainty, and active engagement with stakeholders) have been associated with the robust operation of HTA programs and institutions (Reference Drummond, Schwartz and Jonsson11;Reference Chalkidou, Tunis and Lopert12). Nonetheless, according to Pichon-Riviere et al. (Reference Pichon-Riviere, Augustovski and Rubinstein13) in Latin America, including Colombia where this study was developed, “the current level of application [of these principles] was considered uniformly poor.” Therefore, it becomes relevant to answer “What are the main aspects that may help or hinder HTA development and use in a setting like Colombia?”
MATERIALS AND METHODS
To answer the empirical research question, qualitative enquiry methods were used to explore conducting HTA in the country of interest from the perspective of local academics involved in HTA research over the past decade, and to examine their views of the potential use of HTA in this context.
The in-depth semi-structured interview approach was selected as a means to collect data, and thematic content analysis was used to analyze it. As with many other forms of qualitative methods of data collection, the interviews aimed to collect data from a small sample of participants because the size of sample was not determined by the need to ensure generalizability, but by the desire to fully investigate the topic of interest and gather information-rich data (Reference Grbich14). Another reason for selecting this frequently used approach (Reference Green and Thorogood15) was its usefulness for answering questions about salient issues among particular groups of participants.
Purposive sampling was used; sampling was judgmental and involved the conscious selection of participants to be included in the study (Reference Crookes and Davies16). Individuals with similar characteristics were considered eligible (local academics with at least 5 years’ experience of leading research groups working on HTA in Colombia). A total of eight senior researchers throughout the country were identified as potential target participants. All names were extracted from a list of active research groups conducting HTA, registered and publicly available on the web page of the Administrative Department of Science and Technology of Colombia- COLCIENCIAS. Five of the eligible researchers were invited to participate in a set of semi-structured interviews, as the remaining three senior researchers were not invited because they belonged to institutions already represented by the first five eligible participants.
All individuals received an information sheet and consent form in Spanish which explained the objectives of the study, and stressed that their participation was completely voluntary. Participants were asked to consent to being audio-recorded during the interviews. All forms complied with the London School of Hygiene and Tropical Medicine (LSHTM) ethics approval requirements for noninterventional studies). All participants gave consent to participate and to be recorded.
Each semi-structured interview was planned to last between 30 and 40 minutes. The interview guide followed a predetermined set of topics that included the individualś knowledge and experience of conducting HTA, quantity and skill mix of the teams to which they belonged, availability of resources devoted to HTA; and possible barriers, facilitators, and challenges to conducting HTA in Colombia. Interviews were open-ended to allow the participants to guide the discussion as much as possible, and to diverge if necessary to address ideas and concepts not anticipated by the interviewer (Reference Britten17).
Once finished, each interview was transcribed verbatim by an independent transcriber. Each interview was then uploaded to ATLAS-ti version 7 as a new hermeneutic unit. No preliminary hypothesis was considered, and the researcher aimed to interpret data instead of simply describing it (Reference Braun and Clarke18). Thematic content analysis was selected because it allows categorization of the participantsáccounts in a way that can be summarized. In this type of approach, themes that recur or are common are classified, compared, and categorized.
Before starting coding, a panel comprised of the main researcher and two independent advisors (a psychologist and an engineer, both professors of qualitative methods at Pontificia Universidad Javeriana in Bogotá, Colombia) was set up to decide on a preselected labelling scheme of potentially emerging codes.
The first coding stage was independently performed by the main researcher in March 2013 and carried out in parallel by each one of the advisors. Instead of using the “scissors and paste” method described by Krueger and Casey (Reference Krueger and Casey19), all data were coded using ATLAS-ti7 based on memos and quotes selected by the research panel. Combined quotations considered as relevant were preselected for further analysis by the panel after this first round of coding.
To enhance the rigor of research, second and third panel meetings were held in April and June 2013 to analyze emerging data and decide on a potential taxonomy of codes. After the second meeting, the term “drivers” for conducting and/or using HTA in Colombia emerged instead of “barriers and facilitators.”
RESULTS
It emerged from the interviews that all participants were physicians; four were male and one female. All except one belonged to public organizations such as universities or hospitals. Four were active academics and one a research leader at a public hospital. All except one (a PhD in Health Economics) had an MSc in Clinical Epidemiology. Most of the academics (three out of five) are located in central Colombia. Table 1 provides a detailed profile of participants.
Profile of Participants
Depth and breadth of knowledge about HTA varied among participants. Of worth noting that just recently HTA methods such as economic modelling have been incorporated into their research. Only one of the participants was formally trained in health economics and economic evaluation for health care. Their experience of the methods for producing HTA reports varied.
It also emerged from their discourse, that HTA was first practiced in Colombia during the early 1990s when a limited number of academics started producing isolated pieces of research using methods related to those of HTA. In 1994, the first school of Clinical Epidemiology was established in Colombia further influencing the establishment of another three schools around the country in the early 2000s. Over the past decade, these institutions have promoted the use of evidence-based medicine (EBM) to inform clinical decision making.
Two important projects commissioned by the Ministry of Health and Social Protection (MoHSP) have had an impact on the recent development of HTA, one in 2009 when these groups developed twenty-four Clinical Practice Guidelines (CPGs) that included HTA reports aimed at informing macro and micro decision making. The second in 2011, when the government commissioned a project funded by the Inter-American Development Bank (IADB) to inform a blueprint for the establishment of specialized HTA organization following the mandate of law 1438. The agency (IETS) was established in September 2012 and in November started operations.
The local capacity to produce HTA is limited. The early academic interest in producing HTA has evolved, and HTA is seen now as a potential aid for resource-allocation decision-making and health care planning. Current users of HTA reports within the country are: the MoHSP, regional secretariats of health, hospitals and clinics, pharmaceutical industry and medical device developers, healthcare insurers, and the National Institute for Health. Up to 2015, no academic institution in Colombia had formally established a postgraduate HTA specialized Master's degree.
The Emergence of “Drivers”
Initially the perceived barriers and facilitators to developing HTA were used as static labels to code participants’ quotations. Nonetheless, it emerged after a deeper round of coding, that a sub-set of codes accounting for changes in time and their perceived level of importance were relevant for analysis.
During the second panel meeting, it was agreed that the terms “barriers” and “facilitators” considered before were rather static whilst the aspects that may help or hinder HTA seemed more dynamic and interconnected, hence the term “drivers” for conducting and/or using HTA in Colombia emerged from discussion.
A “driver” was defined as a factor that could potentially have an impact on the development or use of HTA (or both) in Colombia. The potential impact of a “driver” suggested directionality, as either being positive (facilitator) or negative (barrier); the potential impact of a “driver” could be perceived as being high or low (leveled) at specific moments of time. A current positive “driver” could become negative and vice-versa depending on the context and its dynamic evolution over time.
The last round of coding was performed by the main researcher alone and looked at relevant quotations after controlling for duplication and looking for “drivers.” Ten possible “drivers” emerged as codes for the development and/or use of HTA in Colombia. During the last meeting of the research panel the possible inter-connections and relationships between “drivers” were explored; the names of two of the emergent “drivers” were also refined during this meeting.
The list of emergent “drivers” includes the following: availability and quality of data, cultural aspects, financial support, globalization, health system context, implementation strategy, local capacity, policy and political support, stakeholder pressure, and usefulness perception. Figure 1 depicts the taxonomy and interaction of these “drivers” on the development and/or use of HTA in Colombia. Each driver is described below.
Emergent drivers for the development and/or use of HTA in Colombia.
All “drivers” have a potential (positive or negative) impact on HTA in the years to come, but also harbor the dynamic potential to switch the directionality of their impact over time. Some “drivers” were considered capable of operating in association or interdependently; for instance, implementation strategy could be associated with installed capacity, and installed capacity with resources devoted to HTA, but availability and quality of data could have a potential impact with no interconnection with any other “driver.”
Availability and quality of data: availability of data refers to the concerns of participants about the amount and quality of local data available to support the development of HTA. This may have an impact on the internal validity of HTA results, extrapolation or generalization of the findings. This driver accounts for the limitations of local information about the clinical effectiveness of interventions, probabilities, costs, and health state preferences, all necessary to populate robust decision-making models. The uncertainty of results had a potential negative impact that may prevent the use of HTA as a reliable source of information for resource-allocation decision making.
Cultural aspects: emerged as a “driver” related to context-specific conduct or behaviors that may have an impact on HTA. This refers also to social conceptions determined by local custom, meanings and beliefs with the potential to influence the development or use of HTA. Most emerging cultural aspects were identified as having a potential negative impact, for instance the lack of tradition of priority setting or relying on the best available evidence to inform decision making or a lack of long-term planning.
Financial support: this “driver” refers to the potential sources of funding that could promote the development of new and on-going HTA projects. It emerged that most of the funding for research centers is project based and envisioned as short-term; nonetheless the availability of financial resources for HTA projects has increased in recent years. Financial support was usually associated with another “driver,” policy and political support. Participants believe funding within academic institutions and government agencies is associated with policy commitment to support long-term research. The policy of establishing an HTA agency in the country is seen as a new opportunity for financial support and institutional strengthening.
Globalization: emerged as a “driver” that has influenced the development and use of HTA in Colombia. It refers to a perceived global and social trend to produce and use HTA to inform decision-making in different countries. Also the concept of an international “peer pressure” to get involved in HTA or an opportunity to work with international organizations in co-operative networks regarding HTA projects emerged. Globalization may also have an influence on the social perception or “mood” about HTA's potential usefulness, and also on the possibility of international co-operation for research. According to participants’ remarks, globalization may carry a negative impact if skilled researchers leave the country to work overseas, this potential “brain drain” may have a negative impact on the stability of research groups.
Health system context: the health system structure, the current context of financial crisis and the macro performance indicators emerged as a “driver” for potential HTA development and use in Colombia; this was labelled as health system context. The need to regularly update the content of the list of publicly reimbursed benefit package (POS) making use of the best available evidence, emerged as having a potential positive impact on HTA development.
Currently the health system is facing a financial crisis and “lack of trust” that has threatened sustainability. This political instability is envisioned as a factor able to produce further institutional instability that may also impact the future development or use of HTA.
Implementation strategy: the need for an organized and well-planned HTA implementation strategy emerged as a “driver.” This “driver” not only included the challenges regarding the use of evidence-based recommendations to inform policy, but also the expected systematic process of institutional (HTA-related) strengthening in Colombia. This involves dissemination of results, capacity building, institutional strengthening, and introduction of incentives, amongst others. Regarding dissemination of HTA results and knowledge transfer, academics expect a systematic and well-organized process bearing in mind the target audiences of the information.
Capacity building needs to consider strengthening the skills of researchers devoted to HTA, but also of those in charge of using HTA as a source of information to make decisions and policies. Implementation strategy, in contrast to some other “drivers,” emerged more as a challenge that needs to be borne in mind during the operationalization of HTA policies. Its potential impact may be bi-directional depending on a successful or unsuccessful implementation plan.
A well-planned implementation strategy is expected to be incremental and step-by-step regarding the development of methods. HTA reports are expected to be available and “digestible” to a broad audience. The recent establishment of IETS represents, according to participants’ remarks, a new opportunity to improve the rigor of independent HTA work and strengthen local capacity. It has risen great expectations, however, it is challenging to achieve in the short-term.
Local capacity: Local capacity emerged as a “driver” which refers to the existence or absence of human technical capacity, infrastructure, tools, resources, and academic institutions in place to develop and promote the use of HTA in Colombia. Local capacity had three different levels of interpretation: organizational, institutional, and individual. Organizational capacity referred to the establishment of postgraduate schools dedicated to evidence-based medicine (EBM), especially of clinical epidemiology; these have increased the supply of skilled professionals over the last two decades in Colombia. Organizational capacity referred also to the recent establishment of research and other groups, in charge of developing HTA work commissioned by the Government. At the individual level the number of HTA specialized professionals is still insufficient to cope with local demand; according to participants, Colombia has 150 clinical epidemiologists capable of conducting HTA (mainly clinical effectiveness requests), but the estimated number of skilled health economists and bio-statisticians able to conduct economic modelling and evaluation was ten times lower.
The recent development of HTA-related projects has been an opportunity to strengthened local capacity, because funding has also served to bring to Colombia “world class” academics to train local HTA groups. According to participants’ remarks, skilled human resources are still scarce in some knowledge fields and are mostly concentrated in big urban areas.
A perceived challenge for the stability of research groups is the competitive local market for scarce HTA professionals (this includes the pharmaceutical industry), so far in Colombia, there is no steady formal specialized postgraduate training in health economics or HTA to ensure sustainable supply of new skilled specialists. The Infrastructure of research groups tends to be modest (small groups of people, with basic software and hardware) and the availability of resources is still limited and highly dependent on financial and policy support.
According to participants, the need to build up local capacity goes beyond researchers’ and academics’ needs, and also requires raising awareness and strengthening theoretical foundations of local policy and decision makers.
Policy and political support: policy support refers to the local social acceptance of HTA by the academic community and other stakeholders, and politicians’ support to include HTA into their policy agendas. This “driver” is represented by the enactment of laws, explicit policies within the government or any other funding bodies regarding HTA production or use. Although previous studies (Reference Castro4) have tracked the policy interest of Colombia to use HTA back to the early 2000s, it is only in the last 5 years that major policy changes have boosted HTA development and use in this context.
Local policy support from governmental institutions such as COLCIENCIAS, the MoHSP or public hospitals have granted funding for HTA work and capacity building in recent times. Policy and political support is critical for the production or use of HTA. According to participants this policy/political support is still unstable and heavily reliant on the political environment.
Policy and political support are expected to have a broad remit that goes beyond the health sector, for instance to ensure HTA development, policies within the Ministry of Education may enhance capacity building within the country.
Stakeholder pressure: the contested pressure among relevant actors with vested interests in Colombia came up as an important “driver.” This “driver” incorporates market and stakeholder pressure to either use or prevent the use of HTA that may, in the end, induce regulation. Among the stakeholders mentioned by interviewees are patients’ associations, providers, insurers, professional bodies, manufacturers, academics, and government bodies. Stakeholder pressure was perceived as mainly having a potential negative impact on both the development and use of HTA in Colombia.
Usefulness perception: usefulness perception came up as a “driver” that refers to the meaning and values that academics, decision and policymakers, and the whole society put into HTA as a tool to help inform health and healthcare decisions. An incremental shift from pure academic interest to a more pragmatic use of HTA as an aid to inform decision-making emerged from participants. According to participants’ remarks HTA is perceived as having a role to support regulation, improve transparency, allocative efficiency, and equity.
This “driver” also refers to the awareness of HTA limitations and the need to incorporate additional values into decision making that goes beyond HTA itself. Nonetheless, the lack of usefulness perception, especially among clinicians, policy makers, and decision makers may have a negative impact on the future development and use of HTA in Colombia.
DISCUSSION
In recent years, qualitative methods have become more commonplace in areas such as health services research and HTA. This facet of research was considered as relevant and important because it is an attempt to add to the knowledge about the aspects that may enable or stifle HTA development and use in a context like Colombia. This country, as noted previously, is currently facing a financial and lack of trust crisis (Reference Castro4;Reference Caballero-Uribe20;21).
According Rajan et al. (Reference Rajan, Gutierrez-Ibarluzea and Moharra7), the top three motives for HTA initiation are: to support decision making in health care and promote appropriate resource allocation; strengthen credibility, transparency, and accountability of different decision-making levels; and achieve better quality of health services. These motives coincided with the role or usefulness perceived of HTA in Colombia.
The top five enablers published by Rajan were also similar with some of the “drivers” mentioned herein (i.e., availability of human resources to develop HTA [local capacity]; availability of financial resources to perform/run HTA [financial support]; existing good practices and examples from other countries [globalization]; existing international networking, support and collaboration [globalization]; and understanding the local needs and setting priorities [health system context and stakeholder pressure]).
The top five barriers described by these authors (although context specific) also coincided with some of the identified “drivers” in this study: lack of financial resources to perform/run HTA [financial support], resistance to change from existing practice routines and culture [cultural aspects], lack of knowledge about EBM and/or HTA [local capacity and usefulness perception], lack of human resources to perform/run HTA [local capacity]; and lack of interest in EBM and/or HTA [usefulness perception].
Studies by Varela-Lema et al. (Reference Varela-Lema, Merino and Garcia22), Silva et al. (Reference Silva, Spath and Moumjid23), Hyder et al. (Reference Hyder, Corluka and Winch24), and Pichon-Riviere et al. (Reference Pichon-Riviere, Ceballos and Briones25) also coincided that implementation strategies, as well as usefulness perception, cultural aspects, financial and policy/political support are important aspects to consider for the successful implementation of HTA.
CONCLUSIONS
Three models for HTA promotion and initiation have been described (Reference Rajan, Gutierrez-Ibarluzea and Moharra7), top–down whenever HTA is initiated from political interest, bottom–up whenever HTA is initiated and promoted from academic or research interest, and converging which combines both the political and academic/research interests. In Colombia, although academic interest in HTA started in the 1990s as a bottom–up initiative, it is only in recent years that HTA has attracted the attention of policy-makers and been top–down formulated as a policy solution. Both bottom–up and top–down interests converged with the enactment of Law 1438 of 2011 that enabled the establishment of IETS, thus the formal initiation process of HTA could be considered as converging in this country.
The emerged “drivers” could potentially have a positive or negative impact at specific moments in time. The local government has strengthened its institutional capacity with the establishment of an HTA agency with the purpose of informing resource-allocation decision-making. Hence of special interest is the uncovering of those dynamic aspects, reported as “drivers” that should be considered for the successful implementation of HTA in this context of interest.
The conceptual transferability of concepts like “drivers” with caveats may be of interest for similar settings trying to incorporate HTA processes and institutions into decision making.
CONFLICTS OF INTEREST
The authors declare no conflicts of interest.
