Introduction
Pharmaceutical innovations have been defined as those shown to be safe, effective, and addressing an unmet medical need (Reference Morgan, Lopert and Greyson1). Nevertheless, these innovations are associated with a high cost, which impacts access to medicine. Such a challenge raises a question: Is innovation a benefit or a burden?
For innovation to result in added value to society, it should not only address therapeutic needs, but it should also be used by patients to demonstrate its innovative aspects. Hospitals are the main entry point for pharmaceutical innovations, and they need to find the required balance between adopting pharmaceutical innovations and affordability to avoid financial toxicity and improve access to those innovations. Therefore, hospitals need to answer another question: Which health innovations should be adopted by a hospital?
Health technology assessment (HTA) is a policy tool that can be utilized by hospitals to answer this question. It helps hospitals to apply a rational selection of innovative health technologies. Countries can apply HTA on a national, regional, or hospital level. During recent years, the hospital-based HTA gained more attention. It aims to apply the same HTA principles within hospital contexts to inform local decisions on the adoption or disinvestment of health technologies, including pharmaceuticals (Reference Sampietro-Colom, Martin, Sampietro-Colom and Martin2;Reference Sampietro-Colom, Lach, Cicchetti, KIdholm, Pasternack and Fure3).
We believe that hospitals can have an essential role in initiating and implementing HTA. For example, Canada implements HTA at both hospital and national levels (Reference Menon and Stafinski4). The Canadian HTA model shows that HTA efforts at both levels complement each other and provide context-specific assessments when needed. Therefore, hospital-based HTA does not contradict national efforts for developing HTA, but on the contrary, it might strengthen it through providing examples and success stories of HTA implementation. Most of the published experiences about the implementation of hospital-based HTA took place in countries and regions other than the Middle East, which have higher income levels and more developed health systems (Reference Sampietro-Colom, Martin, Sampietro-Colom and Martin2;Reference Attieh and Gagnon5). Those regions have different contexts and healthcare systems compared with Middle East countries. Therefore, this paper aims to describe the establishment of, and assess the implementation of, a hospital-based HTA program in a comprehensive cancer center in Jordan. We believe that this experience could serve as a starting point to other hospitals that would like to initiate a hospital-based HTA in the region. Moreover, we will be sharing our challenges and lessons learned to provide other hospitals with some insights from real-life implementation.
Jordan Health Care System: A Snapshot
Jordan is a country in the Middle East region with a total area of 89,342 km2 and an estimated population of 10,248,069, including refugees (6). Jordan is an upper-middle-income country, according to the recent World Bank classification (7). The healthcare system in Jordan is considered fragmented. It includes public, private, and not-for-profit sectors. The public sector is the leading player in the healthcare system. The total healthcare expenditure as a percent of GDP is 7.89 percent. The pharmaceutical spending as a percent of total healthcare expenditures is around 26.6 percent, which represents 2.10 percent of the GDP (8).
On the other hand, out-of-pocket expenditure is approximately 27 percent. According to the Jordanian national health strategy, the significant challenges facing the pharmaceutical and health technologies are: increased spending on medicine, a high proportion of pharmaceutical waste, failure to use health economics to inform evidence-based decisions, and a lack of data for use in health technology adoption decisions (9).
King Hussein Cancer Center
The King Hussein Cancer Center (KHCC) is a leading comprehensive cancer center in the Middle East region. It is a nongovernmental not-for-profit organization that serves both pediatric and adult cancer patients through providing state-of-the-art comprehensive cancer care. The KHCC is the first and only center in the developing world that is accredited by the Joint Commission International (JCI) as a disease-specific cancer center.
The high cost of new antineoplastic medications triggered a need to have a structured, evidence-based system for formulary listing at the KHCC. We based the development of the hospital-based HTA program on earlier steps that have been conducted by the pharmacy department from 2003 to 2008.
Materials and Methods
We conducted a cross-sectional assessment study of the HTA program at the KHCC during the period from 2008 to 2018. The assessment was based on three types of indicators: structural, process, and outcome indicators. These types of indicators were selected because they are frequently used for assessing the performance of public health programs and in accreditation standards (10–Reference Opadeyi, Fourrier-Réglat and Isah12). As we are assessing a hospital-based HTA program, using indicators that are frequently used in hospitals and public health will be more engaging for stakeholders who are familiar with such type of indicators. The specific selection of each indicator was based on the best practices for performing HTA, which are embodied on the key principles of conduct for HTA and on the AdHopHTA core criteria (Reference Sampietro-Colom, Lach, Cicchetti, KIdholm, Pasternack and Fure3;Reference Drummond, Schwartz, Jönsson, Luce, Neumann and Siebert13). Drummond et al. identified the key principles to improve the conduct of health technology assessments for resource allocation decisions and grouped them under four main dimensions: structure of an HTA program, methods of HTA, processes for conducting an HTA, and the use of HTA in decision making (Reference Drummond, Schwartz, Jönsson, Luce, Neumann and Siebert13). In addition, the AdHopHTA project identified four dimensions: the assessment process, leadership, resources, and impact (Reference Sampietro-Colom, Lach, Cicchetti, KIdholm, Pasternack and Fure3).
The selected indicators for our program were customized to match our hospital setting and were grouped under the three major types of indicators: structure, process, and impact. Using these types of indicators helps in showing the gradual development of a program. For example, a program is expected to meet the structure indicators before proceeding to implementation. The process indicators help hospitals to focus on implementation and learning to improve processes and make them more effective and efficient. The impact indicators show the outcomes of the program and whether it works. Seeing the program as a chain of iterative phases is important for the evolution, the maturity of the program, and its improvement.
The AdHopHTA project identifies collaboration with HTA organizations as a core principle. However, there is no national HTA agency in Jordan and HTA is still in its early stages in the region; therefore, this principle was not included as one of selected indicators. Also, the selection and prioritization criteria were not included as indicators because the scope of assessment was clearly defined for the program targeting only antineoplastic medications.
Using those indicators provided us with an objective way to describe the establishment of our hospital-based HTA program and to assess its implementation.
Structural indicators assessed the availability of required infrastructure and resources that enables the implementation of an HTA program. Process indicators addressed the scoping, assessment, and appraisal processes; as such, they assessed the program's standard operating procedures and methodologies that are required to produce the final outputs. The outcome indicator assessed the impact of the program on formulary listing decisions and price negotiations.
The structural indicators included the presence or absence of a clear organizational structure, the availability of human resources, and the number and type of capacity building activities conducted within the previous 5 years.
The process indicators included the availability of methodological guidelines for the assessment and appraisal of health technologies, the availability of assessment templates, the availability of HTA submission, assessment, and appraisal processes, and the availability of decision-making criteria including a cost-effectiveness threshold.
The outcome indicators included the number of HTA projects conducted during the last 7 years, the HTA reports’ utilization rate by the appraisal committee, the number of HTA reports that were used for price negotiations, and the adoption rate by KHCC clinical practice guidelines (CPGs).
We prepared a data collection form to collect data related to each indicator (Table 1). All related policies, tool kits, assessment reports, meeting minutes, and training activities reports were reviewed to extract the required data and answer the specific questions under each indicator. Most of the questions were yes/no types. Indicators were classified into core and supplementary indicators. We included eight core indicators and two supplementary indicators (Table 1). Core indicators represented the minimum requirements for the program, whereas the more advanced supplementary indicators can be achieved with time and improve the program. This classification of indicators is widely used in public health programs. Meeting all core indicators indicates that the program is functional. On the other hand, meeting supplementary indicators means that there is a level of development in the assessed program (14).
HTA program assessment questions
HTA, Health Technology Assessment.
Having a formulary submission process for manufacturers and a system to monitor the use of listed health technologies were considered supplementary because those processes are seen more at the national level rather than at the hospital level. Those processes were mentioned among the good practice on the national level (Reference Drummond, Schwartz, Jönsson, Luce, Neumann and Siebert13). However, we included them as part of our program process indicators because we implement them and are considered part of the evolution of our program. We considered them supplementary because not having them will not prevent other hospitals from starting a hospital-based HTA. Those two process indicators might be more context-specific to our program than the others.
To determine if the program is considered functional, the answers to all core structural and process indicators should be yes. To determine if the program is effective, there should be a production of HTA reports, and decision makers should use these reports in making health technology adoption decisions.
Results
Structural Indicators
Organizational Structure
The Center for Drug Policy and Technology Assessment (CDPTA) is responsible for conducting HTAs at the KHCC. It operates under the pharmacy department, and it submits final assessment reports to the Pharmacy and Therapeutic (P&T) Committee. The formulary system and the CDPTA policies represent the official mandate for assessors and appraisers of new formulary additions. The center started with one full-time staff and currently has three full-time staff. The adopted HTA model follows the HTA-unit type (Reference McGregor and Brobhy15;Reference Catananti, Cicchetti and Marchetti16). The scope of assessments is expensive antineoplastic medications. Figure 1 shows the CDPTA's organizational structure.
CDPTA organizational structure.
Capacity Building
Investing in capacity building is a core strategic objective for the CDPTA. The CDPTA led five capacity-building activities during the last 5 years. These activities targeted both decision makers and technical assessors and ranged in type, depth, and level. Practical hands-on exercises were a mandatory part of all activities. Two courses, one workshop, and one long-term capacity-building program was conducted. The CDPTA delivered courses and workshops in collaboration with different international universities and health economic centers (17–19). The CDPTA established an annual capacity building program in HTA due to the need for such a program in Jordan and the region. The program aimed to build the capacity of pharmacists to become HTA analysts. It consisted of eight rotations that included evidence-based medicine, biostatistics, health economics, and decision analytical modeling. The participants prepared an HTA report including an economic evaluation of a new health technology as a prerequest to graduate from the program. The first two pharmacists had graduated from the program in December 2017. The program is expected to run every 2 years to generate a pool of competent human resources. A separate paper that described the full structure and outcomes of the first-year implementation of the program is already published elsewhere (Reference Al Rabayah, Al Froukh and Sawalha20). Besides, the pharmacy department has supported the participation of CDPTA staff in international conferences to learn and exchange knowledge.
Process Indicators
Submission Process
The pharmacy department requests formulary submission dossiers from manufacturers for new formulary addition requests according to the P&T committee formulary submission pathway. The formulary submission pathway was developed in 2012. The pathway clarifies to manufacturers the required data for the assessment. The submission is expected to include the following sections: regulatory landscape, disease background, standards of care, comparative safety and efficacy, economic evaluation, and resource implications. An excel-based cost-effectiveness decision analytical model must be submitted as part of the economic evaluation section. Once the submission is received by the CDPTA, a preliminary review will be conducted to assess the submission completeness. Submissions that pass the completeness review will enter the assessment process. Figure 2 shows the formulary submission process.
HTA submission process.
Assessment Process
The center's staff critically appraise the submissions, assess the transferability of economic evaluation models, and adapt them to the hospital context. The CDPTA has built some cost-effectiveness models from scratch before releasing the formulary submission pathway. Currently, the CDPTA staff may initiate internal HTAs and build cost-effectiveness models in cases where the medication is clinically needed for KHCC patients but is not registered in Jordan and where the manufacturer did not submit any HTA dossier or cost-effectiveness models to other HTA agencies. For example, the CDPTA assessed the cost-effectiveness of Pegaspargase versus Asparaginase in treating acute lymphoblastic leukemia (ALL) in pediatric patients. There was a clinical need to expand the use of Pegaspargase to become a first-line treatment rather than to be used only for patients who develop allergy to Asparaginase. Pegaspargase is not registered in Jordan, and at the time of assessment, the manufacture did not have a global cost-effectiveness model that was submitted to HTA agencies. Therefore, we built a cost-effectiveness model along with a budget impact part to assess the value of shifting the use of Pegaspargase to the first-line setting.
The CDPTA follows the following ten steps in assessing new health technologies:
-
Step 1: Clinical efficacy/effectiveness assessment
-
Step 2: Safety evidence assessment
-
Step 3: Quality-of-life evidence assessment
-
Step 4: Published cost-effectiveness evidence assessment
-
Step 5: Critical appraisal of the submitted cost-effectiveness evidence to the KHCC, including the adaptation of the submitted decisions' analytical model to the KHCC context
-
Step 6: External validity of the cost-effectiveness evidence
-
Step 7: Threshold sensitivity analysis and value-based pricing
-
Step 8: Preparation of the final report
-
Step 9: Submitting the report to the P&T committee for consideration and appraisal
-
Step 10: Monitor the implementation and compliance with the P&T committee listing decisions
These ten assessment steps follow a specific guidance document for conducting each aspect of the HTA. A toolkit that includes critical appraisal checklists and assessment tools was compiled to guide the assessment process. For example, critical appraisals of decision analytical models are conducted using a specific checklist by one assessor and reviewed by another assessor (Reference Drummond, O'Brien and Stoddart21–Reference Eddy, Hollingworth, Caro, Tsevat, McDonald and Wong24). Results are released after a discussion by the CDPTA members, and a list of raised questions and required clarifications are sent to the manufacturers. Furthermore, evidence-based Medicine tools and checklists are used to appraise the submitted clinical evidence critically (25–Reference Jansen, Fleurence, Devine, Itzler, Barrett and Hawkins27). The appraisal results are discussed among the CDPTA team before releasing them.
If the submitted decision analytical model is considered valid based on the critical appraisal results, then the adaptation step using local data starts. Therefore, all data requirements will be identified, along with resources. For example, to understand the disease management process, the CDPTA team reviews the related CPG. In addition, specific questionnaires are developed to collect data about specific disease management resources and usage rates (see Supplementary material). These questionnaires are shared with the members of the multidisciplinary clinics (MDCs) and treating physicians.
Auditing the use of newly listed antineoplastic medications is essential to assess the compliance status with the P&T committee decisions and to identify any gaps or required improvements.
Appraisal Processes
The CDPTA submits the assessment reports to the P&T committee for consideration. The report is based on a standardized template that addresses clinical pharmacology, clinical efficacy/effectiveness, safety, and economic evaluation along with expected resource utilization. The results of the adapted cost-effectiveness model, along with assessments of uncertainty and resource implications, are included in the economic evaluation section.
The P&T committee members will make their decisions according to the formulary system policy criteria that consider efficacy, safety, clinical efficacy, cost, and cost-effectiveness. A cost-effectiveness threshold was developed by the KHCC pharmacy department in 2008, and it was reviewed in 2016. Our threshold follows a precedent-based approach (Reference Santos, Guerra-Junior, Godman, Morton and Ruas28). It ranges from $42,000 to $56,000 per QALY. This cost-effectiveness threshold is based on a Jordan-specific cost-effectiveness analysis of imatinib mesylate, a novel cancer treatment accepted as the country's standard of care to manage chronic myelogenous leukemia. The details of the development of the threshold can be found in a separate publication (Reference Jabr and Treish29).
Furthermore, during the committee deliberations, the expected ethical and social consequences of using a new pharmaceutical are addressed. The P&T committee may approve listing the technology, reject listing the technology, list it with restriction, or postpone the decision until further information is available. Figure 3 shows the appraisal process.
HTA appraisal process.
Impact Indicators
Number of HTA Reports
The CDPTA produced thirty-one HTA reports of antineoplastic medications from 2012 to 2018. Twenty-three medications were listed with restrictions for a specific indication or a specific patient's population, whereas eight listing requests were rejected.
Utilization of HTA Results by Decision Makers and Impact on Price Negotiations
All thirty-one HTA reports were used in the decision-making process, with a utilization rate of 100 percent. The prices of twenty-one medications were reduced to meet the KHCC cost per QALY threshold based on the HTA results and were listed in the KHCC formulary.
Adoption of Listed Health Technologies in the Institutions’ CPGs
All approved formulary listings were included within the CPGs according to approved indication and population.
Discussion
Our results showed that our HTA program is functional and effective. As stated by the best practices of hospital-based HTA and key principles for conducting HTA (Reference Sampietro-Colom, Martin, Sampietro-Colom and Martin2;Reference Sampietro-Colom, Lach, Cicchetti, KIdholm, Pasternack and Fure3;Reference Opadeyi, Fourrier-Réglat and Isah12), the minimum core requirements for a hospital-based HTA program are available. The KHCC has embedded HTA activities within the organizational structure of the KHCC, which makes it more structured than hospital-based activities in other countries (Reference Eddy, Hollingworth, Caro, Tsevat, McDonald and Wong24;Reference Gagnon, Desmartis, Poder and Witteman30–Reference Van der Wilt, Rovers, Oortwijn, Grutters, Colom and Martin32).
In comparison with the AdHopHTA guiding principles (Reference Sampietro-Colom, Martin, Sampietro-Colom and Martin2;Reference Sampietro-Colom, Lach, Cicchetti, KIdholm, Pasternack and Fure3), our selected indicators addressed the recommended requirements for founding and running HB-HTA units by the AdHopHTA project. Our assessment methodology is based on best practices (Reference Sampietro-Colom, Martin, Sampietro-Colom and Martin2;Reference Sampietro-Colom, Lach, Cicchetti, KIdholm, Pasternack and Fure3;Reference Opadeyi, Fourrier-Réglat and Isah12) and can be seen as a complementary to AdHopHTA core principles as it facilitates the transition of HB-HTA development through phases. Therefore, the AdHopHTA can be used as a starting point for hospitals to assess their baseline situation and set their action plan. Nevertheless, addressing all recommended dimensions requires time. Our assessment indicators can be applied iteratively to track progress until we have a functional and effective hospital-based HTA system that moves from building a structure to implementing processes to measuring impact.
Furthermore, the primary human resources in our program are hospital pharmacists, which differ from experiences in other countries where physicians are the more dominant human resources. Our experience showed that hospital pharmacists could play an essential role in initiating and sustaining an HTA program due to their experience with the P&T committees and their integral role in conducting assessment for formulary additions. Furthermore, having a functional P&T committee facilitated the introduction of the HTA concept and helped in showing its value.
Our program is different from other programs in the scope of technologies assessed. We started with pharmaceuticals as the infrastructure was suitable for the introduction of the HTA concepts. As discussed previously, having a functional P&T committee and a dedicated center for conducting HTA under the pharmacy department facilitated the implementation of HTA. We preferred to start gradually and to prioritize health technologies that will undergo HTA. Expensive oncology antineoplastic was a priority within our hospital, taking into consideration the available resources and infrastructure. However, there is a need to have a prioritization matrix for selecting which antineoplastic medications will undergo full HTA assessment according to the CDPTA 10-step assessment process.
The economic evaluation part of our HTA includes the appraisal and adaptation of the cost-effectiveness decision analytical model. This component is an essential part of our assessments to demonstrate the value of expensive pharmaceutical technologies. In comparison with other HTA experiences, the economic evaluation component of our program is more comprehensive in scope, which mandated the learning and implementation of decision analytical modeling skills (Reference Roine, Pasternack, Colom and Martin33–Reference Kahveci, Tütüncü, Yüksek, Küçük, Koç, Zengin, Colom and Martin35).
Our program indicators showed that the P&T committee utilized the HTA results in formulary listing decisions. The utilization rate of the HTA results at the KHCC was 100 percent, which was higher than what was reported previously in other countries with hospital-based units like Sweden (92%), Italy (66%), and Iran (35%) (Reference Jivegård, Bergh, Kindblom, Colom and Martin36–Reference Marchetti, Cicchetti, Colom and Martin38).
Our results showed that hospital-based HTA was associated with a positive impact. This result was consistent with what was reported at other hospitals in Europe, North America, South America, Australia, New Zealand, South Africa, and the Far East countries like Singapore and Argentina (Reference Sampietro-Colom, Martin, Sampietro-Colom and Martin2).
Challenges
Having limited experts in HTA and health economics is considered a significant barrier not only in Jordan but in most of the countries in the Middle East and North Africa region (MENA). Therefore, the CDPTA tackled this challenge by establishing an HTA capacity building program for hospital pharmacists. In addition, the pharmacy department at the KHCC delivered nontechnical workshops and training courses to decision makers. Evidently, capacity-building activities are crucial for a hospital-based HTA program; therefore, funding must be secured for its long-term sustainability.
Decision makers and stakeholders demand a shorter assessment time. However, such rigorous assessments require at least 3–6 months. The mean assessment time at the KHCC is around 6 months to 1 year, which is considered a long time by end users. Speeding up the process requires more human resources and a better understanding by the manufacturer of the submission requirements. Furthermore, finding and collecting the required input data and transforming them to be used within the decision analytical modeling part of submitted economic evaluations requires time. Other international HTA units also faced such challenges (Reference Sampietro-Colom, Martin, Sampietro-Colom and Martin2).
Lessons Learned
As stated by other hospital-based HTA experiences (Reference Sampietro-Colom, Martin, Sampietro-Colom and Martin2;Reference Sampietro-Colom, Lach, Cicchetti, KIdholm, Pasternack and Fure3;Reference Opadeyi, Fourrier-Réglat and Isah12), we also learned during these years that political will and leadership support are the prime success factors for the establishment and implementation of an HTA. At the KHCC, the higher administration and the pharmacy department administration supported the initiation and the development of HTA.
Furthermore, the evidence-based culture at the KHCC facilitated the acceptance of HTA as it is rooted in evidence-based medicine. Also, the KHCC has CPGs, which means that there are standard treatment strategies, and this is considered essential for the implementation of HTA and the adaptation of analytical decision models to the KHCC context. Moreover, having a well-defined formulary submission pathway for the manufacturers standardized the processes and provided us with a reference case to guide the assessment of any submission.
Continuous HTA capacity building for both the pharmacists performing HTA and the decision makers is essential. We adopted a learning while doing philosophy, which helped us in developing our knowledge and skills over the years. Moreover, the engagement of other departments at the KHCC, such as the clinical departments including multidisciplinary clinics (MDCs), the cancer registry, the biostatisticians, the finance department, and the IT departments, is considered essential for the implementation of HTA. We also learned that the adaptation of HTA, including the economic evaluation component to the KHCC context, made the HTA results more usable because it is closer to the local context.
Finally, showing decision makers the impact of HTA results within a specific context and the expected outcomes for both patients and the institution is an essential step toward the institutionalization of HTA at any level.
Future Directions
In conclusion, based on our experience, the successful utilization of HTA results requires a context that supports HTA development, an HTA unit within a clear organizational structure, an evolving standard operating procedure, an investment in capacity building, and a culture of learning while doing.
In the future, we aim to publish a summary of our HTA reports and to transfer our experience to assess nonpharmaceutical health technologies. We will be working on selecting medications that will be candidates for disinvestment based on newly generated evidence. Moreover, we will be working on determining the satisfaction of our stakeholders with the HTA process to understand their perceptions and involve them in identifying areas for improvement.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462321000246.
Financial Support
This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.
Conflict of Interest
All authors declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
