Due to resource constraints associated with the usage of health technologies in developing countries, these countries need to lay greater emphasis on the creation of structures and procedures assessing the efficiency and effectiveness of existing and new health technologies (Reference Oortwijn, Mathijssen and Banta1). Such a structure should be able to assess the aspects of using and/or developing a particular technology from various points of view, including its medical, economic, social, and ethical consequences. Furthermore, an health technology assessment (HTA) program must be able to account for the efficiency, return, and direct/wanted results along with indirect/unwanted results and must be able to translate and convey its findings to health policy makers in an understandable way (Reference Facey2;3). The acceptance of HTA recommendations by stakeholders is a key determinant of the rate at which the reports’ suggestions will be implemented (3).
Generally, stakeholders can be defined as “individuals, groups, or organizations which not only share the benefits of the topic under scrutiny, but who can potentially affect the goals or the performance of a sector, plan, or policy” (4, p.85). Based on this definition, stakeholders may consist of various groups of general health policy makers, health experts, consumer organizations, individuals, patients, and industries, which are affected by the HTA reports. The methods through which stakeholders collaborate and the necessities of their collaboration may differ in different countries, depending on the roles of the stakeholder groups and the effects each of them can have (5).
Iran began its systematic HTA program in 2007 and, as a consequence, the HTA Office was established in the Ministry of Health and Medical Education (MOHME) in early 2010 (Reference Doaee, Olyaeemanesh and Emami6;Reference Doaee, Oliyaeemanesh and Nejati7). Accordingly, among the Eastern Mediterranean countries, as classified by the World Health Organization (WHO), Iran is known as a pioneer in promoting HTA. The need for a domesticized model that is both comprehensive and responsive led Iran toward the development of a National HTA Model in 2012. Despite the challenges in the beginning, the use of HTA in decision making is rapidly increasing, thanks to the familiarity of the majority of health policy makers with evidence-based decision making. Moreover, the usage of HTA findings in the preparation of other higher-order documents such as the “Roadmap for Health Sector Reform” has supported and promoted scientific HTA and its usage in decision making (8;9).
Although there is no single agreed-upon formula for the involvement of stakeholders in the HTA program among world health systems, any given model has to be able to respond to three questions: (i) Who should be involved? (ii) How to involve them? and, (iii) In which aspects of HTA should they be involved? (Reference Nielsen, Lauritsen and Kristensen4;Reference Pinson, Thielke and King10). Subsequently, this study was conducted upon demand of the HTA Office, which has been in charge of organizing the HTA program in MOHME, with the goal of improving stakeholder involvement in the HTA program. The study aimed to evaluate the opinions of stakeholders on their roles in the HTA program and to determine both the barriers and the facilitators in their organizations to help increase their collaboration in the HTA program.
METHODS
The study was conducted in two phases: semi-structured interviews, and policy dialogue.
Interviews with Stakeholders
The HTA stakeholders’ organizations were identified using the “Flowchart of HTA in Iran” (Figure 1). Based on the representatives’ roles in their organizations, a representative from each organization was chosen during a meeting with HTA Office experts and several researchers in the field.
Flowchart of HTA in Iran.
A semi-structured guide was used in the interviews (Table 1). The contents of the guide were extracted based on the results of a literature review in four countries including Canada (Reference Hailey11–Reference Menon14), Australia (3;Reference Healy and Pugatch12;15), Britain (Reference Pinson, Thielke and King10;16), and Brazil (Reference Oortwijn, Mathijssen and Banta1;Reference Banta and Almeida17;Reference Ferraz, Soárez and Zucchi18). These countries were chosen based on their achievements in developing HTA systems and the availability of their data. The meetings were held in the representatives’ workplaces. For ethical purposes, verbal consent was obtained from the participants at the beginning of the interviews. The interviews were conducted by two interviewers familiar with HTA (who were not working in the HTA Office). The contents of the interviews were transcribed directly and later analyzed.
Guide to Interviews with Stakeholders
Policy Dialogue
At the second stage, a policy dialogue was held. The main purpose of the dialogue was to bring all the key stakeholders together, to familiarize them with each other's ideas, and to provide them with the opportunity to revise or improve their opinions given in the previous interviews. The session offered a good opportunity to the main authorities of the HTA program (i.e., the Deputy of Curative Affairs in MOHME and the HTA Office) to become familiar with the thoughts and beliefs of other stakeholders. In fact, the policy dialogue provided a chance to those who had not taken part in the interviews. Thus, the participants of this meeting consisted of the representatives of stakeholder organizations, some of which had already taken part in the interviews. Supplementary Table 1, shows the list of the stakeholders who contributed to the interviews or policy dialogue or both.
During the dialogue, first, we explained the experiences of other countries on each section of the HTA program and compared them with what was being practiced in Iran. The participants were then asked to discuss their own and their respective organizations views in this regard. To direct the discussion, a questionnaire that had previously been developed for this purpose was distributed among the representatives (Supplementary Figure 1). The questionnaire was designed based on the literature review, the results of the interviews previously held, analysis of the existing situation, and the existing potentialities for HTA promotion in stakeholder organizations. The participants were also asked to add any issues they had not addressed in the discussions to the questionnaire form.
Analysis
The data collected during the interviews were analyzed through the framework approach (Reference Pope and Mays19). To do this, a matrix was drawn with six stages of HTA in decision making (i.e., topic identification, topic prioritization, conducting assessments, external review of the assessments, publication/dissemination of findings, and use of HTA in decision making) (Reference Gauvin, Abelson, Giacomini, Eyles and Lavis20), which are shown as columns. The decisions of stakeholder organizations requiring HTA, the existing process, considerations of the existing process, the role of the organization in the current HTA process, the benefits of HTA for the organization, the roles of the public and patients in HTA, future opportunities, and the role of MOHME formed the rows of this matrix (Supplementary Table 2). The themes “the expectations from HTA reports,” “objection to the reports,” “policy flexibility,” and “financial resource provision for HTA programs” were added to the matrix from the policy dialogue analysis results.
Themes, Sub-themes, and Relevant Barriers and Facilitators from Stakeholders’ Perspective
The validity of data used in this study was ensured in two ways; the participants of the Policy Dialogue were mostly the same individuals who had taken part in the interviews; therefore, they had the opportunity to modify or explain their previous comments. In addition, in line with the peer debriefing method (Reference Frels and Onwuegbuzie21), the entire data were evaluated by the HTA Office experts who had the appropriate knowledge and relevant experience in working in the HTA program to further clarify the suggestions or to draw more in-depth conclusions. To increase the dependability of the study, an independent researcher repeated the data analysis of 10 percent of the interviews and the results were compared.
This study was approved by the Ethics Board of Tehran University of Medical Sciences, which abides by the Helsinki Declaration.
RESULTS
Interviews were held with ten representatives, each interview lasting approximately 40 minutes. In addition, twenty-one individuals participated in the policy dialogue (Supplementary Table 1). Twelve themes and twenty-seven sub-themes were identified (Table 2). The current issues were discussed in each theme. Then, the facilitators and barriers were identified, which constituted the sub-themes. The main suggestions for each theme are displayed in Table 3.
Stakeholders’ Recommendations on the Themes
All the stakeholders thought that they could be the interest groups and, thus, responsible for all the stages of HTA. However, as mentioned below, the level of participation in each theme was believed to be different.
Theme 1: The Decisions of Stakeholder Organizations Requiring HTA
Some stakeholders had special expectations from HTA, that is, other than the assessments of safety, efficiency, and cost-effectiveness (which are routinely evaluated in HTAs). These expectations included conducting evaluations of the social aspects of technologies and the required number and qualification of the workforces needed to use a particular technology.
Theme 2: HTA Stewardship
One of the health insurance organizations and a representative of the private sector did not accept the central stewardship of MOHME for HTA (which is the present situation). Another stakeholder believed that MOHME should only have a supervisory role and that private medical companies should have their own HTA programs.
Theme 3: Topic Identification
Generally, the process of topic identification in HTA is carried out in two ways: horizon scanning and propositions by individuals or organizations. In Iran, the topics are presented by stakeholder organizations and individuals to the HTA Office in MOHME. Presently, horizon scanning rarely takes place in stakeholder organizations. When it does take place, it is usually unscientific and disorganized, to the extent that these activities cannot be considered as horizon scanning or even needs assessment.
All the stakeholders unanimously held the belief that lack of information on burden of diseases is the main barrier of topic identification (and topic prioritization). In this regard, the Health Insurance Organization (HIO) representative suggested presenting a proposal through which the massive amount of data gathered in the HIO be used in topic prioritization.
There are several specific procedures for topic identification in the Food and Drug Administration (FDA) and Department of Medical Equipment in MOHME, and the Ministry of Welfare, which are among the strengths of these organizations in conducting the HTA program.
Theme 4: Topic Prioritization
Currently, topic prioritization is exclusively carried out by MOHME's HTA Office, and certain criteria such as prevalence of disease, burden of disease, costs, fluctuations in the usage of technologies, and political and ethical acceptability are applied to this end.
According to the stakeholders, sets of criteria for topic prioritization, and that too for internal usage, exist in only three deputies of MOHME, including the FDA, the Office of Stewardship and Accreditation of Curative Affairs, and the Deputy of Research and Technology. However, they are either vague or are neglected by decision makers. Only in the private sector are there well-defined standards and procedures for identification and prioritization of topics in areas such as burden of disease, net costs for the patient & the public, efficiency, effectiveness, and economy of technology. Another point to be noted is that many of the stakeholders’ representatives were unaware of the existence of such criteria in MOHME.
Regarding the procedures of prioritization, most stakeholder representatives believed that prioritization should first be determined inside their organizations and the list should then be sent to the HTA Office. Then, a group of representatives from all the stakeholders should validate those lists under the supervision of the HTA Office and make the final shortlist. One stakeholder believed that prioritization must consist of a mix of two approaches; macro-level and critical case studies should be carried out by the HTA Office, and other smaller and local cases should be carried out in stakeholder organizations.
Theme 5: Conducting Assessments
Currently, the topics selected in the HTA Office are submitted to the National Institute of Health Research (NIHR). NIHR monitors health status and produces scientific evidence for health policy making at the national level (22). These topics are then handed over to those who will work on them.
Most of the stakeholder organizations believed that either their organizations did not have the ability to conduct HTA by themselves or that their capabilities were limited. The FDA believed that the topics could be assigned to its members based on their capabilities. The private sector stated that a part of evidence synthesis should be conducted in the organization itself and another part should be directly ordered to researchers. The rest of the organizations thought that the HTA Office should be responsible for ordering the projects to third parties. The existence of organizations that can potentially conduct HTA in the health system, such as Knowledge Management Units (KMUs), which are currently responsible for developing the country's clinical guidelines, is considered a facilitator at this stage.
Some representatives believed that the rate of usage of HTA results would remarkably improve if the stakeholders were involved in the process of performing the assessment, which is presently not the case. It is worth mentioning that one stakeholder believed the monopolized administration and external review of reports by a single organization (NIHR) to be a barrier.
Theme 6: External Review of Reports
The external review of HTA reports are currently carried out by independent reviewers under the supervision of NIHR. Then, if necessary, the results are further evaluated in MOHME's Science & Technology Committee (which consists of many experts from various academic backgrounds). According to the stakeholders, the quality of external reviews is not acceptable. Although there are scientific forums, individuals, and organizations outside MOHME who are interested in reviewing the reports, their disagreement with each other is a great disadvantage to the review process.
Theme 7: Dissemination of HTA Results
Currently, the summaries of assessment reports are accessible to the public on the HTA Office Web site. In addition, if any stakeholder requests additional details on a report, it will be provided by the Office.
The representative of the private sector believed that HTA reports could inform the producers and importers of technology about priorities. Thus, the accessibility of reports to these groups should be considered a priority.
Theme 8: Use of HTA in Decision Making
Presently, the use of HTA recommendations is not obligatory. The participants pointed out numerous obstacles related to the usage of HTA results, such as, personal or organizational preferences; organizational, social, and political pressures; lack of knowledge on the part of stakeholders on the importance of using HTA in decision making; lack of awareness of stakeholders about the HTA results, and the ways in which the results can be used.
Some of the stakeholders believed that the usage of HTA results in decision making must be obligatory, but some others believed that it would be better if it remained optional. The only matter that was deemed evidently necessary by the stakeholders was that a precisely defined legal procedure should exist for the implementation of HTA recommendations.
Theme 9: Objection to HTA Results
Currently, the stakeholders are able to object to the HTA results reports, but the process is not systematic and clear. The participants believed that it should be possible to object at any stage of the HTA and that the procedures and mechanisms of objection should be simple, transparent, and quick. Moreover, there should be predefined schedules for submitting the objections and a committee consisting of all the stakeholders must be responsible for dealing with the objections. The objections should be grounded on scientific evidence and preferably submitted to the committee by research centers. Similarly, the responses given to the objections must be evidence-based.
Theme 10: Flexibility in Policies Based on HTA Results
Presently, there is no legal procedure for changing the policies grounded on HTA recommendations. According to the participants, if an HTA proves that a technology, which is currently used, is dangerous or harmful or is not cost-effective, MOHME's policies and even those at macro level must change. Therefore, policy makers should remember that flexibility in policies based on HTA results is necessary.
Theme 11: Resource Provision for Conducting HTAs
Resource provision for HTA is currently part of MOHME's responsibility, even when the requesting body is outside MOHME. The stakeholders believed that resources should be provided by both private and governmental sectors and that all the organizations involved in health should contribute to this matter. Special circumstances and considerations must also be taken into account, and, depending on the type of technology, and the monopoly of the client over that technology, the method of provision must change. MOHME should adopt the role of an operator and a regulator in this process. One of the stakeholders from MOHME believed that due to interest-related issues, the private sector must never provide resources. On the other hand, private sector stakeholders were of the opinion that resources must be provided by client organizations, but that they should be the main beneficiaries of the project for a particular period of time.
Theme 12: Role of the Public and Patients in HTA
Other than being informed of the HTA results, which are demonstrated on the HTA Web site, public engagement has no place in the HTA program in Iran at present. The participants in this study believed that public contribution to HTA should be taken into account under certain circumstances.
The stakeholders referred to the “existence of information gap among people, patients, and professionals” as the most important barrier against public/patient participation in the HTA program. They also believed that the existence of associations and groups formed by patients could act as a facilitating factor. A member stated that, because patient associations act as public representatives, the pressures exerted by companies should not affect their judgments.
DISCUSSION
This study was conducted to evaluate the opinions of stakeholders on their roles in the HTA program and to determine the barriers and facilitators in their organizations. Based on the findings, all the stakeholders considered themselves engaged in all the stages of the HTA program.
However, the study illustrated that the status of stakeholders’ participation was not satisfactory. The interest groups were not fully informed of the HTA process, which resulted in their passive involvement. Among the different stages of HTA, participation in topic identification was the most active area for engagement in which the stakeholders (i.e., the organizations and individuals) had the opportunity to suggest any topic. Nevertheless, in some stages, such as “conducting the assessments” and “reviewing the reports,” the standard protocol for stakeholder involvement was not observed. The situation was worse in other stages, including the “implementation of HTA,” “objection to HTA results,” “flexibility in policies based on HTA results,” “resource provision,” and “public involvement” in which stakeholder participation was not even contemplated by the government. All these findings confirm that the stakeholders are not meaningfully involved in the process of HTA in Iran, so there is a gap between Iran's situation and those of other countries.
Britain and Australia are among the pioneers in facilitating the process of engaging stakeholders in various stages of HTA (23;24). In these countries, individuals and organizations can easily make topic suggestions to the relevant offices; topic prioritization happens in committees whose members are stakeholders; HTA protocols are prepared with the cooperation of stakeholder organizations; and stakeholder representatives are active in reviewing the results, and the reports are openly published. Furthermore, creating an opportunity to object to the results is an important factor in increasing public involvement. The European HTA Network recognized the necessity of stakeholder involvement in 2008 and established the Stakeholder Collaboration. The key interest groups in this establishment include stakeholder organizations, the public, and external experts who operate at the European level. Various types of input from stakeholders are delivered mainly through provision of comments or exchange of views on specific issues or through facilitating the presentation of information by disclosing unpublished data (25).
When the HTA stewardship in a given country is centralized and it aims to engage the stakeholders in the process of HTA, it would be much more effective if the barriers and facilitators in each stakeholder organization were identified, as it would accelerate stakeholder involvement. This study demonstrated that there are certain obstacles in every stage of the HTA process in Iran when it comes to interest groups’ participation. Topic Identification and Topic Prioritization are not completely carried out according to the pre-established criteria, and despite having the appropriate information infrastructure, they are not used in these two stages. In Conducting Assessments, there is a shortage of human resources and the quality of report reviewing is not acceptable. However, the study confirmed that, in some stakeholder organizations, certain important infrastructures for each stage of the process exist, which should be used when planning for stakeholder involvement.
The difference between the suggested patterns for engaging private and public sectors is another lesson learned from this study, meaning, the “one size fits all” model for involvement of various stakeholders does not hold true in this case. To overcome the identified barriers, certain interventions were proposed by the stakeholders (Table 3). As shown in the table, the main focus lay on transparency of the process and assisting the participation of various stakeholders throughout its different stages, which both need to be well thought-out by the HTA decision makers.
The fact that stakeholders should have a meaningful involvement is emphasized in most available documents. Because the process of engagement is challenging, generating more evidence for familiarizing other health systems with examples of successful experiences or ineffective interventions in the form of case studies or lessons learned can provide valuable insights into developing practical programs.
The HTA program has been in place for a few years in Iran now, and since its commencement, the health system has been striving to reach its ideals by overcoming various obstacles. To this end, among the basic steps required to improve the status quo are to fully inform the stakeholders about the goals and duties of the HTA Office and make all its stages transparent.
As the main client, the HTA Office had integrated partnership and was involved in all the stages of the study, which can be considered as a strength of this study. Moreover, international audiences can adopt the guide used in the Policy Dialogue. The lack of presence of service providers and patients, who are important stakeholders as well, can be considered a limitation of this study.
In conclusion, stakeholder involvement in the HTA program is not easy and does not happen instantaneously. Various stakeholders hold different interests, responsibilities, infrastructures, and barriers. If a program does not meet these concerns, the chances of its success will decrease. Therefore, to prevent overlooking the needs and expectations of stakeholders from the HTA process, it is essential to create opportunities in which their thoughts and ideas are taken into account.
SUPPLEMENTARY MATERIAL
Supplementary Table 1: http://dx.doi.org/10.1017/S0266462316000167
Supplementary Figure 1: http://dx.doi.org/10.1017/S0266462316000167
Supplementary Table 2: http://dx.doi.org/10.1017/S0266462316000167
CONFLICTS OF INTEREST
The authors declare that they have no competing interest.
