Health technology assessment (HTA) is considered as part of the movement that aims to improve quality and efficiency in health care (24;37). HTA has gained prestige over the past decades and is now considered an important source for supporting decision making (13;23;39). However, a recurrent concern is that knowledge produced from HTA is still not integrated as it should into political, organizational, and clinical decisions (34).
Methods have been proposed to assess the impact of HTA recommendations at different levels of decision making (15;18;35). However, knowledge on the dynamics that affect HTA recommendations integration into organizational and clinical practices is scarce. Rosen and Gabbay (34) have identified three major forces that can hinder the uptake of HTA recommendations: (i) the complexity of organizational milieu, (ii) the changing nature of research evidence on new health technologies, and (iii) the value conflicts between decision makers at different levels. Also, it is important to stress that HTA takes place in a local and social context (23). It is essential, thus, to take human factors and organizational structures into consideration to foster a culture of evidence-based decision making.
CONTEXT
The Catalan Agency for Health Technology Assessment and Research (CAHTAR) was created in 1994. The CAHTAR is involved principally in HTA-related activities and produces a dozen recommendations yearly. Diffusion of recommendations is made through distribution of printed versions of HTA reports and technical documents as well as by accessing electronic documents on the Web. However, it is difficult for the CAHTAR (as for other HTA agency) to assess the actual impact of a given HTA recommendation, because its work usually ends with the publication of a report.
Three criteria were used for selecting HTA recommendations for this case study. First, the recommendations had to be published for at least 1 year to ensure that they have been widely disseminated. Second, the recommendations were selected to represent two broad categories of health technologies: administrative and clinical. This selection was done because the literature reports important differences between administrative and clinical innovations (21). Third, two similar technologies were selected in the administrative category to allow for comparisons.
The clinical technology selected was the external pump for continuous subcutaneous insulin infusion—hereafter insulin pump—in patients with type I diabetes. CAHTAR's recommendations on the prescription of insulin pumps were published in 2000 (30). The report was generally favorable to the prescription of insulin pumps at the clinical level, because it provides increased flexibility to patients, better glycemic control, and improves quality of life. However, the recommendations also highlighted increased costs associated with insulin pumps, time and costs required for patient education, and increased risk of infection.
The administrative technologies consisted of prioritization systems for patients in waiting list for two surgical procedures: cataract surgery and hip and knee replacement. Working in collaboration with scientific societies and various groups of stakeholders (patients, healthcare professionals, population), the CAHTAR processed to a multistaged study to develop systems for assessing priority among patients waiting for cataract surgery and hip and knee replacement. The development process of the prioritization system as well as the prioritization tools were published in early 2003 (9).
These three recommendations were reviewed with complementary documentation. Researchers who had collaborated to the HTA reports were also consulted to provide information concerning the process of the study and dissemination activities.
THEORETICAL FOUNDATIONS
Institutional Level
Institutional environments represent a shared set of rules and requirements to which individual organizations must conform to receive legitimacy and support (27). According to institutional theorists, organizations are likely to adopt an innovation if their environment legitimized it. Furthermore, institutional pressures increase the homogeneity of organizational structures. Institutional theory allows for considering noninstrumental factors, such as symbolic, cultural, and political aspects involved in change processes in hospitals (32).
Organizational Level
The internal structure of healthcare organizations is considered unique in that it consists in various professionals working relatively independently and autonomously, corresponding to Mintzberg's configuration of professional bureaucracy (29). The division of work is usually done according to health conditions or patient groups. Coordination between professionals is guaranteed by standardization of skills and knowledge. Administrative control is limited because the work of professionals is self-regulated through training and collegial associations.
Kimberly and Evanisko (21) have found that autonomy (decentralization of decision making), specialization (number of medical specialties), size of the organization (number of beds), and functional differentiation (number of work subunits) were positively correlated with the adoption of clinical innovations. In contrast, external integration, that is the communication and liaison mechanisms with other organizations, did not significantly influence adoption of clinical innovations.
With respect to adoption of administrative innovations, structural characteristics of hospitals only had a negligible influence (21). Conversely, external factors, such as environmental pressure, influenced the adoption of administrative innovations. Managerial support was also positively correlated with adoption of administrative innovations. With respect to contextual variables, high competition (measured by the number of hospitals in the region) and the size of the community were neither significant predictors of the adoption of administrative nor clinical innovations.
A meta-analysis (6) has identified factors involved in the adoption of innovations across various types of organizations. For nonprofit public organizations, such as hospitals, specialization, vertical differentiation, and favorable managerial attitude toward change are positively associated with adoption, whereas centralization and formalism have a negative influence.
Professional Level
Professions in the healthcare sector are regulated by collegial associations and are organized around fields of specialized knowledge and competencies, referred to as jurisdictions (1). According to Friedson (10), the dominance of the medical profession is expressed through professional autonomy and self-regulation, as well as the control over patients and other healthcare professions. Furthermore, the medical profession is highly valued in the public opinion and physicians, therefore, can enjoy high support to legitimate their claims (38).
Professionals may anticipate that the introduction of a new technology would impact the definition of their roles and responsibilities, and this possibility may create resistance (2). Conversely, when the new technology is perceived in accordance with professional interests, its adoption is easier (23). Learning through modeling is also an important aspect of medical training; thus, peer influence might be a possible determinant of adherence to guidelines (33).
Jorm and Kam (20) have proposed a metaphor that illustrates the strong influence of the medical professional culture: “Like the knights of Camelot, medicine has its own all-pervading culture, occupationally centred, exclusive to the initiated and tending to the traditional, which can inhibit doctors' participation to quality improvement.” (p. 251). An integrated conceptual model was developed from a combination of theoretical concepts used in previous studies (11;21;22). The operationalization of concepts was adapted to the specific nature of the behavior under study (adoption of HTA recommendations), the characteristics of healthcare organizations and professional groups, and the larger sociopolitical context.
METHODS
A multiple case study was used as the method of inquiry, considering each of the three HTA recommendations as a case. This method accounts for the contextual conditions of a given phenomenon and relies on multiple sources of evidence (36;40). In-depth semistructured interviews with key informants are considered an effective means to collect relevant material for case studies (7).
Interview Schema
The interview schema had a total of nineteen questions. The first series of questions collected information about hospital's external characteristics: size (number of acute care beds), age (number of years under the current denomination), localization (central or peripheral), competition (number of hospitals in the Health Region), and interorganizational relationships (collaboration with the CAHTAR).
A second series of questions measured variables related to the internal organizational structure: type of management (public or private nonprofit), level of care (general hospital or teaching hospital), specialization (number of medical and surgical specialties), differentiation (number of departments, units, and services in the hospital), size of units (number of employees working in the service or department), planning and control systems (formalism in the organization of medical work in general and formalism with respect to technology utilization); internal communications (committees or taskforce groups regarding decision making in general and decision making for the specific technology), and decentralization of power (number of administrative positions held by physicians).
Finally, open-ended questions assessed respondent's knowledge of CAHTAR's recommendations and experience regarding the implementation and use of the technology. The interview schema was pilot-tested with one physician from each specialty and two hospital managers to verify length and ensure understanding.
Selection of Study Sites and Key Informants
Participating hospitals were selected through contacts with representatives of each Catalan Health Regions. Criteria used for hospital selection were: location (central/peripheral), level of specialization (low/high), type of management (public/private nonprofit), and presence of one of the three targeted medical specialties (endocrinology, ophthalmology, orthopedic surgery). Fifteen hospitals were finally selected to represent as much as possible the various profiles of Catalan hospitals. In each selected hospital, the head of service or department of each specialty was contacted by telephone. A total of thirty-one persons were contacted.
The contacted specialists were invited to participate in an interview. If they were not available for the interview or did not feel knowledgeable about the topic, they were asked to provide the name of a colleague who was. A specialist from all contacted departments or services agreed to participate in the study. However, six respondents were not available at the moment scheduled for the interview because of professional duties. Among those, two were briefly interviewed by telephone and their opinions were considered as a complement to other interviews.
Furthermore, interviews were conducted with three other key informants, representing decision makers, professional groups, and hospital administrators. Therefore, a total of twenty-eight persons were interviewed. Table 1 presents the profile of participating organizations as well the characteristics of interviewees.
Verbal consent was obtained from all participants before the interview. Interviews were conducted at the respondent's location and lasted from 20 minutes to 1 hour and 15 minutes. Written notes were made from those respondents who refused to have the interview tape-recorded. To ensure the accuracy of data collection, the interviewer summarized the general content of the discussion to the respondent at the end of the interview. Time constraints prohibited returning a written interview summary to the respondents.
Analysis
The method proposed by Huberman and Miles (17) was followed to analyze interview transcripts. In a first step, all material was read to extract general themes corresponding to theoretical constructs. In a second step, codification tables were created and material was organized according to these themes. Free spaces were also added to allow for emerging themes. Then, similar ideas or trends were grouped into categories and linked to the theoretical concepts. This step was done separately for each HTA recommendation. Finally, convergences and divergences within and between the three cases were explored to identify which factors had influenced the integration of HTA recommendations into practices.
FINDINGS
Factors affecting the uptake of HTA recommendations are presented according to the three levels of analysis considered in the study: institutional, organizational, and professional.
Institutional Level
The three technologies considered are embedded in the larger sociopolitical context. Thus, forces that are present in the institutional field have affected their integration into practices. Major factors of influence were classified according to these categories: localization, interorganizational relationships, reimbursement, and financing policies.
Healthcare organizations may share common goals and values, but the importance of local cultures on the potential impact of guidelines has been acknowledged (4;35). In hospitals located at a greater distance of the CAHTAR, the recommendations received less attention. Interviews with specialists practicing in peripheral hospitals have highlighted that these organizations rarely were involved in research activities taking place in the capital region. Thus, the institutional pressure to adopt CAHTAR's guidance regarding health technologies was less among peripheral hospitals.
This observation is closely related to the importance of interorganizational relationships. That some hospitals shared closer links with the CAHTAR had facilitated the dissemination of the recommendations among professionals. This finding was particularly true with regard to the prioritization systems for hip and knee replacement in which the head of orthopedic surgery of a teaching hospital was involved in the elaboration of the system. This hospital was the one in which a greater proportion of patients (75%) already have been assessed based on the CAHTAR's prioritization system.
In the case of insulin pumps, CAHTAR's recommendations have had an indirect impact on practice through their impact on payment policies, because decision makers used them to assess patients' eligibility to reimbursement. As an endocrinologist reported, change in reimbursement modalities increased prescription:
“Now the reimbursement process is easier. That's why they [endocrinologists] prescribe more pumps. It's not a matter of clinical indications.” (Endocrinologist, Hospital 1).
Thus, many endocrinologists followed recommendations to ensure that their patients would receive coverage for insulin pump and other required material (catheter, needles, and so on).
The Catalan Health Service can purchase services from privately managed (profit and nonprofit) hospitals to benefit publicly covered patients. Specific services, such as procedures for which there are waiting lists, can be purchased as a bonus for additional cases. This process is known as selective purchasing. However, in many private nonprofit hospitals, this practice was perceived as a way to increase productivity without considering other aspects. In that respect, prioritization systems for cataract surgery and hip and knee replacement were seen by some physicians as an element to legitimate selective purchasing practices:
“They [the Catalan Health Service] haven't provided us priorisation criteria for other pathologies. So, there are many interventions that we just can't do because we lack the financial resources and time to do them.” (Orthopaedic surgeon, Hospital 1).
Organizational Level
The particular structure of healthcare organizations has an impact on internal mechanisms and decision-making processes. Depending on the type of technology considered—administrative or clinical—some structural characteristics have had different effects on the integration of HTA recommendations in hospitals. The main factors of influence referred to specialization and level of care, formalism, and type of management.
Hospitals with a lower level of specialization were less likely to follow recommendations simply because they had a limited need for the technologies considered. For instance, many smaller hospitals did not have an endocrinology service; thus, patients with type I diabetes were referred to larger hospitals. The situation was somewhat different for the prioritization system, especially for cataract surgery, because some smaller hospitals were important providers for that service. However, the prioritization system for cataract surgery was not generally applied for several reasons, among which the lack of incentive.
In most hospitals, CAHTAR's recommendations have impacted management practices. Some departments have instituted local protocols in accordance with the guidelines. The recommendations have led to a strict control of the clinical indications for the prescription of insulin pumps, because each case is reviewed by two endocrinologists of the Department of Health.
“Recommendations have influenced our practice. Clinicians have adapted themselves to the new technology and a special protocol was established that helped us to prescribe pumps.” (Endocrinologist, Hospital 3).
With regard to prioritization systems, formalism was associated with a possible decrease in interhospital variations for surgical waiting lists. CAHTAR's recommendations allow the standardization of prioritization criteria and, thus, facilitate waiting lists management at the regional level.
“Previously, there were many ‘personal’ waiting lists. They were false lists because the same patient could be on many different lists at the same time. Also, some physicians were only putting the name of patients who didn't need the intervention, just in case.” (Orthopaedic surgeon, Hospital 9).
In other cases, the lack of internal protocols was seen as a major impediment for physicians to use CAHTAR's recommendations. Hence, many departments were still using their own prioritization systems, whereas some proceeded on the “first arrived, first served” basis.
With respect to management type, hospitals with private nonprofit administration were less likely to use recommendations for patient prioritization because waiting lists were not seen as an issue for them, especially for cataract surgery. Often, selective purchasing provided the financial resources required to maintain waiting time quite short (e.g., 1 to 3 months for cataract surgery). Conversely, some publicly managed hospitals experienced an important rise in waiting times for cataract surgery, because they were receiving urgent and more complex cases that other hospitals refused to treat with surgery.
Professional Level
Professionals groups share values and norms that may differ from those of other players in the healthcare sector. These features may impact in several ways on the adoption of guidance and recommendations. Interviews have highlighted five important areas of professional influence: patient interest, professional autonomy, roles and responsibilities, and peer influence.
There is a fundamental opposition between technology assessment perspective, which considers health benefits at a societal level, and clinical perspective, which tends to maximize benefits for the individual patient (34). In that respect, conflicts may arise when specialists are expected to apply societal HTA recommendation to specific patient care:
“There are particular indications for insulin pumps that must be examined case by case. It's a subjective assessment, a question of quality of life and satisfaction. Patients have the impression that they're getting better, whatever the clinical results are.” (Endocrinologist, Hospital 2).
However, specialists acknowledged that the CAHTAR had collaborated with their professional societies in the elaboration of recommendations. Thus, the prioritization system was seen as an acceptable response to healthcare system imperatives. As this quote illustrates, specialists recognized that the recommendations were not only based on economic considerations, but also included clinical evidence:
“This problem is basically a politic one. The resources are not sufficient to meet everyone's needs. Thus, priorisation criteria ought to be fair, demonstrable and equitable.” (Ophthalmologist, Hospital 14).
Nonetheless, some physicians in teaching hospitals believed that the recommendations limited their capacity to expose residents to a variety of clinical conditions:
“There is a lack of diversity in clinical cases. We need different pathologies to teach residents. Before the priorisation system, there was a better repartition of the cases and a greater autonomy on the doctor side.” (Orthopaedic surgeon, Hospital 4).
Thus, the economistic or instrumental logic that favors the concentration of clinical activities into procedures for which there is higher demand was confronted to the professional logic that encourages the diversity of practice for maintaining skills and teaching purposes (31). Some specialists believed that using the recommendations would increase the amount of work spent on administrative tasks. Healthcare professionals might consider “paper work” as bureaucratic and, thus, in opposition to their professionalism (32). For instance, a physician thought that recommendations concerning the prioritization system should not target specialists, but less specialized professionals:
“We don't use the priorisation criteria because the evaluation of patients should be done by non-medical professionals, in ambulatory care.” (Ophthalmologist, Hospital 8).
Peer networks and allegiance to professional associations exert a strong influence on physician behavior. In the case of insulin pump, renowned experts in diabetes were involved in the elaboration of CAHTAR's recommendations.
Conversely, peer influence did not appear as an important factor in the adoption of prioritization systems, even though scientific societies of ophthalmology and orthopedic surgery were involved in their elaboration. According to physicians from both specialties, their professional associations did not make any diffusion of the CAHTAR's recommendations, because waiting lists were not perceived as their problem but that of the healthcare system.
DISCUSSION
These findings are relevant for research development on the impact of HTA in different respects. First, this study was innovative in that it used a multidimensional theoretical framework to assess the factors that have facilitated or hindered the uptake of HTA recommendations. This framework provided relevant concepts to organize findings into a logical interpretation scheme. Second, clinical and clinical–administrative health technologies were considered for allowing comparisons between technologies. The findings support many theoretical assumptions, and close similitude was found between the two clinical–administrative technologies. However, several theoretical concepts did not appear relevant to the present study, such as structural characteristics of healthcare organizations. The relative heterogeneity of internal organization between hospitals, or institutional isomorphism (8), could explain this observation.
Third, fifteen hospitals were involved in the study, providing contrasting experiences and a variety of perspectives. It appeared that hospital localization as well as its collaboration with the HTA agency significantly influenced the uptake of recommendations. Involving hospitals with public and private nonprofit management also helped identify the various meanings that HTA recommendations could have depending on the structure of healthcare organizations. Therefore, some important lessons can be drawn from this study to understand factors affecting the integration of guidance based on HTA into practice.
Improving HTA Uptake
More work on the dissemination of HTA recommendations is needed to improve their impact on decision making (12). One successful means to impact on the dissemination of a technology is through fiscal restriction (19). Our study confirms that economic considerations are central to decision making at the management level. However, at the clinical level, fiscal restrictions may be perceived as a threat to professionalism. Economic incentives are known to have a greater effect if professionals are motivated and perceive other advantages to the proposed change (25).
Coleman and Nicholl (4) have observed that guidelines that were produced in a given region were more likely to be integrated into healthcare settings of the same region. In this study, CAHTAR's recommendations were better known in hospitals that were also located in the capital region. Moreover, physicians practicing in those hospitals were often collaborating in research projects with the CAHTAR. Thus, HTA recommendations could be more difficult to implement in a distant milieu, calling for specific efforts to adapt the recommendations to local context and culture.
Furthermore, the fundamental dilemma between individual patient and population interests has been identified as a major barrier to the adoption of clinical guidelines (4). Also, physicians may consider that evidence based on meta-analyses is poorly relevant to “real-life situations” (5;28). Adapting HTA recommendations to the needs of end-users is essential to promote their use (16). Hospital-based HTA represents a promising option to develop context-relevant policy advice on the introduction of health technologies (26).
Consultation processes, such as the Delphi approach, have been suggested to increase physician participation in HTA process (20). However, physicians are already overloaded and many of them may not be interested in HTA activities. The HTA Ambassador's experience in Sweden, which involves clinical leaders in dissemination activities (3), could be applied upstream by involving respected experts early in the assessment process of health technologies. Finally, various successful strategies have been identified with respect to the adoption of clinical guidelines (14) and could be adapted to improve the uptake of HTA recommendations to support decision making.
CONCLUSION
This study confirms that the uptake of scientific evidence resulting from HTA is shaped by the various forces regulating the healthcare system. Furthermore, the integration of a given recommendation into practice depends on a conjunction of factors that is unique to the technology assessed. Ideally, these specific factors should be acknowledged when disseminating recommendations on a given health technology. This approach represents an amount of work that HTA agencies are unlikely to conduct alone.
HTA is a relatively new field of activity that is gaining recognition to support healthcare decision making. It is important, thus, to carry on further research on the factors influencing the uptake of HTA recommendations. Other studies of the impact of HTA recommendations are needed to extend the validation of the proposed theoretical framework and to develop an integrated method to assess HTA adoption into practice.
POLICY IMPLICATIONS
- Adoption of HTA recommendations depends on a conjunction of factors that is unique to the specific technology assessed. It is important, thus, to adapt dissemination strategies accordingly.
- Economic incentives, implication of users in HTA activities, as well as local adaptation of recommendations could provide effective strategies to increase the uptake of HTA guidance.
- Multifaceted interventions that target various levels of decision making are likely to be the most successful.
- HTA agencies and decision makers at various levels (clinical, organizational, and political) should join efforts in the development of dissemination strategies to increase the uptake of HTA recommendations.
CONTACT INFORMATION
Marie-Pierre Gagnon, PhD (mpgagnon@ucalgary.ca), Postdoctoral Fellow, Health Telematics Unit, University of Calgary Health Sciences Centre, G-204, 3330 Hospital Drive NW, Calgary, Alberta, Canada T2N 4N1; Researcher, Unit of Evaluative Research, CHUQ–Hopital St-François d'Assise, 10, rue de l'Espinay, Quebec City, Quebec, Canada G1L 3L5
Emília Sánchez, MD, PhD (esanchez@aatrm.catsalut.net), Researcher, Joan M. V. Pons, MD (jpons@aatram.catsalut.net), Catalan Agency for Health Technology Assessment and Research, Parc Sanitari Pere Virgili, Edifici Mestral, Esteve Terradas, 30, Barcelona, Spain 08023
This work was made possible with the help of CAHTAR researchers and staff, as well as the physicians, managers, and secretaries who have contributed their precious time to this study.
