No CrossRef data available.
Article contents
“The future should not take us by surprise”: Preparation of an early warning system in Denmark
Published online by Cambridge University Press: 01 August 2004
Abstract
Objectives: To explore and test methods for the operation of a national Early Warning System (EWS) in Denmark and to support decision making by the Danish Centre for Evaluation and Health Technology Assessment on this issue.
Methods: On the basis of literature reviews, information from members of EuroScan, and supported by clinical experts and stakeholders, existing methods were adapted and new methods were developed as part of a feasibility study.
Results: Approximately 200 technologies in 30 specialties were identified on the basis of information by EuroScan. A new instrument was developed to distinguish between important and unimportant technologies (filtering). Clinical experts in six specialties applied the instrument to sixty-two technologies in their respective fields, of which nine (15%) were judged potentially important for the Danish health care system. For priority setting, adapting a Dutch instrument to the Danish context was discussed. In principle, the instrument was acceptable, but several changes were proposed, for example, relating to the decentralized structure of the Danish health care system. For early assessment, the format and methods applied by SBU and Canadian Coordinating Office for Health Technology Assessment (CCOHTA) were compared and applied to pharmaceuticals (glitazones in treatment of type 2 diabetes mellitus) and a procedure (embolization of uterine fibromas). Given the main target group of the Danish EWS, local decision makers, the CCOHTA format was preferred.
Conclusions: The findings of the study have laid the foundation for an EWS using appropriate methods adapted to local circumstances. On the basis of the findings, a decision was made to start an EWS.
Information
- Type
- GENERAL ESSAYS
- Information
- International Journal of Technology Assessment in Health Care , Volume 20 , Issue 3 , August 2004 , pp. 342 - 350
- Copyright
- © 2004 Cambridge University Press
With a history of more than 20 years, assessment of health technologies has become an integrated activity in several health care systems in the industrialized world. The benefits of assessing health technologies to support decision making in policy and practice have been demonstrated in many cases, illustrated by a survey published by the International Network of Agencies for Health Technology Assessment (INAHTA) (11).
Historically, most assessments have focused on technologies that are in relatively widespread use. In the 1990s, however, it has increasingly been recognized that acting proactively, including early identification and timely assessment of new and emerging health technologies, would have considerable advantages over the present situation, where decision makers are often caught by surprise. In general, early notice will allow decision makers time to consider possible approaches to handling the technology within a health care system (9). In particular, early identification could result in improved management of subsequent evaluation and, hence, of the technology's introduction into clinical practice (19). To realize the potential benefits, Banta and Gelijns (1) stated that a systematic approach for early identification is needed, organized in an Early Warning System (EWS).
The objective of an EWS has been formulated as to help prevent the undesirable consequences of the unorganized, haphazard introduction of high-impact technologies and to promote the adoption of beneficial and cost-effective technologies (18). The concepts of early identification and early assessment have gained considerable support, and currently around ten countries have established early warning systems or horizon scanning centres. The respective agencies are collaborating in EuroScan, an international network for the exchange of information and evaluation of new health technologies (8).
In Denmark, health technology assessment has developed strongly in recent years, after a national strategy for Health Technology Assessment (HTA) was issued in 1996 (20). A national HTA institute was established in 1997.
Subsequently, Jørgensen and Larsen (12) carried out a first feasibility study exploring the necessary input sources and the demand for the output of a system for early identification of new health technologies in Denmark. However, the “processes in between,” the filtering, prioritizing, and assessment processes, were left unstudied at that time.
In 1999, the Danish Centre for Evaluation and HTA (DACEHTA) decided to commission a second, one-year feasibility study to explore the practicalities of operating a national Early Warning System in Denmark. Considering the increasing interest in obtaining advance notice on the consequences of emerging and new health technologies for the (Danish) health care system, in particular in the hospital sector (10), the time seemed right to follow up on one of the objectives of the National Strategy for HTA, namely to establish an Early Warning System in Denmark (20).
OBJECTIVE
The objectives of the feasibility study were to explore and test methods for operation of an EWS and to support decision making by DACEHTA on the establishment of a permanent Early Warning System. The study focussed on the following:
- The set-up of an initial organizational and advisory structure
- The exploration and testing of steps involved in systematically identifying and assessing new health technologies
- The establishment of a network of clinical experts to support identification and selection of significant new health technologies
METHODS
Organization and Content
The project organization consisted of a university-based research team supported by a secretariat at DACEHTA. In addition to monitoring progress, the secretariat served as a bridge to an advisory committee, including fifteen stakeholders at the administrative/political level in the Danish health care system representing, among others, health care policy and administration, industry, and medical societies. This structure was considered to allow potential gaps between research and policy making to be bridged. The establishment and selection of the members of the advisory committee was based on the conviction that as many stakeholders as possible should be stimulated to participate in the study, to arrive at an EWS that would both be practical and relevant, while being firmly rooted in the health care system. Four steps in the process of early warning were studied: Identification of new health technologies, filtering, priority setting for assessment, and early assessment. In four joint meetings of the research team, the secretariat and the advisory committee, the latter committee acted as a “sparring partner” for the research team, discussing the material that the research team provided, and advising on subsequent steps in the process. The methods used in each of these steps are summarized in Table 1.
For the step of identification, a letter requesting information on the names of the technologies, their application, and the estimated time-horizon for the introduction of the technologies was sent to all EuroScan members. The objective of collecting information from EuroScan was to test the feasibility of international exchange of information on new health technologies, and the usefulness of the information for a quick start of an Early Warning System. The resulting list of technologies served as an input for the step of filtering.
In Denmark, the process of priority setting is described as a two-phase procedure (13). The first phase is called filtering and is a necessary step in the selection process of relevant technologies when you actively scan information sources to identify new health technologies. The number of technologies that you find will be too high to include them all for further study. Filtering includes a rough selection of emerging technologies in either a category of technologies that are not addressed further (at least for the time being) and a category of technologies that are defined as, at least in principle, in need of assessment.
As there are no publications on the methodology of filtering, a new instrument had to be developed. The instrument should be brief, as filtering is intended to be a quick process and should facilitate the involvement of clinical experts, as experts are considered a valuable resource in this process (17). Taking into account the goals of the EWS, the draft instrument for filtering addressed the novelty of the technology for Denmark, the anticipated time-horizon of the introduction of the technology in the health care system, and the potential impact of the technology on the health care system.
To clearly distinguish between filtering and priority setting, the priority setting process should be applied to those technologies that have been categorized as relevant on the basis of the filtering procedure. All Early Warning agencies use some way of prioritizing new health technologies for assessment, but none of these have been explicitly reported in the literature. Methods for priority setting in HTA but not specifically in Early Warning have been reported internationally, but these have not always been applied in practice (2;6;7;14;17). One of the few instruments for priority setting, and one with which practical experience has been obtained, is a Dutch instrument for priority setting in HTA. This instrument has been developed to assist in a formal procedure for selecting research proposals in HTA for funding (16). A survey was conducted among the members of the advisory committee to adapt the Dutch instrument to the Danish context. The survey covered the instrument's perspective, the nature and content of criteria, the methods of measuring, scoring, and weighing criteria, and the decision rules to arrive at an overall judgment on the policy relevance of a new health technology for assessment in terms of either low, medium, or high relevance. The original instrument is presented in Table 2.
An early assessment summarizes the best available evidence to estimate the clinical, organizational, financial, and social/ethical consequences of a particular technology. To explore this step, two different methods and formats for arriving at an early assessment were compared and applied to two technologies, selected by the advisory committee: (1) the use of two new glitazones in treatment of diabetes type 2 (rosiglitazone and pioglitazone), and (2) embolization of uterine fibromas. These topics were selected because of their perceived relevance and because it was judged important, as a start, to become familiar with early assessment of both drugs and procedures. On the basis of differences in methodology and format, the early assessments of the Swedish Council for Technology Assessment in Health Care (SBU Alert) and the Canadian Emerging Technology Program (CETAP) at the Canadian Coordinating Office for HTA (CCOHTA) in Canada were selected as templates for two early assessments in our study.
RESULTS
Identification
Information on more than 200 technologies, representing approximately thirty specialties and clinical areas, was obtained from individual EuroScan members. On average, these technologies were anticipated to become available within zero to five years. Forty-five percent of the technologies were categorized as procedures, 42% as pharmaceuticals, and 13% as medical devices. The highest number of new technologies was found in the clinical specialty of oncology (n=29). The list of technologies was categorized by members of the advisory committee in accordance with the structure and boundaries of Danish clinical specialties.
Filtering
Figure 1 presents the filtering instrument. The newly developed instrument was pilot tested and finalized on the basis of discussions with the advisory committee. The advisory committee also facilitated the involvement of clinical experts in the filtering process, by identifying leading clinical experts in six areas: oncology (n=8), rheumatology (n=4), neurology (n=4), immunology (n=2), medical imaging (n=4), and robotsurgery (n=1). The instrument was applied in a survey among the clinical experts (n=23) to filter sixty-two technologies in the above-mentioned clinical areas. Fifty-nine percent of the technologies was categorized as pharmaceuticals, 23% as medical devices, and 18% as procedures. Technologies were categorized as “potentially important” when a majority of the experts judged a technology to be either new or experimentally used in Denmark, to be introduced in 0–1 or 1–3 years in the Danish health care system, and to have a large impact on at least one dimension (clinical, organizational, financial, ethical/social). The response rate of the survey was 78%. Of the sixty-two technologies, nine (15%) were judged by the experts to be of importance for the Danish health care system (see Table 3). Table 3 shows that the selected technologies represent only three of the six specialties, with strong emphasis on new pharmaceuticals in the specialties of oncology and rheumatology.
Example of the instrument used for filtering new health technologies.
Priority Setting for Early Assessment
The adaptation of a priority setting instrument is highly dependent on the objective of the involved organization and the actors in the process (16). The objective of a priority setting instrument in an EWS is to support the selection of a technology for early assessment. The outcome of the survey on the adaptation of the Dutch instrument to the Danish EW setting, led to the following suggestions for adaptation:
Perspective of the Instrument
It was suggested that, in principle, the instrument should enable to set priorities from both a societal perspective and a health care perspective. Because there can only be one leading perspective, it was decided that this issue will be elaborated on in the further development of the instrument.
Selection, Weighting, and Scoring of Criteria
It was suggested to accept all six criteria for priority setting in an EWS, as presented in the original instrument (see Table 2) and to incorporate organisational and educational aspects related to the use of a new health technology in criterion F: “Additional aspects.” All criteria should receive equal weights, suggesting that all criteria are equally important, and a simple additive procedure should be used to score criteria, resulting in a minimum score of 6 (if all six criteria were assigned a score of 1 and a maximum score of 30 (if all six criteria were assigned a score of 5).
Cut-off Points of Total Scores to Determine Overall Policy Relevance (Low, Medium, High)
It was considered that the number of prioritized health technologies that actually can be assessed depends on the available financial and personal resources of an EWS. When resources are scarce, only those technologies with the highest absolute score will be assessed. When resources are abundant, one may select technologies for assessment on the basis of a score that is higher than a predefined minimum score (a cut-off point) for high policy relevance. Given uncertainty on how many resources will be available for the permanent EWS, no such cut-off points were defined. Furthermore, the advisory committee judged it would be more appropriate to define cut-off points after the instrument would be applied to several health technologies in practice.
Assessment of New Health Care Technologies
Table 4 presents a summary comparison of the methods and format for early assessments of SBU Alert and CETAP. Table 4 shows that, in the SBU Alert approach, clinical experts write the first version of an assessment, with the SBU Alert staff in a stimulating and supporting role. An external reviewer, supposedly with a slightly different perspective, reviews the assessment (Prof. Dr. P. Carlsson, personal communication). The CETAP method for early assessment is predominantly based on literature review by a researcher associated with the EWS. Selected members of the CETAP scientific advisory panel and, when appropriate, clinical experts review a draft assessment.
In this study, the SBU Alert approach was applied to the case of embolization of uterine fibromas, whereas the CETAP approach was applied to the case of two new glitazones in treatment of type 2 diabetes patients (rosiglitazone and pioglitazone). The assessments served as an input for a discussion with the advisory committee on the preferred format and method for early assessment in the Danish EWS. Additional input was formed by a randomly selected example of an early assessment as performed by SBU Alert, “Acupuncture treatment in stroke”(21), and an assessment of the first type of glitazones (troglitazone) made earlier by CETAP (4).
The advisory committee favored the CETAP format over the SBU Alert format for assessment because they judged the CETAP format to be more concise, while containing all relevant information. In addition, the advisory committee believed that the CETAP format is easier to read for a nonmedical audience and, thus, better suited for policy makers, which are regarded as the prime target group for the Danish EWS. In contrast, the SBU Alert format was judged to be rather detailed, in particular regarding clinical aspects, which may require more insight in medicine than the average decision maker may have. The advisory committee also endorsed the consultation of experts and reviewers in the process of assessment. As a consequence of the advisory committee's preference, the assessment on embolization of uterine fibromas was revised to fit the CETAP format. This assessment has been disseminated, whereas the assessment of glitazones was included in the project report (5).
DISCUSSION AND CONCLUSION
The study demonstrated that it is possible to develop a systematic and transparent process of identifying, selecting, and assessing new health technologies. Furthermore, a wide variety of stakeholders in Danish health care demonstrated an interest to participate in this process, as represented by the advisory committee. In particular, this latter outcome was considered to be important by the commissioning agency, because of the necessity of such support for a future system. Combined with recommendations to the commissioning agency on the further development of methods and operations of the system (5), this has led to the decision to establish a Danish EWS.
To some extent, the study could build on existing methods, incorporating the state-of-the-art knowledge of other EWSs. In other steps of the process, in particular related to filtering and priority setting, the research team had to break new ground. The appropriateness of the selected methods for a local application was determined with the support of a variety of stakeholders in the Danish health care system, representing both the input (industry and clinicians), and the output side (health care decision makers on all levels) of an EWS. The intense, structured involvement of the main stakeholders is considered as a strength for the implementation of the methods in a permanent EWS, and ultimately for the impact of the EWS on policy and practice.
In addition, it is considered to be a benefit that the study could focus on methods in a situation without any pressure to produce immediate output. The focus of activities can now shift to producing high quality output, with associated dissemination and implementation strategies, while building on the experience gained in the previous steps. What does this shift suggest for each of the steps that have been addressed?
Identification of New Health Technologies
When a quick start is needed, it has shown to be effective and efficient to collect information on emerging technologies from existing Early Warning Agencies. Information collection has become more efficient now that a common database containing information on new and emerging health technologies has been developed by EuroScan (8). But for a permanent EWS, this one source is not enough. Robert et al. (18) stated in their review that “a combination of sources is required in order to ensure that all types of technologies and all important technologies are identified. Using more than one source will provide corroboration, increase the likely accuracy of any predictions and increase the amount of useful information regarding a new technology.” This conclusion was also reached by Jørgensen and Larsen (12). This finding suggests that, preferably, in any EWS, more information sources will have to be added to the sources of expert consultation and information from other EW agencies, which constituted the two main sources in the present study. One can, for example, think of using Internet sources (22).
Filtering
The use of the prestructured filtering instrument by the clinical experts proved useful and will, therefore, be continued. The instrument was easy to understand for the experts. In turn, the answers of the experts allowed for meaningful interpretation by the research team, resulting in a rough selection of technologies in two categories. The “potentially important” technologies will be subject to priority setting for early assessment. The technologies that were deemed unimportant will be monitored.
The involvement of clinical experts in clinical specialties other than those selected for this study will be actively pursued. This strategy will facilitate integration of the EWS in the health care system and will allow to study the validity and reliability of the filtering instrument on the basis of a bigger and more representative sample of experts. For example, when consulting more than one expert on a single technology, the possibility of disagreement needs to be taken into consideration. Knowing this, one can either strive for consensus between the experts' opinions or devise a decision rule for the analysis of the responses. In this study, a majority rule was used, that is, when more than 50% of the experts agreed on an option this was considered to hold true for the technology. The outcome of the filtering process, thus, is heavily dependent on the number of experts in a clinical specialty. Our approach, therefore, may have to be balanced against a more labor-intensive consensus approach.
Priority Setting for Early Assessment
Denmark has a decentralized health care system, where the counties are responsible for both financing the services and decisions on the service level (3). This approach allows local preferences and priorities to play a role, which should be facilitated by a corresponding health care perspective in the priority setting instrument. The Ministry of Health is the most important centralized authority with, among others, the task to formulate overall policy goals. It was believed that the priority setting instrument, in principle, should allow for a societal perspective at this level. This recommendation could be translated to the instrument by extending the health care costs associated with a technology (criterion D) to costs borne by the patients and relatives, and costs in other sectors, for example, related to production losses.
In general, transnational and cross-setting adaptation of an instrument for priority setting of assessment seems feasible on the basis of the experience in the study, but it is complicated to arrive at an instrument that is easy in use, acceptable to a wide range of stakeholders, and builds on a sound theoretical basis. For example, any new criteria to be added to the instrument should allow for clear-cut operationalization. In addition, it should be recognized that, in the case of new and emerging technologies at this early stage of the life cycle a technology, only limited information is available that can be used to score each criterion and that the information is usually associated with substantial uncertainty. For example, an initial judgment on the societal costs of a new health technology may be extremely difficult to arrive at this stage. The requirement for data to serve as an input for the priority setting instrument, therefore, should first and foremost be based on the consideration that the data contribute to reduce uncertainty. In that case, the instrument will be of added value compared to a subjective decision whether or not to select an emerging technology for assessment. Although the priority setting instrument has become more definitive by the adaptation process, future experience with applying the instrument in the early warning process will have to define the final characteristics.
Early Assessment
The main output of an EWS is the brief reports or series of assessments of new health technologies. Hailey et al. (9) state that individual jurisdictions of health care systems may have particular requirements for the type of information provided. Also, the way early assessments are produced differs between EWSs. In this feasibility study, both format and methods of arriving at an early assessment were subject to a first exploration.
To secure the methodological quality of future assessments, procedures have to be developed to achieve standardization in the assessment method. Standard search strategies and definition of levels of evidence for applying this information have to be developed for early assessment of drugs, procedures and medical devices. There will be a role for reviewers and experts in the procedure of early assessment. It might be a problem, however, in a relatively small country as Denmark, to find independent expert reviewers. A possibility could be to select reviewers in neighboring Scandinavian countries.
Evaluating the process of the study, several features emerged as particularly important. These are as follows:
- The EuroScan database with common descriptions on new health technologies is an indispensable source of input for an efficient start of an Early Warning System.
- Clinical experts are more valuable as a filter for information on new health technologies that is supplied to them, than as a primary source for information on new health technologies.
- For the process of adaptation of a priority setting instrument to a certain context, one needs to provide practical examples to clarify how the instrument works.
- The process of making an early assessment takes longer and the assessment includes more complex language when a clinical expert is in charge than when staff of the EWS write the assessment.
And in general:
- As the methodology for early warning of new health technologies is in its infancy, the development of an appropriate methodology to be applied by Early Warning Systems deserves the attention of the international HTA research community.
POLICY IMPLICATIONS
Following up on the recommendation of Hailey et al. (9) might be the way to take matters further. These authors state that “integration of this horizon scanning process within the health care planning process will be needed if such a program is to realize its full potential.” One issue related with integration is the positioning of the EWS in the health care system. In Denmark, it has been decided to locate the EWS close to both the policy and HTA process, that is at DACEHTA. Further integration can be achieved by operationalizing and formalizing the relation with each of the stakeholders. For example with the industry, to ensure firsthand information on emerging technologies. And with the Danish regions to be able to provide them with adequate information. In the end, this will be helpful to promote the adoption of beneficial and cost-effective technologies, which can be regarded as the major policy implication of an EWS.
We thank the members of the advisory committee and Finn Børlum Kristensen and Mette Lange of the Danish Centre for Evaluation and Health Technology Assessment for their support of the study. Furthermore, we thank the members of EuroScan for providing input to our study. The authors are indebted to David Banta for his comments on a previous version of this article, and to Wija Oortwijn for advice on the issue of priority setting. Note: At the time of the study K. Douw was affiliated with the Centre for Applied Health Services and Technology Assessment (CAST), University of Southern Denmark, and T. Jørgensen was affiliated as external Associate Professor at the Institute for Public Health – Health Economics, University of Southern Denmark and Odense University Hospital.
References
Banta HD,
Gelijns AC.
1994
The future and health care technology: Implications of a system for early identification.
World Health Stat Q.
47:
140–
48.Google Scholar
Buxton M,
Hanney S.
1996
How can payback from health services research be assessed?
J Health Serv Res Policy.
1:
35–
43.Google Scholar
Christiansen T.
2002
Organization and financing of the Danish health care system.
Health Policy.
59:
107–
118.Google Scholar
Canadian Coordinating Office for Health Technology Assessment (CCOHTA).
Canadian Emerging Technology Program (CETAP).
Issues in Emerging Health Technologies. Troglitazone for Type II Diabetes
(revised January 1998). Available at: http://www.ccohta.ca/newweb/etech.asp. Accessed March 11, 2002.
Danish Institute for Health Technology Assessment.
2001.
Etablering af et dansk system til tidlig varsling af nye medicinske teknologier. Rapport fra et pilotprojekt.
(Establishment of a Danish system for early warning of new technologies. Report of a pilot project).
Copenhagen:
P.J. Schmidt A/S
Donaldson MS,
Sox HC. eds.
1992.
Setting priorities for health technology assessment: A model process.
Washington, DC:
National Academy Press
Eddy DM.
1989
Selecting technologies for assessment.
Int J Technol Assess Health Care.
5:
485–
501.Google Scholar
EuroScan.
The European Information Network on New and Changing Health Technologies.
http://www.publichealth.bham.ac.uk/euroscan/. Accessed March 11, 2002.
Hailey D,
Topfer L-A,
Wills F.
2001
Providing information on emerging health technologies to provincial decision makers: A pilot project.
Health Policy.
58:
15–
26.Google Scholar
Haller J.
2000
Fremskridt må ikke komme bag på os (The future should not take us by surprise).
Ugeskrift Læger.
162:
812–
814
(in Danish).Google Scholar
International Network of Agencies for Health Technology Assessment (INAHTA).
Impact of HTA on Policy: Examples from INAHTA members.
INAHTA Newsletter, nr. 4, 2002. Available at: http://www.inahta.org/. Accessed March 25, 2002.
Jørgensen T,
Larsen GL.
1998
Basis for decisions on emerging health technology: A Danish feasibility study.
Int J Technol Assess Health Care.
14:
624–
635.Google Scholar
Jørgensen T.
1998.
Idé-oplæg: tidlig varsling af kommende medicinske teknologier
(Proposal: Early Warning of emerging health technologies). Institut for Sundhedsvæsen (DSI) (internal report, in Danish).
Copenhagen:
DSI
Lara ME,
Goodman C.
1990.
National priorities for the assessment of clinical conditions and medical technologies.
Washington, DC:
National Academy Press
Mowatt G,
Thomson M-A,
Grimshaw J,
Grant A.
1998
Implementing early warning messages on emerging health technologies.
Int J Technol Assess Health Care.
14:
663–
670.Google Scholar
Oortwijn WJ,
Vondeling H,
Van Barneveld T,
Van Vugt C,
Bouter LM.
2002
Priority setting for health technology assessment in The Netherlands: Principles and practice.
Health Policy.
62:
227–
242.Google Scholar
Rico R,
Asua J.
1996.
The prioritisation of evaluation topics of health.
Vitoria Gasteiz:
Osteba
Robert G,
Stevens A,
Gabbay J.
1999
‘Early warning systems’ for identifying new healthcare technologies.
Health Technol Assess.
3:
13.Google Scholar
Stevens A,
Robert G,
Gabbay J.
1997
Identifying new health care technologies in the United Kingdom.
Int J Technol Assess Health Care.
13:
59–
67.Google Scholar
The Danish National Board of Health.
1996.
National Strategy for Health Technology Assessment.
Copenhagen:
Danish National Board of Health
The Swedish Council for Technology Assessment in Health Care (SBU).
SBU Alert.
Early assessment of new health technologies. Acupuncture treatment for stroke.
Available at: http://www.sbu.se/admin/index.asp. Accessed March 11, 2002.
Trindade E,
Topfer L-A,
De Giusti M.
1998
Internet information sources for the identification of emerging health technologies. A starting point.
Int J Technol Assess Health Care.
14:
644–
651.Google Scholar
Methods Applied in the Study to Explore the Practical Operation of an Early Warning System in Denmark
Definition and Scoring of Policy Criteria Used by the Dutch Health Care Insurance Board in Setting Priorities for HTA
Example of the instrument used for filtering new health technologies.
Results of the Filtering Process of Sixty-two Technologies by Twenty-three Experts in Six Specialtiesa
Comparison of the Method and Format for Early Assessments Used by SBU Alert at the Swedish Council on Technology Assessment in Health Care (SBU) and the Canadian Emerging Technology Assessment Program (CETAP) at the Canadian Coordinating Office for HTA (CCOHTA)a
You have
Access
- 7
- Cited by