Identifying new procedures which are being introduced into health services is difficult. Unlike new devices, there is generally no system of regulation for them. Routine coding is typically not sufficiently nimble or specific to monitor new procedures early in their use. Widespread adoption of new procedures in the absence of an adequate evidence base not only poses the risk of potential harm to patients: it may also limit their access to new interventions which offer greater benefits than established practice. In addition, inappropriate dissemination of procedures can be costly for health services, in terms of purchase of equipment, time spent training, and changes to care pathways.
The late 1990s saw sporadic attempts to register new procedures (Reference Campbell and Maddern1;Reference Babidge and Maddern2). Simultaneously, concerns were publicized about adverse outcomes from inappropriate use of procedures, for example in the areas of laparoscopic surgery and pediatric cardiac surgery (3). These concerns led, in the United Kingdom, to the establishment of the Interventional Procedures (IP) Programme within the National Institute for Health and Clinical Excellence (NICE) (Reference Campbell and Maddern1).
The IP Programme assesses new procedures for their safety and efficacy, using published evidence; input from specialist clinicians and from patients; and the knowledge of a wide range of medical and nonmedical members of an advisory committee which drafts guidance for publication (4;5). Following a period of public consultation, draft guidance is reviewed by the committee, amended as necessary, and then published by NICE for all the UK National Health Services (NHS). Since 2002, when NICE IP work started, 503 guidances have been published (including seventy-four reviews) (6). A variety of systems have developed in other parts of the world for evaluating new procedures and producing differing types of guidance or statements about their use (Reference Babidge and Maddern2;Reference Maddern, Boult, Ahern and Babidge7–Reference Mad, Geiger-Gritsch, Hinterreiter, Mathis-Edenhofer and Wild11).
The assessment of new procedures depends fundamentally on finding out about procedures at the time that clinicians first want to start using them. At its inception, the NICE IP program had access to a previous UK register but it needed to create a system to identify procedures coming into use in the future (Reference Campbell and Maddern1). This study analyzes experience at NICE in identifying new procedures entering the United Kingdom (UK) healthcare system, for assessment and publication of recommendations on their use. It describes the notification system which was developed; reviews the outcomes of the first 11 years of its use; and addresses current and future challenges.
METHODS
NICE Notification System and Evaluation System Processes
Notification of procedures to the NICE IP program is by means of a Web portal (12). Procedures may be notified by anyone, but there was a mandate for clinicians to notify whenever they intend to undertake a procedure which they have not done before (unless it is a well established procedure which, for some reason, they did not undertake during their years of training) (13). This NHS mandate for clinicians to notify was also recognized by private health insurers and private hospitals.
Steps taken to draw the attention of the clinical community and the health service in general to the requirement to notify procedures have included: (a) A directive Health Service Circular was sent to all Chief Executives of NHS hospitals and to a wide range of others, including Medical and Nursing Directors and relevant national agencies (12). (b) Information about the need to notify was sent to all professional medical organizations whose members might be involved in undertaking interventional procedures. Reminders have been sent to the leaders of professional organizations at intervals subsequently (every 1–2 years). The number of organizations contacted has increased as additional relevant ones have been identified and as new specialist societies have been formed. (c) NICE has had direct contact with clinicians in all specialties, about notifying procedures. (d) Manufacturers of medical technologies and patient groups have been informed about the IP program through a variety of channels. (e) The IP team undertakes continual opportunistic surveillance for new procedures, by means of reports in the media, journal scanning and prompts from specialist clinical advisers and advisory committee members. These groups of clinicians are particularly familiar with the need to identify and review procedures and are a special source of information about procedures which may need evaluation or review. (f) There is liaison with a national horizon scanning centre for information about procedures which may be relevant in the future.
However, and by whomever a procedure is identified, the NICE team ensures that it is formally notified by means of the NICE Web portal—either by prompting a specialist to notify it or by notifying by means of the Web portal themselves.
Information Requirements at Notification
Person notifying: Title, name, role (profession) and email address. Procedure: Name, indication. Description of what the procedure involves. References of any published papers/abstracts. Device: Details of any device involved and the manufacturer. Current use: Details of where in the NHS (or private sector) the procedure is in use (including research use). Proposed use: Details of any clinicians known to be interested in using the procedure (if none is already doing so). Comparator procedures. Declarations of any conflicts of interest.
When a procedure has been notified, consideration is then given to whether it is appropriate to be assessed to produce guidance for the health service. Fundamentally, the procedure must fall within the remit of the Program.
To Fall within the Program's Remit, a Notified Procedure Must
To fall within the program's remit, a notified procedure must: (a) involve an incision or a puncture or entry into a body cavity, or the use of ionizing, electromagnetic or acoustic energy; and (b) be available within the NHS or independent (private healthcare) sector, or be about to be used for the first time, outside formal research; and (c) either not yet be generally considered standard clinical practice; or (d) be a standard clinical procedure, the efficacy or safety of which has been called into question by new information or advice.
For procedures that fall within the Program's remit, the decision about whether or not to proceed to a full assessment may then be straightforward. For example a procedure which is clearly novel, and which the notifier has stated is starting to be used in the NHS, would be selected for assessment; but a procedure which is the same as one that has already been notified (typically by a different name) would not be selected. Important questions include: (a) Is the procedure a minor modification of an established procedure, which is most unlikely to make any difference to its safety or efficacy; or is it a more major modification, which might influence safety or efficacy? (b) If a device is integral to performing the procedure, has that device a CE mark which allows it to be marketed throughout Europe for the proposed indication? If it has not got a CE mark for the indication which has been notified then it cannot be assessed by the NICE. (c) If the procedure involves the use of a drug or other active agent, is its novel aspect related to the action of the drug/agent (not appropriate for assessment for NICE Interventional Procedures guidance) or to the nature of the procedure used to introduce it into the body (appropriate for assessment)? (d) Is the procedure in use in the NHS or is there an intention to introduce it into NHS use in the near future, outside a research setting? If there is no current or intended NHS use in the UK, then evaluation for guidance is not appropriate. If the procedure is only being done in the context of research (and there is no current intention to use it outside research) then patients are protected by research governance and producing guidance would serve no useful purpose.
Addressing the questions above often involves communication with specialist clinicians, nominated by their professional organizations (“specialist advisers”): their advice is also sought about miscellaneous other queries which arise during the process of deciding whether to select a notified procedure for evaluation (Reference Lyratzopoulos, Hoy, Veeramootoo, Shanmuganathan and Campbell14).
When selected, a procedure is subject to a process of evaluation which has been described elsewhere (4;5). In summary, an independent advisory committee considers published evidence, the opinions of specialist advisers and patient commentaries. Its draft guidance is subject to public consultation and review. Guidance is then published by NICE with four main types of recommendation.
The four main recommendations made by Interventional Procedures guidance. (reflecting the evidence on the efficacy and safety of procedures) are as follows.
Use with normal arrangements for clinical governance, consent and audit: The evidence is sufficient to show that the procedure works well enough and is safe enough for surgeons or other clinicians to use as part of their normal practice, with the usual local policies for clinical governance, patient consent, and audit. Use with special arrangements for clinical governance, consent and audit or research: This means “tell your hospital; tell your patients; and audit your results with special care.” Hospitals need to ensure that their facilities and risk management arrangements are adequate. There is a greater need for explicit information for patients regarding the uncertainties about the safety and/or efficacy of the procedure as part of obtaining their consent. Follow-up and critical review of outcomes in especially important. Use only in research: The procedure is considered to be experimental and/or particular uncertainties need to be resolved before more supportive guidance can be developed. The procedure should only be done in the context of formal research studies, approved by a Research Ethics Committee (these may take the form of randomized controlled trials, but other designs, such as well-planned prospective observational studies may be appropriate). Guidance specifies the most important outcomes which need to be elucidated. Do not use: The evidence suggests that the procedure is not effective, and/or there are unacceptable safety risks.
Study Period and Outcomes Measures
For the purposes of this study, records from the NICE Web site were analyzed for: (a) Numbers of notifications from the start of the program in April 2002 to the end of 2012. These were subdivided by calendar year (2002 being an incomplete year). (b) Source of notification: clinician/hospital/manufacturer/patient/horizon scanning centre/private insurer/other. There are two important points to note. First, one source of notifications was a pre-existing register (the Safety and Efficacy Register for New Interventional Procedures) which started in the United Kingdom in 1996 ceased its activity in 2000 (Reference Campbell and Maddern1). NICE was directed to consider all the procedures from that register which had been judged by its panel of assessors not to have sufficient evidence of safety or efficacy. Second, if the NICE IP team identifies a procedure and considers it appropriate, then may then ask one of the specialist clinicians whom it has approached for advice to perform the notification or may itself notify the procedure. (c) Specialty most closely associated with the notified procedure. The classification of specialities was largely based on a UK NHS system of specialty categories. The “General Surgery” category included all gastrointestinal surgery, breast and endocrine surgery. The allocation of procedures to specialties was done by consensus of the clinical members of the team (BC and HP). This was not always straightforward: where more than one specialty was involved in doing the procedure a decision was made about which specialty was likely to be undertaking it most frequently. (d) Whether or not notified procedures were selected for assessment, leading to publication of guidance for the UK health services. (e) The main recommendation of the guidance, based on evidence of the procedure's efficacy and safety.
Statistical Analysis
Statistical analysis for any trend in the numbers of procedures notified annually during the years 2003–2012 was done by Pearson's correlation. The data were presented as frequencies and percentages. The year 2002 was excluded from this analysis because it included the substantial number of procedures inherited from a pre-existing register.
Small International Survey (Reference Latif15)
In addition, a small international survey was done, to find out how HTA organizations around the world identify and select procedures for assessment. A short questionnaire was sent to twenty-seven organizations in twenty-seven countries asking three main questions: (a) What are the sources of the procedures which you assess? (b) What is your main product of assessment of procedures? (c) What considerations influence your decision about whether or not to take a procedure for full review?
RESULTS
A total of 1,094 notifications (seventy-nine of which were duplicates) were identified. The numbers of notifications year by year are shown in Figure 1. Analysis of the numbers of procedures notified annually during the years 2003–2012 showed a significant decrease (p = .049, Pearson's correlation test)
The total number of procedures notified by calendar year, in period 2012-2012*.
* 2002 was not a full year and it included 140 procedures inherited from a pre-existing register.
Table 1 shows the sources of notifications (type of person or organization). Clinicians outnumbered all other sources: they included individual clinicians and clinicians who notified procedures on behalf of their specialist organizations. Notifications from “hospitals” were primarily from staff involved in clinical governance activities. Those from “NICE IP team” were for procedures which had come to the attention of the team by means of the media and other sources and which were judged to be worthy of consideration for the production of guidance. “Unknown” represents notifications for which the notifier had not completed this field.
Sources of Notifications and Percentage for Which Guidance Was Published
aThe figures in this column include those for which the source of notification was not specified (“Unknown”). They also include the procedures inherited from SERNIP (a pre-existing register) for which the source of the notifications was also unknown.
Table 2 lists the clinical specialties most closely associated with the notified procedures. Surgical specialties ranked highest among notifiers, with general surgery, urology and orthopedics being the top three, accounting for 328 (30 percent) of all notifications. Interventional radiology and Cardiology were next in frequency. Overall, 44 percent of notifications went on to published guidance: specialties with a noticeably high proportion were cardiothoracic surgery (72 percent), ophthalmology (59 percent), and orthopedics (58 percent).
Clinical Specialties with Which Notified Procedures Were Most Closely Associated, in Descending Order of Frequency.
Table 2 shows the numbers of notifications for each specialty by year. The numbers were too small for any kind of meaningful statistical analysis for trends, but there were no obvious or remarkable deviations. In particular there was no suggestion that certain specialties had been unaware of the need to notify during one part of the study period.
The main recommendations for the 376 procedures with current published guidance (at February 26, 2013) were for “normal arrangements” in 176 (47 percent), “special arrangements” in 153 (41 percent), use in research only in twenty-five (7 percent), “do not use”` in five (2 percent), and “other” in sixteen (4 percent) (the latter had complex recommendations for different patient groups).
The small international survey yielded twenty responses (74 percent response rate). The most common sources of notifications to the organizations were “government bodies” (85 percent), “payers” (75 percent), and clinicians (65 percent). Half of organizations assess appropriateness of procedure from a funding/reimbursement perspective, some organizations produce assessments in different forms depending on the requirement (systematic reviews, HTAs, clinical guidance) and others provide options rather than recommendations to their target audience. Considerations that influence organizations to take on a procedure for full review are most commonly those of concerns about efficacy (85 percent), the involvement of a new device (80 percent) or safety concerns (75 percent).
DISCUSSION
This retrospective study shows that the system we have developed for identification of new procedures has yielded an average of over eighty notifications annually (excluding the procedures inherited from a pre-existing register in 2002). Over half were from clinicians and this degree of engagement with the clinical community is encouraging. It suggests a generally responsible approach of clinicians to the introduction of new procedures: but we cannot be certain whether some clinicians start using procedures without reference to NICE. This may occur in the private healthcare sector which is less directly related to the activity of NICE than the National Health Service (the latter delivers the great majority of healthcare in the United Kingdom).
Our “open” approach to notification (anyone is able to notify a procedure) was designed to maximize the opportunity of identifying all procedures which are entering practice in the UK. Sources other than clinicians accounted for a substantial proportion (49 percent) of the notifications. The wide variety of people and organizations who have notified testifies to the usefulness of having a portal which is open to all. The relevance of procedures notified from these different sources is supported by the fact that around half of the notifications from each source resulted in publication of guidance, including 65 percent of those notified by private health insurers and 40 percent of notifications from patients.
One of the reasons that we embarked on this study was a suspicion that the number of notifications was declining. Our findings have confirmed this impression, showing a trend that was just statistically significant. There are several possible reasons for this decrease. It may be that less procedures are being developed and introduced: the boom of new laparoscopic operations is past its peak and it may be that the surge in new endovascular technologies has now waned. Perhaps the awareness or inclination of clinicians to notify has decreased: experience has shown us that sudden “peaks” in notifications in particular specialties are often related to the activity of individual enthusiastic specialists who become associated with, or more aware of, the NICE IP Programme and who then proffer several procedures from their specialist area. The finding of decreasing notifications has led us to re-examine our communications with clinicians and their professional organizations, to try to ensure that all new procedures are notified as they enter clinical practice.
Designation of procedures to particular specialist areas is not always straightforward, especially when a procedure is done by more than one specialist group. Examples include procedures for female urinary incontinence which may be done by urologists or gynecologists (or by “urogynecologists”); and endovascular procedures which may be done by interventional radiologists or by vascular surgeons. In addition, specialists may be closely associated with procedures without actually doing them (for example stroke physicians or neurologists and procedures for stroke). For the purposes of our categorization, the specialty thought to be most often involved with doing each procedure was selected. The numbers of procedures from different specialist areas influences the choice of specialists on the NICE advisory committee. Specialties with substantial numbers of procedures being assessed are represented by permanent members on the committee while those with small numbers are not (invited specialists offer advice to the committee when required).
The categorization of specialties was one area in which we found that our established system had not worked well over the years: we established a new system and allocated specialties to all 1,094 procedures as part of this study. The categories we chose could be criticized in some respects—in particular the “general surgery” category, which was based on a UK system of specialist classification. Another limitation of this study was some missing data—in particular “unknown” sources of notifications: these had simply not been provided by the notifiers.
The numbers of notifications across the whole range of specialties provides evidence of a great diversity of innovation. The increased use of laparoscopy since the 1990s helps to explain the large numbers in general surgery and urology; development of endovascular technologies underpins many of the procedures in cardiology and vascular surgery; and new technologies also accounted for the substantial numbers in interventional radiology. However, many new procedures seemed to be related to the ingenuity of clinicians in developing new open surgical techniques, and to applying a wide range of existing or new technologies for a greater range of indications.
Our aim in identifying new procedures for assessment is to produce guidance which will be useful for health services, clinicians and patients. The fundamental purpose is to protect patients from harm while enabling them to receive new procedures which have evidence of benefit, in the best possible circumstances. This can be a difficult balance but our guidance offers recommendations about how it can be achieved, by appropriate means of patient selection, good facilities and adequate training and expertise of clinicians (6). It also encourages relevant data collection and research to add to the evidence base for future review. All this depends on identifying new procedures as comprehensively as possible and assessing them at about the time they start to be introduced into clinical practice. We continue to consider ways of improving our current system and maintaining a vibrant engagement with the clinical community—largely through liaison with professional organizations which represent clinicians throughout the United Kingdom.
We have previously published data on the practices in different countries for assessing and producing guidance on interventional procedures (Reference Plumb, Campbell and Lyratzopoulos8;Reference Plumb, Lyratzopoulos, Gallo and Campbell9). These data supported the belief that the same procedures present similar challenges to assessors and healthcare systems across the world (Reference Gallego, Casey, Norman and Goodall10). Our studies revealed that procedures were most commonly referred for assessment by government and professional organizations, or selected by the assessment organizations themselves, but most organizations also received referrals from other types of referring or mandating organizations. None seemed to have an “open” system like ours, to receive notifications from anyone, including patients and manufacturers. Our new survey, conducted as part of the present work, produced broadly similar findings, with government bodies and payers as the dominant sources of procedures and financial implications a priority (although safety and efficacy were important considerations). Two-thirds of other countries did describe notifications from clinicians, but none seemed to have the kind of permissive notification system used by NICE which aims to capture every new procedure entering clinical practice. Even with our “open” system we are uncertain about whether all procedures are notified and we would welcome consideration of regular international collaboration, to share information about procedures.
In conclusion, our “open” system of notification of new procedures has resulted in identification of large numbers of procedures from across the whole range of medical specialties. The fact that similar proportions of procedures notified from diverse sources have been selected for assessment and publication of practice recommendations suggests that this inclusive approach is worthwhile. We will continue to look critically at the ways in which we can best identify all procedures reliably when they are entering practice and we are currently conferring with organizations in other countries about their methods.
CONTACT INFORMATION
Bruce Campbell, MS, FRCP, FRCS, Chairman Interventional Procedures Advisory Committee, National Institute of Health and Care Excellence (NICE), 10 Spring Gardens, London SW1A 2BU, University of Exeter, St Luke.s campus, Heavitree Rd, Exeter EX1 2LU
Rebekah Morris, National Institute of Health and Care Excellence, University of Exeter
Lakshmi Mandava, MSc, National Institute of Health and Care Excellence, University of Exeter
Lakshmi Murthy, MSc, National Institute of Health and Care Excellence, University of Exeter
Helen Gallo, MSc, National Institute of Health and Care Excellence, University of Exeter
Koh Jun Ong, MPh, MSc, National Institute of Health and Care Excellence, University of Exeter
Ali Latif, MFPHM, National Institute of Health and Care Excellence, University of Exeter
Hannah Patrick, MRCP, FFPHM, (hannah.patrick@nice.org.uk), National Institute of Health and Care Excellence, University of Exeter
CONFLICTS OF INTEREST
All authors report that they have nothing to disclose.
