DISCUSSION

There is clearly no shortage of routine data collected in the UK's NHS. But it is not so clear that there is a shortage of good routine data, that is, data that can be used for such exacting purposes as HTA. Although we were able to identify over 270 health-care databases covering the United Kingdom as a whole, one of the constituent countries, or an English region, only thirty-three of these databases both record a health technology and a patient health state—the basic minimum for evaluating the effectiveness of health care. These thirty-three, of course, include some important databases such as the HES and the cancer registries, but worryingly, when put to our two tests, that is, whether they are ever used and whether they are ever usable, there were very few positive findings.

Does the very low use of routine data for such purposes just reflect an ignorance or an unjustified skepticism of routine data or does it reflect their real flaws? In fact, routine data are widely used in clinical audit, that is, measuring the performance of individual units. But this use is not always consistent and well-defined and may not always demand impeccable data. More often that not, the data are supplemented by additional information such as case note review. In the case of HTA, the data requirements are very tight. Indeed, the requirements of randomized control trials for very high quality data throws into sharp contrast the quality of routine data.

The flaws in routine data are five-fold. First, health-care systems have historically lived in an “information-free” culture. The scale and diversity of health-care systems have hitherto limited data collection to highly aggregated returns. Most health-care systems have some data on the number of people admitted to hospital or drugs prescribed, often (particularly in the United States) for reasons to do with payment, but few have data on the effects of treatment.

Second, there are clearly data collection problems. Whereas details of patients' conditions and treatments are noted, they generally do not become available as data for analysis. They are usually held in bulky case notes, usually still hand-written, specific to the place of treatment, difficult to interpret, and sometimes hard to locate.

Third, there is the problem of inadequate coding. The International Classification of Diseases (ICD) and the OPCS 4 Classification of interventions are in general use. It is acknowledged that they are limited not only by the quality of the underlying data but also by the lack of detail in most standard coding definitions. ICD does not cover disease severity, and in practice, many conditions are coded as unspecified, whereas procedure codes (such as OPCS) specify surgical interventions, detail is lacking on the specific technology used. Furthermore, standard classification systems are very poor at identifying new technologies. There is always a lag before new diseases and procedures, the very ones one might wish to evaluate, are recognized. For example, coronary stenting is not picked up as a technology distinct from percutaneous transluminal angioplasty, and even when eventually it is, it is unlikely that drug-eluting stents will be classed as a distinct technology. Furthermore, no systems exist at all for outpatients, for diagnostic and imaging technologies, or for the range of types of contact in primary and community health services of the organization of health-care delivery.

Fourth, confidentiality presents a new challenge. Data on the use of certain health services have long been deliberately anonymized (returns for sexually transmitted diseases, for example) to prevent stigmatization and encourage uptake. Now, increased legal protection of the privacy of individuals has raised major issues around all health-care records and the degree to which consent is required before data can be entered.

Fifth, there is a vicious circle of quality of data and their use. The perceived lack of value of most NHS administrative databases, which creates a barrier to their use and, therefore, creates a barrier to their improvement.

The coding problem in particular is highlighted by our findings on the “potential use” of routine databases. Very few of the UK's HTA priorities could be identified in routine data. However, we have no reason to feel confident that, had the technologies been coded, the level of patient detail (before and after use) would have been satisfactory in any but those relatively specialized clinical registries that carefully collect patient baseline condition and outcome, and in those where the outcome is relatively stark (e.g., a successful pregnancy, a death, or a population screening program uptake rate).

So, do routine databases hold out any hope for the deficit in information on the effectiveness of health care? Are we entirely limited to randomized trials? There are some signs of hope, but these findings may not be sufficient to change the current pattern of use. Some of the health technology databases we found, that is, the HFEA's data set and the National Transplant Database, for example, have found selective value. The Hospital Episode Statistics have been used to explore variations in clinical outcomes, even if seldom linked to specific interventions. Linked routine data in Scotland, when validated, notably as part of the West of Scotland Coronary Prevention Study Group prospective trial of statins, proved as useful as reporting based on direct patient contact (29). Routine data have also been central to analyses of inequity in health care, and routine data are clearly central to analyses of the diffusion of health technology. The use of routine data has also found some success overseas. In the United States, the patient outcome research teams (PORT) studies (30) used administrative claims data to assess the effectiveness of a range of common treatments, although they have been criticized (25) for potential bias. Sweden has over forty quality registers (20), which include patient specific data on diagnoses, medical interventions, and outcome. Denmark has developed standards for use of its biobank, defined as all health-related databases that include personal identifiers (28), and Iceland provides an example of an attempt to develop a single computerized database linking medical records, family trees, and assorted genetic information.

Information Technology, in health care as elsewhere, offers the potential for major change. Some services, such as intensive care units, are collecting routine data electronically with much greater levels of detail than hitherto. Computerization, unique patient identifiers and linkage between data sets offer the prospect of improvements. They make it increasingly possible to chart patients' complex itineraries through health-care systems, provided the appropriate detail has been captured and coded. Against this background, issues around confidentiality of health-care data are working to diminish their use in health services research (24). Issues of confidentiality, coding, and the use of routine data all need addressing urgently.