DISCUSSION

The basis for our study has been the need for reliable effectiveness and cost-effectiveness data to guide future decisions on investments in health care. Such data should be available with as little extra investment as possible and cover, on a routine basis, at least all the most common treatments the hospital provides. As our hospital, like most other hospitals, has long collected rather meticulous data on costs, all that is needed for the estimation of cost-effectiveness is an indicator of effectiveness of treatments. To allow comparison of different interventions across many clinical specialties, this indicator of effectiveness needs to be generic, that is, non–disease-specific.

For measuring effectiveness, we chose the 15D HRQoL instrument due to its many desirable properties. The 15D questionnaire can easily be filled in by the patients themselves. This ease is evidenced by the fact that the patients seldom contacted the project personnel to ask for help concerning the questionnaire, although contact details were readily available on the project information sheet. In some clinical specialties, HRQoL data were supplemented with disease-specific outcome data. However, preliminary analyses show that the disease-specific outcomes do not correlate especially well with HRQoL. We acknowledge that disease-specific outcomes are in many cases important for a given clinical specialty to monitor their treatment outcomes and to control the quality of the treatment given. Disease-specific outcomes, however, rarely are suited for the estimation of cost-effectiveness of treatments as they do not allow the comparison of different clinical specialties with each other. Furthermore, they rarely involve the viewpoint of the patient, that is, the patient's conception of whether a particular treatment has been helpful or not.

In addition to the specific disease and its treatment, HRQoL can be affected by other health problems and sickness experiences a patient may have. In most cases, these conditions will be reflected in the 15D score. However, in some cases more sophisticated analyses will be needed. These analyses can be based partly on unexpected or illogical changes in the 15D profile. Furthermore, our database also includes secondary diagnoses, which can be used to create a system that takes into account comorbidity by using validated comorbidity scores or other similar approaches.

Collection of effectiveness data proved to be fairly simple and did not require major investment. A single person can fairly easily handle the mailing and data collection provided that the mailing is at least partly automated using appropriate software. Both the health-care personnel and the patients appreciated the efforts to produce effectiveness data. This finding was reflected in that none of the specialties refused the offer to participate in our project, even though the distribution of the first questionnaires did in some cases cause a little extra work for those sending out the invitations to the patients. Staff of some specialties even approached us spontaneously with a request to join the project.

Also, patients appreciated the unusual event that the hospital was interested in their well-being afterward. An indication of this appreciation is the unusually high response rate to the first questionnaire compared with surveys in general. This finding despite that the patients were not approached a second time if they had not returned the first questionnaire. Also, the response rate to the repeat questionnaire was high, around 80 percent, showing the willingness of the patients to participate.

The handling of patient data required the design of some new software solutions in the beginning of the project. As the Microsoft Access database was modified to meet our needs, the mailing of questionnaires was sufficiently automated to be easily managed by one person alone. However, if the measurement would take place on an even larger scale in the future, efforts should be made to also automate the data entry, which in our case was manual. One possibility is to make the data collection, at least in part, Internet-based so that patients can enter the data directly into a database. Further automation of the mailing process could be achieved by improving the database, for example, to take into account that the covering letters of the questionnaires need to be tailored to suit the individual needs of different clinical specialties. Furthermore, combining the mailing system with the Finnish population register to automatically ensure that the patient is alive and the address is correct would further reduce the amount of human resources needed for large scale routine data collection.

The merging of effectiveness data with cost data and information on the treatments posed no problems in the Ecomed database. However, when individual patients had been treated multiple times during the follow-up period, it was sometimes difficult to ascertain to which treatment the effectiveness data were connected. Similarly, it is also virtually impossible to tell the effect of a single intervention on the change in HRQoL if the patient has received treatment for two or more illnesses during the follow-up. Some of these problems can be overcome by manually checking that the effectiveness data are linked to the right treatment during the follow-up period. This process does not require much work when the patient material is small but does not solve the problem of how much each treatment has contributed to the change in HRQoL. In large patient materials, hand checking is exceedingly difficult but, on the other hand, may not be that important, as some deviant patients easily disappear into the mass when thousands of patients are studied and, thus, do not bias the results in a significant way.

Whether the cost-effectiveness data produced by our system is able to really influence decision making remains to be studied when enough data are available to make reliable comparison across various interventions. When Wright and colleagues studied outcomes of elective surgery using the SF-36, their results showed some interesting differences in the effectiveness of various surgical procedures. However, only approximately half of the surgeons giving feedback had positive comments about the project and the information it produced. The rest believed it was of little value and did not want to receive such information on their patients in the future (13). It may indeed be, as Wright et al. (13) speculate, that some surgeons may have difficulty accepting that self-reported HRQoL is a more valid outcome than their own impressions and, thus, may be prone to deny the usefulness of such data (13). This finding could be an even bigger problem in our project, where opposed to Wright et al., we also report cost-effectiveness data, which may be seen by some physicians as an attempt to limit their sovereign right to decide what is right for each patient regardless of costs. On the other hand, a limited number of other hospital decision makers so far interviewed considered our preliminary cost-effectiveness data very useful.

The approximate cost of data collection, €16 per patient, may not be considered quite insignificant and is higher than the $12 Canadian per patient reported by Wright et al. (13). One has to bear in mind, however, that our costs include all the initial investments in the development of the system, and we are confident that, once the system is up and running, the cost per patient will be significantly lower. Further cost reductions can certainly be achieved by the automation of the data collection so that, compared with the value of the information received, costs of data collection eventually can be expected to be trivial.

Measurement and interpretation of the cost-effectiveness data is especially suited for comparison of fairly straightforward interventions such as various surgical procedures. We have attempted, however, to collect data also on more vague disorders, where the intervention does not consist of a single, well-defined procedure. These entities include, for example, rheumatology, intensive care, and treatment of obesity. Whether HRQoL data can be used to assess the effectiveness of also such treatments remains to be seen, as data collection in these fields is still in progress. However, preliminary results concerning the treatment of, for example, esophageal cancer, showing clear HRQoL benefits from palliative treatment, indicate that our approach may well work in less clearly defined treatment entities as well.

In conclusion, our project shows that routine collection of effectiveness data is feasible and does not require major extra investment. Data on effectiveness can be easily used to complement the already available cost data to produce estimates of cost-effectiveness of various interventions. Attitudes of both clinicians and patients have been favorable toward data collection, indicating that estimation of cost-effectiveness of treatments is generally considered necessary. Preliminary experiences also allude that other decision-makers appreciate cost-effectiveness data. Whether the data produced really affect allocation of resources, however, must be established in future studies.