Contingent valuation: a ship without a sail?

Contingent valuation (CV) provides a survey-based method for estimating the monetary benefits of non-marketed goods and services (see Mitchell and Carson (Reference Mitchell and Carson1989) and Bateman et al. (Reference Bateman, Carson, Day, Hanemann, Hanley, Hett, Jones, Loomes, Mourato, Ozdemiroglu, Pearce, Sugden and Swanson2002) for fuller description of the method). Although CV has been applied in the environmental and transport literature since the early 1960s, and was applied in the health area in the mid 1970s (Acton, Reference Acton1973, Reference Acton1976), it remained neglected by health economists until the late 1980s and early 1990s (see Diener et al. (Reference Diener, O’Brien and Gafni1998), Klose (Reference Klose1999) and Olsen and Smith (Reference Olsen and Smith2001)). Since that time, the number of CV studies in health care has increased rapidly (see Diener et al. (Reference Diener, O’Brien and Gafni1998), Klose (Reference Klose1999), Olsen and Smith (Reference Olsen and Smith2001), Smith (Reference Smith2003) and Sach et al. (Reference Sach, Smith and Whynes2007)), covering a wide variety of disease areas and countries; although this number remains small compared to the number of cost-effectiveness and cost–utility studies conducted over the same period (for instance, the OHE (Office of Health Economics) Health Economic Evaluation Database holds 35,000 plus references (http://www.ohe-heed.com/)).

Using data from a CV database compiled by the authors following on-going systematic reviews of the literature, the methods of which have been described extensively elsewhere (Olsen and Smith, Reference Olsen and Smith2001; Smith, Reference Smith2003; Sach et al., Reference Sach, Smith and Whynes2007), Figure 1 illustrates that in the 5 years, from 2001 to 2005, 163 CV studies were published.

Figure 1

Number of CV studies per year (1985–2005)

Table 1 shows that the USA and UK remain the largest producers of CV studies in health care (Sweden contributed a number of studies during the 1990s, but activity has declined recently). However, the most obvious change in the last few years has been the application of CV in countries other than the USA, UK, Canada and Sweden (the main, or only, players in the 1980s and 1990s). From 2000, at least one CV study in health care had been conducted in 35 other countries. CV thus appears to be making inroads in a number of different health systems and cultures, and especially so in some developing countries where often these studies are used to estimate possible demand and co-payment rates, and thus have a very specific policy focus.

Table 1

Country of origin by year

Table 2 shows that many of the early studies were focussed on cardiovascular disease – predominantly by Johannesson in Sweden – but the wider geographical use of CV illustrated above coincides with a broadening of clinical area from the mid-1990s.

Table 2

Clinical area/diagnosis over time

*Other includes, for example, different locations for care, irradiation of food, asthma and respiratory illness, blood products, and information from pharmacists on medicine use.

Table 3 shows that a significant, but nonetheless a minority, of interventions evaluated have been pharmaceutical products. However, since 2000, there have been more studies focussed on screening for various diseases – perhaps because a significant component part of screening concerns the value of information, not usually captured in other evaluation methods.

Table 3

Intervention by year

*Other includes, for example, information provisions, unspecified movement between health states, and irradiation.

It can be seen, therefore, that there has been an expansion in total number, breadth of geographical coverage (and hence health systems and cultures), clinical areas, and interventions. The popularity of CV thus shows no signs of abatement – indeed, quite the opposite – and this makes it timely to critically review the development of CV and raise future research directions.

Despite the technique’s growing popularity, there remain a number of critical unresolved issues. These may be categorised as concerning: (i) whether or not there are (or could be) substantial advantages to the use of the technique over alternative outcome measurement strategies; and (ii) the appropriate methodology for conducting CV generally, and in its application in health economics more specifically (Cookson, Reference Cookson2003; Hanley et al., Reference Hanley, Ryan and Wright2003).

Numerous specific CV research questions could be raised as important in a paper such as this. For instance, what the ‘threshold’ income level is for impacting on sensitivity, the impact of prominent numbers on valuations for a wide range of goods, and the impact of response acquiescence or ‘yeah-saying’. These have been considered in more detail, including by the authors, elsewhere (e.g. Smith, Reference Smith2001, Reference Smith2004, Reference Smith2005, Reference Smith2007a, Reference Smith2007b, Reference Smith2008; Yeung et al., Reference Yeung, Smith and McGhee2003, Reference Yeung, Smith, Ho, Johnston and Leung2006; Covey and Smith, Reference Covey and Smith2006; Whynes et al., Reference Whynes, Frew, Philips, Covey and Smith2007). However, focus on such questions, important as they may be for methodological advancement, would miss more significant and fundamental issues facing the technique. The discussion contained in this paper therefore invites health economists to consider: (i) whether there is a role for CV in health economics; and (ii), if yes (or perhaps a weaker not sure), how can CV development progress to a stage to realise the desired potential, in light of developments over the last 20 years of research in the area?

At present, we would suggest that CV is like a ship without a sail – it is adrift in the sea of economic evaluation methodologies, unsure of a direction and without the ability to move in a purposeful manner to achieve its destination. Throughout the paper the metaphor of the ship is used, since for a boat to float it first needs to be built correctly, which requires an in depth understanding of the dynamics and dimensions of a successful boat; that all components are present to make it move; and that once moving, an experienced captain oversees the direction and speed of travel. This is a useful metaphor for CV because uncertainties remain about the exact dynamics and dimensions of the technique, and earlier research has proceeded in a rather ad hoc and uncoordinated manner that has resulted in the technique having limited influence in decision-making in health care.

Is CV seaworthy?

The first major question to ask is whether the good ship CV is ready to be taken to the sea. Is the technique worth further research, time, and money being invested into it so that it can form part of the health economists’ toolkit? This depends on whether there is a realisable advantage to using CV over other techniques. If the answer to this question is no then CV should be abandoned as unseaworthy, and efforts put in to developing other ships.

In this respect, Olsen and Smith (Reference Olsen and Smith2001) advanced three arguments why CV may offer real advantage over other outcome measures. These included its foundations in welfare economics, that it does not restrict outcomes assessed to health outcomes only, and that monetary valuation of benefits and costs is required for decisions about allocative efficiency (that is, it allows a full cost–benefit analysis (CBA) to be undertaken). The first of these arguments was largely refuted as unimportant on the basis that it is more important for a method to fit with societal value judgements, which may be expected to differ across countries and health systems. For instance, one could argue that the failure by the National Institute for Health and Clinical Excellence (NICE), in the UK, to adopt CV as an approach in its methods manual is one indication that UK society does not share the values of CV within health care (NICE, 2004). That this may reflect the situation that health care is free at the point of use, does not seem a logical argument since NICE still has to make a decision about societal willingness to pay for a quality-adjusted life year (QALY; as discussed below); although, removing this judgement from the individual level may in some way make it appear more acceptable.

In terms of the second argument, Olsen and Smith (Reference Olsen and Smith2001) categorised the benefits measured by CV as: health outcomes, where three dimensions were identified including change in health state, duration, and probability (which are those assessed by the QALY); process utility, which may also derive from the use of health care; and non-health outcomes, covering option value and caring externalities. Their review however, found that, in practice, few studies made use of this potential advantage, mainly through poor scenario specification.

In respect of the third argument in favour of CV – that it can inform allocative efficiency decisions through application in CBA – Olsen and Smith (Reference Olsen and Smith2001) found that most studies reviewed were partial evaluations, valuing one programme alone. Together with the fact that most also did not compare CV values to the cost of the programmes, this suggests that few CV studies were making use of their theoretical advantage. Such was the evidence back in 2001, that Olsen and Smith had ‘the distinct feeling of a huge mismatch between the theoretical glory of WTP and the usefulness for public health policy of the majority of surveys which have applied this method’ (Olsen and Smith, Reference Olsen and Smith2001: 47). Updating this review to see if there has been a change over time finds that the same conclusion can largely be reaffirmed.

Looking first at health outcomes, one can see from Table 4 that the majority of studies do not explicitly address the valuation of health outcomes comprehensively. Table 4 also suggests that this situation has not improved, although there are fewer who provide no details, as a proportion, or for whom health is not the focus.

Table 4

Health outcomes description

Table 5 provides a breakdown of non-outcome attributes or ‘process utility’, and provides a comparison between the two time periods.

Table 5

Non-outcome descriptions

In terms of non-current use attributes – reflecting option value and externalities – a more complex picture of the interaction between the scenario specification and the respondent arises. That is, which values are activated in this respect depends upon the role in which respondents are placed. To assess the extent to which CV studies had actually sought to include these elements of value, each study was examined for three types of respondents: users, convenience sample, and the general population. Users are most often patients, but sometimes their relatives. Convenience samples are most often groups of students or seminar participants. General population samples are those who are supposed to be representative of the population. Table 6 shows that most surveys included only use-value, reflecting the fact that most surveys asked patients only. Only 24% of studies have asked the general population. When users may have expressed option value or externalities, this refers to preventive programmes. Table 6 also illustrates that this picture has changed little over the two time periods considered.

Table 6

Types of values by type of respondents

Given that CV studies in health care are overwhelmingly constructed to elicit use-value alone, the question that arises therefore is whether CV studies in health care are misspecified. Empirical research suggests that, if these results are more widely replicable, most CV studies in health care may indeed be misspecified, as a significant element of the value of the good in question is not being captured (Smith, Reference Smith2007). This is important as, ultimately, the cost–benefit calculus and policy recommendation would be biased against the good in question, and in a comparison between goods. Of course, whether studies actually are misspecified depends upon the policy context within which the CV study is conducted. For instance, it may be argued that the most appropriate policy context is often one that is ex ante, and thus that the only source of relevant value is option value.

Updating the contribution that CV can make to CBA and allocative efficiency decisions reveals that in terms of the number of partial evaluations (163 or 62% of studies) this does seem to have improved slightly over time, but the number comparing CV values to the cost of interventions was not much higher (29 studies). The challenge here is that, given the failure of CV to achieve its theoretical advantages, can other outcome approaches devise ways to incorporate these advantages. If they can then clearly CV might become redundant.

Of course, one possible alternative to conducting individual CV studies is to maintain the current practice of conducting cost–utility analysis (CUA), but have a ‘translation’ factor to convert these in to CBA through use of a statistic or formula representing social willingness-to-pay (WTP) for a QALY (Adler, Reference Adler2006; Gafni and Birch, Reference Gafni and Birch2006). Without such a ‘translation’ factor, there is no way in which the cost effectiveness of health interventions can be compared with that of interventions in other sectors, such that an inefficient allocation across sectors is a distinct possibility (noting that, of course, even if such a translation factor were used, basing this on a QALY would still only represent the WTP for health benefits, thus still excluding some of the claimed advantages of CV over other techniques).

Currently, bodies such as NICE, have to make an implicit judgement about whether the QALY gains are ‘worth’ what they will cost (Devlin and Parkin, Reference Devlin and Parkin2004). However, the ‘threshold’ values adopted (such as £20,000–£30,000 per QALY above or below which a new therapy will be rejected or recommended for adoption in England and Wales, and the commonly-mentioned €18,000 in the Netherlands) are more or less ‘arbitrary’ (NICE, 2004; Rawlins and Culyer, Reference Rawlins and Culyer2004), and NICE has recently funded work looking at the assessment of a money value (Devlin and Parkin, Reference Devlin and Parkin2004). Similar work is being undertaken elsewhere, for instance, Australia (George et al., Reference George, Harris and Mitchell2001). There are also a growing number of papers that have either explicitly called for, or at least implied the need for, estimates of the monetary value of a QALY, with empirical estimates ranging from around £1,000 to over £500,000 per QALY, depending on assumptions used and whether values have been imputed from the value of a prevented fatality or a specific survey looking at WTP for changes in health states (Phelps and Mushlin, Reference Phelps and Mushlin1991; Johannesson, Reference Johannesson1995; Garber and Phelps, Reference Garber and Phelps1997; Johnson et al., Reference Johnson, Fries and Banzhaf1997; Bleichrodt and Quiggin, Reference Bleichrodt and Quiggin1999; Hirth et al., Reference Hirth, Chernew, Miller, Fendrick and Weissert2000; Dolan and Edlin, Reference Dolan and Edlin2002; Hammitt, Reference Hammitt2002; Gyrd-Hansen, Reference Gyrd-Hansen2003; Devlin and Parkin, Reference Devlin and Parkin2004; Eichler et al., Reference Eichler, Kong, Gert, Mavros and Jonsson2004; Buxton, Reference Buxton2005; Byrne et al., Reference Byrne, O’malley and Suarez-Almazor2005; Franica et al., Reference Franica, Pathakb and Gafni2005; King et al., Reference King, Tsevat, Lave and Roberts2005; Haninger and Hammitt, Reference Haninger and Hammitt2006; Adler, Reference Adler2006; Van Houtven et al., Reference Van Houtven, Powers, Jessup and Yang2006).

However, although such a calculation is appealing, and interest in generating such a value increasing, it is not as straightforward as it initially appears and there are a range of issues that need to be considered when applying WTP at a societal level. These have been considered in depth elsewhere (Richardson and Smith, Reference Richardson and Smith2004; Smith and Richardson, Reference Smith and Richardson2005), but in brief concern whether the sum of individual WTP equals social WTP, whether individual WTP maps directly into social WTP, whether personal income is the appropriate budget constraint, and whether WTP should be adjusted for ability to pay. The degree to which these considerations can be overcome will determine the degree to which CV may be unnecessary. The recently completed SVQ (Social Value of a QALY) project in the UK has considered a number of these issues (Donaldson et al., Reference Donaldson, Bateman, Chilton, Jones-Lee, Loomes, Metcalf, Mugford, Robinson, Ryan, Shackley, Smith, Sugden and Wildman2003) and will be extended, on a European level, through the EuroVaQ (European Value of a QALY) project (Donaldson et al., Reference Donaldson, Brouwer, Gulacsi, Gyrd-Hansen, Kozierkiewicz, Olsen, Manca, Mataria, Moatti, Persson, Pinto Prades, Ryan and Smith2006). These projects are designed to provide information concerning whether the values/weights attached to gains in quality and length of life should vary according to the characteristics of the patients receiving them, and how much the health system should be spending (at the margin) to achieve such gains. This involves testing alternative procedures in order to: (i) determine whether it is feasible to derive a baseline value of a QALY sufficiently robust to be of use to decision makers; (ii) explore the issues involved in incorporating ‘citizen’ concerns into a WTP framework; and (iii) identify which method(s) (if any) appear to be the most promising candidate(s) for national decision-making (and, in EuroVaQ, look at cross-country differences).

Of course, despite all this effort, whether a ‘social value of a QALY’ is obtainable, or even desirable, is a moot point. It could simply be that economists are trying to combine conceptually incomparable measures – that the conditions for equivalence are excessively restrictive and unrealistic (e.g. Dolan and Edlin, Reference Dolan and Edlin2002). Given that we are almost certainly looking for a non-linear function, rather than a universal constant, WTP for money value of a QALY, it might well be that in trying to value a QALY we simply support the replacement of the QALY with WTP, and thus the replacement of CUA with CBA (Van Houtven et al., Reference Van Houtven, Powers, Jessup and Yang2006).

Therefore, if a ‘social value of a QALY’ is not feasible, or perhaps desirable, then we are left in the position of having to address the issues discussed in this and previous reviews of CV (Diener et al., Reference Diener, O’Brien and Gafni1998; Klose, Reference Klose1999; Olsen and Smith, Reference Olsen and Smith2001, Smith Reference Smith2003, Sach et al., Reference Sach, Smith and Whynes2007). In this respect, the need for a coordinated research effort and the specification of some form of reference case and/or guidelines for conduct of CV in health economics appears unavoidable, and perhaps even more important. That is, if CV is deemed seaworthy, then a sail needs to be constructed.

To summarise, CV does not seem to be capitalising on the major advantages it holds as a technique over other outcome methodologies. Overall, it is unclear why and how a decision maker could use existing CV studies in resource allocation decision-making. This failure to capitalise on the potential advantages of CV by researchers could suggest that training in the technique is inadequate, the technique is fundamentally flawed, the costs of doing CV appropriately are astronomical, or simply that there are inadequate guidelines available to guide researchers. The detachment between the doers and users of this research may also mean that interested individuals do the studies for reasons of personal interest rather than with a policy use in mind. It is quite likely that the answer contains elements of all these arguments. However, these authors would not suggest that the technique is fundamentally flawed; rather, it has not been appropriately developed and applied in health economics.

What sort of sail?

If the foundations of CV are seaworthy, decisions about the construction of CV, or to use our metaphor the construction of the sail, need to be made. Historically, the key specifications have focused on scenario development and presentation, payment vehicle, respondent type, the incorporation of risk or uncertainty, the time period over which WTP/WTA is estimated and the mode of administration (Smith, Reference Smith2003). In order to see what the specifications of the sail should be, it is important to reflect back to see if, over time, a consensus has been reached on any of these design aspects, whether there has been any convergence towards a common agreed standard, as has been the case in other areas of economic evaluation (NICE, 2004). Using the CV database used in previous reviews of CV (see Olsen and Smith (Reference Olsen and Smith2001), Smith (Reference Smith2003) and Sach et al. (Reference Sach, Smith and Whynes2007) for details) it is possible to address this question by looking at a few examples.

To know if the methodology used in CV studies has improved, one would need a standard against which to judge. Within health economics, there has been no formal guidelines established or validated, hence many studies claim to have followed guidelines established within environmental economics, although very much in an ad hoc fashion. Assuming this is acceptable for the moment, we can illustrate change over time with reference to the National Oceanographic and Atmospheric Administration (NOAA) guidelines (Arrow et al., Reference Arrow, Solow, Portney, Leamer, Radner and Schuman1993). Taking an example using question-elicitation format first, the NOAA Panel recommended that dichotomous-choice questions should be used in preference to open-ended questions, as the latter are ‘biased, erratic and unreliable’ (Arrow et al., Reference Arrow, Solow, Portney, Leamer, Radner and Schuman1993). The dichotomous-choice format was developed to attempt to resolve three problems that had been encountered in using the open, bidding, and payment-card formats: low response rates; strategic bias; and starting point (for bidding games) or range (for payment card approaches) bias. This format was argued to have three advantages over other techniques: it more closely resembles consumer choice in an actual market situation; no one respondent can unduly influence the overall WTP value; and it avoids starting point (and range) bias since respondents are only presented with one bid (Smith, Reference Smith2000).

One would expect such a significant recommendation to spill over to the conduct of CV studies in health care. Table 7 demonstrates that between 1990 and 1996, dichotomous-choice questions represented fewer than any other single format, yet for the period from 1996 to 1999 this format represented more studies than were undertaken with any other single format. However, Table 7 also illustrates that the majority of studies (80% across the full period) were still not undertaken using the dichotomous-choice format. In more recent years, since 2000, the payment card and bidding approaches have, especially, been more widely reported, and critical attention has only been paid to the use of different question formats within health economics in the last few years (Frew et al., Reference Frew, Whynes and Wolstenholme2003, Whynes et al., Reference Whynes, Frew and Wolstenholme2003, Reference Whynes, Frew and Wolstenholme2004).

Table 7

Questionnaire format over time

As a second example, the method of survey administration recommended by the NOAA Panel is that of face-to-face interviews. Smith (Reference Smith2003) reported that 41 studies (37%) collected data from face-to-face interview. Of the remaining 70 (63%) who did not undertake face-to-face interview, 12 (11%) used telephone interview, 23 (21%) used postal self-administration and 29 (26%), other self-administration. The technique was not available for six studies. Of the 154 studies published subsequently, the corresponding figures for studies collecting data from face-to-face interviews, telephone, postal, and other self-administration were 45% (70/154), 16% (25/154), 14% (21/154) and 15% (23/154), with the technique not being available in 10% (15/154) of cases. This would seem to indicate some improvement – there are more studies using face-to-face and telephone interview, and less using postal or other self-complete forms of administration, which should improve the quality of studies in this latter period. There is, therefore, some hope, but the degree of change has appeared non-universal across different aspects of design, perhaps indicative of a lack of coordination or direction.

Of course, it may not be appropriate or desirable to apply the NOAA guidelines, which were developed within a very specific context – valuing environmental damage caused by pollution for use in legal settlement. By its very nature, environmental economics is mostly concerned with ‘non-use’ valuation of unfamiliar commodities, such as wildlife, areas of natural beauty or air quality, and irreversible effects. In contrast, CV surveys in health care are valuing a ‘good’ that is generally more familiar, the context (health system) is familiar, the good (usually) providing direct ‘use-value’, and the good itself being of a private (i.e. rival and excludable, and therefore tradable) rather than public nature. Thus, for instance, there may be very justifiable reasons why health economics may not wish to adopt NOAA recommendations for elicitation formats, given the differences in what is being valued (Smith, Reference Smith2006). However, the critical issue is that there is no systematic exploration of this justification, or application of alternative, health-specific, guidelines adopted in health economics. As with other economic methodologies, there is a need for readily identifiable criteria by which the quality of CV studies may be judged.

At the most fundamental level, this has meant that CV studies are not currently comparable, as health-specific CV studies lack agreement and adherence to a specific design choice for each dimension (including, inter alia, how to develop CV studies appropriately, what and how information should be provided in the scenario, elicitation format, respondent type, payment vehicle, time frame for valuation, partiality, administration mode, and how risk and uncertainty should be incorporated). The result of this is an inability to compare across CV studies because they use incomparable methods on two or more dimensions. This leads to an inability to look at the existing CV literature to try to establish the impact of different design choices, as it is not clear which design choice may have impacted the result across studies, even for studies of the same or similar interventions, as they are intrinsically confounded. It therefore seems critical that more systematic work be undertaken in order to consider how key dimensions should be handled in these studies, and thus enable us to craft a sail that is ready to drive the ship.

Setting sail: which direction?

This paper has thus far argued that applications of CV have failed to deliver on the theoretical advantage of the technique, which, at least in part, has reflected the absence of agreed guidelines in health economics about how to conduct CV studies. The consequence has been a methods-policy gap where the technique has had limited influence in decision-making processes. If the further pursuit of CV is seen as a worthy cause then there is some work needed to make CV useful. In this section, we make some tentative suggestions as to some ways forward, and whilst readers may disagree with the detail, the key point being made is the need to make some decisions on where to set our sail, otherwise the direction of travel will never be determined and the good ship CV will continue to drift about, not really being of any use to anyone.

For this, there has to be a common standard in the reporting of CV studies in the published peer-reviewed literature. For example, Sach et al. (Reference Sach, Smith and Whynes2007) found important design details missing in CV studies of pharmaceutical interventions published since 2000. Here, 25% failed to report the elicitation method used and 23% failed to report the price year. Both of these details are required in order to interpret and compare the results in a useful manner. An obvious and easily implemented solution to this problem would be the adoption by health economists and journal editors of an agreed checklist, listing what details should be included in published CV studies. Box 1 suggests what such a checklist might include, covering all aspects from development, design and results, through to tests of reliability or validity undertaken (see also Yeung and Smith (Reference Yeung and Smith2006)).

Given the issues highlighted in this paper, we suggest that guidelines need to be produced in a coordinated manner that can be used by authors, reviewers and policy makers alike, which should also include guidance concerning how to deviate from the guidelines appropriately when undertaking methodological work. For instance, isolating the change of interest, rather than confounding the issue with multiple simultaneous changes. The importance of guidelines is clearly demonstrated by looking at the application of CV in the environmental and transport safety literature where NOAA was so influential in shaping the development of the technique, and the production of government guidelines for CV conduct in these areas. Given the methodological uncertainties that remain in CV as applied to health care, any guidelines developed need to be formed on the basis of some consensus so that they are adhered to. The simplest approach might, in the first instance, be to adapt the NOAA guidelines in health (outlined in brief in Box 2).

Importantly, however, there has been little consideration of whether the methodology found in the application of CV in the environmental or transport safety literature is directly translatable to health care and, if so, whether this is even desirable (in most areas of the application of economics to health care, we consider ‘health’ to be sufficiently ‘different’ to warrant specific adaptations to theory and/or practice (Mooney, Reference Mooney1986, Reference Mooney1994), so why not here?). It would seem sensible for research to first address the applicability of CV guidelines in other branches of economics before developing and coordinating work to create new ones in health. The main point to note is that there is no point tinkering if there is no common purpose. A coordinated research effort, that ensures research is reported appropriately and isolates the one aspect being varied, would go some way towards making CV useful to policy.

Others might fundamentally disagree with using NOAA as a starting point on the basis that health is sufficiently different from the environment and that it makes no logical sense to start from that point. Such an argument could be seen as quite persuasive in terms of the misspecification issue raised earlier in the paper, for instance. Because health CV values extend beyond non-use and irreversible decisions, there could be a recommendation that the default assumption be that all elements of value are of importance such that researchers must explicitly consider the relevance of use, externality and/or option value in their design and discuss how this issue was tackled. Similarly, research around elicitation format in health by no means necessarily supports the use of the dichotomous-choice questions over other formats and, therefore, health economists may wish to agree on the use of payment scales or another format as their preferred base case option. The key point being made here is that this requires systematic discussion and application.

Clearly, guidelines will never produce a ‘perfect’ CV study. By their very nature, CV studies are flawed representations of the world. Their purpose is not to achieve perfection but rather to provide a structure to guide movement toward obtaining more accurate values, a belief presumably driving all forms of research into aspects of the valuation exercise, such as embedding effects, payment vehicles, or scenario specification. Indeed, it is perhaps somewhat ironic that not only despite, but perhaps because, of such high-profile guidelines, and the wealth of research undertaken into CV surveys in general, there is not greater consistency in the conduct of CV surveys. However, this paper indicates not only that CV studies in health care have performed poorly in the construction, specification and presentation of the contingent market, but also that there has been little, if any, improvement in this respect over the last 20 years.

Who will captain the ship?

Of course, setting direction and staying on course requires a captain. The paper has thus far considered the what-needs-to-be-done question, but we also need to consider who should, or could, captain the ship.

Health economists could choose to be proactive and coordinate themselves so as to develop the technique themselves or in collaboration with researchers from other disciplines. Alternatively, health economists could wait for provocation to do something. This might come in the form of a major health accident, akin to the impetus for the NOAA recommendations, or rather more likely as a policy agenda that forces change. From a policy point of view, if health economists were to focus narrowly on evaluating the value for money of curative technologies that only have health benefits, then arguably CV might be unnecessary, other than in helping determine the overall allocation of resources to health from the treasury perhaps. In reality, of course even clinical interventions often have spillover effects onto family carers or children in terms of educational benefits for example. Such benefits are not captured in traditional cost per QALY analysis, which could be justified by organisations such as NICE in the UK, on the grounds that an NHS and social services perspective is taken. However, in 2005 the remit of NICE expanded to incorporate public health, as it took on the functions of the Health Development Agency, an area where the costs and benefits very clearly extend beyond the health care arena and have consequently challenged standard economic evaluation methods.

To deal with this, NICE has extended the scope of costs captured in economic evaluations and whilst benefits remain assessed by QALYs, a cost-consequent approach has also been adopted to take account of the wider benefits where possible (Drummond et al., Reference Drummond, Weatherly, Claxton, Cookson, Ferguson, Godfrey, Rice, Sculpher and Sowden2007). Such an approach does not seem ideal since potentially non-health benefits could be attached less weight than the health effects, biasing decisions towards those with larger health gains. Potentially, this agenda could provide an impetus to develop CV methods in health care. This being the case then NICE might be an appropriate lead. However, the CV approach then risks not being captained anywhere, except to the bottom of the sea perhaps, because certain elements of the health economics community may not approve of the technique. This paper has shown that on current evidence such an opinion of the technique might actually be well founded. However, it is one thing to demonstrate that a technique has failed in the past but another to argue that it could never work in the future, which is what concerns us in this paper.

To conclude, it seems to us that now would seem an appropriate time to take stock of the history of CV in health economics, in part because the ad hoc application of CV can no longer be ignored and in part because new opportunities such as the public health economics agenda may renew the potential importance of the technique. This paper has attempted to be provocative in the hope of stimulating response and debate about the future of CV because the status quo seems clearly inadequate, and as a profession it is arguably our responsibility to debate whether this means it should be put in to port and never sailed again, or is given the ability to set sail and travel to new and exciting places!