“There is nothing in the laws of value which remains for the present or any future writer to clear up; the theory of the subject is complete”
John Stuart Mill, 1848 (Reference Mill19)
Demand theory used in modern economics, and hence in technology assessment, is the culmination of a millennium long debate related to value (Reference Kauder16). By the mid 19th century, following the contributions of what we now call the classical economists, it was assumed that all issues surrounding the notion of economic value had been resolved (Reference Mill19). Despite this assertion the economics of valuing goods—especially in nonmarket settings—continues to be highly debated in applied economics, and especially in health (Reference Günther and König13;Reference Lindholm, Rosén and Hellsten18).
Economic valuation is conducted to acquire objective, reliable data on the benefits and/or costs of a good, service or intervention (Reference Bridges6). Like other forms of technology assessment, economic valuation is conducted to inform decision making and can promote patient centeredness by allowing the scientific enquiry into the patient point of view (Reference Whynes24). Many political decisions are supported by economic valuation and there is evidence that economic valuation can greatly impact policy decisions (Reference Whynes24). A lack of consensus concerning valuation methods persists, with many economists actively and independently approaching economic valuation projects, especially nonmarket goods like health, in a multitude of ways. Although variations in methods are accepted, and even encouraged, among economist (Reference Graff Zivin and Bridges12), such variations impede the ability of policy/decision makers to distinguish between robust studies and weak studies as valuation methodologies are entirely variable.
The initial aim of this study was to conduct a relatively simple technology assessment to determine the economic value of a pneumococcal vaccine in Bangladesh. For this particular intervention domestic and foreign policy makers were especially interested in parents' willingness to pay for such a vaccine, an approach that has been previously applied in several vaccine applications (Reference Yeung and Smith26). To answer this question, a more fundamental question needed to be answered, “Of the available methods to estimate monetary measures of value, what is the most suitable?”
While willingness to pay for a good or service can be inferred by revealed actions in the market, the majority of healthcare interventions exist in nonmarket settings and, for the purposes of technology assessment, evaluations most often occur relatively early in the product lifecycle, before reliable demand data are available (Reference Bridges and Jones7). Hence, stated-preference methods such as contingent valuation (survey methods used to directly estimate marginal benefits and demand for a particular good) and conjoint analysis (survey methods used to estimate trade-offs across various attributes of a particular good) are used to estimate willingness to pay (Reference Bridges6;Reference Herbild, Gyrd-Hansen and Bech14). Given that the purpose of this study was to estimate the willingness to pay for a specific vaccine, it was clear that contingent valuation methods were more appropriate than conjoint analysis, but it was not clear what was the most appropriate contingent valuation elicitation method. To shed light on this subject, the focus of the study turned to one of examining the theoretical and, most importantly, the practical difference between contingent valuation elicitation formats.
DEMAND AND VALUE
While issues of demand and value are inevitably linked, it is important to note that they present two very different traditions in economics and philosophy. Demand for a good is the relationship between the price of the good and the amount or quantity that the consumer is willing and able to purchase, that is, for a given price what is the quantity demanded. Well before the establishment of the demand function—where quantity could be derived as a function of price—the primary focus of economics, moral philosophy and many other disciplines had been focused on a very different question—one centered on value (Reference Kauder16). In modern economics, the understanding of the economic value for a particular good or service is referred to as the inverse demand function. For the inverse demand curve, the focus is on determining price (or marginal benefit) as a function of quantity demand. Such analysis is focused on quantifying the underlying worth or benefit of each unit to be consumed.
Contingent valuation studies—especially in health—have ignored this dichotomy between demand and value. This neglect can account for some of the confusion about what elicitation methods are preferred in different settings and for different research questions. In health, contingent valuation studies predominately focus on issues related to inverse demand (i.e., issues of value rather than of quantity). Especially in technology assessment, studies aim to elicit an individual's maximum willingness to pay for a good or intervention in a hypothetical marketplace. This differs to other sectors that focus more on demand estimation (i.e., estimating the quantity demand for given price) and on estimating price elasticity (the responsiveness of quantity demand for a given chance in price) (Reference Arrow, Solow and Portney2).
Contingent Valuation Elicitation
Contingent valuation methodology has four commonly used elicitation formats: Open-ended, Dichotomous Choice, Payment Card, and Bidding Game. Although some other elicitation methods exist, they can be considered as extensions of hybrids for these four methods.
Open Ended
The open-ended elicitation format simply asks the respondent how much they would be willing-to-pay, that is, “For this good, what is the most that you would be willing to pay?”. It has been argued that the methods produce more valid (Reference Onwujekwe22) and less valid results (Reference Arrow, Solow and Portney2;Reference Klose17)—albeit for different reasons related to the objective of the study (i.e., whether the focus of the analysis is on valuation or demand estimation). The open-ended elicitation format has also a reputation for producing conservative results (Reference Johannesson, Jonsson and Borgquist15;Reference Whynes24), but is not subject to framing effects as it does not suggest an answer. This said, it is possible that a lack of context might lead respondents merely to pick a number out of the air, and hence results might lack construct validity. Given that the focus if on maximum willingness to pay (a monetary measure of value), the open-ended elicitation method it is best considered as estimating inverse demand.
Dichotomous Choice
The dichotomous choice elicitation format—often referred to as the take-it or leave-it—focuses on a simple yes/no question regarding whether an individual would pay a specified price, that is, “If the price were $100, would you purchase it?”. Given that prices are set exogenously, it has been argued to be more representative of market (demand) situations (Reference Arrow, Solow and Portney2;Reference Klose17). It produces less conservative results compared with open-ended (Reference Johannesson, Jonsson and Borgquist15;Reference Ryan, Scott and Donaldson23), potentially because it does not seek maximum willingness to pay. The limitations of the dichotomous choice elicitation method are that it is subject to yea saying—the tendency of the respondent to respond positively to a hypothetical scenario regardless of the content or scenario presented (Reference Blamey, Bennett and Morrison3;Reference Blumenschein, Johannesson, Blomquist, Liljas and O'Conor4;Reference Frew, Whynes and Wolstenholme9)—and requires a much larger sample size than the other elicitation formats (Reference Boyle, Johnson and McCollum5). Given that prices are determined exogenously, and that demand is the focus of analysis, the dichotomous choice elicitation format is best considered as demand estimation rather than inverse demand (valuation) estimation.
Payment Card
The payment card elicitation format presents a list of prices to the individual and asks them what value represents their maximum willingness to pay, that is, “Of the prices listed on this card, can you please circle the one that indicates the highest price that you would pay?”. The method circumvents the need to produce a single starting point, but unlike the open-ended method it does provide the respondent with a context for determining their willingness to pay. However, given that one has to present a maximum and minimum price, the payment card elicitation format can introduce an alternative framing effect known as range bias, that is, that the maximum and/or minimum price has an influence on the result (Reference Boyle, Johnson and McCollum5). As value estimation is the focus of the payment card elicitation format, it can be considered as estimating inverse demand.
Bidding Game
The original contingent valuation elicitation method (Reference Whynes, Wolstenholme and Frew25), the bidding game is a series of yes/no questions that aims to find the maximum willingness to pay. Like the dichotomous choice method, the respondent is presented with an initial price, but if the respondent accepts that initial price, then they are offered higher and higher prices until the respondent says no. Alternatively, if the respondent is a nondemander, then the prices are repeatedly decreased until the respondent accepts (even if that requires negative prices). Whereas the bidding game can be subject to starting point bias, the repeated nature of the elicitation method allows a greater amount of time for the respondent to contemplate their answer, and hence could produce results that have greater construct validity. Given that it attempts to estimate maximum willingness to pay, it can be considered as inverse demand estimation.
Comparing Elicitation Methods
Although it has been argued that there is no theoretical difference between the different formats (Reference O'Brien and Gafni20), our hypothesis is that two classes of methods exist—those that aim to at demand estimation and those that aim at value estimation. This similarity of methods has been confirmed among some pair wise comparison of elicitation methods (Reference Frykblom11;Reference Onwujekwe21) and disproven in others (Reference Frew, Whynes and Wolstenholme9;Reference Onwujekwe22–Reference Whynes24). Whereas there has been some attempt to compare more than two elicitation methods, such studies have seen a merger of several distinctly separate studies (Reference Frew, Whynes and Wolstenholme9;Reference Frew, Wolstenholme and Whynes10;Reference Whynes24).
Methods
A household level survey was conducted in Dhaka, Bangladesh, among 361 families with at least one child under 5 years of age. Twenty-six of Dhaka's ninety wards were identified for this study using a combination of convenience sampling (near to the hospital location) and random sampling. Households were included in the study if there was at least one child under 5 years of age. A parent was interviewed and the under-5 child was the subject of the discussion. If there was more than one under-5 child in the household, one of the children was chosen at random to be the subject of the analysis.
The survey questionnaire collected basic social and demographic information about the household, socioeconomic data, vaccination history of the child (including the price paid for previous vaccinations) and basic health information of the child. Respondents were given background information regarding some of the infectious diseases causes by Streptococcus pneumoniae, were presented with a description of the child mortality rate and the causes of mortality for children under 5 (Reference Arifeen, Akhter, Chowdhury and Macro1) and a description of a hypothetical pneumococcal vaccine that could reduce overall mortality by 16 percent (Reference Cutts, Zaman and Enwere8). Respondents were there asked about their total valuation for all three doses for the pneumococcal vaccine.
As seen in Figure 1, respondents were randomly assigned elicitation formats: open-ended, dichotomous choice (with prices of 100TK [∼US$1.47] and 300TK [∼US$4.41]) and payment card (one of two cards with ranges of 0–1,000TK [∼US$0–$14.71] and 0–2,000TK [∼US$0–$29.41]). The dichotomous choice was then followed up with a bidding-game elicitation method, with the price of the dichotomous choice acting as the starting price for the bidding game. To minimize potential interview bias, the elicitation formats were continually re-randomized by interviewer. To minimize regional cluster bias, every cluster had each elicitation method tested within it.
Experimental design of elicitation formats. (100TK ~ $1.47, 300TK ~ $4.41, 1000TK ~ $14.71, 2000TK ~ $29.41)
Each of the elicitation formats were tested for sensitivity to household income and range/starting point bias (where appropriate) by applying ordinary least-squares multiple regression models. Here, ability to pay—the effect of income on willingness to pay (Reference Bridges6)—was seen as an expected result and confirmatory of the construct validity of the elicitation method. The dichotomous choice and the bidding game were subsequently adjusted for their starting point. Clustering was accounted for by calculating Huber-White standard errors to correct for the correlation of observations within wards. For simplicity, these many regressions are not reported here, but serve as the basis for validity tests of the elicitation methods.
Health characteristics of the child, past family experience with pneumonia and meningitis, socioeconomic variables, vaccination history, and demographic characteristics of the respondents were summarized and compared between the different elicitation format groups using F-statistics produced by one-way analysis of variance.
Results
Three hundred sixty-one households were asked to participate in the survey. Thirty-one households refused to participate, leaving 330 households who were interviewed (a 91.4 percent response rate). The vast majority of the households were Muslim (95.2 percent), 3.0 percent were Hindu, 1.5 percent were Christian, and the remaining 0.3 percent were Buddhist. The majority of respondents were women (92.4 percent), and the average age of the respondent was 28.6 years. Half of the mothers had less than a secondary level education. In terms of number of children per household, 43.5 percent of had one child, 33.4 percent had two children, and the remaining 23.1 percent had three or more children. The respondents were asked their family history with pneumonia and meningitis, after descriptions were described and 47.9 percent of the households had experienced pneumonia in their family and 10.9 percent of their family's had experienced meningitis. The median household income was $172 per month and the medium consumption was $167 per month. A comparison of the descriptive statistics across the elicitation formats can be found in Table 1. There was a significant difference in the percentage of male respondents across the groups, with experience with cough, and BCG vaccination history (but not with being fully vaccinated).
Comparison of the Descriptive Statistics across the Elicitation Formats
aPercentages unless otherwise noted.
bCough with difficulty breathing, past 2 weeks.
cIn Bangladesh, a fully vaccinated child is one who has received the following: BCG vaccine; three doses of DPT; three doses of OPV, given with DPT; and one dose of measles vaccine. All vaccines must be administered by the time the child is 1 year of age.
dFamily/Friend experience with the disease.
Willingness to Pay
As seen in Table 2, the mean willingness to pay is much higher in the open-ended elicitation method ($18.01) as compared to the bidding game ($5.94) and the two payment cards of 0–1000TK ($2.38) and 0–2000TK ($6.40). This result contradicts the hypothesis that the open-ended method is the most conservative.
Willingness to Pay across Elicitation Format (US$)
Given the dichotomous choice focuses on a quantity demanded rather than price, and that we only used two starting points, this method did not lead to a mean willingness to pay, although one could have been estimated if a specific functional for of the demand curve had been assumed (Reference Whynes24).
Demand Curves
The traditional demand curve with price on the Y-axis and quantity on the X-axis relates quantity as a function of price. If one wanted to look at the inverse demand (where willingness to pay is a function of quantity), we could reverse this graphic, as seen in Figure 2, with (log) price on the X-axis. Comparing the inverse demand elicitation methods, we find that the open-ended produces an inverse demand curve that has much higher willingness to pay for all levels of demand. Again, without assuming a functional form for the dichotomous choice elicitation method, a continuous demand curve cannot be estimated (Reference Whynes24).
Inverse demand (adjusted for income and starting point). (4 ln Takas ~ $0.80, 5 ln Takas ~ $2.18, 6 ln Takas ~ $5.93, 7 ln Takas ~ $16.13, 8 ln Takas ~ $43.84)
Yea-saying Bias
Dichotomous choice and bidding game elicitation formats both use yes/no questions and hence were both tested for yea-saying bias. With regard to the dichotomous choices methods, of the fifty-eight households who were asked if they would be willing to pay 100TK, 84 percent said Yes, and of the seventy-three households who were asked if they would be willing to pay a total of 300TK for all three doses of the pneumococcal vaccine, 67 percent said Yes. Comparing these two proportions, there was a significant difference detected (p > .023), with more people saying yes to the lower price, indicating that the method has some construct validity. To determine whether there was yea saying present at both prices, the proportion of demanders were compared with the other elicitation formats for the same prices. Compared with the level of demand from the other elicitation formats, we found some evidence of yea-saying with the dichotomous choice format. There was less evidence of yea-saying for bidding game as the bids varied.
Starting Point Bias
Again focusing on the dichotomous choice and bidding game elicitation formats, it appears that there is starting point bias, and the starting bias is certainly related to the initial presence of yea-saying. For the bidding game, only 34 percent (20/58) of the group that started with $1.47(100TK) eventually bid up to $4.41(300TK), while 67 percent (49/73) of the group that started at 300TK bid to that level (p > .001).
Range Bias
There is strong evidence of range bias based on the format of the payment cards. The mean of the short payment card format was $2.38 (SD 3.60) as opposed to the long payment card format $6.40 (SD 8.72). This is a significant difference between the two formats (p > .001), whose only difference was the range of prices presented to the respondent.
Strengths and Weaknesses of the Methods
Each elicitation format has strengths and weaknesses. Unlike reported data in the literature (Reference Johannesson, Jonsson and Borgquist15;Reference Whynes24), this study found the open-ended format produced the least conservative estimates. This was a surprise as most families currently receive childhood vaccinations for free; it was hypothesized that the open-ended question would be very conservative as the concept for paying for vaccines is not familiar with the majority of this community. The open-ended elicitation methods failed the construct validity tests when regressed against income, drawing caution when considering its validity for decision making. The payment scale format appeared to be extraordinarily influenced by the scale itself. Although both payment cards were equally sensitive to household consumption, they were highly prone to range bias and were not considered useful for the purposes of decision making without a clear justification for the range used. The dichotomous choice format was sensitive to the starting point bias, indicating that families did understand the question and based their decision on the price. For the bidding game, elicitation format was less sensitive to starting point, indicating that respondents had more time to contemplate their response, and hence were less influenced by the starting point. There was some indication that the dichotomous choice format was prone to yea-saying and the sensitivity of the dichotomous choice format to income increased as the price of the questions increased, notwithstanding a potential yea-saying bias and starting point bias. The bidding game and dichotomous choice questions were both highly sensitive to tests of household consumption. The bidding game elicitation format was more sensitive to income than the other results, and although it is influenced by the starting point, such bias can be adjusted when using multivariate regression. Therefore, in terms of selecting an elicitation format to use for this policy decision making, the bidding option proved to be the only option that provided tests which both passed the validity tests and for which potential bias can be adjusted.
CONCLUSION
The findings strongly support previous studies that have suggested that different elicitation formats give rise to significantly different demand curves (Reference Ryan, Scott and Donaldson23). This is important for policy/decision making as these data are then used to populate benefit-cost ratios, the profitability of an intervention, consumer surplus, and demand elasticity. When determining which elicitation format to choose, it is important to first consider the use of the outcome data and which format is most biased and then determine whether there are biases, if they can be adjusted for. If the data will be used to calculate elasticity, quantity, or market shares, one should use a format that will produce a demand curve, that is, the dichotomous choice elicitation format. If, however, the focus is on value, then one of the alternative methods should be considered.
CONTACT INFORMATION
Rebekah R. Heinzen, PhD (rheinzen@jhsph.edu), Research Assistant, GAVI's PneumoADIP, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street E8536, Baltimore, MD 21205
John F. P. Bridges, PhD (jbridges@jhsph.edu), Assistant Professor, Department of Health Policy & Management, Johns Hopkins Bloomberg School of Public Health, 624 N. Broadway, Room 689, Baltimore, MD 21205
