I. Introduction
Every spring, wine enthusiasts and professionals turn to Bordeaux for the en primeur market. From March to April, tastings take place while the en primeur campaign itself usually runs from late April to June. Over the past decade, however, the en primeur market has evolved. More and more wine tasters and journalists take part in the en primeur tastings and create enthusiasm for a vintage. Buyers have become a heterogeneous mix of wine lovers, collectors, and investors, who are sometimes more concerned about the brand or the potential for price appreciation than consumption. Deciding which wines to buy has therefore become challenging. This is reinforced as the taste and quality of Bordeaux wines can strongly vary from one estate to another and from one vintage to another. Thus, Bordeaux wines are considered experience goods. One has to experience (i.e., consume) the wine to evaluate its quality.Footnote 1 Storchmann (Reference Storchmann2012) argues that the very nature of wine as an experience good generates an information void for customers that wine experts can help to fill. Rosen (Reference Rosen2012) illustrates this influence that wine critics enjoy, writing that Robert Parker “has been called guru, emperor, pope … his name is now a verb (parkerize) used for the creation of a style of wine that pleases his taste.” In this article, we therefore study wine critics, their ratings, and the influence they exert on en primeur prices.
So far, the existing academic literature has mostly addressed questions on the relation between producers, vintage, climate, or expert opinions and wine prices. Most papers have also focused on the relation between scores and prices and, more specifically, the influence of Robert Parker on the en primeur market. This article goes further by taking a broader perspective. Before examining the influence of the different expert scores on prices, we want to specify the way in which experts rate wines. Our dataset includes the scores of 12 leading critics for the vintages 2003–2012. We first study the rating approach of these experts. This analysis encompasses both their rating scale and philosophy. We then identify and quantify variables that may explain scores to gain a better understanding of the predictability of expert scores and their added value. We further investigate the degree of consensus among the different experts. Good experts may disagree when it comes to a particular wine, but on average they should display converging opinions. In a last step, we test whether expert surprises in scores correlate with prices.
We find that European critics are less transparent and use scoring systems that are often more difficult to interpret than do their American counterparts. Over the past decade, several critics have decided to change their rating scale, which has a direct influence on the score distribution. In general, European critics appear to be more severe than their American counterparts. Experts also reach a relatively strong consensus on overall wine quality, but have more diverse opinions on wines that achieve a surprising level of quality given the vintage, the ranking, or the appellation from which they originate. Our evidence also suggests that Robert Parker and Jean-Marc Quarin are the most influential critics, as a 10% surprise in their scores leads to a price increase of around 7%. We further find that their impact is higher for appellations and estates that are not covered by the official 1855 classification and for the best vintages (2005, 2009, 2010). Both appear to be complementary, catering to the French and U.S. market, respectively.
Our main contributions are twofold. First, we examine how 12 of the most respected experts in the world rate wines over a 10-year period. To the best of our knowledge, this article is the first to consider a dataset with such a large number of experts over such a long period. This allows us to investigate the precise differences in the rating methodology of each expert. We believe it is an important step toward better understanding the critics' work and influence on the wine market. Second, we analyze the relation between wine scores and prices using an approach that is robust toward endogeneity and unaffected by different distributional characteristics of the scores. Moreover, as we study multiple critics, we are able to contrast the various tasters and their influence on prices. Notably, our paper contributes to the literature by demonstrating not only that Robert Parker and Jean-Marc Quarin scores and en primeur prices are strongly correlated but more importantly that this correlation is much higher than it is for other experts.
The remainder of the paper is structured as follows. The next section reviews the existing literature while section 3 presents our dataset. Section 4 examines how experts rate wines. Section 5 displays an analysis of the relation between expert scores and en primeur prices, and section 6 concludes.
II. Literature
To date, the literature has focused mainly on the relation between climate or expert opinions and wine prices. Different studies, such as Ashenfelter (Reference Ashenfelter2010) or Ashenfelter et al. (Reference Ashenfelter, Ashmore and Lalonde1995), show that the final quality of a wine depends strongly on the climatic conditions that prevailed during the growing season. They further indicate that climate appears to predict wine quality better than do wine experts. Haeger and Storchmann (Reference Haeger and Storchmann2006) suggest that for U.S. Pinot Noir, expert ratings improve the explanation of wine prices by weather data only marginally. Ashenfelter and Jones (Reference Ashenfelter and Jones2013) study the relation among climate, expert opinion, and prices. They illustrate that expert opinions on Bordeaux wines not only correctly reflect available climatic data but also display an independent effect and private information for the highest-rated wines.Footnote 2
Jones and Storchmann (Reference Jones and Storchmann2001) specifically look at Parker ratings and show that they have stronger influence on prices for wines from small châteaux cultivating Cabernet-Sauvignon that have been highly rated in the past. In a theoretical paper, Mahenc and Meunier (Reference Mahenc and Meunier2006) indicate that asymmetric information could be a source of inefficiency that leads en primeur prices to be set too high by the châteaux. In particular, they argue that the usual assumption of perfect competition, while acceptable for bottled wine, is not appropriate for en primeur wine. Ali and Nauges (Reference Ali and Nauges2007) demonstrate that the marginal impact of Wine Spectator and Parker scores on en primeur prices is small but significant. Ali et al. (Reference Ali, Lecocq and Visser2008) examine the impact of Parker by exploiting the fact that he rated the 2002 Bordeaux vintage after its price release and confirm a limited but significant impact of Parker points on prices. Hay (Reference Hay2010) finds evidence that each additional Parker point is worth GBP 10 a bottle while Dubois and Nauges (Reference Dubois and Nauges2010) also find a significant influence of Parker on prices. Roberts and Reagans (Reference Roberts and Reagans2007) find evidence that new world wines sold on the U.S. market display a positive rating-price relationship that increases for producers with critical exposure. San Martín et al. (Reference San Martín, Troncoso and Brümmer2008), by contrast, indicate that expert opinions have less influence on prices of Argentinian wines than labeling, location, or variety. Friberg and Grönqvist (Reference Friberg and Grönqvist2012) look at the effect of favorable reviews on wine demand in Sweden. They document that, following positive reviews, demand rises by 6% and remains significant for the following 20 weeks. Hilger et al. (Reference Hilger, Rafert and Villas-Boas2011) refine these results and suggest that demand decreases for wines with low scores, while it increases for wines with average or high scores. They argue that expert opinion labels therefore transmit not only shelf visibility but also some information on quality to consumers. Ramirez (Reference Ramirez2010), in a study on Napa Valley wines, shows that not only expert ratings but also their comments have a positive impact on wine prices.
Another avenue that has gained attention considers the way in which wine prices act as quality signals on the market. To convince consumers to buy their products, producers can send quality signals in different forms (price, advertising, brand name, warranty).Footnote 3 This also holds true on the market for fine wine in which consumers may use the label (Gluckman, Reference Gluckman1990) or awards (Orth and Krška, Reference Orth and Krška2001) to infer product quality. Lockshin and Rhodus (Reference Lockshin and Rhodus1993) show that wine prices are also used as strong quality signals. Evidence by Ashenfelter (Reference Ashenfelter2010) and Ashenfelter et al. (Reference Ashenfelter, Ashmore and Lalonde1995) indicates that, at release, wine prices diverge from their full information prices. However, as additional information is released, these wine prices tend to move toward their full-information equilibrium. Schnabel and Storchmann (Reference Schnabel and Storchmann2010) find some support for the hypothesis put forward by Bagwell and Riordan (Reference Bagwell and Riordan1991) that prices can be used to signal product quality. Their results show that price signals display a negative relation with additional information and a positive link to wine quality. Gergaud and Livat (Reference Gergaud and Livat2007) analyze a sample of 6,000 European wine consumers and find that prices substitute for umbrella branding as a signal. These prices acting as signals of wine quality are finally less used by wine connoisseurs than nonconnoisseurs.
III. Data
Our dataset includes information on en primeur scores and prices for a total of 122 Bordeaux estates over the vintages 2003 to 2012. These estates have been selected on the basis of the number of available price and score observations as lesser-known estates are rated by very few experts. These 122 estates represent more than 90% of the en primeur market.Footnote 4 More specifically, our sample includes 67 reds from the Médoc, 10 reds and 6 whites from Pessac-Léognan and Graves, and 39 reds from Pomerol and Saint-Emilion.Footnote 5
A. En Primeur Prices
In Bordeaux, the relation between producers and final customers is complicated. Producers typically use the services of courtiers to help them set their release prices. They then sell their wines to négociants (also known as the négoce or the trade), which take a percentage before selling the wines to merchants all over the world. Merchants then add their own margin, before selling it to the general public. In this article, we use ex-VAT (value-added tax) final customer prices that we have obtained from different sources.
Table 1 reports descriptive statistics on prices. As apparent from the first line, our dataset contains release prices for almost all wines included in the sample, except for some rare wines that were not released en primeur (e.g., in 2012, Latour stopped selling its wines en primeur). The average is always higher than the median because a minority of top estates sell their wines at high prices. One can also observe an asymmetric relationship between price changes and quality from one vintage to another. Excellent vintages, such as 2005 or 2009, lead to high price increases. However, if subsequent vintages are of lower quality, prices decrease slowly. The gap between the most expensive and cheapest wines has also widened since 2003.
Wine Prices (in Euros): Descriptive Statistics
B. En Primeur Scores
We have hand-collected en primeur scores from 12 internationally acknowledged wine critics. Table 2 provides basic information on each critic, including name, number of wines tasted (in our sample), country of origin, and a brief description. Not all critics have rated all wines, and some have skipped entire vintages. For instance, the Grand Jury Européen has not rated the 2003, 2011, and 2012 vintages. Similarly, no information is available for Bettane and Desseauve scores for the 2003 vintage. James Suckling only started rating en primeur Bordeaux in 2010, while Farr Vintners only provides ratings for a limited number of wines. Among these 12 critics, four are from France, four from the United Kingdom, and four from the United States.
Presentation of the Wine Experts
Magazines such as La revue des vins de France, Decanter, or Wine Spectator employ different experts to rate the different Bordeaux appellations or vintages. Notably, James Suckling was the Bordeaux expert at Wine Spectator before launching his own website in 2010. Similarly, Michel Bettane and Thierry Desseauve were the editors of La revue des vins de France before leaving the magazine in 2004. The Grand Jury Européen is also special as it is not a single expert but an association that includes several well-known experts. Each tasting session involves at least 15 experts who rate each wine and is subject to strict conditions. According to François Mauss, the founder of the association, the scores emanating from their tasting sessions should be less subjective than those from individual critics as they include and aggregate the opinions of different experts, thereby reducing idiosyncratic biases.
IV. Analysis of Expert Scores
A. How Do Experts Rate the Wines They Taste?
(1) Rating Systems Used by Experts
Comparisons are difficult to make, as not all experts use the same scale to establish their scores. Table 3 shows the various rating systems that are currently used. It is apparent that two distinct approaches are followed: the American 100-point scale and the European 20-point scale.
Rating System Used by Wine Experts
Notes: FV: Information is not available, but the scale used is apparently similar to that of JR.
GJE: Information is not available, but the scale used is apparently similar to that of RP.
BD: The scale is based on the one provided in their book (Le grand guide des vins de France). However, it is not clear whether this score is valid for both in-bottle and en primeur wines.
All American critics (including Neal Martin [NM]) use a similar scoring approach, with six or seven quality tiers and scores ranging from 50 to 100. James Suckling (JS) states on his website: “I rate wines using the 100-point scale. I have used this point system for close to 25 years. I still believe it is the simplest way to rate a wine, with its origins from grade school in the United States.”Footnote 6 The definitions of the various tiers have only minor differences . In particular, Stephen Tanzer is more severe, as can be seen from his scale. For instance, a wine deemed excellent will get a score marginally lower than or equal to 89 from Stephen Tanzer (ST), while the same wine gets a better score from Robert Parker (RP) or James Suckling (who do not have a specific category for excellent wines: they are either very good or outstanding). The Grand Jury Européen (GJE) uses the same scale as its American counterparts with some differences in the definition of the various tiers.
In Europe, most critics apply a 20-point scale. However, their scores are more difficult to compare than those from their American counterparts. For instance, a wine rated 16 points would be considered excellent by Jean-Marc Quarin (JMQ), very good by the Revue des vins de France (RVF) and Bettane and Desseauve (BD), and just good by Decanter (DEC) and Jancis Robinson (JR). In general, British critics (DEC and JR) can be considered less severe than French critics (JMQ, RVF, and BD). Theoretically, scores could be lower than 10, but, in practice, this possibility does not occur. This ambiguity and cultural specificities explain why scores by French critics are often lower than those of American critics. Moreover, some scales may sometimes be inconsistent or have gaps. In particular, some critics, like Jean-Marc Quarin and Decanter, while primarily using a 20-point scale, also provide the equivalent scores on a 100-point scale. The relationship between the two scales is, however, incoherent at times.
Another key difference between American and European critics regards the degree of transparency of their scoring method. The former clearly disclose and explain their rating system. In Europe, it is more complex. First, three critics (GJE, Farr Vintners [FV], and BD) do not disclose their scoring system. It is, however, possible to infer some basic information on the basis of the tasting notes that are published with the observed scores.
Second, over the past decade several critics have changed their rating scale, sometimes multiple times . The Grand Jury Européen used a 5-star scale for 2004 and 2007 vintages but a 100-point scale in 2005 and 2006. The Revue des vins de France switched from a 5-star system to a 20-point system in 2006 after having used a 5-point scale in 2003–2005. Decanter used a 5-star rating scheme in 2003, 2004, 2006, and 2008 but a 20-point scale for all other vintages. Neal Martin is the only critic who used a 25-point scale, in 2003 and in 2005. The remaining six critics have never changed their scoring system (JMQ, FV, JR, RP, ST, and Wine Spectator [WS]).
(2) Effective Distribution of Expert Scores
Table 4 reports key statistics on the scores delivered by the various experts. Several critics have opted either for the 20-point scale (RVF and DEC) or the 100-point scale (GJE and NM). For those critics, we report two sets of statistics, one for each scale used. Figure 1 shows the entire distribution of scores for each critic.
Cumulative Distribution of Expert Scores
Key Statistics on Expert Scores
Notes: STD denotes standard deviation, and CV denotes coefficient of variation (defined as the ratio of s/m); the line on the right of “m – 1s” shows the average (m) minus one standard deviation (s), and the line below it reports the percentage of observed scores that are lower or equal to this score.
Several critics have changed their rating scheme over the last decade. A star (*) has been added next to the names of those critics in order to distinguish their old scheme from the new one.
As can be observed in Table 4, European critics sometimes give perfect scores to en primeur wines. At the exception of James Suckling, American experts generally wait until the wines are bottled before eventually awarding such a score. Scores from European experts can also be quite low. They are often lower than 16 (or 80 points on the American scale) and some of them even below 14 (or 70). The higher coefficient of variation further indicates that their scores are more variable than those of American tasters.Footnote 7 These observations do not necessarily mean that European experts are more severe than Americans but, rather, indicates that they use a larger proportion of the available score range. These observations reflect a cultural difference between American and European experts.
Among the critics who use a 100-point scale, James Suckling and Robert Parker have the highest average scores, while Stephen Tanzer has the lowest scores. Neal Martin is in the middle of these two extremes. Scores from the Grand Jury Européen are relatively low on average and have a limited degree of variability, which is due to its aggregation of scores from several critics. Figure 1 further confirms that Stephen Tanzer is, on average, more severe than his U.S. colleagues. For critics who rate on a 20-point scale, Decanter is the most generous and Farr Vintners the most severe. This last observation is surprising given the fact that Farr Vintners is a wine merchant, but it is consistent with its reputation of impartiality.
Expert scores have a kurtosis that is often slightly above 3, and the skewness strongly varies from one expert to another. The assumption of normality has to be rejected (at the 5% level) for most experts, except for the Grand Jury Européen and Decanter. Scores from these two experts are distributed almost linearly along the portion of the scale they use (see Figure 1). The Revue des vins de France exhibits the most negative skewness due to the larger number of rather low scores relative to the number of high scores. Stephen Tanzer presents the opposite situation. Being more severe than some of his counterparts on average, he also awards high scores to wines he considers exceptional. The high kurtosis exhibited by Robert Parker scores is explained by a significant number of very low and very high scores, although most of his scores are centered around the mean.
The rating scheme has a direct impact on the way in which experts rate wines. For instance, the Grand Jury Européen has used a 5-point scale for the 2004 and 2007 vintages and has opted for a 100-point scale for all the others. Figure 1 shows that this change led to a shift in the scores toward the right-hand side of the graph, thereby indicating that scores, when expressed on a similar scale, look more generous today than in 2004 or 2007. Interestingly, if one considers the case of the Revue des vins de France, the situation is reversed, with an increase of the number of low scores and a reduction of high scores. Finally, the distribution of Decanter scores did not significantly change after its decision to change its scoring approach. Overall, these observations reinforce the impression that scores expressed on a 20-point scale look less generous than those expressed on a 100-point scale.
B. Determinants of Expert Scores
In this section, we examine the determinants of expert scores. To make scores comparable for the different critics and facilitate the interpretation of the results, we have rescaled all scores to a range between 1 and 100 points. We consider variables such as the appellation (used as a proxy for natural endowments), the rankingFootnote 8 of a wine (used as an estimate of reputation and also to complement the information provided by the appellation), and its vintage (used to take into account climatic conditions and their impact on quality). We further assume the relation between all these explanatory variables and expert scores to be linear and of the form
$$Score_{i,t}^{\left( j \right)} = \alpha _0^{\left( j \right)} + \sum\nolimits_{v = 2003}^{2012} {\alpha _v^{\left( j \right)} D_{\left\{ {vint_t = v} \right\}}} + \sum\nolimits_{a = 1}^9 {\alpha _a^{\left( j \right)} D_{\left\{ {app_i = a} \right\}}} + \sum\nolimits_{r = 1}^8 {\alpha _r^{\left( j \right)} D_{\left\{ {rank_i = r} \right\}} + \varepsilon _{i,t}^{\left( j \right)}} $$
where
$D_{\left\{ {vint_t = v} \right\}} $
takes the value 1 if the wine is from vintage v and 0 otherwise; similarly,
$D_{\left\{ {app_i = a} \right\}} $
and
$D_{\left\{ {rank_i = r} \right\}} $
take the value 1 if the wine is from the ath appellation and is part of the rth ranking category. We estimate the model using a conventional ordinary least squares (OLS) regression. We also consider a model, in which all scores are pooled together:
$$Score_{i,t} = \alpha _0 + \sum\nolimits_{v = 2003}^{2012} {\alpha _v D_{\left\{ {vint_t = v} \right\}}} + \sum\nolimits_{a = 1}^9 {\alpha _a D_{\left\{ {app_i = a} \right\}}} + \sum\nolimits_{r = 1}^8 {\alpha _r D_{\left\{ {rank_i = r} \right\}}} + \varepsilon _{i,t} $$
Model 1 enables us to assess how the different experts appraise wines from various appellations, vintages, and ranking categories. Model 2 provides additional information on the average preferences of all experts. These two models can be refined to determine which wines attract the best scores. Thus, we replace the appellation and ranking dummies in models 1 and 2 by a wine producer dummy to obtain:
$$Score_{i,t}^{\left( j \right)} = \alpha _0^{\left( j \right)} + \sum\nolimits_{v = 2003}^{2012} {\alpha _v^{\left( j \right)} D_{\left\{ {vint_t = v} \right\}}} + \sum\nolimits_{k = 1}^{122} {\alpha _k^{\left( j \right)} D_{\left\{ {ID_i = k} \right\}} + e_{i,t}^{\left( j \right)}} $$
$$Score_{i,t} = \alpha _0 + \sum\nolimits_{v = 2003}^{2012} {\alpha _v D_{\left\{ {vint_t = v} \right\}}} + \sum\nolimits_{k = 1}^{122} {\alpha _k D_{\left\{ {ID_i = k} \right\}} + e_{i,t}} $$
where
$D_{\left\{ {ID_i = k} \right\}} $
takes the value 1 if the wine under consideration is the kth from the list and 0 otherwise;
$D_{\left\{ {vint_t = v} \right\}} $
is defined as before. Table 5 reports detailed results for the estimation of models 1 and 2. All coefficients are to be interpreted as the score difference between a wine from any ranking, any appellation, any vintage, and a 2010 Pauillac first growth (used as the reference).
Determinants of Expert Scores (Results from the Regression Analysis)
Notes: The first row indicates expert names for the first 12 columns and is estimated with model 1. The last column, All, pools all experts and is estimated with model 2.
The reference is a first growth from Pauillac, 2010 vintage.
Others indicate second wines, cru bourgeois, and grand cru classé (in Saint-Emilion).
— denotes that the respective expert has not tasted the specific vintage
***, **, and * denote significance at the 1%, 5%, and 10% level respectively.
In Bordeaux, status, tradition, terroir, and climate have great influence on both perceived and effective quality. However, unobservable variables (e.g., specific viticultural or oenological decisions made by a winemaker or the influence of a particular microclimate in a given vintage) may lead to positive or negative surprises. Customers expect wine critics to be able to identify such surprises and to be sufficiently precise in their tastings to ensure that their scores are truly reliable. Cao and Stokes (Reference Cao and Stokes2010) argue that wine experts should be judged on the basis of three characteristics: bias (difference between the scores of an expert and the average score of a panel of experts), variation (degree of randomness in an expert's scores), and discrimination (ability to distinguish wines on the basis of their quality).
In the present context, a low R 2 for model 1 would indicate that scores are erratic (or random), which may cast doubt on their reliability. However, an R 2 that is too high would suggest that scores do not provide information on the unobservable quality of a wine and thereby do not carry added value (low discrimination ability). Scores from the Grand Jury Européen being aggregated from the opinions of several experts should only have a minimal bias and exhibit a low degree of randomness (the noise imbedded in the appreciation of different judges is cancelled out). Moreover, they are obtained under strict blind tasting conditions, these scores cannot be influenced by information such as appellation or ranking and should therefore facilitate the identification of genuinely successful wines in a given vintage.Footnote 9 We therefore use the R 2 obtained on the Grand Jury Européen scores as a benchmark. Indeed, the Grand Jury Européen exhibits an R 2 of 0.53, the third highest among all critics.Footnote 10 Farr Vintners and James Suckling display similar R 2, at 0.54 and 0.55. The other R 2 are generally close to 0.40–0.50, indicating that about 40% to 50% of expert scores can be explained by generic variables related to quality and reputation. Jancis Robinson and Neal Martin seem to be the most independent critics, with R 2 of 0.29 and 0.38, respectively. This low R 2 suggests that their scores are less reliable than those from other critics.
Model 1 also allows us to assess whether an expert favors wines from a particular appellation or rank. If expert opinions were true measures of a wine's objective quality, they should all lead to similar conclusions concerning the quality of a wine and therefore share the same preferences. In general, wines from the right bank, in particular, those from Pomerol, obtain better scores than those from the left bank. On the left bank, Pauillac, Saint-Estèphe, and Saint-Julien are the preferred appellations of most critics. Margaux and Médoc are less appreciated. The rare whites from the Graves appellation enjoy high scores from all critics.
Although there is a general consensus regarding the best and the worst appellations, some significant differences among critics can be observed. American critics are more negative about wines from the northern part of the Médoc (in particular, Saint-Estèphe) but more positive about wines from the right bank and Pessac-Léognan. Robert Parker and, to a lesser extent, Stephen Tanzer are the most positive about these wines, while Jancis Robinson exhibits almost opposite preferences. These results are in line with Cardebat and Livat (Reference Cardebat and Livat2013), who conclude that American experts (Robert Parker, Stephen Tanzer, and Wine Spectator) favor wines from the right bank and Pessac-Léognan, while Jancis Robinson gives on average better scores to wines from the left bank.
The relation between the 1855 classification and expert scores is statistically and economically significant. That is, a second growth from the Médoc will typically obtain a lower score than a first growth. First growths “A” from Saint-Emilion are generally less appreciated than first growths from the Médoc, but the difference is statistically insignificant. When it comes to first growths “B,” the difference widens and becomes statistically significant. Overall, critics appear to demonstrate the same preferences in terms of ranking and appellation, but the amplitude of those preferences varies from one critic to another.
We also examine vintage regression coefficients in order to determine the qualitative hierarchy of the last ten vintages. In general, the preferred vintage is 2009, which is followed closely by 2010. Most experts also rank 2005 quite high, except for Decanter and the Grand Jury Européen; 2006, 2008, and 2012 are good to very good but relatively heterogeneous. Appreciation varies quite significantly from one expert to another. In some instances, 2003 can be outstanding but is more generally only average to good while 2004, 2007, and 2011 seem to be weaker.
The estimation of models 3 and 4 allows us to identify the wines that are the most praised by experts.Footnote 11 Latour displays the best scores over the sample period. It is followed by Margaux, Haut Brion, and Lafite Rothschild, with Mouton Rothschild and Léoville Las-Cases not far behind. Among the first growths “A” from St-Emilion, Ausone, and Cheval Blanc are preferred and are close to Latour. Angélus and Pavie, which have been recently promoted, attract lower scores. In general, the results are in line with the official classification but nevertheless some striking differences can be observed. For instance, Palmer (third growth) and Pontet-Canet (fifth growth) fare much better than their ranking suggests, while Rauzan-Gassies or Durfort-Vivens (both second growth) have disappointing scores. This illustrates the inherent limitations of the rigid 1855 classification. Moreover, models 3 and 4 allow a finer analysis of the Pomerol appellation, for which no classification exists. Pétrus and Lafleur are clearly the preferred wines from this appellation, followed by Eglise-Clinet, Vieux Château Certan, and Trotanoy. The higher degree of precision in models 3 and 4 is further reflected in their R 2, which are on average close to 0.75 for model 3 and 0.40 for model 4 versus 0.50 for model 1 and 0.26 for model 2.
C. Consensus Among Wine Experts
In this subsection, we concentrate on the consensus criterion put forward by Ashton (Reference Ashton2012, Reference Ashton2013). As stated by Hodgson (Reference Hodgson2008), “good judges [should] agree with each other.” In the case of wine, an objective measure of quality does not exist which makes consensus an essential criterion to distinguish good from average experts.
In order to examine the degree of agreement among the various wine experts, we compute the correlations between their scores and also between their residual scores of models 1 and 2. By definition, a residual is the part of an observation that a model cannot explain and can therefore be interpreted as a surprise. In the present context, a positive residual would thus indicate that the expert has been positively surprised by the quality of a wine. Residuals from model 1 capture the score surprises, while those from model 2 capture both these surprises and the preferences (in terms of appellation and ranking) of each expert compared to the others. The corresponding correlation coefficients are reported in Table 6. We use Spearman's rank correlations, as the distribution of scores is not normal and varies from one expert to another.Footnote 12 Panel A indicates correlation coefficients of expert scores. Panel B uses the residuals of equation 1 (lower triangle) and of equation 2 (upper triangle).
All correlation coefficients in Panel A are positive and statistically significant. The average correlation is close to 0.60. This indicates a strong consensus among all experts on overall wine scores. U.S. experts seem to agree more with one another than European critics do. James Suckling has the highest correlation with other critics. Jancis Robinson and the Revue des vins de France, by contrast, are much less correlated with their colleagues.
The significantly positive correlation coefficients in the upper triangle of Panel B further demonstrate that experts generally share similar preferences in terms of appellations, ranking categories and vintages. These correlations are, however, lower than in Panel A, which suggests that subtle differences in preferences among the various experts nevertheless exist. This is in line with the results in the previous subsection. James Suckling and Stephen Tanzer are the critics whose preferences are the most correlated with the others. Jancis Robinson exhibits the most idiosyncratic preferences.
The lower triangle of Panel B shows that the divergence among critics on unexpected quality is more pronounced, as the correlation coefficients calculated on the basis of score surprises are much lower than those obtained earlier. This indicates that score surprises differ substantially among the various critics. Thus, while experts appear to reach a relatively strong degree of consensus on overall wine quality, they tend to have more diverse opinions on wines that achieve a surprising level of quality given the vintage, the ranking, or the appellation from which they originate. As before, Jancis Robinson appears to be the most independent and least correlated with respect to her colleagues while James Suckling and Stephen Tanzer display the highest correlations.
V. Expert Scores and En Primeur Prices
In the previous section, we showed that a few key variables explain a substantial part of the variation in expert scores. We now focus on the relationship between expert scores and en primeur prices. Variables such as appellation, ranking, and vintage account for more than 65% of the variation in en primeur log prices. If we add individual producer information, this figure increases to 95%.Footnote 13 These variables capture a very large proportion of the variation in both expert scores and prices, as they are intrinsically related to climate, natural endowments, vineyards, and winemaking techniques, or, in sum, to quality. Moreover, the fact that these variables appear to explain prices better than scores is due to the prevalence of status and tradition in Bordeaux: between two wines of similar quality, the one from the more renowned appellation or with the higher ranking will sell for a higher price.Footnote 14
A. Methodology
Even if a substantial part of quality can be captured by a limited set of observable variables, expert scores may still be informative and potentially exercise influence on prices. That is, part of the quality of a wine is not directly observable and known to the general public. In any vintage, some wines achieve a surprisingly high (or low) quality level that cannot be fully explained by observable variables. Presumably, only the wine producer and some critics who have tasted the wine should be capable of fully appraising its quality. Ratings of trustworthy experts should therefore reflect the overall quality of a wine and contain information related to both observable and unobservable quality attributes.
A common specification used in literature regresses wine prices on a set of explanatory variables, including expert ratings (see Haeger and Storchmann, Reference Haeger and Storchmann2006). In the present context, we use the same explanatory variables as in model 3 to control for observable quality attributes and propose a model that relates wine prices to the vintage, the estates, and expert ratings:
$$p_{i,t} = \theta _0^{\left( j \right)} + \sum\nolimits_{v = 2003}^{2012} {\theta _v^{\left( j \right)} D_{\left\{ {vint_t = v} \right\}}} + \sum\nolimits_{k = 1}^{122} {\theta _k^{\left( j \right)} D_{\left\{ {ID_i = k} \right\}}} + \gamma ^{\left( j \right)} Score_{i,t}^{\left( j \right)} + \varepsilon _{i,t}^{\left( j \right)} $$
where p i,t = log (P i,t ). For low correlations between the scores and the exogenous variables (the dummies for wine producers and vintages), the coefficients θ (j) should be similar for all experts. In this article, we focus on a modified version of equation 5. Instead of focusing on prices, we analyze their evolution. We define the percentage change in price (the log-return) between year t-1 and t as r i,t = p i,t −p i,t −1. By taking the first difference of equation 5, we obtain the following specification:
$$r_{i,t} = \beta _0^{\left( j \right)} + \sum\nolimits_{v = 2004}^{2012} {\beta _v^{\left( j \right)} D_{\left\{ {vint_t = v} \right\}}} + \gamma ^{\left( j \right)} {\rm \Delta} Score_{i,t}^{\left( j \right)} + \varepsilon _{i,t}^{\left( j \right)} $$
where
${\rm \Delta} Score_{i,t}^{\left( j \right)} = Score_{i,t}^{\left( j \right)} - Score_{i,t - 1}^{\left( j \right)} $
. To ensure that the β coefficients capture the average return between each pair of vintages, we further define
${\rm \Delta} ^s Score_{i,t}^{\left( j \right)} $
as
${\rm \Delta} Score_{i,t}^{\left( j \right)} - \displaystyle{1 \over N}\mathop \sum \nolimits_{i = 1}^N {\rm \Delta} Score_{i,t}^{\left( j \right)} $
and substitute it into equation 6 to obtain:
$$r_{i,t} = \beta _0^{\left( j \right)} + \sum\nolimits_{v = 2004}^{2012} {\beta _v^{\left( j \right)} D_{\left\{ {vint_t = v} \right\}}} + \gamma ^{\left( j \right)} {\rm \Delta} ^s Score_{i,t}^{\left( j \right)} + \varepsilon _{i,t}^{\left( j \right)}.$$
Equation 7, while conceptually similar to equation 5, displays advantages. First,
${\rm \Delta} ^s Score_{i,t}^{\left( j \right)} $
captures the part of expert ratings that is not related to observable variables (e.g., wine producer or vintage), which is the truly informative and valuable part of expert scores. This can also be considered a surprise in quality, estimated by a particular expert. We therefore refer henceforth to this variable as the surprise in score. If the expert is correct or if his scores have an influence on customer choices, such surprises should correlate positively with prices. Second, this approach reduces multicollinearityFootnote
15
issues, as it uses a more parsimonious set of explanatory variables, which are uncorrelated with the surprise in score. As such, the coefficients in model 7 are not only easier to interpret but also estimated in a more robust manner than those in model 5.
A positive relation between
${\rm \Delta} ^s Score_{i,t}^{\left( j \right)} $
and r
i,t
does not necessarily imply that ratings influence prices. It may simply indicate that experts, as wine producers, are able to identify good and bad wines and to score, and respectively price, them accordingly. Prices and scores are both related to quality and may therefore lead to potential endogeneity problems that can be solved using one of the following two methods. First, equation 7 can be estimated using an instrumental variable technique. Instruments used in the existing literature are climate-related variables (see Cardebat et al., forthcoming; Haeger and Storchmann, Reference Haeger and Storchmann2006) or the lag of the potentially endogenous variable (see Malter, Reference Malter2014). In the present case, it is not possible to find an appropriate instrument as the variable, surprise in score, cannot be anticipated or instrumentalized by definition. A second approach is based on mediation theory (see Baron and Kenny, Reference Baron and Kenny1986; Hayes, Reference Hayes2013), which is widely used in human and social sciences. The underlying idea is to introduce a variable that controls for the surprise in quality into equation 7 to obtain:
$$r_{i,t} = \beta _0^{\left( j \right)} + \sum\nolimits_{v = 2004}^{2012} {\beta _v^{\left( j \right)} vint_t} + \gamma ^{\left( j \right)} {\rm \Delta} Score_{i,t}^{\left( j \right)} + \lambda ^{\left( j \right)} x_{i,t} + \varepsilon _{i,t}^{\left( j \right)} $$
x i,t serves as a proxy for the effective surprise in quality. In order to estimate it, we exploit the fact that our dataset contains scores of the most acknowledged experts in the world. As talented as an individual expert can be, he may nevertheless sometimes be overly optimistic (or pessimistic) about a wine, thereby leading to a positive (or negative) surprise in score for this wine. But if a wine receives positive surprises-in-scores from most experts, this suggests that its quality is surprisingly good. We therefore use the average surprises in scores as an estimate of x i,t . This average should not only be less noisy but, more importantly, also more informative than individual surprises in scores about the effective quality surprise. We, however, do not invoke the law of large numbers and do not argue that this average is equal to the surprise in quality. We only assume that it can serve as a proxy for it. Models 7 and 8 are further estimated using OLS and robust standard errors (see Arellano, Reference Arellano1987).
B. Main Results
Table 7 reports results on the relationship between surprises in scores and price changes following model 7. The different vintage coefficients give indications on the price evolution over the 2004–2012 vintages. Prices have strongly increased for the 2005 and 2009 vintages. In 2010 they increased slightly and, for all other vintages, they decreased. The last row displays the effect of the various experts on price changes. The surprise in score of eight critics significantly correlates with price changes while four critics (GJE, JR, NM, and RVF) do not appear to have an influence on prices. Robert Parker and Jean-Marc Quarin are the most influential experts, with a 10% increase in their scores leading to about 7% higher prices. These results confirm previous results on the strong influence that Robert Parker has on Bordeaux prices. Jean-Marc Quarin, though less known, has in-depth knowledge of the Bordeaux market, a region in which he lives, and is widely regarded as one of the most authoritative expert on Bordeaux wines in France. For the remaining six critics, prices go up by around 2% to 4.5% for a 10% surprise in score. The last column of Table 7 reports the results with the average surprise in score of all experts used as an explanatory variable in model 7. Results suggest that the average surprise has a stronger impact on price than any individual surprise, as a 10% higher average score leads to a 10% increase in prices. This is consistent with our hypothesis that the average surprise in score can be used as a proxy for the unobservable surprise in quality. When quality is especially high, producers will be inclined to increase their prices to send a signal to the market and presumably also because they believe their wine deserves to be sold at a higher price.
Score Surprises and Price Changes
Notes: Column “Average” reports the results for the surprises-in-score averaged over all experts.
2004 to 2012 denote the respective vintages with vintage 2010 as reference year.
— indicates that the respective expert has not tasted the specific vintage.
***, **, * shows significance at the 1%, 5%, and 10% level respectively.
In order to determine whether the positive relation between surprises in scores and price changes is due to a genuine score effect or to a quality effect, we resort to model 8. The purpose of this model is to distinguish the influence of an individual expert (captured by γ in model 8) and of quality (λ). Results are reported in Table 8. Coefficients attached to each vintage are again very similar and in line with the results in Table 7. Consistent with the results in model 7, λ is always positive and statistically significant. Controlling for surprises in quality, we find that only four critics have a positive and significant influence on prices. The two most influential critics are Jean-Marc Quarin and Robert Parker, who lead to a 5% price increase per 10% surprise in score. They are followed by Stephen Tanzer and Wine Spectator, which lead to 1.75% higher prices for a 10% surprise in score.
Average Surprises, Score Surprises and Price Changes
Notes: 2004 to 2012 denote the respective vintages with vintage 2010 as the reference year.
— indicates that the respective expert has not tasted the specific vintage
***, **, * shows significance at the 1%, 5%, and 10% level respectively.
In order to further refine our findings, we re-estimate model 8 and allow γ to be different, depending on the specific region, appellation, vintage, or ranking considered. The results indicate that in general few experts exert genuine influence on prices of wines from the left bank (Saint-Estèphe, Pauillac, Saint-Julien, Margaux, and Haut-Médoc). The situation is quite different, however, on the right bank (Saint-Emilion and Pomerol) and in Pessac-Léognan, as, in these areas, Robert Parker and Jean-Marc Quarin strongly affect prices. Wine Spectator also has a significant, albeit more limited, impact on the prices of wines from these appellations.
Wines included in the 1855 classification are generally not very sensitive to surprises in scores. By contrast, unclassified wines and those from Saint-Emilion (where the classification is regularly updated) show a strong relation between surprises in scores and prices. A most pronounced effect is once again found for Robert Parker and Jean-Marc Quarin and, to a lesser extent, for Stephen Tanzer and Wine Spectator. These results suggest that in appellations such as Pomerol, Saint-Emilion, and Pessac-Léognan and for unclassified wines in general, expert scores may substitute for the presence of a long-standing and rigid classification system, such as the 1855 classification that exists on the left bank, and may therefore affect prices more heavily.
Finally, the relation between surprises in scores and prices is dependent on the vintage. The effect is most pronounced in the best vintages (2005, 2009, and 2010). It thus appears that producers justify part of their price increases on the basis of the scores and comments of experts. Robert Parker is the expert who has an effect most often, as only in 2008 and 2012 does he fail to display significant influence. The year 2008 constitutes an interesting context and explains the finding obtained. For that vintage, many producers started selling their wines before Parker published his scores, and thus it is not surprising that his scores do not have an impact on prices.
In the same vein, it is possible that the relationship between surprises in scores and price changes is nonlinear. Very positive surprises in scores could have a larger impact on prices than negative surprises. We therefore re-estimate our model to account for the possibility of a nonlinear relation. Adding a quadratic term or a dummy taking the value 1 for positive surprises in scores to our model does not alter our results. Only Jean-Marc Quarin displays some slight nonlinearity, but the economic impact on the score-price relation is nearly insignificant.
Finally, we run a regression in which we include only the four critics (JMQ, RP, ST, WS) that have been found to influence wine prices significantly. This allows a comparison of their relative influence using a sample of wines that they have all rated. Once again, Robert Parker and Jean-Marc Quarin show very strong influence. Their influence remains stable compared to their respective univariate regressions, indicating that their scores are complementary. This suggests that Robert Parker may reflect the taste of American customers and Jean-Marc Quarin that of French consumers. Wine Spectator, by contrast, loses influence, and the impact for Stephen Tanzer turns insignificant. This tends to indicate that the two other critics dominate them.
Overall, our results suggest that Jean-Marc Quarin and Robert Parker are the two most influential experts on the Bordeaux market. Both appear to be complementary, catering to the French and U.S. markets, respectively confirming results by Cardebat and Livat (Reference Cardebat and Livat2013).
C. Robustness Tests
To verify the robustness of the results obtained in the previous section, we perform a series of tests.Footnote 16
-
1. Model misspecification: our results might be affected by the fact that we examine the relation between differences in scores and log prices instead of using these variables expressed in levels. To ensure that our results are robust to the particular specification used, we re-estimate them using equation 6 (levels) instead of equation 8 (differences). The results remain consistent, with Robert Parker and Jean-Marc Quarin still having a positive and significant effect on prices. It should be noted, however, that the coefficients associated with the producer and vintage dummies are less stable due to multicollinearity problems.
-
2. Definition of the surprise in score variable: the definition we use to estimate the surprises in scores may also influence the results. We consequently construct an additional surprise in score measure using the residuals of model 3. To account for the fact that the surprises are estimated in a first regression, we estimate the coefficients of the second regression and their standard errors using a bootstrapping approach. The results, and, in particular, the relation between surprises in scores and returns remain similar to those obtained with our initial definition of the surprise in score variable.
-
3. Definition of the surprise in quality variable: our results could also be affected by the proxy used to control for the unobservable surprise in quality. We therefore re-estimate model 8 using two additional proxies. We first use the median surprise in score instead of the average. We then use a common factor extracted from all surprises in scores using a principal component analysis (PCA). Replacing the mean surprise in score by either proxy does not affect the results (RP, JMQ, and, to a lesser extent, WS and ST still have a significant influence on prices). However, the relation between surprises in quality and changes in log-price becomes stronger when a proxy based on a PCA is used. This suggests that this proxy is more closely related to surprise in quality than the proxy based on the average surprise in score.
-
4. Influence of extreme observations: some experts may award extremely high or low scores. We examine whether such scores have an impact on our results by either discarding or winsorizing extreme values at the 5% and 10% level and re-estimating our different specifications. Overall, these two procedures do not alter our results (RP, JMQ, ST, and WS are the only experts that influence prices in both settings). More precisely, (i) when extreme surprises are first winsorized, the relation between surprises and price changes becomes slightly stronger (for all experts), but (ii) when extreme surprises are discarded, the influence of RP and ST increases, while JMQ and WS see their influence on prices decrease.
-
5. Sample size and missing values: we analyze the impact our sample and its definition has on results. In a first test, we restrict our sample and consider only the wines that have been rated by the four experts who have a significantly positive influence on prices (WS, JMQ, RP, and ST). The results are essentially unchanged. In a second test, we assess if the fact that four experts (DEC, GJE, NM, and RVF) have changed their rating scale during the sample period alters the results of the analysis. We thus re-estimate the regressions for these four experts using a restricted sample that only includes the data based on the scoring system each expert currently uses. In all cases, the results remain qualitatively similar.
VI. Conclusion
In this article, we provide an empirical analysis of 12 international wine experts on the market for Bordeaux wines over the period 2003–2012. We first analyze their rating approach and their influence on en primeur prices. We illustrate that score comparisons between various experts are difficult, as they use different approaches. European critics are less transparent and use scoring systems that are often more difficult to interpret than their American counterparts. Scores generally follow complicated distributions, and the assumption of normality has to be rejected for all critics, except for the Grand Jury Européen and Decanter. Over the past decade, several critics changed their rating scale. We show that this change has a direct influence on the score distribution, with the Revue des vins de France looking more severe than it used to be, and the Grand Jury Européen appearing to have become more generous. In general, European critics appear to be more severe than their American counterparts.
All critics demonstrate similar preferences in terms of ranking and appellation, but the amplitude of those preferences varies from one critic to another. In particular, American critics generally prefer Pessac-Léognan and Saint-Emilion to Pauillac, while European critics are more indecisive. Experts also reach a relatively strong consensus on overall wine quality, but have more diverse opinions on wines that achieve a surprising level of quality given the vintage, the ranking, or the appellation from which they originate.
Wine experts play an important role on the wine market by delivering valuable information for an experience good for which true quality cannot be precisely determined. We use a mediation technique as proposed in Baron and Kenny (Reference Baron and Kenny1986) to discriminate between the quality surprise in wines and pure surprises in scores by critics that measure the true influence a critique has on wine prices. We finally estimate different alternative specifications to test the robustness of our results.
Compared to most studies that focus on Robert Parker, we explicitly contrast and compare his influence with other critics for a sample of 122 Bordeaux estates. Our evidence suggests that Stephen Tanzer and Wine Spectator display significantly positive albeit limited influence on prices. By contrast, Robert Parker and Jean-Marc Quarin are the most influential critics, as a 10% surprise in their scores leads to a price increase of around 7%. We further find that Robert Parker and Jean-Marc Quarin are especially influential under certain conditions. Their impact is higher for appellations and estates that are not covered by the official 1855 classification, and for the best vintages (2005, 2009, 2010). Overall, our results suggest that Jean-Marc Quarin and Robert Parker are the two most influential experts on the Bordeaux market and appear to be complementary in catering to the French and U.S. markets, respectively.
Appendix 1: Map of Bordeaux
Appendix 2: List of Estates
