Public demand and politics

Theoretically, public policy scholars hold different views on which “workhorse” models of public policymaking offer more robust explanations of policy choices. Those models differ significantly in terms of what types of demand are most relevant. On the one hand, the median voter model (Downs Reference Downs1957) presumes that voters have single-peaked preferences and public demand can somehow be aggregated into a single dimension, and that the voter in the middle of an observable distribution is at the centre of attention from the policymakers’ perspective. On the other hand, interest group models agree that the median voter matters, but they also claim that in many cases interest groups facing large and concentrated opportunity costs and/or benefits are easier to organise and are likely to fight harder to get what they want, relative to more latent interests in society(Olson Reference Olson1965; Persson and Tabellini Reference Persson and Tabellini2002). Empirically, these two basic models of democratic policymaking have different implications for the classical measurement of demand for policies. In the following sections, we will introduce classical measurements of different facets of demand and briefly elaborate on their main advantages or shortcomings.

Scholars seeking to explain policy choices from an interest group perspective usually engage in stakeholder surveys. Such surveys measure the preferences of stakeholders (e.g. interest groups), their position within policymaking networks and their self- or other-assessed influence on public policy (Fisher et al. Reference Fisher, Leifeld and Iwaki2012). Scholars seeking to explain policy choices from a median voter model perspective tend to rely either on simple structural proxies, on public opinion survey data or – more recently – on internet search data.

With respect to structural proxies for public demand, in studies on environmental policies, many scholars rely on Gross Domestic Product (GDP) per capita, arguing that at low levels of income people prefer economic growth over environmental protection and that, at some point, this trend reverses and people demand more postmaterial goods such as environmental protection (Dasgupta et al. Reference Dasgupta, Laplante, Wang and Wheeler2002; Bättig and Bernauer Reference Bättig and Bernauer2009; Spilker Reference Spilker2011; Kim and Wolinsky-Nahmias Reference Kim and Wolinsky-Nahmias2014). Similarly, levels of democracy are presumed to be associated with stronger public demand for environmental public goods provision (Neumayer Reference Neumayer2002; Bättig and Bernauer Reference Bättig and Bernauer2009). Other studies have used environmental Non-governmental organization (NGO) density (Bernauer et al. Reference Bernauer, Kalbhenn, Koubi and Spilker2010), NGO counts (Frank Reference Frank1999; Fredriksson and Millimet Reference Fredriksson and Millimet2004; Roberts et al. Reference Roberts, Parks and Vásquez2004) and share of green parties in the legislature (Neumayer Reference Neumayer2003) as proxies for public demand.

However, to explain policy output in many countries over time, we are in need of more direct measures of public demand without having to rely, for instance, on the assumption that the theoretical linkages stipulated by the environmental Kuznets curve (Dasgupta et al. Reference Dasgupta, Laplante, Wang and Wheeler2002; Bernauer and Schaffer Reference Bernauer and Schaffer2012) hold true.

As stated before, well-designed and executed public opinion surveys can be regarded as the gold standard when seeking to capture public demand. Thus, the fact that public opinion is measured directly from the source is the main advantage of public opinion surveys. However, the main problem with existing survey data is that they fail to provide consistent time series data for longer periods and that they do not fully cover the various dimensions of policy problems. These limitations exist even more for questions regarding climate change policy. Although the issue has gained some prominence for the latter part of the time period and for advanced industrialised countries (with extensive survey activity particularly in the US), it still cannot be compared with more salient topics such as pensions or the welfare state in general.

Public demand and the media

Often, the relationship between politics, the media and the public is depicted using a triangle (Bernauer et al. Reference Bernauer, Jahn, Kuhn and Walter2014). We follow this notion with our assumption that direct interactions between public and politics can happen, but that most of the interactions happen indirectly through an intermediary actor – the media. On the one hand, the media reflects viewpoints of politicians and parties in the form of interviews and reports, and, on the other hand, also channels the interests of ordinary citizens and other nonstate actors.

Organised interest groups often make their demands public in a way that ensures media attention (e.g. the animal protection organisation People for the Ethical Treatment of Animals (PETA) uses naked celebrities to raise awareness on the issue of fur). Newspapers give their (average) readers space to express opinions in the form of letters to the editor and other commentary sections. In addition, all media outlets provide some form of “comment”, where journalists give their personal opinion on relevant topics. Thus, the media presents a rich and multifarious sample of opinions and demands. In this study, we build on this in order to create our proxy for the general public demand on a particular topic.

The scholarly community on media effects thus scrutinises one direction of the aforementioned triangle: the effect of media on public opinion. With regard to demand, however, media studies have traditionally emphasised the salience component of public demand (Weingart et al. Reference Weingart, Engels and Pansegrau2000; Brossard et al. Reference Brossard, Shanahan and McComas2004; Carvalho and Burgess Reference Carvalho and Burgess2005; Boykoff Reference Boykoff2007; Boykoff and Mansfield Reference Boykoff and Mansfield2008; Ahchong and Dodds Reference Ahchong and Dodds2012; Schmidt et al. Reference Schmidt, Ivanova and Schäfer2013). Studies on media impact, for instance, agenda-setting theory (McCombs and Shaw Reference McCombs and Shaw1972; McLeod et al. Reference McLeod, Becker and Byrnes1974), argue that one principal role of the media is to communicate to people what topics to think about. Empirically, salience is usually measured by frequency of reporting and placement (Andrewsa and Caren Reference Andrewsa and Caren2010).

Although salience is relatively easy to measure empirically and quite uncontroversial at the conceptual level, the concept of opinion is more challenging. Proponents of agenda-setting theory postulate that the media shapes what issues people think about, but that it does not necessarily have a strong influence on their opinions per se. Other media researchers such as Zaller (Reference Zaller1992) argue, however, that the media actually influence public opinion. Krosnick et al., for instance, note that “media coverage sometimes increases national seriousness assessments by altering existence beliefs, sometimes by conferring knowledge and thereby increasing certainty, or sometimes by altering people’s perceptions of a phenomenon’s consequences” (Reference Krosnick, Holbrook, Lowe and Visser2006, Reference Habel36). With respect to our substantive issue interest – climate change policy – some studies show that media usage positively affects knowledge and awareness on several aspects of climate change (e.g. causes, possible solutions, see Stamm et al. Reference Stamm, Clark and Eblacas2000; Krosnick et al. Reference Krosnick, Holbrook, Lowe and Visser2006; Zhao Reference Zhao2009). Several studies find evidence for effects of media content on public opinion (Gamson and Modigliani Reference Gamson and Modigliani1989; Vliegenthart et al. Reference Vliegenthart, Schuck, Boomgaarden and De Vreese2008; Sampei and Aoyagi-Usui Reference Sampei and Aoyagi-Usui2009). For example, Vliegenthart et al. (Reference Vliegenthart, Schuck, Boomgaarden and De Vreese2008), in a study covering nine countries, find that media reporting that highlights benefits is positively correlated with support for the European Union (as measured by surveys), and reporting on contested issues is negatively correlated with support. Overall, this research shows that published opinion, the aggregate of opinions represented in the media, and public opinion, as measured by surveys, are not identical.

This comes as no surprise given our above-mentioned arguments that media reporting also reflects claims and opinions of interest groups and not solely the general public. However, by affecting knowledge and awareness, the two phenomena (published opinion and public opinion) arguably are related, and thus, for our purposes, to find a general demand measure that can ultimately serve to explain policymaking, the multifarious published opinion is more relevant than the survey-based public opinion. Moreover, as the omnipresent published opinion is (compared with opinion polling) rather easily available, we claim that this demand measure is likely to be observed closely by policymakers (Herbst Reference Herbst1998).

Even though it remains unclear how much direct influence the media has on public opinion (Habel Reference Habel2012; Lee Reference Lee2014), most researchers agree that at least indirect effects exist. One such indirect effect is likely to be that “people will tend to overestimate the influence mass communications have on the attitudes and behavior of others” (Davison Reference Davison1983, 3). This is called the third-person effect. Empirically, the third-person effect is well documented (Perloff Reference Perloff1993; Sun et al. Reference Sun, Pan and Shen2008). Gunther (Reference Gunther1998), for example, showed in an experiment that the slant of news reporting causes differences in perceived public opinion by the participants: although the participants did not change their own opinion on their respective topics, they expected that others had changed their opinion because of the reporting. Policymakers who are interested in the current state of public opinion rely on the media exactly for that reason (Herbst Reference Herbst1998, 86). They also face the problem of insufficient survey data. Herbst (Reference Herbst1993, 101) reports that, in the 1930s and 1940s, 86% of congressmen relied on the printed press to gauge public mood, and a quarter of congressmen actually counted newspaper editorials to get insights into opinion on specific topics. As outdated as this finding might seem at the first glance, more recent research supports this result (Herbst Reference Herbst1998). It is the “influence of presumed influence” (Gunther and Storey Reference Gunther and Storey2003) that matters – for politicians as well. For example, policymakers try to get access to and coverage in the media to present themselves in a good light because they believe that their media coverage influences their reelection chances (Cohen et al. Reference Cohen, Tsfati and Sheafer2008). A prerequisite for that expectation is that politicians believe in the influence the media has on public opinion. Politicians even take specific news reports as a call for action (Cook et al. Reference Cook, Tyler, Goetz, Gordon, Protess, Leff and Molotch1983; Herbst Reference Herbst1998, 79). However, “it is the totality of media attention, the general slant or tendency of coverage (…) that is synonymous with public opinion” for them [Herbst Reference Herbst1998, 67 (emphasis in original)].

Much like politicians, we can therefore rely on media content analysis to generate high-frequency data for public demand that can also be used to reconstruct public demand backwards in time – which is impossible with public opinion surveys. Our content analysis follows this idea by looking at frequencies and claims, which are opinion statements of nongovernmental actors in the news, in order to grasp what is discussed by the public. Various authors have argued that these claims influence public opinion the most (Koopmans Reference Koopmans2004; Howland et al. Reference Howland, Becker and Prelli2006). Our assumption is that claims are those parts of media content that reflect the perceived published opinion best.Footnote ⁴

As we established above, the published opinion is likely to differ from public opinion measured by surveys. Furthermore, in contrast to large parts of the media effects literature (McCombs and Shaw Reference McCombs and Shaw1972; Zaller Reference Zaller1992), we are not interested in trying to explain whether public opinion shapes news media reporting and the published opinion, or vice versa. Hence, if it would turn out that survey results change first and public opinion follows suit, this would still be perfectly in line with our expectations as we only want to show that the relationship exists, independent of its direction. However, public opinion remains hidden and abstract most of the time. Politicians who try to gauge the public demand for certain policies must therefore rely on what is available – which is mostly the published opinion. Published opinion consists of single opinions made public through the media, and can also be described as elite opinion, as it results from a selection process by journalists.Footnote ⁵

The public demand spectrum

In developing an empirical measure of the general public demand, we need to disentangle the theoretical dimensions of the concept we are interested in. We consider public demand as a multifaceted umbrella term. Public demand towards a topic is composed of a salience and an opinion component (see Figure 1). Salience captures how important a given issue is from the viewpoint of members of a given social unit (e.g. a country). Empirical demand measures combine the two elements – salience and opinion – in varying ratios. In the communication sciences, for instance, “[e]mpirically, object salience on the media agenda typically is measured by the amount of media coverage over time” (Chyi and McCombs Reference Chyi and McCombs2004, 22). In survey research, respondents are often confronted with a list of social problems and are asked to rank these problems in terms of how important they think these are.

Figure 1

Spectrum of public demand with the three media-based demand concepts. Note: CCTI=climate change threat index; MIP=most important problem.

Opinion is located on the other axis and captures the position people take about a given social problem, what they think should be done about it and by whom. The ends of the spectrum are distinct in the sense that individuals or groups that make up the general public may hold opinions on a given issue, whereas the underlying issue may or may not be important to them.

We propose three measures: media salience, politicisation of the public debate and published opinion. These can all be placed at different points of the demand spectrum. As the name indicates, our media salience measure represents the salience aspect of demand, whereas we capture the opinion aspect of demand via published opinion in our data.

Politicisation of the public debate covers the middle ground between salience and opinion on the public demand spectrum by combining the extent of coverage of the topic as well as the specific opinions presented. Politicisation thus provides information on how contested or politicised the respective issue is within the media debate.Footnote ⁶ In contrast to the positive (pro) and negative (con) directionality of the published opinion measure, politicisation accounts for the general level of debate that exists over the topic of interest. Thus, a politicised debate can be characterised by a high number of opinionated (pro or con) statements on a certain topic presented within the public discourse.

Although we may observe times where factual coverage of the issue is high (thus resulting in a high media salience for the issue) and times when there are only a few articles – but these are full of positive or negative claims for policy change from the government (thus, resulting in a high positive or negative published opinion towards climate change policies), the politicisation measure is able to distinguish whether for a given salience of the topic there is more debate (i.e. more pro or constatements). The politicisation measure thus proxies how contested a debate is in terms of statements given (independent of their direction).Footnote ⁷

We compare our measures with existing public opinion survey data and data from Google Trends – a new data source that is increasingly used in social sciences research.

The next section starts with a discussion on the relationship between media, opinion and politicians, presents the data generated by our newspaper content analysis and subsequently offers a brief description of the public opinion survey data and the internet search requests data we use for the comparison.

Measuring public demand in the media

This section describes the coding of our theoretically established dimensions of public demand – media salience, published opinion and politicisation – by means of a media content analysis. The latter measure combines salience and opinion by capturing the politicisation of public debate in terms of the relative frequency of claims (pro or con) in comparison with the frequency of media reporting on climate change issues. Future research will have to establish which of these three demand measures performs best in the sense of explaining and predicting policy output over time and across countries. In any event, the politicisation measure is the most encompassing as it includes both the salience and opinion component of public demand.

Media salience

To measure media salience, we use the number of climate change-related (see Appendix A for definition) articles relative to the total number of articles published in the respective newspaper within a given time. Figure 2 displays the resulting data for the two newspapers from which the data for the US were coded.

Figure 2

Media salience of the climate change (CC) issue, 1995–2010, monthly data. Note: Measured as the number of CC-relevant articles as a proportion of all articles published within a given month in the USA Today and the New York Times (NYT). Scale=0 … 1, where 1 is the empirical maximum of the variable (occurs in 2009m12). Events: A: IPCC assessment report 2 (AR2) (1995m12); B: Conference of the Parties (COP) Kyoto (1997m12); C: AR3 and United States withdrawal from the Kyoto Protocol (2001m3); D: COP Bonn (2001m7); E: Kyoto Protocol enters into force (2005m2); F: AR4 (2007m2); G: COP Bali (2007m12); and H: COP Copenhagen (2009m12).

It is noteworthy that the two series parallel each other quite closely with similar peaks. Five events coincide with (and are presumably causing) spikes in our media salience measure: the signing of the Kyoto Protocol in December 1997; the conference of the parties in Bonn in July 2006; the publication of the 4th assessment report of the Intergovernmental Panel on Climate Change (IPCC) in March 2007; and the Conference of the Parties in Bali in December 2007 and in Copenhagen in December 2009 (see also Sampei and Aoyagi-Usui Reference Sampei and Aoyagi-Usui2009; Schaffer Reference Schaffer2011). We also observe a strong upward trend in media salience from about 2005 to 2009.

We will compare media salience to Google Trends and MIP, as these are the two most established indicators that reflect salience. Hence, the fit should be best among these three measures (see below for a detailed discussion of the reference demand measures).

Published opinion

Assuming that debate in the media reflects the distribution of published opinion, we have coded claims in favour (for more climate protection) or against (preservation of the status quo or less climate protection). In general, claims are statements, demands or opinions given by an individual or an organisation or summaries of position papers, press statements, comments and so on. In addition, a claim is always directed towards the future and at something that lies beyond the sphere of influence of the claim maker. This means that claims by government officials have not been recorded. To the contrary, a major share of our claims stems from letters to the editor written by ordinary citizens, as we do not differentiate by claim maker. With regard to our definition of claims, we deviate from the classical claims analysis approach by Koopmans and Statham (Reference Koopmans and Statham1999). This adaptation is necessary, as we pursue a different goal with our claims data. In fact, for our purpose, claims are an intermediate, and only their direction and frequency are examined.

Claims can be either general or specific. For the pro claims, this can look like, respectively, “The major countries that emit should take domestic action, and not have these loopholes” and “We need a moratorium on coal now”.Footnote ¹⁰ The same applies to con claims: “This [limit on greenhouse gas emissions for the industry] has the potential to be very damaging to the California economy” and “For years, Dr. Goklany, an electrical engineer by training, has written in papers and books that it may be better not to force cuts in greenhouse gases because the added prosperity from unfettered economic activity would allow countries to exploit benefits of warming and adapt to problems”.Footnote ¹¹

We use the ratio of pro and con claims, defined as pro minus con claims divided by the total number of claims, as our published opinion measure. This measure captures whether news media content within a given time interval is leaning more for or against climate protection. Figure 3 displays the data generated by this approach for the New York Times and USA Today. The upper panel shows the five-month moving average of our published opinion measure, whereas the lower panel gives information on the actual number of claims made in the respective newspaper at a given point in time.

Figure 3

Published opinion (ratio of pro- and con climate protection claims over total claims), 1995–2010, five-month moving average. Note: Measured as pro- minus con climate protection claims divided by the sum of positive and negative (total) claims per month. Scale=−1 … 1; where “−1” means that only negative claims were made and “1” that only positive claims were made. Using five-months moving averages makes it easier to visually compare the two series. Events: A: assessment report 2 (AR2) (1995m12); B: COP Kyoto (1997m12); C: AR3 and United States resigns of Kyoto (2001m3); D: COP Bonn (2001m7); E: Kyoto Protocol in force (2005m2); F: AR4 (2007m2); G: COP Bali (2007m12); and H: COP Copenhagen (2009m12). NYT=New York Times.

As one might expect, there is less similarity between the two newspapers, compared with the salience measure, as this measure taps more into editorial preferences of the newspaper and how certain issues and positions of stakeholders are reported. Contrary to the media salience measure, the two series do not move in parallel. The New York Times has the largest proclimate protection claims peak in 2001 (probably in view of US withdrawal from the Kyoto Protocol), and, from about 2004 onwards, includes on average more pro climate claims in relation to total claims than USA Today. The rough shape of the time series in earlier years stems from the lower absolute number of claims. There is a general upwards trend in the pro climate protection claims ratio for the 1995–2009 period.

As discussed above, we do not expect that published opinion correlates perfectly with public opinion measures. However, we do expect that our measure of published opinion and the two survey-based measures CCTI and MIP should at least generally move into the same direction. Nevertheless, we want to stress that other survey items asking people whether, for example, the government should do more on climate change would be more directly comparable with our more detailed measure.

Politicisation of the public debate

Although the media salience measure captures the attention climate change issues receive in the media in general terms, it does not really measure how intense or contested the public debate on the subject is. We submit that the total numbers of claims (positive and negative) in relation to salience can proxy for debate intensity, or politicisation, of the issue. This measure includes both salience and opinion, and thus offers a rather encompassing measure of public demand.

We use the total number of claims (both pro and con climate protection) divided by the total number of climate change-relevant articles in a given time interval to that end. Figure 4 displays the resulting data. Similar to the salience measure, the two series move quite closely in parallel. Furthermore, the overall level of politicisation rises over time.

Figure 4

Intensity of the public climate change (CC) discourse, 1995–2010, monthly data. Note: Measured by total number of claims per month (sum of pro and con claims) divided by the number of CC-relevant articles in USA Today and the New York Times (NYT) (monthly). Scale=0 … 1, where “1” is the empirical maximum on the variable (occurs in 1999m7). Events: A: assessment report 2 (AR2) (1995m12); B: COP Kyoto (1997m12); C: AR3 and United States resigns of Kyoto (2001m3); D: COP Bonn (2001m7); E: Kyoto Protocol in force (2005m2); F: AR4 (2007m2); G: COP Bali (2007m12); and H: COP Copenhagen (2009m12).

As stated above, this novel concept of politicisation of the public debate is difficult to compare with existing measures that either serve as public opinion measures or salience measures. Nevertheless, in the comparison (see further below), we expect to observe some coevolution of politicisation of the public debate with Google Trends and particularly with Gallup’s MIP, as it is survey-based and thus closer to public opinion but asks respondents directly about salient issues.

Comparison

The main purpose of this article was to find out whether our proposed historical reconstruction of public demand on the basis of media content analysis provides a useful alternative where other types of measures of public demand (notably data from public opinion surveys or internet searches) do not exist, cannot be historically reconstructed or are not commensurable enough to be patched together from different data sets. Although some caveats apply concerning the comparability of the survey measures, we believe this comparison can provide an indication of whether our data represent a valid measure of public demand that can be generalised past the US and whether there are dimensions of public demand where there is more or less overlap.

We thus compare our new data with the best available alternatives – namely, public opinion survey data and internet search data from Google Trends. As noted above, the Google Trends data likely capture primarily the saliency dimension of the climate issue, whereas the public opinion data tend to capture both saliency and, to some extent, public opinion. The new data are more explicit in measuring media salience, the politicisation of the public debate and published opinion. This raises the issue of what should be compared. We opted for an inductive, data-driven approach and compare the three new measures with both public opinion and internet search data. In order to make our data comparable with the different survey and Google Trends measure, we transformed all series to have a common mean of 0 and a SD of 1.Footnote ¹²

The following part visually identifies the most obvious similarities and differences, and uses cross-correlation analysis (see Sampei and Aoyagi-Usui Reference Sampei and Aoyagi-Usui2009) to obtain some indication of whether and how the series correlate. The latter method reveals patterns of correlation between time series and displays correlations for various lags immediately. If two series move exactly in parallel, we will see a peak at a lag of 0 (contemporaneous). It also enables us to spot lags of one variable that may serve as predictors of the other variable. If two series correlate at lag 0, this means that we can reject an implicit null hypothesis of independence or noncorrelation between the series. This is in fact what we observe for most of the crosscorrelations examined in the remainder of this section.

Media salience

As expected, the media salience measure generated by media content analysis seems to perform quite well, presuming that the MIP and Google Trends data are valid indicators of saliency (see Figure 1), and can thus provide the basis for a meaningful comparison (Figure 5, top row). At least until 2007, the series appear to be quite closely correlated.

Figure 5

Comparison and cross-correlation of saliency data with survey and internet search data. Note: Top row: comparison of the relative frequency of media reporting on climate change by two United States (US) newspapers with the Gallup most important problem (MIP) measure and Google Trends data. Scale: normalised with mean=0 and SD=1. Events: A: assessment report 2 (AR2) (1995q4); B: COP Kyoto (1997q4); C: AR3 and US resigns of Kyoto (2001q1); D: COP Bonn (2001q3); E: Kyoto Protocol in force (2005q1); F: AR4 (2007q1); G: COP Bali (2007q4); and H: COP Copenhagen (2009q4). Bottom row: cross-correlation coefficient plus 95% confidence intervals between Gallup MIP and Google Trends and our salience measure [lags in quarters: 10 (Gallup); lags in month: 15 (Google Trends)]. For the cross-correlation analysis, we try to show the cross-correlations over a sufficient number of times, given the length of the time series; we have thus opted to show roughly between one and two years’ worth lags.

In the cross-correlation analysis, we are mostly interested in whether our media salience measure has a high and significant contemporaneous correlation with the target series for salience, the MIP and the Google Trends.Footnote ¹³ As shown in the bottom row of Figure 5, we can see that the serial cross-correlation between the Google Trends monthly data and our media salience measure is always very high (above 0.6). The Gallup MIP as a survey measure also seems to compare well with our media salience measure, although the contemporaneous correlations are somewhat lower, peaking at around 0.34. Given that the available data relate only to “the environment” and not to “climate change” or “global warming” as such, this may explain the lower cross-correlation. Furthermore, besides the contemporaneous correlations, we can observe that a four-period (one year) lag of the Gallup MIP seems to be a good predictor of our media salience measure at time t, meaning the Gallup MIP leads our media salience measure. Overall, we can quite confidently establish that our salience measure can serve as a valid proxy.

Published opinion

In determining which measures are comparable, we have argued that the opinion dimension (see Figure 1), which we intend to cover with our published opinion indicator, can best be compared with the two survey measures (CCTI and MIP). Overall, as shown in Figure 6, it appears that our opinion indicator indeed lines up to some extent with the two survey-based measures. This suggests that the published opinion measure based on the pro/con claims ratio is actually closely linked to the general direction of public opinion concerning climate policy. We observe a generally good fit for both of the survey-based measures, but also observe major differences in published opinion around major events or major amplitudes (positive and negative).

Figure 6

Comparison and cross-correlations of opinion data with survey and internet search data. Note: The top row compares the ratio of positive to negative claims in media reporting on climate change (CC) by two United States (US) newspapers with the climate change threat index (CCTI) and the Gallup most important problem (MIP) measure. Scale: all series normalised with mean=0 and SD=1. Events: A: assessment report 2 (AR2) (1995q4); B: COP Kyoto (1997q4); C: AR3 and US resigns of Kyoto (2001q1); D: COP Bonn (2001q3); E: Kyoto Protocol in force (2005q1); F: AR4 (2007q1); G: COP Bali (2007q4); and H: COP Copenhagen (2009q4). Bottom row: cross-correlation coefficient plus 95% confidence intervals between CCTI and Gallup MIP and our opinion measure [lags in quarters: 10 (CCTI and Gallup)].

The cross-correlation (bottom row) analysis confirms these visual impressions. The opinion measures against which we compare our published opinion measure are the CCTI and the Gallup question (MIP). Although there seems to be no significant correlation for the CCTI and our opinion measure, we observe a significant, although small (0.27), correlation at lag −1 between published opinion and the Gallup MIP (Figure 6). This indicates that a one-period lag of the Gallup MIP (t−1) is a good predictor of our published opinion measure at time t, and thus the Gallup MIP leads our opinion measure.

We find two potential reasons for these mixed results. On the one hand, the three alternative measures for public demand available tend to primarily capture concern about climate change issues, and not so much public demand for or against political responses to the problem (our substantive interest). We believe that our opinion measure comes closer to measuring the opinion aspect of public demand than the three best available alternatives used here. However, further research should investigate in greater detail how our measure relates to the very few survey items seeking to capture attitudes towards climate policy more directly. On the other hand, the low cross-correlation could also be due to the fact that editorial policies are producing differences in our opinion measure across media outlets.

Politicisation of public debate

Finally, as indicated by Figure 7, our measures referring to the politicisation or contestation of an issue lines up quite well with the amplitudes, trends and peaks of the survey data and the internet search data. As this measure conceptually and empirically covers the middle ground between salience and opinion (see Figure 1), we have argued above that the best measures to compare are Gallup MIP and Google Trends.

Figure 7

Comparison and cross-correlation of intensity data with survey and internet search data. Note: The three panels compare the ratio of total claims (positive and negative) with total climate change (CC)-relevant articles two United States (US) newspapers with the climate change threat index (CCTI), the Gallup most important problem (MIP) measure and Google Trends data, respectively. Scale: all series normalised with mean=0 and SD=1. Events: A: assessment report 2 (AR2) (1995q4); B: COP Kyoto (1997q4); C: AR3 and US resigns of Kyoto (2001q1); D: COP Bonn (2001q3); E: Kyoto Protocol in force (2005q1); F: AR4 (2007q1); G: COP Bali (2007q4); and H: COP Copenhagen (2009q4). Bottom row: cross-correlation coefficient plus 95% confidence intervals between Gallup MIP and Google Trends and our intensity measure [lags in quarters: 10 (Gallup); lags in month: 15 (Google Trends)].

The cross-correlation (bottom row) analysis supports the visual impression of the comovement of the series. The contemporaneous correlations between the Gallup MIP and the Google Trends and our politicisation of the public debate measure are significant and substantial. It is noteworthy that the Gallup MIP measure, which we think is the most plausible alternative measure for politicisation as it asks for the importance of the topic on the political agenda, correlates significantly and consistently with our politicisation measure. Put differently, the perception of an issue as an important problem seems to go hand in hand with the politicisation of the debate. Regarding the Google Trends measure, we can see that our politicisation measure at time t−1 is a good predictor of the Google Trends measure at time t, meaning that the Google Trends measure leads our politicisation of the public debate measure. It also makes intuitive sense that an increased politicisation of an issue within the media leads people to search for it.

Robustness check: how does the US case compare with other countries?

This article has proposed a way of measuring different dimensions of public demand for public policy, focussing specifically on the issue of climate change policy. One reason for our efforts to come up with a media-based content analysis was that commensurable and comparable public opinion or stakeholder survey data between countries and over time are hard to find. Although we have collected our media-based data for six countries over the time period from 1995 to 2010, we were not able to find sufficiently comparable survey data for reliability checks on our data in countries other than the US. However, as a robustness check, this section will first show how our US newspaper data compare with our media-based data from GermanyFootnote ¹⁴ and Spain,Footnote ¹⁵ and, second, will compare our data on the public demand for climate change policies to the best available proxies from Germany and Spain. This should increase the confidence that the US case can be regarded as representative of our larger population.

Figure B1 in Appendix B shows our three measures – media salience, published opinion and politicisation of the public debate – for the US, Spain and Germany. In the case of media salience (Panel A), the descriptive comparison of the three time series shows that – generally, but especially since 2005 – the salience for climate change has increased in all three countries, which coincides with the inception of the Kyoto Protocol. Similar to the US case, media salience in Germany has also been elevated sharply from about 2005 onwards. With regard to our published opinion measure, we observe that, although there is a lot of comovement of the three series, published opinion in Germany seems to include more pro claims on average and also more claims in general (see lower part of the graphs in Panel B). Yet, the number of claims sharply increased in all three countries from about 2007 onwards. The third measure, politicisation of the public debate, however, increases over time in the US and stays around the same level in Spain and Germany. This could imply that problems of climate change and potential policy solutions were debated earlier in Spain and Germany compared with the US. All in all, the US case does not differ starkly from the evolution of public demand in climate change leader countries such as Germany, or in other countries that are part of the Kyoto Protocol, such as Spain.

Are our new measures in these two additional countries correlated with the best available proxies concerning public opinion or salience towards climate change? To this end, we again compare our three measures with two different time series: first, the equivalent of Gallup’s MIP question on “the environment” surveyed in Germany (Breunig Reference Breunig2014) and SpainFootnote ¹⁶ (Chaqués-Bonafont and Baumgartner Reference Chaqués-Bonafont and Baumgartner2013; Chaqués-Bonafont et al. Reference Chaqués-Bonafont, Palau and Baumgartner2015)Footnote ¹⁷ and, second, Google Trends data on internet searches in Germany and Spain.

With regard to the respective cross-correlations for our media salience measure, we find that, similar to our US data, the Google Trends data show a significant contemporaneous correlation ranging from 0.4 in Spain to 0.65 in Germany (cf. Figure B2 in Appendix B). Concerning Gallup MIP, for Germany as well as for the US, the contemporaneous correlation is significant (cf. Figure B3 in Appendix B). However, for Spain, the two time series are not seemingly correlated. Furthermore, our measures of published opinion and politicisation do not correlate as well as the media salience measure (see Figures B4–B7 in Appendix B). However, given that the available data relate only to “the environment”, and not to “climate change” or “global warming” as such, this may explain the lower cross-correlation.

In sum, the descriptive analysis of our media-based data in three countries finds a lot of parallel movement across all three measures. In comparison with already-existing (but not completely comparable) data, our measure of media salience empirically performs best. The other theoretically important dimensions of public demand we have proposed, published opinion and the politicisation of the debate, do not compare as well. Their importance lies in the potential to use these more fine-grained and multifarious demand measures to explain a government’s responsiveness.