Introduction
Since the time of Aristotle’s delineation of argumentation into the categories Ethos, Pathos and Logos, scholars have struggled to understand the power of persuasion. This is particularly true in the literature on public policy debates where participants in the policy process have incentives to frame policy issues in different ways in order to gain an advantage. Riker (Reference Riker1986, Reference Riker1996) focused our attention on the ability of individual protagonists in the policy process to destabilise debates through strategic framing. Baumgartner and Jones (Reference Baumgartner and Jones1993, Reference Baumgartner and Jones2009) document countless examples of this in the United States during the post-war period and show how shifts from positive to negative images can change venues and drastically alter interest group mobilisation patterns, with significant consequences for public policy outcomes. For instance, based on a content analysis of newspaper articles, Baumgartner et al. (Reference Baumgartner, De Boef and Boydstun2008) show that the public debate about the death penalty has been completely transformed. The long-dominant morality frame has been replaced by an innocence frame highlighting the errors in the criminal justice system. The authors convincingly demonstrate the power of framing, as the reframing of the issue has led to an important change of public opinion and public policy towards the death penalty.
However, up until now, there are few studies that systematically analyse how interest groups employ framing to reach their policy goals. Baumgartner et al. (Reference Baumgartner, Berry, Hojnacki, Kimball and Leech2009) have analysed interest group framing, covering a random sample of 98 policy issues in the United States. They found that fewer than 5% of the issues were significantly reframed. Mahoney (Reference Mahoney2008) similarly found limits to the ability of lobbyists to reframe issues in her study comparing interest group lobbying in the United States and the European Union. Even though these studies have considerably enhanced our understanding of the role of framing in the policy process, we have little systematic knowledge about the determinants of successful framing. Significant reframing may be uncommon, but the question still remains: which types of frames are more successful in realising public policy outcomes that are in line with an interest group’s preference?
There are furthermore a number of important case studies that have generated invaluable insights concerning the role of framing in public policy that analyse framing on specific issues, such as the death penalty (Peffley and Hurwitz Reference Peffley and Hurwitz2007; Baumgartner et al. Reference Baumgartner, De Boef and Boydstun2008), the Gulf War (Althaus and Kim Reference Althaus and Kim2006), the Kosovo crisis (Berinsky and Kinder Reference Berinsky and Kinder2006), pornography (Sharp and Joslyn Reference Sharp and Joslyn2003) and the Monica Lewinsky scandal (Shah et al. Reference Shah, Watts, Domke and Fan2002). However, it is important for two different reasons that the empirical analysis of interest group framing is complemented by large-N designs allowing for studying interest group framing across a large number of interest groups and policy issues. First, the cases selected for the case studies tend to be highly salient, controversial and partisan, yet the majority of policy issues that are the object of lobbying do not exhibit these characteristics (Baumgartner and Leech Reference Baumgartner and Leech2001) – so there is a question of whether the findings regarding framing from these specific cases are generalisable to less sensational political debates. Second, the recent interest group literature has demonstrated that interest group strategies and lobbying success vary importantly across different policy debates depending on characteristics that vary on the level of a policy debate, such as the salience of a debate, the complexity of a debate, the degree of conflict and the power of issue-specific lobbying coalitions (e.g. Dür and De Bièvre Reference Dür and De Bièvre2007; Mahoney Reference Mahoney2007; Baumgartner et al. Reference Baumgartner, Berry, Hojnacki, Kimball and Leech2009; Klüver Reference Klüver2011). Thus, it is important to also study the effect of interest group framing across a large number of policy debates. This systematically takes into account the cross-level interactions between the framing strategies of interest groups and the characteristics of the policy debate to better understand how interest group framing affects policy outcomes.
The lack of large-N studies is, to a large extent, due to methodological difficulties in measuring interest group framing. We therefore introduce a new methodological approach that has previously been applied to parliamentary debates, committee deliberations and presidential speeches (Schonhardt-Bailey Reference Schonhardt-Bailey2005b, Reference Schonhardt-Bailey2008; Bailey and Schonhardt-Bailey Reference Bailey and Schonhardt-Bailey2008; Weale et al. Reference Weale, Bicquelet and Bara2012) that allows for systematically measuring interest group framing across a large number of interest groups and policy debates.Footnote 1 The proposed text analysis technique combines cluster and correspondence analysis using the software package T-LAB to identify interest group frames, assess the dimensionality of policy debates and evaluate the impact of interest group framing on public policy outcomes. In this article, we introduce this new methodological approach and demonstrate its validity on the basis of two case studies in the areas of environmental and transport policy in the European Union. We show that it is possible to identify frames employed by interest groups and to assess their effectiveness through this computer-assisted quantitative technique. This new approach opens the door to studying framing strategies across a large number of cases. Such large-N designs allow for a better understanding of how interest groups choose their frames and the identification of the determinants of successful framing that affect the course of public policy debates.
Considerable ambiguity characterises the literature on framing, as scholars in different disciplines or subfields of political science have used a variety of terms and definitions. Entman (Reference Entman1991, 53) has described framing as “selecting and highlighting some features of reality while omitting others”. We follow Entman (Reference Entman1991) and define a frame accordingly as a specific aspect of a policy proposal that is emphasised in a policy debate by a specific actor. A dimension, in contrast, refers to the underlying structure of conflict according to which interest groups can be aligned in a legislative debate. The unit of analysis of the interest group framing analysis that we propose is the framing activity of an interest group regarding a specific policy initiative. Hence, when we talk about interest group framing, we are interested in studying which frames interest groups use in a specific policy debate to achieve their policy goals in that debate. Framing can vary across interest groups and across policy debates. The same interest group can employ the same frame in different policy debates (e.g. environmental NGOs typically use an environmental frame), but interest groups can also use different frames in different policy debates (e.g. producers of biodiesel could use an environmental frame in one debate but an economic frame in another debate).
In the following section, we briefly discuss the state of the art in quantitative text analysis before introducing the text analysis approach employed in this study. Subsequently, we test this approach on the basis of two case studies before the concluding section summarises the findings.
Text analysis in political science
Textual data is arguably the most widely available data source in the study of politics. Political documents have great potential to reveal information about the positions, attitudes and activities of their authors at a precise point in time. Unlike interviews, texts can be analysed as many times as one wishes, and the information extracted from texts does not get less reliable as time passes. Political scientists have therefore developed promising quantitative text analysis techniques to gather systematic information about political actors and processes. The most important applications in political science fall into two different categories: first, starting with the well-known Comparative Manifesto Project that relies on manual hand-coding to analyse the content of texts (Budge et al. Reference Budge, Klingemann, Volkens, Bara and Tanenbaum2001; Klingemann et al. Reference Klingemann, Volkens, Bara, Budge and McDonald2006), scholars have developed promising text analysis techniques to measure the positions of political actors, such as Wordscores (Laver et al. Reference Laver, Benoit and Garry2003) or Wordfish (Slapin and Proksch Reference Slapin and Proksch2008). Interest group research has made use of these novel content analysis techniques to measure the positions of interest groups with regard to specific policy proposals and federal agency rules (e.g. Yackee Reference Yackee2005; Yackee and Yackee Reference Yackee and Yackee2005; Klüver Reference Klüver2009, Reference Klüver2012, Reference Klüver2013a, 2013Reference Klüverb; Bunea Reference Bunea2013). Second, another important application of quantitative text analysis in political science is the classification of documents into policy areas (e.g. Grimmer Reference Grimmer2010; Quinn et al. Reference Quinn, Monroe, Colaresi, Crespin and Radev2010). These techniques allow for automatically classifying thousands of documents, such as press releases, bills and speeches, into issue areas.
However, the systematic analysis of framing has only received little attention in political science. Cheryl Schonhardt-Bailey has successfully used quantitative text analysis to study framing and the dimensionality of parliamentary debates, speeches of presidential candidates and transcripts of the Federal Reserve’s Federal Open Market Committee (Schonhardt-Bailey Reference Schonhardt-Bailey2005a, Reference Schonhardt-Bailey2006, Reference Schonhardt-Bailey2008; Bailey and Schonhardt-Bailey Reference Bailey and Schonhardt-Bailey2008). More specifically, she has applied a combination of cluster and correspondence analysis in order to identify frames and to assess the dimensionality of political debates. Weale et al. (Reference Weale, Bicquelet and Bara2012) have analysed legislative debates on abortion in the House of Commons relying on the same technique. While this quantitative text analysis approach has been successfully applied to speeches and committee deliberations, there are, to our knowledge, no studies that employ quantitative text analysis to study framing and political argumentation of interest groups, which is precisely what we are aiming for in this article.
Studying interest group framing using quantitative text analysis
This article applies the methodological approach employed by Bailey and Schonhardt-Bailey (Schonhardt-Bailey Reference Schonhardt-Bailey2005a, Reference Schonhardt-Bailey2006, Reference Schonhardt-Bailey2008; Bailey and Schonhardt-Bailey Reference Bailey and Schonhardt-Bailey2008) to the study of interest groups. This methodological technique allows for measuring interest group framing across a large number of cases, drawing on a text analysis implemented with the software package T-LAB. Following the work by Bailey and Schonhardt-Bailey (Schonhardt-Bailey Reference Schonhardt-Bailey2005a, Reference Schonhardt-Bailey2006, Reference Schonhardt-Bailey2008; Bailey and Schonhardt-Bailey Reference Bailey and Schonhardt-Bailey2008), we combine a cluster and a correspondence analysis, which we apply to a text corpus made up of interest group submissions to legislative consultations, as well as official policy documents produced by decision-makers. First, a cluster analysis is conducted in order to identify the frames employed by interest groups and, second, a correspondence analysis is carried out, which allows for assessing the dimensionality of policy debates. We use the software package T-LAB to carry out this text analysis.Footnote 2
T-LAB relies on co-occurrence analysis, which is the statistical analysis of words that appear together in designated spans of a text corpus (here: documents). The underlying assumption justifying the use of co-occurrence analysis is that words that co-occur “in similar contexts tend to have similar meaning” and “documents that contain similar word patterns tend to have similar topics” (Lancia Reference Lancia2007, 25). Using the presence or absence of words in each document, the program generates a term-document matrix upon which to base the classification process. This matrix contains documents in rows and the occurrence of words in each text in columns.
Based on a bisecting K-means clustering algorithm, T-LAB identifies clusters of documents. T-LAB hereby follows standard procedures in text mining, as clustering techniques are currently one of the most popular ways to automatically classify texts into different categories. The bisecting K-means cluster analysis is an augmented version of the k-means clustering technique in which k is fixed to 2. The bisecting K-means cluster analysis works as follows: T-LAB starts with a single cluster of the words mentioned in all the documents in the text corpus on the basis of the term-document matrix. T-LAB then splits the cluster into two different sub-clusters using the bisecting K-means algorithm (for further details about the algorithm, see e.g. Steinbach et al. Reference Steinbach, Karypis and Kumar2000). While the traditional bisecting K-means algorithm requires that the desired number of clusters is fixed in advance, T-LAB stores a number of cluster partitions based on the intracluster correlation coefficient (i.e. between cluster variance/total variance, where total variance corresponds to the sum of the between-cluster and the within-cluster variance) from which we chose the most parsimonious solution. In order to run the cluster analysis, it is necessary to indicate the maximum number of clusters that should be obtained. By default, T-LAB uses 10 clusters. This is a reasonable choice given that we analyse very specific legislative debates, and it is therefore plausible to assume that not more than 10 frames occur in the same policy debate.Footnote 3 The clusters identified by T-LAB can be interpreted as frames used by various actors in a policy debate (see also Miller Reference Miller1997; Schonhardt-Bailey Reference Schonhardt-Bailey2005a, Reference Schonhardt-Bailey2006, Reference Schonhardt-Bailey2008; Bailey and Schonhardt-Bailey Reference Bailey and Schonhardt-Bailey2008).
In the second step, correspondence analysis is used to assess the dimensionality of these frames. Correspondence analysis is a multidimensional scaling technique that allows for the spatial representation of the relationship between the clusters as distances in dimensions (for further details, see Greenacre Reference Greenacre1984).Footnote 4 T-LAB cross-tabulates document clusters and words in order to create a second matrix that can be used for factor correspondence analysis. This contingency table has the words in rows and the clusters in columns with the word occurrences in each cluster indicated in cells. Correspondence analysis provides a measure that indicates the amount of variance explained by the dimensions. It aims to account for a maximum amount of variance along the first dimension. The second dimension then seeks to account for a maximum amount of remaining variance and so forth. The correspondence analysis provides coordinates for individual interest groups, the frames and decision-makers in the (potentially) multidimensional policy space.
Research design
In order to test our proposed framing measurement approach, we conducted two case studies of two different policy debates in the European Union. First, we analysed the policy debate surrounding the 2007 proposal by the European Commission to reduce CO2 emissions from cars, as a previous study has already analysed this debate using three manual as well as automated text analysis techniques (Klüver Reference Klüver2009). We are therefore able to compare the results obtained by our framing analysis with independent estimates to cross-validate our findings. Second, we analysed a second policy debate leading to the policy proposal for a regulation of international rail passenger rights adopted by the European Commission in March 2004 to further illustrate the proposed quantitative text analysis technique.
In order to reduce CO2 emissions from cars, the European Commission published a Communication in February 2007 that laid out the plans for an upcoming proposal on this issue. On the basis of the Communication, the European Commission launched a public consultation in which stakeholders could submit position papers expressing their views on the proposed legislative framework before the Commission adopted its official proposal in December 2007. In order to enhance rail passenger rights, the European Commission released a working paper in October 2002 in which it suggested a legislative framework improving the rights of passengers in international rail transport and on which basis a public consultation was launched. In March 2004, the European Commission then adopted its final legislative proposal on this issue. By comparing the interest group position papers with, on the one hand, the Communication and the working paper and, on the other hand, the final Commission proposal, we can examine the framing strategies of interest groups and their effectiveness during the policy formulation stage.
The frames employed by interest groups were extracted from their submissions in the public consultations. In order to assess which frames were most successful in shaping the official position of the European Commission, we analysed the Communication and the working paper, as well as the preambles of the proposals finally adopted by the European Commission. We solely focused on the preambles rather than the entire legislative proposal due to considerations related to the comparability of the texts. The Communication and the working paper are continuous texts drafted without any restrictions. By contrast, the legislative proposals follow strict structural guidelines and employ highly technical legal terminology. Given that quantitative text analysis requires that documents use a similar pool of words, we therefore cannot use the entire legislative proposals for the computerised content analysis (Laver et al. Reference Laver, Benoit and Garry2003, 315). However, the proposal preambles are suitable for quantitative text analysis, as they summarise the content of the proposals and, at the same time, are written like a continuous text.
In addition, two further limitations of quantitative text analysis more generally and the framing measurement approach that we suggest more specifically have to be noted. First, it is necessary that all documents that are analysed are written in the same language. As the identification of frames is based on the co-occurrence of words, it is not possible to simultaneously analyse texts written in different languages. Second, the requirement that texts need to rely on a similar pool of words as mentioned above not only implies that texts need to be of a similar kind (e.g. do not compare speeches with laws), but also that the vocabulary is comparable. This could become a problem if framing is analysed over a longer period of time, as the use of words in a language typically change over time.
Before text analysis can be conducted, several preparatory steps are required. First, all the documents have to be transformed to machine-readable txt files. Second, all the text passages not directly referring to the policy debate have to be manually removed from the documents, such as contact details or repetitions of consultation questions. Third, spelling errors have to be corrected. While these modifications are standard practice in quantitative text analysis, several additional modifications have to be conducted that are specific to T-LAB. First, the documents have to be collapsed into one single file with each original document tagged with identification variables. Second, a list of key terms is selected, which is the basis upon which the cluster and correspondence analysis is carried out. T-LAB allows for an automated or manual selection of key terms. In this analysis, key terms were automatically selected on the basis of the χ 2 criterion after stopwords had been removed from the documents.Footnote 5
Framing the CO2 emmissions debate
In this section, we present the framing analysis of the CO2 emissions debate conducted with the proposed quantitative text analysis approach. First, the results of the cluster analysis are presented, which allows for identifying the frames employed by interest groups in this debate. Second, the findings of the correspondence analysis are reported, which maps interests groups in a multidimensional policy space to assess the effectiveness of interest group framing. Finally, the results obtained using the T-LAB analysis are cross-validated by comparing them to estimates obtained in a related study by Klüver (Reference Klüver2009).
Cluster analysis
Table 1 reports the most typical words per cluster (frame) according to their χ 2 value. Three document clusters could be identified: the first and smallest cluster (12% of the documents) comprises texts using words such as “advertising”, “press” and “media”. The list of typical words in this frame (cluster) clearly indicates its focus on the impact of the legislative proposal on the press and advertising industry. The following excerpt from FAEP (European Federation of Magazine Publishers) underscores how this frame deals with the implications for the press and advertising business:Footnote 6
Publishers would strongly oppose any political measure that has the potential to create an imbalance in the advertising revenues of the press as this would have a severe impact on the independence and diversity of the press.
Most prominent words distinguishing clusters of actors in the CO2 emissions debate
The second cluster, which encompasses 28% of the documents, is marked by words such as “automotive”, “segments” or “product”. The key terms show that this cluster comprises documents emphasising the negative impact of the proposal on automobile manufacturers. This frame is illustrated in the following text passage taken from the position paper of the VDA, which is the German automobile manufacturers association:
A policy discriminating against premium vehicles would damage a key area for generating value added and employment in the European automotive industry, and primarily in the German automotive industry.
The third and largest cluster (60% of the documents) is represented by words such as “LPG”, “biodiesel” and “natural”. Further analysis using the keyword-in-context function of the text analysis program Yoshikoder reveals that these terms are used to discuss the negative effects of global warming on the environment and to highlight the environmental superiority of alternative technologies, such as hybrid or electric cars and biofuels. The following excerpt from the contribution of Transport & Environment, an environmental NGO, highlights the nature of this frame:
Legislation on CO2 from cars will oblige car makers to implement CO2 saving technology on their vehicles. (…) They appear not to be willing to pay to avoid climate change, and do not even consider lifetime fuel savings, even if to do so would be in their own best interests. (…) CO2 regulation will lead to a quicker and more widespread adoption of fuel saving technology across Europe’s car fleet. (…) Strong regulation will to slow climate change, strongly reduce our oil bill and bring high tech development to Europe.
In order to check the validity of the framing analysis, we compared a manual coding of actor type to the clusters in which the text analysis classified the contributions of each interest group. The underlying assumption of comparing cluster membership and actor type is that interest groups of the same kind should use similar framing strategies. For instance, it is plausible to assume that environmental groups would rely on a similar framing strategy, as they share the same fundamental policy goal. Drawing on information gathered from the interest group submissions and websites concerning their interests and their organisational structure, we coded the interest groups into five different categories: traditional automobile industry groups that represent the interests of the car manufacturers (n=6); alternative automobile industry groups that, among others, represent the interests of manufacturers of electric cars and producers of biofuels (n=4); environmental NGOs fighting for environmental protection (n=6); press groups representing the interests of the print media and the advertising industry (n=2); and a variety of other groups (n=5).
Table 2 compares the clusters obtained by the text analysis with the coded group type. Each row represents an interest group together with the cluster membership of the document it submitted to the consultation. Cluster scores represent the degree to which each document is a member of the various clusters, as well as the best cluster solution according to these scores. The results show that the automated identification of clusters corresponds very strongly (though not perfectly) with a manual coding of group type. Of all the interest groups classified into substantial actor type categories, 89% were grouped into the same clusters as their fellow groups.Footnote 7 For instance, all environmental NGOs except for FOE (Friends of the Earth) were grouped into the environmental cluster. The cluster analysis also classified the initial (Comm1) and final (Comm2) Commission document. The cluster membership scores of the Commission have changed over time. While membership in the environment cluster has decreased from 0.42 to 0.38, membership in the industry cluster has increased from 0.36 to 0.39. The cluster analysis therefore indicates that the European Commission was responsive to the industry frame at the expense of the environment frame.
Comparison of actor type and cluster membership of interest groups
Note: Full names of the associations are available in the codebook accompanying the replication data set.
Correspondence analysis
In the second step, the underlying dimensions of the frames are identified and the frames are mapped spatially using correspondence analysis. The correspondence analysis identifies a two-dimensional space in which the frames are located (see Figure 1). The first dimension displayed on the x-axis of Figure 1 accounts for 58% of the variance and the second dimension accounts for 42%. The “environmental” and “industry” frames mainly oppose each other on the first “environmental regulation” dimension concerning the extent to which CO2 emissions should be limited by European legislation. On the second dimension, the “press” frame largely differs from the other two frames. The conflict on this dimension focuses on the regulation of European advertisement with regard to cars with high CO2 emissions, which the press groups AAUK and FAEP strongly oppose while the other groups largely ignore this issue.
Two-dimensional policy space of the CO2 emissions debate.
Finally, as we have two measures of the location of the European Commission, we can assess the direction of any movement in the official position. In this example, we compare the initial location (t 0) of the February 2007 Communication to the final location (t 2) of the legislative proposal adopted by the Commission in December of that same year, after the consultation materials described above had been submitted and reviewed (t 1). It should be noted that the interest group comments were submitted after the Commission published its communication in February 2007 but before the Commission adopted its final legislative proposal in December 2007. The movement of the European Commission between the two time points is marked by an arrow. Figure 1 shows that the European Commission moved primarily on the first dimension away from the environmental and alternative industry groups and towards the traditional automobile industry.
Cross-validating the results
In order to further check the validity of the proposed framing analysis approach, we compare the results obtained by the cluster and correspondence analysis with position estimates obtained by Klüver (Reference Klüver2009) using three established text analysis techniques: manual hand-coding, Wordfish and Wordscores. In a manually hand-coded content analysis, the texts are divided into units of analysis (here: natural sentences) and human coders assign every single text unit to categories contained in a predefined coding scheme. Position estimates are obtained by subtracting the mentions of positive categories from the mentions of negative categories. By contrast, Wordscores and Wordfish are fully automatised text analysis techniques that estimate policy positions of texts based on the relative frequency of words within and across documents (for more detailed information about the text analysis procedures, see Laver et al. Reference Laver, Benoit and Garry2003; Slapin and Proksch Reference Slapin and Proksch2008; Klüver Reference Klüver2009). Since these techniques were used to estimate the policy positions rather than the framing strategies of interest groups, we are not able to cross-check the identification of frames based on the cluster analysis. However, given that the correspondence analysis produces coordinates of interest groups and the European Commission in the policy space, we are able to cross-validate these estimates.
Since manual hand-coding, Wordscores and Wordfish estimate policy positions only for a single dimension, which corresponds to a “pro-anti environmental control dimension” in this particular policy debate, we compare them separately to the two dimensions identified by our framing analysis (see Table 3). The estimates obtained by T-LAB strongly correlate with the hand-coding (r=0.76), Wordfish (r=0.74) and Wordscores (r=0.50) estimates on the first dimension. However, on the second dimension, the correlation between our framing analysis estimates and the results obtained by each text analysis technique is only moderate. The comparison of the different estimates therefore shows strong correspondence for the first dimension, but little for the second one. In addition, according to all three text analysis techniques, the European Commission moved towards the traditional automobile industry over the course of the policy debate, which corresponds to the Commission movement detected by the correspondence analysis.
Correlation of T-LAB coordinates with hand-coding, Wordfish and Wordscores
Source of hand-coding, Wordfish and Wordscores estimates: Klüver (Reference Klüver2009).
*p<0.1, **p<0.05, ***p<0.01.
Framing the passenger rights debate
In line with the illustration of the CO2 emissions debate, we will now demonstrate the findings of the framing analysis of the passenger rights debate. First, we will present the results of the cluster analysis that identified three different clusters. Then, we will report the results of the correspondence analysis, which maps interest groups and the European Commission in a multidimensional policy space.
Cluster analysis
The T-LAB analysis has identified three different frames employed by interest groups in the passenger rights debate. Table 4 presents the top 15 key terms associated with each frame. The first frame is expressed with words such as “disabled, “blind” and “accessibility”. These key terms clearly indicate that this frame focuses on the implication of the policy proposal for disabled people. For instance, the European Disability Forum highlights in its position paper:
the absolute necessity of ensuring full and equal access for disabled persons to all stations, not just the major international stations, and to all trains. Disabled persons and other PRMs want to be able to travel as independently as possible. By ensuring an accessible rail service it will ensure a service accessible for all customers not just for disabled persons.
Most prominent words distinguishing clusters of actors in the rail passenger rights debate
The second frame is associated with key terms such as “charter”, “voluntary” or “commitment”. The key terms indicate that the frame is employed to advocate for a voluntary passenger rights scheme instead of a regulatory approach. For instance, TRENITALIA, the largest railway operator in Italy, argues as follows:
Should a regulatory approach be adopted, which remains highly questionable, several topics regarding rail passenger services should be left to subsidiarity. A voluntary charter allows to overcome all these difficulties. This Charter has the further advantage to be applied also by railways undertakings from non-EU countries.
The third frame is associated with key terms such as “passengers”, “compensation” and “protection”. Interest groups employing this frame highlight the necessity for enhanced passenger rights and protection. BEUC, the European consumer organisation, for instance, makes the following argument:
We strongly believe that legislation at the EU level is the only way to provide clear, legally enforceable levels of passenger protection in the passengers/rail operator relationship and removal of legal uncertainty. We therefore support a legislative proposal regarding rail passengers’ rights.
Hence, the inspection of the key terms and the ways they are used by interest groups provides a high degree of face validity for the three frames that are identified by the text analysis. To further validate the frame measurement, we shed light on which interest group types employ the three different frames. Based on information gathered from consultation submissions and interest group websites, we coded interest groups into four different categories: road operators (n=5); interest groups representing passenger interests, which include general consumer organisations and generic passenger federations (n=9); interest groups representing the interests of disabled persons (n=3); and other groups (2).
Table 5 reports the interest group name, the group type and the frame identified by the text analysis. The text analysis results strongly correspond with the manual coding of interest group type. All the interest groups representing the interests of handicapped people employ the “accessibility” frame, while all interest groups representing the interests of passengers use the “passenger” frame. Moreover, four out of five rail operators employ the “voluntary” frame advocating for a voluntary self-commitment instead of a regulatory approach. Thus, also with regard to the comparison of actor type and frame choice, the results of the text analysis exhibit a high degree of face validity. In addition, the cluster analysis also classified the working paper and the legislative proposal adopted by the European Commission. Both documents are primarily associated with a “passenger” rights frame, even though the Commission proposal is slightly more similar to the “voluntary” frame than the initial working paper, as reflected in the increase of the economic cluster membership score from 0.35 to 0.37. The cluster analysis therefore indicates that the European Commission remains closest to the passenger organisations’ position, but that it has slightly adjusted its argumentation in response to the “voluntary” frame employed by rail operators.
Comparison of actor type and cluster membership of interest groups
Note: Full names of the associations are available in the codebook accompanying the replication data set.
Correspondence analysis
Drawing on correspondence analysis, interest groups and the European Commission are furthermore mapped spatially (see Figure 2). The correspondence analysis identified two different dimensions. The first dimension, illustrated on the x-axis in Figure 2, accounts for 61% of the variance, while the second dimension, plotted on the y-axis, accounts for 39% of the variance detected in this debate. Figure 2 shows that rail operators are opposing interest groups representing passenger interests on the first dimension. Hence, in line with the framing excerpts from the interest group documents discussed above, conflict on this dimension centres on the question to what extent passenger rights should be legally regulated by the European Commission or instead solely based on voluntary self-commitments of the railway companies. While railway operators employ a “voluntary” frame to highlight the advantages of a voluntary charter of passenger rights, their opponents use a “passenger” frame to push for strict legislative regulations.
Two-dimensional policy space of the rail passenger debate.
On the second dimension, the conflict primarily takes place between interest groups representing handicapped people on the one hand and railway operators and passenger organisations on the other hand. While NGOs representing disabled persons use this policy debate to fight for better accessibility for people with handicaps in European railway transport, neither railway operators nor passenger organisations discuss this issue in their consultation submissions.
The correspondence analysis furthermore allows for assessing the movement of the European Commission over the course of the policy debate, as the text analysis indicates the position of the European Commission based on its initial working paper and its final legislative proposal. While the European Commission has remained fairly stable on the second “accessibility for handicapped people” dimension, it has moved slightly towards rail operators on the “regulating passenger rights” dimension as indicated by the arrow in Figure 2. Hence, one can conclude that rail operators were moderately successful, as the European Commission has accommodated at least some of their demands.
Conclusion
The way interest groups frame a policy initiative can have significant implications on the outcome of a legislative debate. Even though framing is at the core of understanding political outcomes, we have very little systematic data on the framing strategies of interest groups and their impact on public policy. The lack of research and data on interest group framing is largely due to methodological difficulties in systematically studying framing. This study has therefore introduced a new methodological approach to analyse interest group framing. Drawing on recent advances in quantitative text analysis, we have employed a combination of cluster and correspondence analysis implemented in the software package T-LAB to identify frames put forward by interest groups and to assess the dimensionality of policy debates. By studying interest group position papers as well as official legislative documents adopted by decision-makers, we were furthermore able to empirically evaluate the success of different frames with regard to shaping the outcome of legislative debates.
We have tested this new methodological approach on two policy debates in the areas of environmental and transport policy in the European Union. These case studies have shown that the new methodological approach is able to systematically extract the frames that interest groups employ based on the relative frequencies of words within and across position papers of interest groups and decision-makers. We have cross-validated these results by comparing the frames identified in the debates with a coding of interest group types and found a strong correspondence – interest groups of the same kind tend to employ similar frames. These two case studies therefore show that our new measurement approach is associated with a high degree of face validity.
As we were able to draw on independent estimates from a related study of one of our cases, we were moreover able to compare our findings to the results from three more established text analysis techniques, namely manual hand-coding, Wordfish and Wordscores. Our results strongly correlate with estimates obtained using these three methods, therefore showing a high degree of convergent validity. However, while manual content analysis, Wordfish and Wordscores are restricted to examining one dimension, the methodological approach introduced in this study allows for multiple dimensions. As the most encompassing study of EU policy-making by Thomson (Reference Thomson2011) has shown that legislative debates in the European Union are inherently multidimensional, the text analysis technique introduced in this study is beneficial, as it allows one to adequately capture the multidimensional contestation in EU policy debates.
The results of this study have major implications for the study of interest group framing. While methodological difficulties have largely prevented scholars from examining framing strategies and their impact on public policy across a large number of policy debates so far, the approach introduced in this study paves the way for the large-scale analysis of interest group framing. The methodological approach proposed in this article will enable interest group scholars to study the role of interest group framing as long as consultations are held before a legislative act is adopted. Legislative consultations are, for instance, similarly organised in Germany, the United Kingdom, the Netherlands and Denmark. In addition, it would also be possible to apply the measurement approach to position papers released by interest groups on their websites. The measurement approach introduced in this article therefore allows for studying interest group framing on a large empirical scale in a variety of different political systems. What is more, the measurement approach is not limited to studying the framing of interest groups, but rather can be used to analyse framing in a variety of different contexts, such as in political campaigns, parliamentary speeches or international negotiations.
Acknowledgements
The authors thank Frank R. Baumgartner, Cheryl Schonhardt-Bailey, Peter John and the three anonymous reviewers for their valuable comments and suggestions. They thank Marc Opper for impeccable research assistance. The authors also thank the National Science Foundation for generous research funding (Grant number 1102978).
