Everyone interested in democratic politics—social scientists, journalists, and the public alike—would like to know how policy is affected by the many people and organizations struggling for influence: interest groups and social movement organizations, manufacturers and unions, the National Rifle Association and gun control groups, as well as many other groups and organizations. And we would like to know how strongly policy is affected by the methods they use to win influence, either through lobbying and protest, campaign contributions and forming alliances with those in power, as well as the many other tactics used by participants in democratic politics.
One critical way in which social scientists contribute to our understanding of policy change is developing and testing theories about how political organizations and their activities affect policy. Since the turn of the twenty-first century, major journals in political science and sociology have published a plethora of articles testing 19 theories about the impact of political organizations (not including political parties) and their activities on policy. There are theories about the impact of interest groups; different theories about the impact of social movement organizations (SMOs); still others about advocacy organizations, nongovernmental organizations (NGOs), unions, and other types of organizations; and separate theories about the impact of lobbying and protest.
What are these theories? What is distinctive about each of them? How well do they predict the effects of political organizations and activities on policy? And how do they help us generalize about the forces that influence policy, which is a core element of democratic politics?
At this point, we have no idea how to answer these questions. There has been no review of the tests of relevant theories, and articles that test particular theories provide little context: they are vague about what previous tests of their theory have shown and say little about what their own tests add to what we know collectively about support for the theory.
This article provides such a review, based on an analysis of all articles testing theories about advocacy and public policy published in 25 high-impact and specialized journals in political science and sociology from 2000 through 2018. It considers what these theories predict about the impact of forces that hypothetically affect policy, the mechanisms seen as linking those forces to policy, the theories’ scope, why there are so many theories, how often hypotheses associated with the theories are supported by the data, what generalizations about the policy process do the theories help us reach, and what are likely to be the most productive next steps toward improving and applying the theories.
The theories generally make the same predictions—that many types of political organizations and activities affect policy—and propose the same mechanism of influence: politicians’ concerns about reelection. The theories are typically narrow, focusing on the impact of just one type of political organization or activity. The hypotheses are consistent with the data just over half the time. Researchers neither test theories against each other nor keep track of how accurate their theories’ predictions are. They seldom try to generalize, and their efforts to propose productive next steps are modest.
After analyzing how theories are tested and reviewing the results, I suggest how the theory-testing process can be improved—specifically, how researchers can create a more coherent research community, improve how they report and evaluate their findings, and make better use of their results—and in this way contribute to the collective accumulation of knowledge.
Theories about What?
Imagine wanting to find out what forces affect policy and how strongly they do so. One way to proceed would be to study the impact of the civil-society organizations and political activities often said to be crucial to democratic responsiveness—including specific types of organizations such as businesses, unions, and environmental organizations; organizations defined more abstractly, such as interest groups and SMOs; and activities including lobbying and protesting—while developing theories to explain their influence.
That is not how the study of policy change works for the most part. Instead, there is a division of labor. There are separate literatures on interest groups (see Baumgartner and Leech Reference Baumgartner and Leech1998; Bunea and Baumgartner Reference Bunea and Baumgartner2014; Hojnacki et al. Reference Hojnacki, Kimball, Baumgartner, Berry and Leech2012), social movements (Amenta et al. Reference Amenta, Caren, Chiarello and Su2010; Giugni Reference Giugni1998; Uba Reference Uba2009), advocacy organizations (Andrews and Edwards Reference Andrews and Edwards2004), unions (Ahlquist Reference Ahlquist2017), and lobbying (de Figueiredo and Richter Reference de Figueiredo and Richter.2014; Leech Reference Leech, Maisel, Berry and Edwards2010). But relatively little is written about how the different types of organizations and activities collectively influence policy (Burstein Reference Burstein2019; Burstein and Linton Reference Burstein and Linton2002). For those wanting to learn about theories explaining how organizations and activities affect policy, there is the additional problem that literature reviews say little about theory. They may address theories in a general way (e.g., Hojnacki et al. Reference Hojnacki, Kimball, Baumgartner, Berry and Leech2012, 380–83), but they do not describe what specific theories predict, compare theories to each other, or assess how often theories’ predictions are supported empirically.
This article is both unusual in that it analyzes work on the impact of a wide range of organizations and activities on policy, and unique in that it analyzes work that tests theories attempting to explain the impact. Ultimately, it will try to come to a judgment over whether there are good reasons for having separate literatures and different theories for different types of organizations and activities, or whether the existing division of labor is problematic because the different theories share an underlying logic and make the same predictions. In that case, it is possible that the number of theories could be consolidated and their scope increased, so that we could move toward a more unified approach to the study of policy change.
Here it is necessary to address a conceptual and terminological problem. We have many types of political organizations and activities—interest groups, SMOs, lobbying, protest, and so on—but no standard term that encompasses them all. “Political organizations and activities” is too generic; it includes political parties and is vague about what the organizations and activities have in common. “Advocacy organizations and activities” seems better; those organizations advocate for policies. Unfortunately, similar terms are already in use. Andrews and Edwards (2004, 485) use the term “advocacy organizations” to refer to organizations making “public interest claims either promoting or resisting social change,” whereas Burstein (2014, 8) defines advocacy as “all kinds of publicly reported activity supporting or opposing specific policies.” This article uses the Burstein definition because it is broader and easier to apply, but extends it in two ways—to organizations, as well as activities—while encompassing organizations and activities seen by researchers as supporting or opposing specific policies.
Finding and Analyzing Tests of Theories
The intent here is to analyze all articles testing theories about the impact of advocacy on policy that have been published in a substantial set of major political science and sociology journals. This article is a “research synthesis,” a quantitative analysis of central aspects of articles about the impact of advocacy on policy similar to the work of Bunea and Baumgartner (Reference Bunea and Baumgartner2014) and Burstein (Reference Burstein2019). Footnote 1
To be included in this analysis, an article had to include an explicit test of a specific theory about the impact of advocacy on policy. The dependent variable—the aspect of policy that advocates may influence—involved the final stage, or something close to it, of a legislature’s or administrative agency’s decisions about a proposed policy, focusing on specific policy-making actions. These actions include, for example, congressional enactment of pro-environmental laws (Johnson, Agnone, and McCarthy Reference Johnson, Agnone and McCarthy2010); the adoption in US states of laws pertaining to women’s suffrage (Cornwall et al. Reference King, Legerski, Dahlin and Schiffman2007; King, Cornwall, and Dahlin Reference King, Cornwall and Dahlin2005), gay rights (Kane Reference Kane2003; Lax and Phillips Reference Lax and Phillips2009), and gun control (Steidley Reference Steidley2018); adoption internationally of laws on abortion and human rights (Boyle, Kim, and Longhofer Reference Boyle, Kim and Longhofer2015; Koo and Ramirez Reference Koo and Ramirez2009); social welfare policies in OECD countries (Jensen Reference Jensen2012); specific policies in Norway (Sorensen Reference Sorensen2003) and India (Uba Reference Uba2005); and action on policy proposals across a wide range of issues in the United States (Gilens and Page Reference Gilens and Page2014).
The view of theory testing is a practical one. I viewed articles as explicitly testing a theory if the authors said before their data analysis that they were doing so, or they stated that a theory guided their research and discussed it after analyzing their data, or they stated after the data analysis that a theory had been tested. Testing a “specific theory” meant the theory had a name—“democratic theory,” “social movement theory,” “the theory of legislative logic,” and the like. There was no second-guessing about what constituted a theory or a test of a theory.
All types of advocacy seen as attempting to influence the final stage of the policy process were included, provided there were statistical estimates of their impact on policy. The usual conventions were followed: I considered hypotheses that advocacy has an impact as being supported by the data if the relevant coefficient was described as significant at the 0.05 level. Questions are sometimes raised about reliance on significance tests, but they are the most practical and widely used method for comparing the findings of different studies.
I first sought articles in high-impact political science and sociology journals appearing between 2000 and 2018. The political science search began with the 15 highest-ranked journals, as calculated for 2016, according to Incites Journal Citation Reports Footnote 2 : the American Journal of Political Science, the American Political Science Review, the Annual Review of Political Science, the British Journal of Political Science, the European Journal of Political Research, International Organization, the Journal of Conflict Resolution, the Journal of European Public Policy, Perspectives on Politics, Political Analysis, Political Psychology, Public Administration, Regulation & Governance, the Review of International Political Economy, and World Politics. I did not search the Annual Review of Political Science, because it does not publish original research. To select articles, I read the table of contents of every issue and, when necessary, the abstract and then the full article.
The sociology search started with the top 20 journals. Footnote 3 I then searched these nine journals: the American Journal of Sociology, the American Sociological Review, European Sociological Review, Gender & Society, Journal of Marriage and Family, the Population and Development Review, Social Forces, the Socio-Economic Review, and the Sociology of Education. The others were not included because they were very unlikely to include articles on policy; for example, Cornell Hospitality Quarterly and Sociological Methods and Research.
I also searched two specialty journals: Interest Groups & Advocacy (first published in 2012), which focuses on interest groups, and Mobilization, which highlights protest and social movements. Thus, 25 journals were searched: 9 high-impact sociology journals, 14 high-impact political science journals, and 2 specialty journals. I tried other approaches to searching for articles, including searching in major academic journal databases; there turned out to be technical problems in such searches, however, and they failed to find many of the articles included here.
This search was wider than those in other reviews. Amenta et al. (Reference Amenta, Caren, Chiarello and Su2010) searched 6 sociology journals; Baumgartner and Leech (1998, 128), various “major political science and economics journals”; Bunea and Baumgartner (Reference Bunea and Baumgartner2014), 22 policy and political science journals; Burstein and Linton (Reference Burstein and Linton2002), 6 journals in political science and sociology; Uba (Reference Uba2009), 11 journals; and Hojnacki et al. (Reference Hojnacki, Kimball, Baumgartner, Berry and Leech2012) considered 5 political science journals along with books from Brookings and major university presses.
There were two units of analysis: (1) articles, relevant to analyses of empirical focus, authors’ theories, generalizations, and conclusions, and (2) hypotheses, central to theory testing. Hypotheses were included only if derived from the theories being tested.
The search discovered 77 articles analyzing the impact of advocacy on policy. Twenty-four (identified by an asterisk in the References) tested theories: 19 in sociology (6 in Mobilization, 5 in Social Forces, 4 in the American Sociological Review, 3 in the American Journal of Sociology, and 1 in the Socio-Economic Review) and 5 in political science (2 in International Organization and 1 each in the American Political Science Review, the European Journal of Political Research, and Perspectives on Politics). The imbalance between political science and sociology may surprise some readers, but is not a product of the search procedure; a somewhat similar search for articles published from 1990 through 2000 found considerably more political science articles than sociology articles (Burstein and Linton Reference Burstein and Linton2002).
The way to code certain variables, including the countries and issues studied, is readily apparent. The coding of other variables must be spelled out, however. A key guideline was that coding should reflect the authors’ views. Thus, authors’ descriptions of their theories were accepted at face value. The “theory of representative democracy” (Giugni Reference Giugni2007) may appear very similar to “democratic theory” (Lax and Phillips Reference Lax and Phillips2009), but I treated theories with different names as different theories. The coding of organizations and activities proceeded similarly. If one author described a particular organization as an interest group, a second described the same organization as an SMO, and a third considered the organization on its own without putting it into a broader category, that organization was categorized in each instance as the authors did.
What, Where, When
I begin with the policies, governments, and historical periods analyzed in the research.
The 24 theory-testing articles addressed a wide variety of policies. Four focused on environmental policy (Agnone Reference Agnone2007; Giugni Reference Giugni2007; Johnson, Agnone, and McCarthy Reference Johnson, Agnone and McCarthy2010; Longhofer et al. Reference Longhofer, Schofer, Miric and Frank2016), three on social policy broadly construed (Amenta, Caren, and Olasky Reference Amenta, Caren and Olasky2005; Amenta and Halfmann Reference Amenta and Halfmann2000; Jensen Reference Jensen2012), two on women’s suffrage (Cornwall et al. Reference King, Legerski, Dahlin and Schiffman2007; King, Cornwall, and Dahlin Reference King, Cornwall and Dahlin2005), two pertaining to gay rights (Kane Reference Kane2003; Lax and Phillips Reference Lax and Phillips2009), and one each on monetary policy (Bearce Reference Bearce2003), gun control (Steidley Reference Steidley2018), the ERA (Soule and Olzak Reference Soule and Olzak.2004), intergovernmental grants (Sorensen Reference Sorensen2003), taxes (Martin Reference Martin2010), abolition (Budros Reference Budros2011), trade policy (Gawande and Hoekman Reference Gawande and Hoekman2006), business regulation (Ingram and Rao Reference Ingram and Rao2004), human rights (Koo and Ramirez Reference Koo and Ramirez2009), privatization (Uba Reference Uba2009), and defense or foreign policy (McAdam and Su Reference McAdam and Su2002). One considered policies on 1,779 issues (Gilens and Page Reference Gilens and Page2014), and one touched on health (Boyle, Kim, and Longhofer Reference Boyle, Kim and Longhofer2015).
Most articles focused on the United States: 6 on federal policies and 11 on state policies. Two articles were about OECD countries (Bearce Reference Bearce2003; Jensen Reference Jensen2012), two on other individual countries (Sorensen Reference Sorensen2003; Uba Reference Uba2005), and three on many countries globally (Boyle, Kim, and Longhofer Reference Boyle, Kim and Longhofer2015; Koo and Ramirez Reference Koo and Ramirez2009; Longhofer et al. Reference Longhofer, Schofer, Miric and Frank2016). Even though several European journals were searched, they did not yield theory-testing articles about advocacy and policy in Europe.
How Many Theories?
Recent reviews of research on interest groups suggest that progress in a field is likely to be greatest when researchers adopt “a common theoretical outlook as they address shared concerns” (Hojnacki et al. Reference Hojnacki, Kimball, Baumgartner, Berry and Leech2012, 382), together forming “an academic community that works within a unified theoretical framework” (Bunea and Baumgartner Reference Bunea and Baumgartner2014, 1428). Scholars may disagree about the utility of various theories and other issues, but they do so within a context in which they generally agree about which theories deserve the most attention. This logic should apply not just to studies of interest groups but also to studies of advocacy in general.
There is no established standard for deciding when there is “considerable agreement” about which theories deserve attention, but those studying the impact of advocacy agree hardly at all. The articles tested 19 theories. The theory most often tested—that is, most often seen as important—was political mediation theory, tested in four articles (Amenta, Caren, and Olasky Reference Amenta, Caren and Olasky2005; Cornwall, et al. Reference King, Legerski, Dahlin and Schiffman2007; Johnson, Agnone, and McCarthy Reference Johnson, Agnone and McCarthy2010; Steidley Reference Steidley2018). Democratic theory and social movement theory were tested in three articles each (the former by Agnone [Reference Agnone2007], Lax and Phillips [Reference Lax and Phillips2009], and McAdam and Su [Reference McAdam and Su2002]; the latter by Budros [Reference Budros2011], Ingram and Rao [Reference Ingram and Rao2004], and Soule and Olzak [Reference Soule and Olzak.2004]). Three more were tested in two articles each: institutional politics theory (Amenta and Halfmann Reference Amenta and Halfmann2000; Martin Reference Martin2010), resource mobilization theory (Kane Reference Kane2003; Steidley Reference Steidley2018), and world society theory (Boyle, Kim, and Longhofer Reference Boyle, Kim and Longhofer2015; Longhofer et al. Reference Longhofer, Schofer, Miric and Frank2016). The remaining 13 theories were tested in one article each: biased pluralism, economic elite domination, majoritarian electoral democracy, and majoritarian pluralism by Gilens and Page (Reference Gilens and Page2014), economic pluralism theory by Bearce (Reference Bearce2003), Grossman-Helpman general equilibrium theory by Gawande and Hoekman (Reference Gawande and Hoekman2006), the theory of legislative logic by King, Cornwall, and Dahlin (Reference King, Cornwall and Dahlin2005), political economy theory by Sorensen (Reference Sorensen2003), political opportunity theory by Kane (Reference Kane2003), power resources theory by Jensen (Reference Jensen2012), the theory of representative democracy by Giugni (Reference Giugni2007), signaling theory by Uba (Reference Uba2005), and world polity theory by Koo and Ramirez (Reference Koo and Ramirez2009).
Why So Many Theories?
There could be many reasons for there being so many theories. Here I consider four possibilities: (1) competing theories make different predictions about the impact of advocacy on policy; (2) competing theories propose different mechanisms linking advocacy and policy; (3) different types of advocacy organizations or activities—for example, interest groups and SMOs—differ in ways warranting different theories; and (4) scholars develop theories independently, paying little attention to existing theories when they develop their own.
What Theories Predict
Most often, different theories make the same prediction: that advocacy will directly affect policy. Eleven theories do so: biased pluralism (Gilens and Page Reference Gilens and Page2014), economic pluralism (Bearce Reference Bearce2003), Grossman-Helpman general equilibrium (Gawande and Hoekman Reference Gawande and Hoekman2006), institutional politics (Amenta and Halfmann Reference Amenta and Halfmann2000), majoritarian pluralism (Gilens and Page Reference Gilens and Page2014), political economy (Sorensen Reference Sorensen2003), political opportunity (Kane Reference Kane2003), resource mobilization (Kane Reference Kane2003; Steidley Reference Steidley2018), signaling (Uba Reference Uba2005), world polity (Koo and Ramirez Reference Koo and Ramirez2009), and world society theories (Boyle, Kim, and Longhofer Reference Boyle, Kim and Longhofer2015; Longhofer et al. Reference Longhofer, Schofer, Miric and Frank2016). The 24 articles collectively include 108 hypotheses, of which 73 predict a positive relationship between advocacy and policy.
Some other theories make different predictions. Three contend that particular forms of advocacy will not affect policy. Theories of majoritarian electoral democracy (Gilens and Page Reference Gilens and Page2014) and the theory of representative democracy (Giugni Reference Giugni2007) predict that interest groups and protest, respectively, will have no effect because public opinion is more powerful; economic elite domination theories (Gilens and Page Reference Gilens and Page2014) say the same about mass-based interest groups vis-a-vis economic elites.
Five theories argue that the impact of some forms of advocacy depends on context. Democratic theory, political mediation theory, and the theory of representative democracy contend that advocacy and public opinion will interact, each strengthening the impact of the other (Agnone Reference Agnone2007, 1597; Giugni Reference Giugni2007, 55; Lax and Phillips Reference Lax and Phillips2009, 380; Steidley Reference Steidley2018, 107). Political mediation theory and the theory of representative democracy also propose that the impact of advocacy will increase when a supportive political party is in power, as does social movement theory (Giugni Reference Giugni2007, 55; Johnson, Agnone, and McCarthy Reference Johnson, Agnone and McCarthy2010, 2282–83; Soule and Olzak Reference Soule and Olzak.2004, 482; Steidley Reference Steidley2018, 107); political mediation theory adds that the impact of advocacy will increase when the public’s political ideology is favorable and elections are competitive (Steidley Reference Steidley2018, 108). Social movement theory (Budros Reference Budros2011) also contends that organizations will have less impact on issues they care relatively little about, as does power resources theory (Jensen Reference Jensen2012), and the theory of legislative logic (King, Cornwall, and Dahlin Reference King, Cornwall and Dahlin2005) suggests that, although social movements may influence legislators early in the policy process, they will have little or no influence on enactment.
How Advocacy Affects Policy
Theories may disagree not only about what they predict but also about what connects advocacy and policy. Different mechanisms may call for different theories.
For the most part, however, the theories do not disagree. Advocacy works because of its connection to election outcomes. In one view of democratic theory, demonstrations influence policy because they raise the salience of issues, making elected officials afraid to oppose the public (Agnone Reference Agnone2007). Political economy theory sees lobbying as effective because it convinces elected officials that supporting particular policies will lead to their reelection (Sorensen Reference Sorensen2003). Social movement theory proposes that SMOs get what they want when politicians sympathetic to their views win office (Soule and Olzak Reference Soule and Olzak.2004). Similar arguments are made by economic pluralism theory (Bearce Reference Bearce2003), Grossman-Helpman general equilibrium theory (Gawande and Hoekman Reference Gawande and Hoekman2006), institutional politics theory (Amenta and Halfmann Reference Amenta and Halfmann2000; Martin Reference Martin2010), the theory of legislative logic (King, Cornwall, and Dahlin Reference King, Cornwall and Dahlin2005), political opportunity theory (Kane Reference Kane2003), the theory of representative democracy (Giugni Reference Giugni2007), resource mobilization theory (Steidley Reference Steidley2018), and signaling theory (Uba Reference Uba2009). No theory argues that electoral politics is unimportant. Some suggest that other factors may matter, including the spread of ideas through organizational networks (in world polity and world society theories; Boyle, Kim, and Longhofer Reference Boyle, Kim and Longhofer2015; Koo and Ramirez Reference Koo and Ramirez2009; Longhofer et al. Reference Longhofer, Schofer, Miric and Frank2016).
Thus, many theories make comparable predictions and agree about what links advocacy to policy. So how do they differ? If researchers were aware of differences among theories not captured by the analyses presented so far, they could test the competing theories against each other. But among the 24 articles, only Bearce (Reference Bearce2003) and Gilens and Page (Reference Gilens and Page2014) do so.
How Theories Differ
There actually is one way in which the theories really differ: in the type of advocacy to which they apply (see table 1). In most articles, the main independent variables are organizations, which vary in terms of attributes (their number or resources, for example) or activities. In some articles, the activities themselves serve as the main independent variables, independent of any organization.
Theories, and Organizations and Activities to Which They Apply
a Specific organizations are organizations not categorized, including Share Our Wealth (Amenta and Halfmann Reference Amenta and Halfmann2000), the American Civil Liberties Union, and the Lambda Legal Defense and Education Fund, as well as gay bars and restaurants (Kane Reference Kane2003).
b Institutional influence activity includes press conferences, lobbying, litigation, testifying before congressional committees, and public education (Johnson, Agnone, and McCarthy Reference Johnson, Agnone and McCarthy2010).
c Conventional action includes lobbying and PAC expenditures (Giugni Reference Giugni2007).
d International organizations include both intergovernmental organizations and international NGOs (Koo and Ramirez Reference Koo and Ramirez2009).
For the most part, theories link advocacy and policy for only one type of organization or activity. Seven theories apply only to interest groups, three only to SMOs (plus specific named organizations), one to unions, one to international organizations, and one to NGOs. Only two theories are said to apply to more than one type of organization: social movement theory is proposed to apply to both interest groups and SMOs, and institutional politics theory to advocacy groups and unions. With regard to activities, democratic and signaling theories apply only to protest, political economy theory only to lobbying, and social movement theory only to petitioning. Just two theories are said to apply to more than one type of activity: political mediation theory to protest and institutional activity, and the theory of representative democracy to protest and conventional action.
Why do different theories focus on different types of organizations and activities? No authors say why they limit their theories to one type of organization or activity, nor with one exception, do they mention any differences among organizations or activities that matter theoretically. Thus, we have to infer why theories are linked to particular types of organizations and activities.
Perhaps different theories predict that different types of organizations will affect policy differently. But the theories do not differ. All predictions about the impact of NGOs, advocacy groups, international organizations, gay bars, specific organizations, lobbying, conventional action, and petitioning are the same: they all will influence policy change. Disagreements between theories are about the impact of the same type of organization or activity; for example, Gilens and Page (Reference Gilens and Page2014) show that some theories predict mass-based interest groups will affect policy, whereas others predict that they will not.
Perhaps there are differences among types of organizations or activities that warrant different theories. Early work on SMOs, for example, argued that they were very different from interest groups, possibly justifying different theories; yet others argue that the differences are minimal (e.g., see Andrews and Edwards Reference Andrews and Edwards2004; Burstein Reference Burstein, Costain and McFarland1998b; Fraussen and Halpin Reference Fraussen and Halpin2018; Jordan, Halpin, and Maloney Reference Jordan, Halpin and Maloney2004; McAdam, Tarrow, and Tilly Reference McAdam, Tarrow and Tilly1996). How do the theory-testing articles describe the different advocacy organizations and activities?
Five articles—Amenta, Caren, and Olasky (Reference Amenta, Caren and Olasky2005), Johnson, Agnone, and McCarthy (Reference Johnson, Agnone and McCarthy2010), Kane (Reference Kane2003), Soule and Olzak (Reference Soule and Olzak.2004), and Steidley (Reference Steidley2018)—focus on SMOs. Yet none distinguishes between SMOs and interest groups; indeed, they highlight similarities rather than differences. For Kane, SMOs “include … political organizations,” which must encompass interest groups. SMOs include “highly institutionalized issue advocacy organizations” (Johnson, Agnone, and McCarthy Reference Johnson, Agnone and McCarthy2010, 2275) that “strategically use institutionalized tactics” (Soule and Olzak Reference Soule and Olzak.2004, 482)—yet traditionally interest groups were seen as highly institutionalized, SMOs less so (Andrews and Edwards Reference Andrews and Edwards2004, 482–83; Burstein Reference Burstein, Costain and McFarland1998b, 42-43). The National Rifle Association is described as both an SMO (Steidley Reference Steidley2018, 103–4) and an interest group (Gilens and Page Reference Gilens and Page2014); so is the National Organization for Women (Barakso Reference Barakso2005, 317; Soule and Olzak Reference Soule and Olzak.2004).
Five articles—Bearce (Reference Bearce2003), Gawande and Hoekman (Reference Gawande and Hoekman2006), Gilens and Page (Reference Gilens and Page2014), Ingram and Rao (Reference Ingram and Rao2004), and Lax and Phillips (Reference Lax and Phillips2009)—consider the impact of interest groups, yet none defines “interest group.” Bearce (2003, 379) says that interest groups get what they want through lobbying, which enables people to “make their preferences known to the state”; that is exactly what McAdam and Su (2002, 702) say about protesters. Interest groups are said to get what they want by exerting “pressure” on elected officials (Lax and Phillips Reference Lax and Phillips2009, 367); so do SMOs (Amenta, Caren, and Olasky Reference Amenta, Caren and Olasky2005, 525; Ingram and Rao Reference Ingram and Rao2004, 479; Kane Reference Kane2003, 318).
Nor is it clear what theoretically relevant differences distinguish other types of organizations. Andrews and Edwards (2004, 485) define a new type of organization—advocacy organizations—distinguished from other types by making “public interest claims.” But they do not define “public interest claim,” nor does the sole article (Martin Reference Martin2010) assessing the impact of advocacy organizations. Therefore it is not apparent why the organizations identified as advocacy organizations, such as Chambers of Commerce and anti-tax organizations, warrant their own theories. Nor do those who write about NGOs and international organizations (Boyle, Kim, and Longhofer Reference Boyle, Kim and Longhofer2015; Koo and Ramirez Reference Koo and Ramirez2009; Longhofer et al. Reference Longhofer, Schofer, Miric and Frank2016) provide any hint as to what distinctive features warrant their own theories.
Seven articles address political activities not connected to specific types of organizations: protest (Agnone Reference Agnone2007; Giugni Reference Giugni2007; Johnson, Agnone, and McCarthy Reference Johnson, Agnone and McCarthy2010; McAdam and Su Reference McAdam and Su2002; Uba Reference Uba2005); “institutional influence activity” and “conventional action,” which are very similar (Giugni Reference Giugni2007; Johnson, Agnone, and McCarthy Reference Johnson, Agnone and McCarthy2010); lobbying (Sorensen Reference Sorensen2003); and petitioning (Budros Reference Budros2011). Only one article says why its theory applies to just one kind of activity: Johnson, Agnone, and McCarthy (2010, 2282–83) claim that, according to political mediation theory, institutional influence activity will be more effective when a supportive party is in power, and protests will be more effective when the party in power is hostile.
The theory-testing articles provide no theoretical or empirical justification for there being separate theories for different types of organizations or activities. The likeliest reason is mundane: scholars often confine their concerns to their own subfields and pay little attention to developments elsewhere. Hojnacki et al. (Reference Hojnacki, Kimball, Baumgartner, Berry and Leech2012) title their review “Studying Organizational Advocacy and Influence” and write that it includes works that “give substantial attention to the activities of organized interests or social movements” (p. 380); yet their subtitle refers only to interest groups, and all the journals they survey are in political science. Amenta et al. (Reference Amenta, Caren, Chiarello and Su2010) care only about the consequences of social movements. The journal Interest Groups & Advocacy says it will address advocacy by groups, movements, and lobbying professionals, but has published just two articles on movements since its inception in 2012. Even though the articles reviewed here are all concerned about the impact of advocacy on policy, those studying SMOs cite only a handful of works on interest groups, and of the five articles on interest groups, only those by Ingram and Rao (Reference Ingram and Rao2004) and Lax and Phillips (Reference Lax and Phillips2009) cite works on social movements. “Interest groups” and “social movements” are recognized subfields, but there is no subfield for advocacy.
Testing Hypotheses
Now we come to the question of whether the theories’ predictions are consistent with the results. If they are, the theories may be proving themselves useful and worth testing further. If they are not, perhaps the theories should be rethought or even abandoned.
The results from the tests of the 108 hypotheses are presented in table 2. Just over half of the predictions, 56%, are correct. This seems disappointing.
Theories, Predictions, and Findings
Note: In italicized theories, more than two-thirds of the predictions are correct.
What about specific theories? With 108 hypotheses divided among 19 theories, the number of correct predictions for any particular theory seems to tell us very little. Yet they could tell us more if we knew how often the theories had been tested before and with what results. For theories not tested before, any findings would matter. For theories that have seldom been tested, the findings from a single article could matter quite a bit, adding substantially to what we know about the theories’ explanatory power. For theories that have been tested often, in contrast, a few more findings from an article or two would likely not add much to what we already know, unless there is something especially innovative about particular tests.
Thus, the value of the findings presented in table 2 depends on what is known from previous tests of particular theories. Unfortunately, authors rarely describe exactly what previous tests have shown. They therefore cannot tell us what their findings add to what we already know: they cannot tell us what is the value added of their work. The importance of proceeding iteratively, keeping track of what is already known, has been highlighted in Perspectives on Politics by Fairfield and Charman (Reference Fairfield and Charman2019), writing about qualitative research, and Lieberman (Reference Lieberman2016) in discussing biomedical research.
Most authors’ conclusions naturally focus on their own findings, but we might also expect them to discuss broader implications of their work. Gilens and Page (2014, 568), for example, consider what their findings imply for our understanding of democracy, even as they describe limitations on their ability to generalize.
What do the 24 theory-testing articles say about the implications of their findings? Three argue that their conclusions help us understand policy change in a particular issue domain: abolition (Budros 2011, 201, 450), social policy (Jensen Reference Jensen2012, 235–36), and the environment (Johnson, Agnone, and McCarthy Reference Johnson, Agnone and McCarthy2010, 2284). One proposes that it contributes to our understanding of how an important variable—policy-specific public opinion—affects policy (Lax and Phillips Reference Lax and Phillips2009, 382–83). And Gilens and Page, the most ambitious, sees their findings as contributing to the literature about how Congress responds to external forces (2014, 576–77). Nineteen articles only narrowly discuss their own findings; they say nothing about any broader implications.
Next Steps
A key way for researchers to influence a field is to suggest how research could move forward most productively (Merton Reference Merton1987). If proposals for future research are nothing more than generic—studying more countries or organizations, for instance—they will not be very helpful. But if researchers delineate the most critical concerns raised by their research and propose specific steps to address them, their advice is likely to be especially fruitful. What do the theory-testing articles propose for subsequent research?
Seventeen include proposals for future work. Most are reasonable but generic: future research should add variables (Giugni Reference Giugni2007, 70; Johnson, Agnone, and McCarthy Reference Johnson, Agnone and McCarthy2010, 2285; Soule and Olzak Reference Soule and Olzak.2004, 492–93), analyze more countries (Giugni Reference Giugni2007, 70; Soule and Olzak Reference Soule and Olzak.2004, 492–93), study additional issues (Lax and Phillips Reference Lax and Phillips2009, 383; Longhofer et al. Reference Longhofer, Schofer, Miric and Frank2016, 1761; Uba Reference Uba2005, 393), improve measurement (Soule and Olzak Reference Soule and Olzak.2004, 492–93), and analyze additional time periods (Bearce Reference Bearce2003, 407; Johnson, Agnone, and McCarthy Reference Johnson, Agnone and McCarthy2010, 2285).
But there are three areas in which suggestions for improvement seem especially promising. Two are empirical, proposing to expand the range of what those studying advocacy and policy should consider.
The first suggestion seems commonsensical: analyses of the determinants of policy change should take both supporters and opponents into account (Giugni Reference Giugni2007, 70; Soule and Olzak Reference Soule and Olzak.2004, 492–93; Steidley Reference Steidley2018, 120). To understand a competition for influence, surely all the competitors must be included. Nevertheless, just five articles—Budros (Reference Budros2011), Gilens and Page (Reference Gilens and Page2014), Ingram and Rao (Reference Ingram and Rao2004), Lax and Phillips (Reference Lax and Phillips2009), and Soule and Olzak (Reference Soule and Olzak.2004)—estimate the impact of advocacy on both sides of an issue. Doing so proves to be a good idea. When support and opposition are measured separately, policy is most often influenced by both. The only exception is Budros (Reference Budros2011), who finds opposition to policy change has no impact. (Gilens and Page [Reference Gilens and Page2014] combine advocacy on both sides into a single, net measure and so cannot gauge their impact separately.)
The second suggestion is to expand the scope of work on advocacy: research should consider not just the final stages of the policy process but also earlier stages, because particular types of advocacy may have more impact at one stage than at another (Agnone Reference Agnone2007, 1609; Giugni Reference Giugni2007, 71; King, Cornwall, and Dahlin Reference King, Cornwall and Dahlin2005, 1211; Johnson, Agnone, and McCarthy Reference Johnson, Agnone and McCarthy2010, 2284). There is enough evidence supporting this claim to support further work (Johnson, Agnone, and McCarthy Reference Johnson, Agnone and McCarthy2010; King, Cornwall, and Dahlin Reference King, Cornwall and Dahlin2005; see also Soule and King Reference Soule and King.2006).
Finally, there is one proposal for improving theory. Agnone (Reference Agnone2007) claims that democratic theory usually sees protesters and the public competing for influence (cf. Gilens and Page [Reference Gilens and Page2014] on interest groups and the public). His research, he argues, shows that, rather than competing, public opinion and protest interact, with protest amplifying the impact of opinion. Giugni (2007, 66) and Steidley (2018, 114) make the same claim, and Lax and Phillips (2009, 377) argue that the same holds for interest groups.
Agnone’s proposal has enough empirical support to warrant further testing. Lax and Phillips’s (Reference Lax and Phillips2009) and Steidley’s (Reference Steidley2018) data are consistent with the hypothesis (though Giugni’s [Reference Giugni2007] findings are not). Burstein (1998a, ch. 4) found protest and public opinion interacting in the leadup to enactment of the Civil Rights Act of 1964, and two recent studies in European countries found policy affected by an interaction between advocacy and public opinion (Kluver and Pickup Reference Klüver and Pickup2019; Rasmussen, Mader, and Reher Reference Rasmussen, Mader and Reher2018).
Six relatively recent articles in the 25 major journals include data on both public opinion and advocacy, but do not consider whether an interaction between the two affects policy; their data could be reanalyzed to see if it does. These include four of the theory-testing articles (Gilens and Page Reference Gilens and Page2014; Johnson, Agnone, and McCarthy Reference Johnson, Agnone and McCarthy2010; Martin Reference Martin2010; Soule and Olzak Reference Soule and Olzak.2004) and two others: Scheitle and Hahn (Reference Scheitle and Hahn2011) on gay rights and Lax and Phillips (Reference Lax and Phillips2012) on 39 policies in US states.
Conclusions
This research synthesis points to a number of strengths in research that tests theories on the impact of advocacy on policy. The substantial number of theories suggests a high level of creativity among researchers. Some theories propose interesting ways in which hypothetical determinants of policy interact with each other. Many theories make similar predictions and agree on the basic mechanism linking advocacy to policy, and there are some potentially fruitful proposals for future work.
There are problems in the theory-testing literature as well. However, often the identification of a problem can lead to fairly straightforward proposals for solutions.
There are arguably too many theories attempting to explain the connection between advocacy and policy, impeding the ability of researchers to create a coherent research community. The number of theories is not a function of the number of hypotheses that researchers test; most theories propose similar hypotheses. Nor is it a function of disagreements about what links advocacy to policy; most theories focus on the same (electoral) connection. What does seem to matter is the organization of subfields; there are different theories for different types of organizations and activities, not because the phenomena differ in ways warranting different theories, but because those studying each type do not pay adequate attention to those studying other types. Even more importantly, researchers rarely test theories against each other, neglecting an important tool for deciding which theories are worth pursuing and which are not. A potentially helpful step would be the consolidation of disparate lines of research into a new subfield—“advocacy studies”?—that would combine extant approaches and synthesize or eliminate redundant theories. In the short term, this would mean more work for everyone, because researchers would have to read a wider range of work. In the long term, it would lead to a more efficient collective research effort with wider implications and greater generalizability.
A second problem is that the extant theories do not predict the impact of advocacy on policy very well. Actually, this breaks down into two problems: the theories do not do a great job, and we do not readily acknowledge this point. Our failure to keep track of what we have already learned about theories’ power to explain outcomes means we are unable to plan research effectively. If a theory we propose to test has already been tested many times, with consistent results, there may be little value to an additional study. If the theory has rarely been tested, any new findings would add significantly to what we know. If a theory’s credibility is somehow hanging in the balance, a well-designed study could be especially valuable. If researchers began their articles with careful descriptions of previous findings and concluded with an updated account that included their own findings, we would know more not only about how valuable the theory is but also about whether it makes sense to devote resources to further testing.
A third problem is that researchers hesitate to generalize. They most often present their findings without considering how they fit in with the results of other researchers. Does their theory explain how advocacy affects policy in the setting they have studied but not in others? Are their results subject to specific scope conditions or a particular set of circumstances? Researchers most likely know about research comparable to their own, and they should use that knowledge to put their findings in context. And if they are concerned about overgeneralizing because of the limited scope of their sample, they should consider how others in the field have tried to address these concerns (Barabas Reference Barabas2016; Baumgartner et al. Reference Baumgartner, Berry, Hojnacki, Kimball and Leech2009; Burstein Reference Burstein2014; Gilens and Page Reference Gilens and Page2014).
Finally, many researchers stop just short of making what is potentially one of their most valuable contributions. They know better than anyone what they have learned, what problems they faced, and what might be done to solve them—but they fail to take advantage of the opportunity, in their conclusions, to both enlighten their colleagues how to move beyond these difficulties and, even more so, propose what might be the most fruitful directions for future work.
All 24 theory-testing articles had to meet high disciplinary standards before they could appear in these high-impact and specialized journals. They are all excellent in many ways. Collectively, though, they do not seem to have accomplished what they might have: there are too many theories and barriers to cooperation across subdisciplinary boundaries, many disappointing empirical results, and a reluctance to generalize and take stock as to where the field as a whole stands, as well as to think about the most fruitful directions for subsequent research. But the description of each problem contains within itself steps toward solutions. It would take some effort to begin pushing theory testing forward more rapidly by consolidating theories, putting tests of theory in the context of past research, devoting more effort to generalization, and more self-consciously suggesting improvements in future research. Given the quality of the work and of the researchers, and the time and effort they have invested, it would not be that much harder to find ways to build on previous work and accumulate a more coherent body of knowledge. It is not difficult to imagine how greater attention to the steps I outlined here could move the field forward in even more productive ways. It would help us move ahead in our effort to understand one of the most important elements of the democratic political process: how advocacy affects policy.
