Agendas and activity: attention, substance and styleFootnote ¹

Most people think of the policy of government with respect to what it actually entails in terms of state activity and how it affects the public. Different policies have different distributional effects on goods and services; they affect markets, safety or human welfare in different ways. Furthermore, the structure of government affects both how pressures are applied to state actors and how policies are implemented. When policies are considered failures, it is because the impacts they have on people are not the ones intended. Therefore, a policy on crime fails when it fails to reduce the illegal activity it targeted, or perhaps succeeds in that aim but creates other unintended costs. We usually think about how important legislation is in terms of such impacts. We call them the substance of policy.

PAP/CAP generally measures policy attention – what is being discussed in various forums – rather than what the government is actually doing. Most writers refer to Kingdon (Reference Kingdon2003, 3) in defining the policy agenda as the set of issues to which political actors are, at any given time, paying serious attention. Although attention might be correlated with importance in the sense that people talk about issues that do or will have a significant impact on citizens, this is not necessarily the case. Although PAP/CAP codes in terms of “content” by issue domain, what it actually measures is the amount of attention given to a code, not the substance of the discussion or legislation nor how that substance impacts or would impact the public if implemented. Measuring how large the impact of given policies is might be difficult, but this is what people who discuss policy change are discussing, and it is not what PAP/CAP is measuring.

Political actors can pay attention to issues in several ways. The most prominent of these is by talking about particular issues. In this sense, the agenda is what government and other political actors are talking about and not necessarily what they are doing. Attention here includes legislation that will lead to activity; however, even so, the issues the government spends most of its time talking about at any given time might be very different from the issues that are routinely being acted on by state actors. In other words, the relative amount of time the government (verbally) devotes to a given issue does not necessarily correlate with the relative amount of time the state spends acting on that issue, nor with how effective that action is. Attention and impact are very different things. Indeed, on occasion, and as May et al. (Reference May, Workman and Jones2008) demonstrate with regard to terrorism threats in the US, increased attention may result in the proliferation of agencies undermining output efficiency. Certainly, the substance of policy will often flow from policy attention but the relationship is far from a perfect one.

There are, then, different aspects of the policy agenda. Attention can be measured in terms of the number of words devoted to particular issues within speeches, the number of questions asked in parliamentary sessions, the number of committee inquiries or the number of relevant pieces of legislation. Within PAP/CAP, attention is the relative amount of time devoted to issue areas by topic code. Attention here takes no account of how much, for example, legislation will impinge upon people’s lives. A lot of attention might be devoted to some industrial regulation that might affect those working in that field, but which most members of the public would not notice. A single line of legislation, however, might restrict the freedom of every citizen. Attention does not measure the distributional effects of legislation or its ideological content (see discussion below). Two governments might devote the same amount of attention to an issue but with very different consequences. Generally speaking, PAP/CAP does not consider these aspects of the agenda. Therefore, we distinguish the subject of the policy agenda (i.e. policy attention) from what we call the substance of the agenda (whom it affects, in what way and by how much), and ideological content will be an aspect of that substance in the manner in which it impacts people.

The third aspect of the agenda after attention and substance concerns how the government goes about implementing policies. Essentially, the same policy might be implemented through completely different instruments run by different government departments or agencies. For example, help for the aged might be run though a welfare department, a health department or a veterans’ agency. We think of this particular issue in terms of implementation style (which is similar but not identical to “policy style”; Richardson et al. Reference Richardson, Gustafsson and Jordan1982). The instructions given to PAP/CAP coders encourage a focus upon the main provisions of a bill, but many clauses within a bill will have specific implications for different portfolios and ministries. The complex effects of particular pieces of legislation can be tracked within case studies. For example, Talbert et al. (Reference Talbert, Jones and Baumgartner1995) track jurisdictional changes across Congressional committees covering drug abuse, nuclear power, pesticides and smoking, and show how non-jurisdictional hearings are especially important for issue re-definition (see also Baumgartner et al. Reference Baumgartner, Jones and MacLeod2000). Nevertheless, implementation style complicates CAP’s efforts to compare the dynamics of issue attention across countries using a general coding scheme.

Measurement issues

Measurement is the centrepiece of PAP/CAP. The policy agenda is coded in terms of 19 major and 250 minor policy areas (see PAP/CAP websites for precise codes). These codes relate to particular and functional areas of policy, such as macroeconomics, health and agriculture (major codes) and agricultural trade, government subsidies to agriculture, unemployment and higher education (minor codes). This coding system was initially developed in the US context. With only few revisions, however, it has proven possible to use the same coding categories to measure the policy agendas in other countries. Furthermore, where changes have been made to the coding system, these are at the subtopic level and reflect differences in the salience or character of particular issues. The UK PAP, for example, created new minor codes for immigration (within the major topic code of minority issues and civil liberties), animal welfare and fisheries and fishing (within agriculture), domestic and international terrorism (within defence), relations with the US and Canada (under international affairs and foreign aid), the monarchy and British nobility and prime ministerial scandals and resignations (under government operations), as well as UK-specific descriptions of other minor policy codes (see http://www.policyagendas.org.uk/).

The details of coding vary depending upon the source of information. In the case of executive speeches, quasi-sentences – which constitute an expression of a single policy idea or issue – are coded. With legislation, the unit of analysis is the bill placed before Parliament, or in some analyses its subclauses. In the case of media coverage, stories in leading newspapers like the New York Times have been coded. What remains constant in all of this is the use of the standardised PAP coding scheme: the fact that the material coded, whatever the unit of analysis employed, is assigned a single major and a single minor policy code, and the fact that coding is undertaken manually by trained research assistants subject to standard reliability checks.

The precise codes and the fact that they are coded by functional categories can always be criticised. However, in coding, some decisions have to be made. Moreover, although some of the codes and subcodes betray historical US-centric concerns, keeping essentially the same coding frame enables substantive comparison across time and country. If topic categories keep changing, researchers risk confusing shifts in the substance of policy attention with changes in coding protocol (Baumgartner et al. Reference Baumgartner, Jones and Wilkerson2002). Thus, one of the great contributions of the coding scheme is ensuring temporal constancy and avoiding the use of ephemeral language, which are the problems that afflicted previous policy studies.

The goal of PAP is to develop “systematic indicators of issue attention within nations’ political systems” (http://www.comparativeagendas.info/). Judged in such terms, CAP deploys a plausible system of classification corresponding to how governments generally organise policy (into named portfolios) and how academics and journalists write and think about policy (as experts on health policy, defence policy and so on). In that sense, the classification adopted is by the “surface phenomena” of public policy. That is what we should expect from initial attempts at classification.

Using a consistent and transparent coding scheme, PAP measures the attention being devoted to different issues in different settings. This data set can be used to explore the process of agenda setting. It can be used to tell us whether the priorities the government identifies for itself in set-piece speeches at the start of its term in office are subsequently taken up in the legislation it seeks to pass. It can be used to tell us whether opposition parties or legislative committees can prime the significance of particular issues for governments. For example, the data can show whether the government devotes more attention to issues that have previously been the subject of critical questions or reports. They may illustrate something about the relationship between public opinion and government attention. Do governments respond to shifts in public opinion about the importance of particular issues? Or, does this relationship run in the opposite direction – does the public come to care more about those issues in which the government or opposition parties have invested attention? PAP can also tell us something about the relationship between legislative inputs and budgetary outputs. When governments start to devote more attention to particular issues – whether in the form of speeches or legislation – do they also tend to start spending more money in those areas?

Furthermore, by looking at the absolute rather than the relative amount of attention governments devote to certain issues, the PAP/CAP data can also be used to address important questions about the size of the policy agenda. Is legislative reach greater than it was in the past? Do governments now try to govern more things? Have executive speeches grown longer? Have the number of pieces of legislation submitted to parliaments increased? What is the relationship between the length of time a party has been in government and the absolute size of its agenda or between the size of its majority and the scale of its policy agenda? Do newly elected governments address more policy issues than older ones? And what is the relationship between the size of a government’s agenda and its electoral success? Do governments that try to do too much during their first term in office dilute their political capital and achieve very little?

We cannot, of course, expect to find any simple answers to these questions. Relationships are likely to vary over time and across countries and between different policy areas. This does not matter. One of the big issues in the policy sciences (and political science more generally) is how and why some issues get on the government agenda, whereas others fall by the wayside (classic contributions include Schattschneider Reference Schattschneider1960; Cobb and Elder Reference Cobb and Elder1972; Downs Reference Downs1972; McCombs and Shaw Reference McCombs and Shaw1972; Kingdon Reference Kingdon2003). PAP/CAP provides data that can be used to address these questions.

The PAP/CAP data can show whether the government, opposition parties or the media are devoting more attention to one issue relative to others. However, they cannot tell us whether, in the course of doing so, they are also changing the policies themselves. Moreover, they cannot confirm whether policy has shifted ideologically to the right or left or is more authoritarian or liberal. Baumgartner et al. (Reference Baumgartner, Jones and Wilkerson2011) list occasions where policy ideas initially failed but were subsequently and successfully re-introduced; however, these examples relate to the substance of policy, not to attention as such. The specific coding in PAP/CAP will not pick out these ideological shifts. Governments will often, but not necessarily always, devote more attention to an issue precisely because they want to change the status quo. Conversely, stability in substance might mask considerable discontinuity in policy direction. The attention aspect of PAP/CAP is, thus, orthogonal to some models of the policy process that consider the substantive impact of policy. Policy heritage accounts suggest that governments spend most of their time implementing the policies of previous governments: a claim about the substance of policy output rather than patterns of attention (Rose Reference Rose1990; Rose and Davies Reference Rose and Davies1994). Similarly, incrementalist accounts of the policy process, which we discuss in more detail at present, are primarily about policy substance and not attention. This is not to say that CAP researchers are unable to say anything about the relationship between policy attention and policy substance; however, they have primarily done so through detailed case studies of the dynamics of agenda change (Mortensen Reference Mortensen2005; Albaek et al. Reference Albaek, Green-Pedersen and Nielson2007; Green-Pedersen and Krogstrup Reference Green-Pedersen and Krogstrup2008; Walgrave and Varone Reference Walgrave and Varone2008; Chaqués and Palau Reference Chaqués and Palau2009; Daviter Reference Daviter2009; Schiffino and Varone Reference Schiffino and Varone2009).

The validity of the PAP coding scheme must also be judged in terms of what is being measured. Politicians in France and the US may be talking about very different things when they discuss healthcare reform or industrial policy, in the sense that their ambitions and normative ideals about what constitutes good health or industrial policy are very different (Capano Reference Capano2009). However, this is a question of differences in substance and not in attention. Whatever the differences in the substance of what they are saying, we can, nevertheless, meaningfully classify what these politicians are talking about in – for instance, industrial policy to the extent that they are talking about manufacturing strategy, technological capacity of industry, assistance to specific industries, national industrial policy, industry re-vitalisation and growth, decline in industrial productivity and plant closings and relocation (as per the definition of the minor codes for industrial policy).

However, implementation style might make cross-national comparison of what is being attended to problematic. Different countries might have very different implementation styles. Two governments might see poverty relief as an important part of their agenda, for example. One might address poverty by means-tested direct welfare benefits. The other might tackle it through a full-employment policy, trying to ensure that everyone has access to work. Although guides to coders try to ensure compatibility across countries, substantially different policies with the same impact might be coded differently (Dowding et al. Reference Dowding, Hindmoor and Martin2013). In another example, government support for the agricultural sector in Australia (particularly but not exclusively drought relief) has at least partly been directed at maintaining rural income stability (Cockfield and Botterill Reference Cockfield and Botterill2013). This can be plausibly coded under agriculture or social welfare, given that poverty relief is a key purpose of the policy. Otherwise, with its emphasis upon drought relief, it could be coded under Public Lands and Water Management, or, given its focus upon regional development within the policy, under Community Development. Coders use filters to try to answer these questions, but clearly the manner in which essentially the same policies are adopted across policy domains can lead to discrepancies across countries because of the manner in which fundamentally the same substantive content is framed in legislation; however, this is the nature of the comparative enterprise and not a problem specific to CAP. Implementation style, the way in which policy is delivered, will have implications for the way in which policy attention is classified. However, explaining implementation style is best done through qualitative accounts of specific policy areas within the policy agenda rather than by attempting to develop a coding scheme for this purpose.

Issues relating to the boundaries between coding categories become important in considering framing effects. Baumgartner and Jones (Reference Baumgartner and Jones1993) argue that policy agendas tend to remain frozen so long as they remain trapped within tight-knit policy networks. Sudden policy change occurs when policy entrepreneurs reframe an issue and persuade policymakers in another arena to devote their attention to it. Yet, the way an issue is coded may depend upon the way it is framed. The coding system cannot track framing if the codes themselves are subject to framing effects. For this reason, framing is perhaps best examined through case studies in which an issue is identified and changes in frames are then tracked and perhaps linked to either changes in attention or changes in substance. This approach was employed by Baumgartner and Jones (Reference Baumgartner and Jones1993) in some of their earliest work on policy agendas, and has also been used by Albaek et al. (Reference Albaek, Green-Pedersen and Nielson2007) and Schiffino and Varone (Reference Schiffino and Varone2009). However, these case studies cannot easily be connected to more general findings about the patterns of shifting attention. Unsupervised machine-learning techniques might enable more systematic analysis of shifting attention (see below).

PAP/CAP informs us of the level of attention devoted to different issues at different times by different actors within these settings. It tells us what governments, opposition parties, the media and other actors are talking about and focussing upon and what legislation is being passed and amended. The policy agenda is measured in various ways, some of the most prominent examples of which include executive speeches (Jones and Baumgartner Reference Jones and Baumgartner2005; Breemen et al. Reference Breeman, Lowery, Poppleaars, Resodihardjo, Lantink and Lettinga2009; Jennings and John Reference Jennings and John2009; Dowding et al. Reference Dowding, Hindmoor, Iles and John2010; John and Jennings Reference John and Jennings2010; Jennings et al. Reference Jennings, Bevan, Timmermans, Breeman, Brouard, Chaqués-Bonafont, Green-Pedersen, John, Mortensen and Palau2011; Mortensen et al. Reference Mortensen, Green-Pedersen, Breeman, Jacques, Jennings, John, Palau and Timmermans2011), legislation (Wilkerson et al. Reference Wilkerson, Feely, Jens, Nicole and Sue2002; Adler and Wilkerson Reference Adler and Wilkerson2008, Reference Adler and Wilkerson2012; John et al. Reference John, Bertelli, Jennings and Bevan2013, Chapter 4), committee hearings (Adler Reference Adler2002; Wilkerson et al. Reference Wilkerson, Feely, Jens, Nicole and Sue2002), parliamentary questions (Vliegenthart and Walgrave Reference Vliegenthart and Walgrave2011), public opinion (Jones and Baumgartner Reference Jones and Baumgartner2004; John et al. Reference John, Bertelli, Jennings and Bevan2013, Chapter 7; Bevan and Jennings Reference Bevan and Jennings2014) and media coverage (Walgrave and Aelst Reference Walgrave and Aelst2006; Walgrave Reference Walgrave2008; Green-Pedersen and Stubager Reference Green-Pedersen and Stubager2010; Boydstun Reference Boydstun2013; John et al. Reference John, Bertelli, Jennings and Bevan2013, Chapter 8). Change in the policy agenda is measured not by the absolute amount, but by the proportion of attention devoted to particular issues relative to other issues. Why? The key theoretical assumption here is that, given time constraints, the proportion of attention can stand as a proxy for the importance the government attaches to the issue at that time. Therefore, increases in attention at that time measure increases in importance of the issue.

Is this a reasonable assumption? After all, the government might consider several issues of equal importance but have no alternative but to deal with them one at a time. In parliamentary systems, a department may be able to secure priority for its legislation because of the adeptness of its minister or senior civil servants rather than because of any overall sense of the urgency of an issue. At times, a significant policy issue of considerable importance to the government may be the subject of only a few well-chosen words, whereas on other occasions governments may deliberately downplay an issue in order to minimise political conflict (Dowding et al. Reference Dowding, Hindmoor, Iles and John2010, 538–539) or devote a great deal of attention to marginal issues if they are symbolically popular. Nevertheless, lacking the government equivalent of polling companies asking the public to identify the “most important problem”, proportional attention can be a reasonable proxy for importance over time.

Significant attention has been devoted to measuring the “productivity” of Congress in terms of the number of laws it enacts (e.g. Clinton and Lapinski Reference Clinton and Lapinski2006; see Adler and Lipinski Reference Adler and Lapinski2006). Again, such studies cannot measure the impact of legislation. One statute might transform healthcare; five might simply slightly amend educational policy. PAP/CAP tries to overcome this problem by examining the complexity of legislation (Jones and Baumgartner Reference Jones and Baumgartner2013) or the press coverage of different bills (John and Bevan Reference John and Bevan2012). Other researchers have compared changes in attention to the importance historians have devoted to issues (Adler and Wilkerson Reference Adler and Wilkerson2012; John and Bevan Reference John and Bevan2012). There are other measures of “important”, “significant” or “major” legislation in the US (Mayhew Reference Mayhew1991; Clinton and Lapinski Reference Clinton and Lapinski2006) based on attention in Congress and weighted contemporary and retrospective subjective ratings, but there is no straightforward relationship between subjective rating and legislative attention (Adler and Wilkerson Reference Adler and Wilkerson2012). Media attention offers another way of measuring importance (see John et al. Reference John, Bertelli, Jennings and Bevan2013). The media may, however, largely ignore issues that are important but not newsworthy either because they are complex or are not politically divisive. Conversely, issues of only marginal significance to a government may receive disproportionate amounts of coverage if they touch upon the particular interests of an editorial team or reveal political divisions within a governing party. There are likely to be overall relationships between media attention and the importance of a piece of legislation but no simple correlations.

The costs (in terms of both time and money) of manual coding PAP/CAP material and advances in computer technology have stimulated interest in automated coding techniques. Two issues dominate here: first, the use of automated coding processes to replicate the existing manual coding scheme with equal or greater reliability, and, second, the potential of unsupervised automated coding and machine-learning techniques to take PAP/CAP in new directions. We consider each of these possibilities in turn.

The first process utilises supervised learning systems (Mitchell Reference Mitchell1997; Russell and Norvig Reference Russell and Norvig2002) that seek to reproduce human coding. Such systems learn how to code by following the lead of human coders. Various supervised learning algorithms are in the public domain. Hillard et al. (Reference Hillard, Purpura and Wilkerson2007) describe using several supervised learning algorithms to replicate the human coding of 379,000 Congressional bills assigned to the PAP coding scheme; they report that, although some algorithms work better than others, combining them proves greater accuracy than any single method. Collingwood and Wilkerson (Reference Collingwood and Wilkerson2012) laid out how to ensure that new cases are machine-classified accurately, also using the Congressional Bills Project and the PAP coding. Using off-the-shelf algorithms with training protocols, it is possible to achieve accuracy levels almost as high as those of manual coding. The best Collingwood and Wilkerson achieved was 76% accuracy over 100% coverage, which they increased to 86% accuracy where three out of four algorithms agree on coding (although this provided only 85% coverage). In sum, content-based, supervised learning systems are a useful addition but work best when a relatively large manual component is also employed.

A second approach is to use unsupervised techniques. Here, algorithms are let loose on the words and the machine itself picks out patterns. In this manner, the data are not coded by a preordained set of categories; rather, categories are found through association. Such techniques require splitting a text into the learning sample and the holdout that is then used to test the patterns found. These processes must be used with care, but by using hierarchically ordered systems that utilise sophisticated thesauruses to match concepts rather than just words, there is the possibility that frames around issues can be systematically tracked over time. Alternatively, text software such as Leximancer or Discursis can identify the major concepts and terms employed within texts by their associations with clusters of other concepts, not just in terms of their simple word-count frequency (Angus et al. Reference Angus, Smith and Wiles2012). In other words, new techniques open up opportunities to examine the changing discourse surrounding policy attention, leading to hypotheses about the nature of change within the sphere of policy substance. It is true that these techniques may be subject to time-bound categories that centre on ephemeral language rather than on more enduring policy matters, which makes it harder to track attention changes over time. However, the use of a sophisticated thesaurus reduces this problem, and categories can be tracked through association even as the language changes. Importantly, however, combinations of techniques can allow a more systematic account of the framing of issues by the terms used in speech or legislation.

Punctuations and turning points

A strong motivation for Baumgartner and Jones in establishing the PAP data collection exercise was to test incremental accounts of policy change that purport to show that “democracies change their policies almost entirely through incremental adjustment” and that “policy does not move in leaps and bounds” (Lindblom Reference Lindblom1959, 80). The key finding from PAP is that policy attention has a pattern of what is now routinely called punctuated equilibrium (True et al. Reference True, Jones and Baumgartner2007; Jones and Baumgartner Reference Jones and Baumgartner2012). Punctuations are proportionate increases or decreases in attention. The punctuatedness of the policy agenda varies cross-nationally, but the general pattern has been found to hold across countries. Changes in budget allocations across departments tend to be more punctuated than legislative change, which itself is more punctuated than committee enquiries and media stories relating to particular issues (Baumgartner et al. Reference Baumgartner, Green-Pedersen, Jones, Mortensen, Neytenmans and Walgrave2009, 612). Countries with multiple institutional veto points also tend to have more punctuated agenda change (Baumgartner et al. Reference Baumgartner, Green-Pedersen, Jones, Mortensen, Neytenmans and Walgrave2009, 968). Although more recent work has gone beyond concentrating on punctuated equilibrium, especially with regard to the CAP (see Comparative Political Studies 2011 special issue), the idea that the policy agenda is punctuated is, nevertheless, a robust “empirical generalisation” (Jones et al. Reference Jones, Baumgartner, Breunig, Wlezien, Soroka, Foucault, François, Green-Pedersen, Koski, John, Mortensen, Varone and Walgravee2009, 855).

The fact that the policy agenda is punctuated is an important result that is not obvious, although the existence of punctuations or paradigm shifts is a recurring theme within scientific and social scientific research (Kuhn Reference Kuhn1962; Krasner Reference Krasner1984; Hall Reference Hall1989; Carmines and Stimson Reference Carmines and Stimson1990; Thurow Reference Thurow1996). It is important, however, to be clear about what punctuation is. The punctuations that are found in PAP/CAP mark proportional increases or decreases in attention to particular issues, not changes in the absolute amount of attention to any issue. However, absolute attention to issues can also be measured by PAP/CAP data. They show that the distribution of changes in the proportion of attention devoted to issues takes the form not of a normal Gaussian (bell curve) distribution but a leptokurtic distribution, characterised by the presence of a large number of very small changes that are accompanied by a small number of very large changes (Jones et al. Reference Jones, Sulkin and Larsen2003; Jones and Baumgartner Reference Jones and Baumgartner2005, 127–132; Baumgartner et al. Reference Baumgartner, Green-Pedersen, Jones, Mortensen, Neytenmans and Walgrave2009). These “fat tail” changes are punctuations. The details of punctuation measurement might seem arcane, but they matter in terms of the significance of the findings. As the yardstick of measurement is increases in the rate of change, PAP data often identify as punctuations instances of agenda change that do not seem that significant. For example, an increase in the attention devoted to space, science, communications and technology from 1% in t ₁ to 3% in t ₃ counts as just as much of a punctuation as an increase in the attention devoted to health from 10% to 30% over the same period. Moreover, it is also partly for this reason that John et al. (Reference John, Bertelli, Jennings and Bevan2013) do not find evidence of punctuations in macroeconomics, an issue that routinely consumes a large amount of attention every year. Given Baumgartner and Jones’s objective of testing whether the distribution of changes takes a Gaussian or leptokurtic distribution, this arguably does not matter; however, it is important to remember that, so defined, punctuations need not constitute significant policy events.

Our more general argument about PAP measuring changes in attention rather than substance also applies to measuring punctuations. Punctuations in attention can arise without significant changes in the substantive content of policy and vice versa. Punctuations in legislative attention can, for example, arise from the passage of pieces of legislation that do not change existing policy but tidy up and render consistent previous legislation (John and Bevan Reference John and Bevan2012). Alternatively, “procedural” punctuations may occur as a result of modest increases in the attention invested in very different issues that, nevertheless, happen to be grouped together under the same major or minor policy code (John and Bevan Reference John and Bevan2012). Conversely, significant changes in policy substance need not correspond with measured changes in attention. Governments may pass one piece of legislation in a year to which they devote most of their political capital and attention. However, if in the same year the government passes a series of bills amending existing legislation, the passage of the significant piece of legislation may not show up as a significant variation in the distribution of attention.

How punctuations are defined in terms of increases or decreases in attention bears upon the critique of incrementalism. This is for the simple reason that incrementalism consists of a particular set of claims about the way in which the substance of policy (and not attention) changes. It is true that PAP/CAP work on budgeting directly addresses the issue of policy substance. Yet, even here, extreme caution needs to be exercised in concluding that the existence of punctuations invalidates incrementalist arguments. The existence of sudden punctuations in budget allocations has previously been identified and framed as being consistent with the basic parameters of the incrementalist model (Schulmann Reference Schulmann1975; Gist Reference Gist1982; Berry Reference Berry1990; on the general relationship between incrementalism and punctuated equilibrium, see Howlett and Migone Reference Howlett and Migone2011). To this extent, PAP might be seen as offering a re-statement of existing incrementalist theory rather than an altogether different story. Furthermore, it should be noted that welfare budgets can sometimes rise suddenly in a recession in the absence of any change in either policy attention or substance.

The term “punctuated equilibrium” is also misleading as PAP does not demonstrate any equilibrium conditions – only periods where relative change in attention remains constant, punctuated by ones where change occurs more rapidly. Constancy does not equate with equilibrium. To be sure, constant attention could occur as a result of a balance of forces with different policy entrepreneurs all trying to get their items onto the agenda – that is, the view of government in the old group theory of Bentley (Reference Bentley1908) and Truman (Reference Truman1951). However, constancy does not have to be the result of equilibrium. Punctuated equilibrium also recalls the evolutionary account of Gould and Eldredge (Eldredge and Gould Reference Eldredge and Gould1972; Gould Reference Gould2002). Indeed, some see a direct analogy here (Kingdon Reference Kingdon2003, 225). However, Gould’s argument is very different. For Gould, punctuated equilibrium is about speciation: there are periods of rapid evolution where species form, but outside of each period evolution halts and there is no change in species. The vision of PAP, rather, is that there is incremental policy change, marked by some government change in attention, but then periods of rapid change. Analogously, that is closer to the more standard view of evolution expressed by Dawkins in his debate with Gould (Dawkins Reference Dawkins1986; Sterelny Reference Sterelny2007). Of course, what is in equilibrium in Gould’s account is species (with equilibrium provided by natural selection within an unchanging environment); the analogy here to the policy process is policy substance (for a critique, see also Prindle Reference Prindle2012). However, what is constant in the measures of punctuation in PAP is proportional change in attention, which is at best a proxy for change in substance.

What is more interesting when it comes to changes in the proportion of government attention is what happens after the change. To illustrate this point, imagine two issues where the relative level of attention over five time periods follows that shown below. In the first case, we have relative stability followed by punctuation and then a stable return to the level of pre-punctuation attention. In the second case, we have relative stability followed by punctuation, but with attention then sustained at a higher level.

In the first case, we have two punctuations in the form of changes in the rate of change of attention (at t ₃ and t ₄). In the second, and perhaps more empirically interesting case, the punctuation also marks a turning point in the level of attention and not just punctuation in the rate of change of attention (Dowding et al. Reference Dowding, Hindmoor, Iles and John2010). How often do punctuations in the rate of change of attention preface increased levels of attention over a longer period of time? We do not know the answer to this question because, for the purpose of measurement within the PAP, punctuations and turning points are treated synonymously (but see John et al. Reference John, Bertelli, Jennings and Bevan2013). This is not to suggest that we should downplay the inherent value of finding punctuations, but rather underlines the need for follow-up work (often qualitative) to examine the long-run effect of punctuations.

How patterns of punctuated equilibrium are explained is also problematic in PAP. Baumgartner and Jones (Reference Baumgartner and Jones1993) initially accounted for the existence of punctuations in terms of the stultifying effects of policy networks and the liberating efforts of political entrepreneurs in reframing issues. Jones and Baumgartner (Reference Jones and Baumgartner2005) (see also Jones and Baumgartner Reference Jones and Baumgartner2012) have gone on to offer a very different and more general argument. Boundedly rational policymakers must economise on their attention. This means that signals warning of an impending policy failure are ignored until it is too late and a crisis erupts, at which point policymakers’ attention is suddenly drawn to an issue and radical policy change is enacted. Policy does not respond simply or directly, and certainly not “automatically”, to changes in public demands, needs or signals about the failings of an existing policy. It is “sticky” (Jones et al. Reference Jones, Sulkin and Larsen2003, 152). Government tends to underreact to emerging policy problems, and then overreact to avoidable policy crises once they occur, in an often ill-considered effort to be seen to be doing something (Jones and Baumgartner Reference Jones and Baumgartner2005, 87, Reference Jones and Baumgartner2012, 7).

The argument that politicians ignore signals until it is too late and a crisis erupts and forces a sudden shift in attention provides a theoretically plausible account of policy change; however, it overlooks important factors. It is not hard to think of instances in which warning signals are ignored, a crisis occurs and a sudden surge of attention to a particular issue follows. Operating with the obvious benefit of hindsight, the National Commission on Terrorist Attacks Upon the United States (2004) and the Financial Crisis Inquiry Commission (2011) each pointed to warning signals that, if heeded, might well have prevented future catastrophes, but were either not communicated effectively or not accorded sufficient priority. There is, however, a hidden evidence problem here. Politicians are bombarded with signals warning them of imminent crises, and we see those cases where those signals are ignored and a crisis occurs, because newspaper commentators, opposition politicians and boards of enquiry can go back and point to those signals and document what happened to them (Hindmoor and McConnell Reference Hindmoor and McConnell2013). Yet, governments do sometimes respond to warning signals and, in doing so, prevent crises from occurring. However, we cannot empirically weigh instances in which governments ignore warning signals and a crisis occurs against those where warning signals are heeded and a crisis is averted, because we cannot know when failure to heed a warning signal would have led to a crisis.

Bounded rationality or constrained maximisation might explain the existence of punctuations but not, as we have described them, turning points. Disproportionate information processing leads to crises and crises lead to punctuations in attention; however, when politicians devote large parts of their attention to dealing with the aftermath of a crisis, they are likely to be paying insufficient attention to other signals. The result is, eventually, another crisis in another area, resulting in a further shift in the distribution of attention. In other words, frequently changing attention means that turning points rarely occur, because attention is constantly shifting from one issue to another.

However, one could imagine cases where an issue becomes important enough for attention to remain fixed (albeit to a lesser extent) on it over an extended or indefinite period of time, hence marking a turning point. Baumgartner and Jones (Reference Baumgartner and Jones2002) provide just such an explanation of turning points in an argument about the existence of positive feedback effects within the policy process. These occur when an increase in policy attention to an issue results in new legislation that serves to keep an issue on the agenda in the longer term. For example, environmental legislation might involve setting up a new regulatory agency that then leads to greater and more regular oversight from the legislature. The agency itself will become an interest that might press the government for more legislation or greater discretionary powers and attempt to influence other interconnected policy areas. This might lead other interests to organise to reduce the power of the agency. The interaction of agenda attention leading to new institutions then feeds into keeping attention deployed upon those new issues, forming the positive feedback loop. Here, we have a punctuation followed by incremental increases in attention. McDonagh (Reference McDonagh1958, 1961) provides an account of the formation of the British administrative state largely along these lines. Walgrave and Vliegenthart (Reference Walgrave and Vliegenthart2010) call short-term increases in attention through panic or imitation “cascades”.

Jones and Baumgartner (Reference Jones and Baumgartner2012, 9) also evoke a seismological “stick-slip” model to explain the existence of punctuations. The basic argument here is a simple one: geological pressures build up until there is an earthquake, perhaps foretold with some minor seismic activity sometimes followed by a major shock. The distribution of geological movements takes a leptokurtic form. The earthquake, however, is the mechanism through which a new stability is established until, once again, pressures accumulate and another earthquake occurs. By analogy, the forces creating pressures upon an existing state of affairs might include changing social attitudes, the shifting social or ethnic makeup of a nation, the building up of international tensions or the slow aggregation of pollution, traffic problems or bad debt. These pressures may accumulate until, for whatever reason, there is a sudden shift in attention followed by a return to the original level. In other words, we do not need to invoke bounded rationality, crises, policy entrepreneurs, framing or even the details of decision-making processes to explain why policy attention takes the shape PAP has identified. Rather, these processes might need to be invoked to explain why the punctuations in attention occur when they do and to explain the changing substance or implementation style of policy.

What is the relationship between punctuations in attention and changes in substance? We previously suggested that there may be a strong association between them insofar as governments are more likely to devote attention to those policy issues where they also want to change policy substance; however, we can now say something more about the dynamics of this relationship. A slip-stick perspective suggests that punctuations in attention will be temporary and reversed to the extent that they lead to the establishment of a new policy that balances underlying social forces, but long-lasting insofar as no such policy is developed. Consider an issue like pensions. Over time, the population in most European countries has aged considerably. Combined with changes in employment conditions, this created additional pressures upon existing pension systems, manifested in concerns about the sustainability of existing policies. At some point, politicians in different countries began to invest attention in this issue – the result often being the introduction of a new system for funding pension entitlements. New policies will be successful to the extent that they effectively address underlying pressures. When they are successful, attention will fade away. In such a case, a change in the substance of policy (a new policy is developed but then remains in place for a protracted period of time) will be associated with punctuation in attention (there is a sudden surge of interest in an issue resulting in the development of a new policy, after which attention fades away). Here, dramatic change in policy substance may be required to ensure that attention fades. Conversely, where for whatever reason it is not possible to develop new policy substance or the new policy substance does not deal with and reflect the underlying problems, attention is likely to remain focussed upon an issue over a period of time. Cultural change and positive feedback are also more likely to lead to turning points rather than punctuations. The distinction between punctuations and turning points also helps here by assisting us in identifying the long-run effect of policy change.