It is a pleasure to respond to the thoughtful comments of Bryan Jones (Reference Jones2015) and James (Reference Adams2015). We first want to explain the motivation behind our review article. As we began a project on policy agendas in Australia, we set our intentions to audiences at conferences in Australia, Europe and the United States and ran a workshop with policy specialists in Australia (which resulted in a special issue of the Australian Journal of Public Administration 2013). What we discovered was that many of the political scientists we were talking to were extremely sceptical about the value of the Policy Agendas Project (PAP)/Comparative Agendas Project (CAP) approach. We think this scepticism arose because these critics do not understand what it is that PAP/CAP is trying to do and, perhaps more importantly, what it is not trying to do and indeed cannot do.
It was at this point that we developed the distinction between policy attention, policy content or substance and implementation style (Dowding et al. Reference Dowding, Hindmoor and Martin2013). In doing so, we disarmed many of the criticisms. Policy attention refers to what is being discussed by various groups of political actors. Policy substance refers to the effect of government policy. The ideological content of policy is one aspect of its substance. Finally, implementation style refers to the manner in which government implements policies. The critics we spoke to mistakenly believed that PAP/CAP was trying to measure attention and substance and criticised it for failing to achieve the latter. They are mistaken. PAP/CAP measures attention with reference to the kinds of issues politicians and other actors are focussed upon. It does not measure policy substance. The best way of defending PAP/CAP to others in the discipline is to be very clear about what it is doing. Our review was designed to explain what PAP/CAP is measuring, but in writing it we decided that many of the attempts to explain the patterns of attention (and, in a more complex fashion, budgets) were misplaced, and were misplaced because writers within the PAP/CAP tradition sometimes (but not always) interpreted findings about patterns of attention as though they were about content or substance.
It is true that sudden shifts in policy attention may well, as James Adams suggests, precede sudden shifts in policy substance and implementation style. Politicians and other actors may well start to focus intensely upon an issue when there is a perception that the status quo is failing and that a new policy approach is needed. However, there is no necessary connection here. At times, the government may spend a lot of time talking about an issue in order to create the impression that it cares and is doing something. On other occasions, governments may suddenly, and in the absence of protracted debate, pass a measure that has a huge impact upon many people’s lives. We do not yet know nearly enough about the relationship between policy attention and policy substance. Adams suggests addressing this problem using Comparative Manifestos Project data and PAP/CAP data to assess the relationship between changes in policy attention and changes in policy substance. This was precisely our strategy with regard to the governor-general speeches in Australia (Dowding et al. Reference Dowding2010, Reference Dowding2012). There, we found that Gough Whitlam’s government was a turning point with regard to policy attention. The social issues he brought to prominence in his executive speech never left the agenda, although his ideological take on those issues were soon swept aside both by the opposition and his own party when it returned to power.
We are grateful for Bryan Jones’ comments on the use of measurement systems and of course are in complete agreement. Our discussion of the theories associated with PAP/CAP was not meant to veer off from the measurement issues but rather is central to them. PAP/CAP describes the pattern of attention and budgetary expenditures over time. The theory enters in two ways. First, the data have been used to critique theories of the policy process, and, second, processes to explain those patterns are theorised. Our argument is that the patterns discovered in the PAP/CAP data are not inconsistent with those earlier theories. They are not inconsistent because, in the main, what is being measured – policy attention – is not the subject of those theories. Second, most of the theoretical ideas that have been used to explain patterns of data within PAP/CAP are, in our view, more suited to explaining changes in the substance of policy and not policy attention.
Bryan Jones thinks it unfortunate that we identify CAP/PAP with punctuated equilibrium (PE). Indeed, citing conversations with colleagues, he suggests that PE has “little or nothing to with CAP”. We do not accept this. In our review article, we are very careful to say that “more recent work [within PAP/CAP] has gone beyond concentrating on PE”. Yet, the claimed existence of PE is a key finding within the PAP/CAP literature. The CAP website lists 71 papers or books using PAP/CAP data (http://www.comparativeagendas.info/?page_id=16), with 16 of these having the word punctuation or punctuated in their title. Bryan Jones recently published a chapter in an edited volume on Theories of the Policy Process called “Punctuated Equilibrium Theory: Explaining Stability and Change in Public Policymaking” (Baumgartner et al. Reference Baumgartner, Jones and Mortensen2014), which cites over 300 articles. Textbooks by Peter John (Reference John1998) and Paul Cairney (Reference Cairney2012) identify PAP with PE, and two key articles with 14 authors from the PAP/CAP teams focus squarely upon PE (Baumgartner et al. Reference Baumgartner2009; Jones et al. Reference Jones2009). There is no necessary and logical connection between PAP/CAP and PE; however, the fact that the policy agenda is punctuated is, nevertheless, claimed to be a robust “empirical generalisation” (Jones et al. Reference Jones2009, 855) and one that a great deal of attention has been devoted to explaining.
We are pleased, however, that Bryan Jones is happy to leave PE behind, as nothing in PAP/CAP has ever established any equilibrium conditions for policy attention or for budgets. Logically, therefore, no punctuation of equilibrium has been empirically established. What PAP/CAP has empirically established is that attention dips and spikes, and some of these spikes mark important departures from previous attention. Although budgets do follow historical trajectories, changes in budgets are not always incremental but subject to large variations. Describing these as dips, spikes and punctuations from previous patterns of attention is more accurate than labelling it PE.
Under the rubric of PE (at least in Baumgartner et al. Reference Baumgartner, Jones and Mortensen2014) come various theoretical explanations of the dips and spikes and large variations in attention and budgetary appropriations. We do not believe that these sets of hypotheses are all on a par. Some are irrelevant. Bounded rationality no more explains shifting attention than optimising under time constraints. Clearly, crises will often lead to big shifts in attention insofar, as political actors often compete to attract media attention and public support by proclaiming some or the other issue to be a crisis. Slip-shift, in its geological application, is about the build-up of stress that can lead suddenly to large-scale movement. Applied to policy change, it is a useful metaphor. Given how common power-law distributions seem to be across a range of not only physical but also human activity, it may well be that we do not need a specifically public policy explanation of these kinds of patterns.
Our more general point is that some of the theoretical explanations offered to account for the patterns mapped by the measuring tools employed by PAP/CAP are irrelevant to the patterns themselves insofar as they relate to shifts in policy substance rather than policy attention. Changes in frames or policy images can help us explain changes in the substance of policy. Indeed, political actors may well seek to create new frames in order to secure changes in policy substance (Baumgartner and Jones Reference Baumgartner and Jones1993). Such shifts need not bear any relationship to the dips and spikes found in the policy agenda data. Implementation style can also mask continuities or discontinuities in attention within a country if, as in our example, drought relief is shifted from one departmental portfolio to another or included in wide-ranging bills.
Coding by “minor” as well as “major” issue topic mitigates but does not resolve this problem. If policy is framed in different ways, over time coders might, quite legitimately following the coding instructions, code them separately. This, as Jones rightly points out, is a problem for all such measurement devices. What it shows, however, is that this quantitative device cannot be used to track framing. Framing might help explain why the relative attention across codes occurs as issues shift across policy domain, but it cannot explain the general patterns (the claims of PE).
We began our review article by saying that PAP/CAP has “generated a rich set of easily accessible data” that can be used to “measure the evolution of the policy agenda across time and between countries”. We end our review by saying that PAP/CAP has made an “invaluable contribution” to the policy sciences. The concerns we raise about measurement issues and the relationship between PAP/CAP and PE should be seen in this light. By identifying weaknesses and highlighting misunderstandings, we hope to be able to better highlight the undoubted strengths of the PAP/CAP approach.
