There has been much discussion in recent years of ‘evidence-based policy’ (Black, Reference Black2001; Boaz and Pawson, Reference Boaz and Pawson2005; Wyatt, Reference Wyatt2002). Tony Blair became Prime Minister in 1997 promising to replace ‘outdated ideology’ with a concern for ‘what works’ (Labour Party, 1997). But disillusion soon set in, summed up in the phrase ‘policy-based evidence’ (Glees, Reference Glees2005; Stimson, Reference Stimson2001; Tombs and Whyte, Reference Tombs and Whyte2003). Despite repeated criticisms of the use made of evidence in making policy (see Barton, Reference Barton2003; Berridge and Thom, Reference Berridge and Thom1996; Gendreau et al., Reference Gendreau, Goggin, Cullen, Paparozzi and McGuire2002; Naughton, Reference Naughton2005; Stevens, Reference Stevens2007a; Tonry, Reference Tonry2004; Young, Reference Young, Matthews and Young2003), there is still a ‘perplexing silence . . . about the social world of the policy-making process’ (Tombs, Reference Tombs2003: 5), although there have been occasional interruptions (for example, Page and Jenkins, Reference Page and Jenkins2005; Smith, Reference Smith2007). Hill's (Reference Hill2009) impressive summary of work in this area also notes the paucity of direct studies of the policy process, which tends to be examined through analysis of its outputs or of accounts given by their producers, rather than through close observation.
Hill also notes the difficulty of pinning down a precise definition of policy-making. He warns that, while this is widely assumed to be a rational process of fitting means to ends, this assumption should not be made in advance of empirical examination of the policy process. Similar difficulties arise over the definition of evidence. What counts as evidence is itself a politically loaded discussion (Monaghan, Reference Monaghan2008; Nutley et al., Reference Nutley, Walter and Davies2007). In this article, policy-making will be taken to refer to the organised attempt to select goals and methods for governmental action, whether rational or not. What counts as evidence will be taken as one of the empirical topics to be addressed (see the section on the ‘oversaturation of evidence’ below). This article contributes to the development of a coherent explanation of the use of such items of evidence in policy-making by combining previous theoretical contributions with ethnographic data from a study of the practice of policy-making in the UK government. The main research question that it seeks to inform is: how do policy-makers use evidence?
Method and ethics
In 2009, I worked as a policy adviser for six months on a placement in a policy-making section of the UK civil service. The team I worked in had responsibility for advising the highest levels of government about policy on a range of social policy and criminal justice areas. I worked at the middle-ranking level, just below the ‘senior civil service’ grades. This is the level where Page and Jenkins (Reference Page and Jenkins2005) found that much policy-making is actually done in the UK civil service. I carried out desk research, responded to requests for information, developed policy proposals, represented the section at inter-departmental meetings, seminars and conferences, and generally shared the working life of the team. While carrying out this work, I observed the work of my colleagues and their interactions with other civil servants and with special advisers to ministers. I made fieldnotes as soon as I could after these observations took place. At the end of the process I also interviewed five of the civil servants with whom I had worked. The aim of these interviews was to ask for their reflections on the use of evidence in policy-making, which I could then compare with the observations that I had recorded.
As I became convinced that I would want to publish an analysis of my experience, I asked permission from colleagues within my team to undertake interviews and write about my work. I decided not to ask for informed consent from all the people I observed. The British Sociological Association's (2002) code of ethics states that covert research can be justified when overt research would change the behaviour of participants and where access is controlled by powerful interests. It would have been impractical to seek formal consent from everyone I worked with. But, more importantly, it would have increased reactivity. I wanted to observe and participate in the work of policy-making with the least possible disturbance to the usual process. And the people I was observing could hardly be described as vulnerable. They are powerful people, with high levels of financial and cultural capital, engaged in work that, while hidden from public view, was carried out on behalf of the public and paid for by the taxpayer. The people quoted in this article were among those who gave verbal informed consent to take part in this research. I have attempted to protect their anonymity by using false names for them in this article. I do not give details of the units they were working in or the particular policies they were working on where that could lead easily to their identification. The study was approved by the University of Kent's internal ethics committee.
Methodologically, ethnography can provide a picture of only a limited range of reality. I cannot claim that the data I obtained are representative of the entire body of British civil servants, let alone of other actors (inside and outside government) who play a part in making policy. The areas I worked on are highly politicised (Monaghan, Reference Monaghan2010; Young, Reference Young, Matthews and Young2003). It may be that evidence is used differently in other policy areas, with lower levels of political salience (Tonry, Reference Tonry2004) and where there is less public conviction that certain, evidently ineffective, interventions should be used (Tilley and Laycock, Reference Tilley and Laycock2000). I hope, however, that this study reveals at least some of the mechanisms that characterise the policy-making process. By participating in the use of evidence for policy purposes, and in discussing and observing this process with people who do it every day, I was able to gather data that we can use to develop and test theory in this area.
Theoretically, this is hardly uncharted territory. It would have been wasteful to ignore all the previous attempts to explain the evidence-policy link by limiting myself to the development of theory from the data alone (Glaser and Strauss, Reference Glaser, Strauss and 1967) or by replicating the ‘barefoot empiricism’ of Page and Jenkins (Reference Page and Jenkins2005: vii). So I decided to follow Layder's (Reference Layder1998) approach of ‘adaptive coding’. This involved generating a list of provisional codes from my previous work in this area (Stevens, Reference Stevens2007a) and from the existing literature. This included: the sophisticated pluralist approaches of Weiss (Reference Weiss1977, Reference Weiss1999), Dunn (Reference Dunn, Fischer and Forester1993), Gans (Reference Gans and Horowitz1971), Sabatier and Jenkins-Smith (Reference Sabatier and Jenkins-Smith1993), Nutley and her co-authors (Reference Nutley, Walter and Bland2002) and John (Reference John1998); the ‘post-foundationalist’ approaches of Hajer (Reference Hajer1995) and Valentine (Reference Valentine2009); and various contributions from critical theory, including the work of Thompson (Reference Thompson1990), Bourdieu (Reference Bourdieu2000) and Habermas (Reference Habermas1984, Reference Habermas and Cooke2002).
As I coded the fieldnotes and interview transcripts, using QSR NVivo software, I highlighted observations and quotes that related to these provisional codes, as well as generating new codes as other concepts emerged from the data. With initial coding complete, I then went through the codes and their content to reconsider, reorganise and reformulate the analysis to which the coding contributed. This process resulted in the identification of eight key themes, which are discussed below.
Commitment to the use of evidence
My observations and discussions with civil servants suggested that they are not very different from academics, at least to the extent that they have to display competence in the use of scientific reason in order to win reputations which lead to personal and professional status and reward (Bourdieu, Reference Bourdieu2000). The civil servants I worked with were highly committed to the use of evidence. One of them told me, ‘it's the job of officials to tell truth to power’. Evidence was ever-present in the development, discussion and presentation of the policies whose construction I observed. So the first casualty of my emerging analysis was the idea that policy is ‘evidence-free’ (The Economist, 2009). As for the policy-makers that Weiss (Reference Weiss1977) interviewed, civil servants thought it proper to use evidence. One of my interviewees told me, ‘evidence is a prerequisite for policy’. Another said, ‘evidence should be the basis for options we put to ministers . . . evidence-based policy is part of the way that we work’. So we need to examine how this part of the work operates. However, as Weiss also found, they often saw evidence being used to justify existing policy rather than to develop wholly rational interventions. All my interviewees were able to report instances where evidence had not been used. So we also need to explain this gap between the normative commitment to the use of evidence and these policy outcomes.
The oversaturation of evidence in policy
The UK government, through its managerialist attempts to control the performance of public sector actors, operates a massive exercise in the collection and storage of data. The many reports that result from analysis of these datasets have to jostle for attention with a host of other evidence. In coding my fieldnotes, I counted 15 types of evidence that were entered into policy debates. In addition to internally collected government data and externally produced academic analysis, the list included opinion polls, reports by thinktanks (such as ippr (Institute for Public Policy Research), Policy Exchange, Centre for Social Justice and so on) and from management consultancies, previous policy papers produced by various parts of the civil service, independent inquiries, reports of the inspectorates of police and prisons, internal and externally commissioned evaluations of policy initiatives, various kinds of reports from abroad, press reports, television programmes (examples from HBO's The Wire were mentioned several times), personal experience and opinion. Information from parliamentary debates and committees was conspicuously absent from this list. Evidence was usually sought through Google searches.Footnote 1
A colleague told me, when I asked him how he looks for evidence when working on a new policy, about:
a depressingly similar pattern where you look for the best – usually quantitative data – you can find, and then, as you work through the policy problem you establish that there is not the best evidence that you want and you work your way down until, at the end, you're left with the odd case study, something which was kind of half evaluated, some anecdotal information and then what you can garner through a few field visits.
Thousands of articles and books have been written on the policy areas on which this civil servant works. The ignoring of these academic outputs can be partly explained by the ‘unsuitabilities’ of social science research for use in evidence that Gans (Reference Gans and Horowitz1971) highlighted. The questions that policy-makers tend to ask include: what should be done in practice; how will it work; what will the effects be; how much will it cost; will there be any adverse consequences, and on whom will they fall? Very little of the evidence that is available to policy-makers – including only a very small minority of academic papers – provides conclusive answers to such questions (Tilley and Laycock, Reference Tilley and Laycock2000). In using evidence, policy-makers therefore have to pick and choose from the available evidence. Evidence becomes a solute in the oversaturated solution of policy problems. There is more evidence than can be absorbed, so some is taken in for use and some is not. The problem with this naturalistic metaphor is that it cannot illuminate the role of human agency. From my observations, the key to understanding this process is to pay attention to the way people use evidence in forming human relationships, and so to the process of telling policy stories.
Intra-government relations
As one of my civil service colleagues told me, policy-making civil servants know they have done their job when their proposals are accepted as government policy. In order to do this job, they have to persuade other people, within and outside government, that their proposals are worth acting on. Other writers have concentrated on the relations between government and external actors (Hajer, Reference Hajer1995; Sabatier and Jenkins-Smith, Reference Sabatier and Jenkins-Smith1993). During my fieldwork, the most important influences on policy acceptance that I observed were the relations between people within government. The state is not the neutral arbiter between opposing policy proposals that is presented in Dunn's (Reference Dunn, Fischer and Forester1993) ‘jurisprudential’ model of policy arguments. It is the source of many of these ideas, and the daily life of policy-makers is spent in discussion and argument with other actors within the state.
To achieve acceptance, policy proposals have to be agreed by a wide range of senior civil servants, special advisers and, eventually, ministers in the various departments that have an interest. I observed determined efforts to ensure that these agreements would favour the proposals that my team was working on. The focus of these efforts was often on the ‘narrative’ of a particular proposal. Indeed, narrative was a constant theme in the creation of new policies, echoing Hajer's (Reference Hajer1995) insistence on the importance of ‘story-telling’ in influencing which proposals will be taken up. It was interesting that this emphasis on narrative had relevance outside the field of environmental policy, within which Hajer developed it.
In working up policy documents, colleagues would examine – often in very great detail – the internal coherence of drafts and the way in which they led the reader to the conclusion that the suggested policy was the only alternative that made sense. One example that I worked on was the document which presented a new policy initiative. In order to demonstrate that urgent action was necessary, I was asked to draft a short section that would lay out the scale of the problem (in money terms) and the recent trends in its development. I found it difficult to be precise, given the very wide uncertainties that are acknowledged in the academic literature on the topic, but I produced a summary of the available evidence. This became the focus for weeks of refinement, inter-departmental argument and revision. The document was passed between many colleagues in my team and others which had an interest in the policy area. The caveats that I had inserted on the uncertainties in the literature were quickly removed. Whenever the document came back to me, with more questions in the accompanying email, I would try and reinsert some of them.
When the document that resulted from this repetitive process was sent to the analytical section of another department, a fierce email exchange broke out. The other department's analyst was very concerned at some of the estimates that we had used, as they might conflict with what his unit had published in the field. At this point, Phillip, my colleague who was leading work on the document, sent him a placatory email. He asserted that the figures – which he had denuded of accompanying caveats – were reliable, as they had been produced by an academic (that is, me). This attempt to justify the figures on the basis of academic authority rather than their evidential status failed when I insisted that I would not have included the figures without caveats in any academic publication. Phillip told me, ‘we have to strike a balance. We don't want to overclaim, but we need to sell the policy’. In the end, the document included a headline estimate which had no supporting evidence, but had previously been published in a government report. Some of the caveats were included, but placed in an annex at the back of the published document.
This episode exemplifies how evidence is used as a tool for persuasion (to ‘sell the policy’) within government, as well as in relations between government, the public and other agencies. An even more striking example was reported to me of a high-level meeting where an extremely senior civil servant had used the promising results of an evaluation of a particular scheme. The evaluation itself was not particularly rigorous. It had no control group, but did show – in common with many other social interventions – that people who are subject to intervention at the peak of their problems tend afterwards to improve. This lack of methodological rigour did not prevent the use of the evaluation by the top civil servant who led the meeting in persuading a number of departments to make savings from their existing budgets in order to expand the favoured scheme.
The control of uncertainty
One of the apparent purposes of creating coherent policy narratives was to reduce the role of uncertainty as a barrier to action. As predicted by Sanderson (Reference Sanderson2004), uncertainty was seen by many of my colleagues as the enemy of policy-making. If we are unsure what the real problem is, and we cannot predict the effects of our actions, then we would not, they implied, get anything done. My discussion of caveats in the work with Phillip was characterised by our team leader, Mike, as ‘verging on the philosophical’. He evidently saw them as an obstacle to the practical issue of what action to take, right here, right now. Uncertainty, which is – of course – a fundamental feature of knowledge (Bhaskar, Reference Bhaskar1978) threatens to disrupt the narrative of a policy. It was rarely entertained in the policy documents which I worked on and saw being ‘put up’ to special advisers and ministers. Indeed, the very form of these documents denied uncertainty. Many policy documents transmitted between policy-making civil servants were in the form of PowerPoint ‘packs’, stuffed with bullet points, diagrams, short text boxes and simple graphs. None of these permitted lengthy discussion of the uncertain or imprecise nature of the knowledge they presented.
The use of graphs was particularly interesting. In an induction session early in my fieldwork, they were referred to as ‘killer charts’. We were taught to construct these instruments of persuasion by choosing data carefully and by restricting the number of cases and categories that were shown. The policy implications of the data should be immediately apparent from the graph alone. This last recommendation obviously ignores the common academic criticism that statistics, even where they represent an underlying reality, are socially and selectively constructed (Prior, Reference Prior2003), and cannot (or should not) simply ‘speak for themselves’. A graph is a visual representation of the results of that construction, which renders invisible the process of construction. In my subsequent work, many hours were spent searching for data that could be used to make such graphs, and many discussions were had on how to construct them. I knew I had succeeded in making a ‘killer chart’ the day that a senior civil servant responded unprompted to a graph that I presented. He used it to argue for more urgent action than his colleagues were currently pursuing. The hours that I spent manipulating the data had not been in vain.
The control of uncertainty strengthens the narrative of a policy document. But it also plays a wider role in structuring the context in which civil servants operate. Mike's comment on caveats reflected the reality that if we were to worry incessantly and individually about the profound limits to our knowledge of the world we inhabit, then collective action could not emerge. Social order, according to Douglas (Reference Douglas1987), rests on our abilities to sacrifice some of our individual freedom of thought in order that peace and predictability can prevail. This enables collective action and also satisfies individual needs. Collective styles of thinking come to shape beliefs in order ‘to satisfy the individual demand for order and coherence’ (ibid: 19). People find in following the strictures of institutional thinking that they also maximise the utility they gain from transactions with others. The civil service ‘thought world’ (ibid) shuns uncertainty. And there are understandable, utility maximising reasons for this.
Bureaucratic competence and the civil service career
Utility has at least two meanings. One general sense is usefulness, as in something that is of use in carrying out a task. Another more narrowly economic sense is the use that something has for the pursuit of personal goals. For civil servants, the maximisation of both types of utility can coincide. If they make themselves useful to the task of creating and carrying out policy, then they are more likely to achieve their own goals of professional advancement. This became clear in discussing the civil service career with my colleagues and from observing civil servants performing their tasks. The pressures and incentives that they experience in developing their careers are important in understanding how they came to use evidence for policy stories.
In pubs and bars around Whitehall after working hours, civil servants gather to relax, drink, tell jokes and speculate on the progress of each other's careers. As an uninitiated outsider, colleagues would take the trouble to explain to me the rules that were taken for granted when they discussed who was going for what job. These can be crudely summarised as: do not specialise; do be useful; and do find superior supporters. At the end of the project on which I worked with Phillip, we went out for a celebratory drink. Another colleague explained to me that the last two projects he had worked on had been on the same topic, so it was time to move on. On a web chat page hosted by a government department for its staff, I saw a discussion of how long was too long to spend in a job. ‘Never less than two years, never more than three’ was a typical contribution. Some civil servants I spoke to shared sentiments that were reported to Page and Jenkins (Reference Page and Jenkins2005). They felt discouraged from developing specialist knowledge. As they moved quickly between policy areas, they were less likely to develop a thorough knowledge of the evidence base in any particular area. They were incentivised to become what Gendreau et al. (Reference Gendreau, Goggin, Cullen, Paparozzi and McGuire2002) pejoratively describe as ‘fart catchers’: generalists who do not have enough expertise to resist whatever the minister decides will be the latest version of common sense, although it should be said that many civil servants do develop a deeper knowledge of their particular policy area.
Page and Jenkins (Reference Page and Jenkins2005) suggest that the avoidance of technical specialisation is one way to avoid creating a conflict between bureaucratic hierarchy and expertise. Such conflict could introduce even more complication and delay into the policy process, which is already ‘elongated and ramshackle’ (ibid: 81). The specialist knowledge that civil servants are incentivised to develop is not of the outside world. It is of the complex inner workings of Whitehall and of how to solve problems within it. In order to get ahead, civil servants need to get recognised for ‘adding value’ to the policy process. They do this by creating connections and solving policy problems. They need to combine familiarity with the detail of current policy with dynamism, certainty and a degree of personal charisma. Of course, as in other professions (academia is no doubt one of them), there are many civil servants who lack these qualities. It is possible to advance through civil service ranks at a slower pace. But rapid advancement, of the type aspired to by my young colleagues, was dependent on building a reputation for usefulness.
One female civil servantFootnote 2 spoke of her frustration that, despite the formal commitment of the civil service to equality of opportunity, promotion often still depended on personal connections with superiors in the hierarchy. She said, ‘it's supposed to be open and equal but it's not. People go for people they know. If you've worked with someone senior, you try and stay in touch with them. They can help you.’ This need to develop and maintain connections with more senior people requires that superiors come to think of potential protégés as both useful and reliable. The way to build trust is to provide evidence of trustworthiness. Another civil servant told me about an incident early in his career. He had been working on a policy area that had initially been based on North American evidence and been a high profile part of the government's ‘agenda’. But evidence that it was not working was emerging. He reported that his boss had:
kind of jokingly said, ‘Well you're young. Why don't you suggest we look again at [policy area] and see how far that takes you in your career?’ So there are certain areas where officials will self-censor and they won't suggest to ministers to change policy on certain areas even though the evidence suggests it.Footnote 3
The third face of bureaucratic reason: getting the story to fit
The combination of the need for effective persuasion, the control of uncertainty and the career incentives which operate within the civil service points towards an explanation of how civil servants use evidence in making policy. Policy-making civil servants have to make a selection from the huge amount of information that they have at their disposal. They tend to see uncertainty as an obstacle to the development and agreement of policy proposals. And if civil servants are to contribute to the policy process and to their own career development, they must produce proposals that are useful to their superiors and can be accepted into government policy. These proposals must therefore fit with the existing narrative of government policy. It must be used to tell a story that is simultaneously about the policy area and about the story teller.
The discursive categories and solutions that the story draws on will often have been set already by the general thrust of government policy, within the thought world that structures the approach that policy-makers take. One such general story line was the belief in the power of the purchaser/provider split to increase value for money in public services. This belief had been applied in the creation of the National Offender Management System (NOMS), but this was followed by years of delay, confusion and waste (see, for example, National Audit Office, 2009). I encountered several civil servants who had bad things to say about NOMS, but nobody who questioned the underlying wisdom of the purchaser/provider split that it is supposed to achieve. This perhaps reflects the predominance of thinking in the style of the ‘new public management’ that has been noted by other analysts of British social policy (Greener, Reference Greener2009; Taylor-Gooby, Reference Taylor-Gooby2009). When I looked for the evidence that had been used to justify the initial and continued pursuit of this policy, I could find very little. A minister had replied to a parliamentary question that the rationale was to be found in the original Carter review (Goggins, Reference Goggins2005). When I read the report he referred to (Carter, Reference Carter2003), I could find only a vague reference to the supposed benefits of quasi-markets, with no evidence provided to demonstrate where benefits had or could be achieved. When I raised this problem, and some of the academic analysis which questions the worth of quasi-markets in the public sector with Mike, he responded that ‘it just feels intuitively right that introducing competition would focus more on cost and quality’.
This intuitive faith in the application of private sector logic to public sector problems may seem to be what Sabatier and Jenkins-Smith (Reference Sabatier and Jenkins-Smith1993) would call a ‘policy core belief’. It cropped up in many other meetings, where civil servants would present policy dilemmas as a failure in the structure of individual incentives, or ask how to encourage the private sector to provide solutions to public sector problems. However, in line with Hajer's (Reference Hajer1995) critique of Sabatier and Jenkins-Smith, these ideas did not arise solely from the operation of individual belief in bringing like-minded people together in ‘advocacy coalitions’. Rather, the use of these expressions suggested to me the transfer and performance of discursive tropes among people who learn how to show both that they belong in this thought world and that they are worthy of promotion within it.
There is an ideological process of ‘subjection-qualification’ at work here (Therborn, Reference Therborn1980: 17). The currently dominant modes of thought constrain the limits of what civil servants think is possible. By reproducing these limits, policy-makers can qualify themselves to take up and perform their role. In telling policy stories, they can signal that they hold appropriate, acceptable beliefs. Through the things that they were told, and the careers they saw acted out, the civil servants I worked with learnt that the stories they tell should not include uncertainty, complexity or opposition to the narrative that already dominates a policy field. For example, after I gave my views on the failures of current policies in a team meeting (views which none of my colleagues had disagreed with), Phillip said ‘I'd love to see you say that to a minister. You'd blow their mind!’ This jocular remark and the laughter which ensued reveal an implicit understanding which goes against my interviewee's comment on telling ‘truth to power’. The joke implies that to tell people in power that their current narrative is fundamentally mistaken is (a) never done and (b) impossible to imagine doing, except in the absurd scenario of the joke. A different colleague said on another occasion, ‘I think if you always use the evidence [when it conflicts with current policy] then you're always going to be the awkward person that's saying, “the Emperor has no clothes”.’ This would not be the way to be recognised for being useful, or to build connections with superiors who can support career progression.
So for policy-making civil servants, we can add a third face of the use of reason to the two that Bourdieu perceived among academics (2000: 109). The first face is the use of reason and empirical data to create knowledge. The second is the use of this performance of rational aptitude in order to show that one is worthy of respect and status. The civil servants I encountered did both these things, but there was also a third face of bureaucratic reason. It was the performance of only certain sections of the range of rationally justifiable positions in order to win the type of respect and personal connections that are necessary to achieve higher status. Acceptable positions to perform were those which reinforced rather than challenged the fundamental assumptions and tropes of current policy narratives.
The silent silencing of inequality
The analysis so far suggests that there is a surfeit of – mostly inconclusive – evidence and a distaste for uncertainty, complexity and contradiction within policy-making circles. It suggests that civil servants learn to avoid such problematic features when they construct policy stories. This does not mean that they deliberately avoid, neglect or misuse evidence, but they are influenced in their use of evidence by the constraints of a particular thought world, whose limits they reproduce in their turn. Civil servants in the corridors of power, just like the people targeted by the policies they develop, take part in the ‘structuration’ (Giddens, Reference Giddens1979) of their social world.
So let us test this emerging theoretical approach against a particular instance of the use of evidence on inequality in the policy areas I worked on – or rather its non-use. Power also operates by excluding certain ideas and possibilities from those that are considered in taking political action (Bachrach and Baratz, Reference Bachrach and Baratz1962; Lukes, Reference Lukes1974). Professor Richard Wilkinson was invited to give a presentation at a seminar held in the very grand rooms of the Admiralty. Surrounded by portraits of long-dead naval grandees, and an audience of senior policy-makers from the Home Office, Cabinet Office, Prime Minister's private office, the Department of Health and others, he summarised the findings of The Spirit Level (Wilkinson and Pickett, Reference Wilkinson and Pickett2008). This book lends itself well to PowerPoint presentation. It is, for instance, brimfull of ‘killer charts’. These scatterplots have clear, diagonal lines of best fit. They show that greater levels of inequality are associated with a very wide range of social problems, including mortality, morbidity, mental illness, obesity, poor educational performance, teenage pregnancy, imprisonment, drug use and murder. A previous study has suggested that Wilkinson's work is known in some governmental circles, but is discussed in ways that minimise the attention he pays to the causal impact of inequality itself. Instead, emphasis is put on the psycho-social processes that he implicates in the production of health inequalities (Smith, Reference Smith2007). At this seminar, Professor Wilkinson ran through his charts, explaining as he went that there are very good reasons to think that these associations are not just correlative; inequality causes these ills through various mechanisms. He competently dismissed alternative explanations that were put to him by some members of his audience. The very senior civil servants who took the roles of chair and discussant of the seminar both said that they found the presentation ‘compelling’ and ‘convincing’.
The next morning, fascinated by the reaction to this presentation, I made my only attempt to start off one of the email discussions that occasionally broke out among colleagues. I sent an email, attaching a paper that Wilkinson uses to back his assertions (Rothstein and Uslaner, Reference Rothstein and Uslaner2005), and asking colleagues what policy proposals would make an appropriate response to this evidence. The result was a tumbleweed-blowing silence. Just as this electronic silence began, the civil servant who worked opposite me sat down at his desk. He asked if the previous evening's seminar had been any good. When I told him that Wilkinson had argued that inequality causes virtually all social problems, he answered ‘didn't we already know that?’ I replied, ‘if we know it already, why aren't we doing anything about it?’ The response was, again, silence.
Since the creation of the Social Exclusion Unit at least, there has been an explicit governmental focus on the reduction of problems that are related to inequality. However, this Unit and other government initiatives have been criticised for failing to counteract the political and economic sources of relative poverty (Chanan, Reference Chanan and Wilson1999; Palmer et al., Reference Palmer, MacInnes and Kenway2007). The concept of social exclusion has been used, it has been argued, as a way of shifting attention away from inequality itself and particularly from the social harm caused by increases in income at the top end of the distribution (Béland, Reference Béland2007). These issues were not linked, by the civil servants I worked with, to the problems that our team was dealing with. Silence on inequality prevailed.
Here we see the downfall of the enlightenment model of the use of evidence in policy (Weiss, Reference Weiss1977). This model, simply stated, suggests that evidence affects policy indirectly by influencing the climate of opinion in which policy decisions are made. The problem is that, at this level of simplicity, it assumes that all evidence has a chance of influencing policy (Stevens, Reference Stevens2007a). It misses a mechanism to explain why some evidence content is consistently ignored. In this example, we can see that evidence of the harmful effects of inequality – which is so comprehensively presented in Wilkinson's work and a host of other sources – has informed people who play a significant role in making policy. They did not dispute that inequality is directly harmful. Indeed, they claimed this was something they already knew. But this knowledge consistently failed to make a significant impact on policy-making in the fields that my colleagues worked on.
This is not because the importance of inequality in these policy areas is a recent discovery that has not yet had time to percolate into policy (see, for example, Bonger, Reference Bonger1969 [first published 1905]). The causal and detrimental impact of inequality is a long-standing current of thought that has been consistently excluded from the ‘policy streams’ that have influenced policy agendas (Kingdon, Reference Kingdon1995) in these areas. Even though relative poverty was an explicit focus of Labour's policy in the areas I worked on while it was in opposition (Straw and Michael, Reference Straw and Michael1996), its importance as a focus of these policies had been minimised years before I started my work in government (Stevens, Reference Stevens2007b). When I tried to insert it in a meeting where my team discussed what the government could do to reduce the prevalence of a social problem we had been asked to look at, I was told that ‘the Gini coefficient is not a policy lever that we can pull’. So instead we looked for programmes that could ‘keep a lid’ on the effects of inequality, rather than doing anything to address its causes.
Totemic toughness
While inequality had, in the phrase of Mathiesen (Reference Mathiesen2004), been ‘silently silenced’, the search continued for policies to shout about that were tough. On my very first day of fieldwork, I took part in a meeting which discussed what else the government could do – beyond final warnings, referral orders, anti-social behaviour contracts and orders, penalty notices for disorder, juvenile curfews, parenting orders, nurse–family partnerships, family intervention projects and the various other forms of ‘naughty step’ to which the government has tried to send this country's unruly children and their parents (Gelsthorpe and Burney, Reference Gelsthorpe and Burney2007) – to reduce bad behaviour by young people. ‘We need,’ the meeting was told, ‘to come up with tough, totemic policies.’
This need to make a totem of toughness was consistently referred to throughout my fieldwork. Pressure was applied from the most senior levels to create policies that would signal the government's willingness to be nasty to bad people.Footnote 4 At one meeting, the prime source of these pressures emphasised the need to keep coming up with ‘totemic’ policies on crime and disorder, because of the high political salience of these issues. A totem, according to anthropologists following Lévi-Strauss (Reference Lévi-Strauss and Needham1964), is a cultural symbol that is used to provide a metaphorical representation of collective identity. Totems are central to the stories that cultures tell themselves about who they are. The transfer of this metaphor from the study of remote tribes to the everyday language of civil servants is somewhat remarkable. Its application to toughness is even more so. The phrase ‘totemic’ was often used alongside the word tough, and never mentioned in the context of reducing poverty or inequality. The use of this term reveals a central, symbolic thrust of government policies. It tells us about the stories that the civil servants I worked with have been encouraged to tell. These stories were most likely to find favour when they told us that, as a nation, we are a robust and unforgiving people, divided between a group of law-abiding innocents who are worthy of protection and support, and a group of threatening outsiders who are worthy of little but material exclusion and symbolic expurgation. At that early meeting on young people, one of my colleagues said, ‘we know who we're talking about. It's not the public schoolkids waiting at the bus stop, it's those other kids’. Totemic toughness applies only to the visible, poor and excluded social groups who have been so consistently ‘othered’ in contemporary crime policy (Young, Reference Young2007).
Conclusion
Telling stories about the policy process runs the risk of replicating some of the mechanisms that I have described. I have tried to shape a coherent narrative out of the messy business of policy-making. I have, however, tried to show my own methods and uncertainties so that readers can judge whether my narrative fits the reality of this process, or just the tropes and assumptions of academic discourse on policy-making. I sought examples which contradict my suggestion that evidence is used selectively in creating convincing, acceptable policy stories. There are indeed many policy discussions and documents that make explicit use of evidence, but few use available evidence that challenges the contemporary distribution of power. I found some instances of the use of evidence that challenged currently dominant tropes of government policy (for example, the ideas that the imprisonment of children and a high number of warranted police officers are important policy tools in creating safer communities) in policy discussions and documents. These instances did not, during my fieldwork, lead to adoption of policies that were based on the challenging evidence. They were selected out of the policy process during the iterative discussions between policy-makers.
The data I have presented suggest that there are inherent difficulties in providing and using evidence on policy issues. These include the vast amount of available information, as well as the indeterminacy and the lengthy timescales of academic research and other evidence. Such difficulties are often seen as causes of the failure to base policy on evidence (Nutley and Davies, Reference Nutley and Davies2000; Nutley et al., Reference Nutley, Walter and Bland2002; Valentine, Reference Valentine2009). But the problem lies deeper than this. This article suggests that governmental communication on the policy areas on which I worked is ‘systematically distorted’ (Habermas, Reference Habermas and Cooke2002).Footnote 5 It does not have to involve deliberate falsehood or direct manipulation of the evidence in order to count as ideological, in the sense that it supports systematically asymmetrical relations of power (Thompson, Reference Thompson1990). The exercise of power and the desire of policy-makers for the maintenance and enhancement of the prevailing order and of their own status within it can short-circuit the thorough use of the available evidence in what I have described (following Bourdieu) as the operation of the third face of selective, bureaucratic reason. Interpersonal interactions within the state contributed to this distortion. Wider analysis of policy and its effects would be necessary to examine all its effects. But one effect we can see from this analysis is that the use of evidence tended to support a certain way of thinking about the world; a thought style which is produced through the unequal distribution of power and then plays a part in its reproduction.
As Hill (Reference Hill2009) has noted, different theoretical perspectives may have more explanatory potential in analysing some policy areas than others. For example, he argues that analysis of recent British government policy on child poverty takes us ‘directly into the critique of pluralism’ (ibid: 112) which he summarises in the statement, ‘[p]ower is distributed unequally both inside and outside government, having an impact as much as what is on the agenda and the context in which decisions are taken as on the decision process itself’ (ibid: 107). This position is apposite to my experience of working within government, as it is also to Smith's (Reference Smith2007) study of the roles of ideas and research evidence in making policy on health inequalities. The pluralist approach tends to pay less attention to the structuring of power in and around the state. Nutley et al. (Reference Nutley, Walter and Davies2007) note the politically tactical uses of research and present useful recommendations for researchers and policy-makers who hope to increase the use of evidence. But they do not go as far as the analysis presented here in examining the strategic, ideological use of evidence which reduced the potential of such efforts in the policy areas I worked on.
The stories I have presented do not tell us all we need to know about the policy process. They cannot, for example, explain policy change, as would need to be done by a comprehensive theory of policy formation (John, Reference John1998). Six months is too short a time to analyse policy change, which typically takes place in cycles as long as a decade (Sabatier and Jenkins-Smith, Reference Sabatier and Jenkins-Smith1993). A longer timescale and additional methods would be necessary to add an explanation of policy change to the insights developed here.
Faith in the ideal of evidence-based policy dies hard. An eminent scientist with long experience of advising policy-makers has recently been quoted as saying that ‘nobody rational could possibly want a government based on any other type of policy-making’ (Colin Blakemore, quoted by Doward et al., Reference Doward, Hinsliff and McKie2009). This ethnographic analysis of the process of making policy suggests that holders of this faith may have to wait a while to see the ascension of their preferred method of policy-making. It has shown that there is little conclusive evidence available for policy-makers in the vast mass of information that is available to them. A certain group of policy-makers chose some of this information as evidence to tell stories. They selected evidence that fitted with the stories that have previously constructed their unquestioned concept of public value. This arose ideologically from the extant distribution of power, which structured their capability to take part in policy decisions. They used evidence to tell stories that were likely to be accepted within a thought world that favoured certainty over accuracy and action over contradiction. They attempted to transform issues of ethical value into questions of financial value. They usually displayed a utilitarian lack of normative engagement with ideas on the fundamental aims of public policy, leaving the pursuit of bureaucratic competence and career advancement as primary goals. The stories they told, and the evidence they used in telling them, were biased towards the rationalisation and reification of the uneven balance of power that constitutes the British state. The civil servants I worked with were not encouraged to select evidence which challenged the unequal status quo for use in telling these policy stories. These stories therefore ultimately supported the consolidation of power in the hands of the people who already hold it. This made their policy-making an intensely ideological activity – not in the sense that it used the kind of political doctrine that the New Labour project explicitly rejected, but because it reinforced the unequal distribution of power which it did so little to challenge.
Acknowledgements
The fieldwork on which this article is based was funded by the Economic and Social Research Council. I am also grateful for the useful comments that I received from Geoffrey Stevens, Jan Pahl, the editors of the Journal of Social Policy and two anonymous reviewers. A longer analysis of this study is presented in my book Drugs, Crime and Public Health: The Political Economy of Drug Policy (Routledge, 2011).