In this contribution we focus on regulatory reform. We present the agenda in the field and the research trajectory of three political scientists rather than a single story. This is because we believe that the story of our collective effort is more important that the individual trajectories. Indeed, impact in the social sciences is often generated by the sustained collective research focus and wider discourse coalitions.

So, to begin with, Alessia is a policy analyst with an interest in qualitative methods. In the last 15 years, she has worked on effective design in regional development, infrastructure, green growth, and budgeting, with a domestic and comparative focus. Her experience with regional and national Italian decision-makers increased her awareness of how diversity in information about possible consequences of public choice is crucial for balanced policy solutions. But such a diversity is hardly attainable when elected politicians alone are in control of instrument design, and information is biased by business and interest groups.Footnote ¹ The justification for information gatekeeping was normatively ingrained in a certain understanding of representative democracy that equates popular sovereignty with the absolute, unconditional control of elected politicians on decisions – provided that the law is not violated. As she was told while discussing the limited role of evaluation in the governance of infrastructural projects, “if a mayor wants a football pitch on the roof of the bell tower of St Mark’s basilica in Venice, s/he must be allowed to go ahead”. No policy instruments seemed capable of changing this mindset. A desire to pin down the variables that could really make a difference in public policy led her to shift the focus from decision-making to accountability constraints on the administrative dimension, and to Qualitative Comparative Analysis (QCA) as a suitable method for testing their difference-making power. Accountability opens alternative channels for bringing minority views into the policy process – Galileo would not have been listened to by authorities looking only for views articulated by the powerful constituencies of the time. These views and information are often discarded, whilst strong pressures and patronage fuel crony relationships. The configurational rationale of QCA promised to support the intention, and identify complete “recipes” of good policy designs for transfer. She began refining and testing the intuition using available data on budgeting practices and environmental policies.Footnote ² The quantum leap took place when Alessia was approached by Claudio and Claire to analyze their data on procedural devices of accountability. We’ll get to back to this project in a minute.

Claire wrote her PhD on risk regulation and epistemic knowledge in the EU at the height of the controversy about the precautionary principle. It was the transatlantic controversies over hormone growth promoters and bovine somatotrophin (rbST)Footnote ³ that sparked an enduring interest in the use of scientific evidence in policy-making. In 2004, she joined Exeter University to continue her research on science and public policy, environmental policy, food regulation and regulatory reform. Most recently, Claire is working with the UK’s Health and Safety Executive (HSE) exploring public risk perceptions of health and safety “regulatory myths”.Footnote ⁴

Claudio got into the field of regulation around 1996 thanks to his mentor, Professor Bruno Dente, who drew his attention to a set of regulatory reform instruments that had been officially endorsed by the OECD in 1995. This did not deter him from writing a doctoral dissertation on the role of knowledge in EU corporate tax policy – definitively not a project on regulation. But it led him to explore a tiny bit of the world of regulation with early papers and readings on governing regulation via policy instruments like consultation, impact assessment, benefit-cost analysis and retrospective evaluation of legislation, followed by a first hand-on project to roll out impact assessment in Italy, in 1999–2000. In 2004, DG ENTR asked him to work on regulatory indicators. Claudio turned the request for “a list of good indicators” into a design of a process of knowledge utilization leading to the choice of indicators as revealing of the underlying preferences of the Member States for regulatory reform. He argued that the design of the process in which indicators would be used was as important as the indicators chosen. The project was so successful that it evolved into an academic volume co-authored with Fabrizio De Francesco,Footnote ⁵ special sessions with the Directors of Better Regulation of the EU, three editions of training of senior officers at Exeter University (organized with Anne Meuwese), a major role in EU-funded projects like ENBR (European Network for Better Regulation, led by Andrea Renda) and Evia (Evaluating Impact Assessment, led by Klaus Jacob), and input to the design of the World Bank Institute’s first global core course on Regulation (Washington, 2009).

More recently Claudio and Claire have pushed learning theory and models in public policy analysis, connecting these constructs empirically to regulatory humility, regulatory culture and the role of regulatory impact assessment in policy processes, using experiments, process-tracing, statistical analysis and qualitative methods. Our Exeter team has grown during the last ten years including some emerging bright stars in the firmament of regulatory analysis like Anne Meuwese (now at Tilburg), Oliver Fritsch (now at Leeds), Lorna Schrefler (now at the European Parliament), Jonathan Kamkhaji and Thibaud Deruelle. Our Exeter-based colleagues Madalina Busuioc, Alison Harcourt, Oliver James and Duncan Russel have generated a fabulous research environment for projects on different strands of regulatory analysis.

The ERC grant Analysis of Learning in Regulatory Governance was the major springboard for our theoretical, methodological and empirical advances.Footnote ⁶ Whilst most literature had looked at the accuracy of economics contained in impact assessments and other procedural regulatory policy instruments, with the ERC advanced grant Analysis of Learning in Regulatory Governance Claudio and Claire focused on its usage and lack-of-usage by policy-makers, drawing on organizational theory and theoretical arguments on varieties of learning. This led us to question established notions of “quality of impact assessment”, “benchmarking”, and “policy diffusion” as interpreted by advocacy think tanks and international organizations. Several organizations have benefited from this work. A good example is the OECD: in 2012–2013 Radaelli and FritschFootnote ⁷ wrote an expert paper and coached delegates on a systematic, context-sensitive approach to regulatory policy evaluation, recently extended and endorsed in the 2014 OECD publication Framework for Regulatory Policy Evaluation. The ALREG work culminated in the Handbook of Regulatory Impact Assessment Footnote ⁸ where practitioners and academics explore different dimensions of impact assessment across 29 chapters.

However, one strong finding that keeps showing up in the field of regulatory reform is that individual policy instruments “explain little”. The effects in terms of knowledge utilization, policy reforms and ultimately accountability are determined by combinations of instruments, the social mechanisms they trigger and the context in which they operate. ALREG provided the first opportunity to join forces with Alessia Damonte. Alessia’s work in the field of set-theory seemed the right way forward to tackle the questions raised by the “ecologies”. This led the three of us to test some initial conjectures about ecologies of policy instruments on a small number of countries and seven policy instruments. More importantly still, we believe that there is a missing link in the field of regulatory reform and more generally the “policy mix” across countries: how to integrate the empirical findings on the mix of policy instruments with an explicit, sound approach to causation and a method consistent with this.

Consider these two crucial questions: how does a given regulatory policy mix generate accountability? How does accountability generate (or not generate) effects on the business climate, control of corruption and trust in government? After all, we are interested in regulatory reform because we care about final outcomes like growth and institutional stability, yet the challenge is how to establish causation. Research on policy ecologies is often descriptive: we know about the policy mix and how governments have chosen this or that mix, but much less about the effects produced by the particular configuration of policy instruments in a country or group of countries, at a given time or across time.

The problem is not the lack of theories – there are many potentially useful explanations in political science. Instead, the problem is that we usually test theories as single determinants. We look at explanatory factors A, B and C and find which one has coeteris paribus the power to increase the probability that, say, corruption or doing business indicators reach a certain level. We end up with the curse of so many empirical estimations of the factors that contradict each other, or show that given factors are causally significant only in a given context – the context is indeed the black box of this type of analysis.

Thus, we argue that the future lies in grasping the “mediating” or “moderating” effect from contextual conditions to unfold the mechanism. There are two ways ahead. The first is to investigate the generative process as a chain of observable antecedents and consequents.Footnote ⁹ The second approach does acknowledge the generative process, but only as an unobserved hypothesis. Instead, it focuses on the conditions which together enable, trigger or hinder it – were the causal claims true.Footnote ¹⁰ This second route to explanation is the one we choose.

If we proceed from these philosophical underpinnings, the next step is to align this ontology with a suitable methodology. Co-variational approaches assess the average causal potential of single policy instruments – and of limited interaction terms – across a population, but cannot explain why such potential led to different results at the case level. Explanations – we argue – are configurational instead. Thus, we maintain that corruption, doing business and trust in government are the result of instrument ecologies. We also have to be open to the possibility that more than one ecology or combination of instruments proves effective in special subpopulations – or even in a single case. A methodological shift is then required: from the correlational tests of average net effects, to the identification of those configurations proved to “enable or disable specific connections between causes and outcomes”.Footnote ¹¹

Qualitative Comparative Analysis (QCA)Footnote ¹² provides this alternative methodological approach. In fact, QCA relies on a variation of the well-known logical canon of agreement and difference to identify all those minimal configurations of generative conditions which “chemically” prove to be associated to the occurrence of an outcome. Its strategy hence assumes the highest level of explanatory complexity as its starting point. Given a number of conditions which, together, can account for the occurrence of an outcome, the analysis considers all the possible configurations resulting from the combination of presence and absence of each condition. Each “primitive” configuration is therefore represented as a row in a truth table, or as an intersection in a Venn diagram, to which actual cases are then assigned. This makes primitive configurations observed and associated to the occurrence of the outcome – or its non-occurrence. Pairwise comparisons then identify which condition proves irrelevant for the mechanism to obtain – because a single changing condition in two otherwise similar configurations to the same outcome is clearly a redundant part of the explanation. The logical induction of “minimizations” stops when every irrelevant condition has been dropped, leaving a number of “prime implicants” which still cover all the cases yet are made of fewer conditions – hence, are more general. As a result of minimizations, cases from different primitive configurations often group under the same prime implicant, and constitute the subpopulation where such “causal path” explains the outcome – so preventing the fallacy of considering any tested model as some single-best.

QCA is a theory-driven method. It starts from a purely deductive moment – the identification of the many possible conditions deemed to trigger and enable the mechanism of interest. Yet, it leaves to the induction of logical minimizations the task of sorting more general explanations, and of identifying to which cases each “causal path” applies. Given its ex-post non-probabilistic rationale, the technique cannot claim any external validity for its solutions; yet, it provides detailed accounts of the local reasons for the mechanism to (not) work which are highly internally valid – at least, when the research design secures the variation of both the explanans and the explanandum; the population under analysis satisfies some theoretically meaningful scope condition; and solutions leave no consistency outliers which would falsify the model.

We do not know whether this will be the future of research on regulatory reform, “better” and “smart” regulation. Other options are possible to capture the mixes and ecologies, for example the analysis of inter-dependence across time and space (as shown by the field of diffusion studies, a good application to the case of impact assessment is De FrancescoFootnote ¹³ ). But this is certainly where we are going with our new ERC Advanced Grant program, Protego, Procedural Tools for Effective Governance, which will keep us amused for the next four years at Exeter and Milan.