1. Introduction
In this article, we develop a framework to analyze the relationship between evidence and policy. First, we define a normative criterion based on cost–benefit analysis and the value of a piece of information. Second, we use standard game-theoretic tools to study the (Nash) equilibria of an interaction between experts and citizens in providing information for a decision maker. Finally, we study three institutional arrangements (evidence-based policy (henceforth EBP), deliberative governance, and negotiated conflict) that differ in terms of how heavily we rely on experts and citizens in providing information to formulate normative prescriptions. This contribution addresses the current discussion on populism (a term that we will not be using)Footnote 1 and post-truthism (Robert, Reference Robert2010), though we adopt a different approach and show that most polarized views for or against experts are generally biased.
We follow and generalize the approach presented by Fischhoff and Eggers (Reference Fischhoff, Eggers and Shafir2013), who argue that any form of policy requires a delicate balance of normative choices to define the best options, a descriptive account of the behavior, and a prescriptive identification of the gap between desired and actual outcomes.
We start from this tripartite analytical framework and move from individual policy to institutional arrangement. From a normative point of view, the definition of an optimal science–policy interaction rule must reckon with issues arising from both scientific discourse's foundationalism and the interpersonal comparison of preferences concerning how collective decision-making should be organized.
From a positive point of view, identifying the expected outcome of the science–policy interaction depends on the clear mapping of the potential sources of conflicts related to the nature of the evidence at hand, the complexity of the interests at play, and the constellations of individual values and preferences. We introduce the concept of the policy space (Bogliacino et al., Reference Bogliacino, Codagnone and Veltri2016), comprised of evidence, values, and interests. Besides the mapping of the roots of conflicts, an effective positive theory of the science–policy interaction requires a reliable theory of human behavior, i.e. a theory of the driving motivation behind human decisions.
From a prescriptive point of view, the way out of conflicts depends on the understanding of the set of relevant constraints in the implementation of institutional architectures and the policy cycle's timing; it is most likely that such ways out require formulation on a case-by-case basis.
So we start by postulating a simple efficiency criterion (based on cost–benefit evaluation), employing the economist's jargon, though we discuss its limits and potential critiques. From there, we discuss the difficulties inherent in the science–policy interaction by working out a toy model. Finally, we discuss alternative institutional arrangements (including EBP) as a way out of conflicts to foster greater efficiency.
2. What should the evidence–policy relationship look like?
A simple framework
We start from a very simple model, based on standard Bayesian decision rules (Morris and Shin, Reference Morris and Shin1993).
We assume that, in a certain policy domain, the choice is to intervene or not. In the case of policy intervention, society should bear a cost c; if things are left to decentralized choices, a bad event can occur with probability p, inducing a loss K > c. A simple policy rule in this case is to intervene whenever p > c/K.
As can be said of all models, this is simplified but captures a large array of societal situations: regulation of the financial sectors is costly but facilitates the avoidance of major financial distress, the occurrence of which would generate large losses for society. Promoting sustainable development is expensive because of short-run employment effects or taxes, yet extreme climate events also cause very expensive emergency interventions.
To complete the formalization, following the aforementioned argument in Morris and Shin (Reference Morris and Shin1993), we should introduce the concept of information. Information is equivalent to a test that produces a false positive (i.e. raises a problem when there is none) at a rate η and a false negative (i.e. does not raise concerns when doing so would have been desirable) at a rate ε.
Information in this case could refer to a campaign organized by groups to sway public opinion or a study by a researcher possibly commissioned in support of a prospective policy intervention. For the sake of simplicity, we can indicate the duples (ηe, εe) and (ηc, εc) to express the precision of information produced by experts (superscript e) and citizens (c), respectively.
Assume for a moment that the prior probability (p) is known. Therefore, if a concern is raised, the posterior probability of actually facing a bad event would be equal to:
$$\gamma ^{\,i} = \displaystyle{{\,p\lpar {1-\varepsilon^i} \rpar } \over {\,p\lpar {1-\varepsilon^i} \rpar + \lpar {1-p} \rpar \eta ^i}}i = e,c$$whereas if a concern is not raised, the posterior probability of having to deal with the bad event would be equal to:
$$\delta ^i = \displaystyle{{\,p\varepsilon ^i} \over {\,p\varepsilon ^i + \lpar {1-p} \rpar \lpar {1-\eta^i} \rpar }}i = e,c$$The information is valuable when δ i < p < γ i because the intervention is carried out whenever it should be and is not carried out when it should not.
In this case, EBP is an optimal rule when δ e < δ c < p < γ c < γ e, for evidence produced by experts is the most valuable. Collective deliberation, using evidence produced autonomously by citizens, is based on valuable information when δ c < p < γ c.
It can be shown (Appendix A.1) that ε i + η i <1 is a necessary and sufficient condition for δ i < γ i. If ε i + η i = 1, then δ i = p = γ i, i.e. the information does not improve on the prior probability. Yet if ε i >1 − η i, the information is biased, i.e. δ i ≥ p ≥ γ i, which means that we induce a selection bias, either intervening when we should not, or not intervening when we should.
Discussion
The normative criterion we propose is a version of cost–benefit analysis. In the literature on policy design and evaluation, this criterion has been criticized on various grounds (Sunstein, Reference Sunstein2000). We can discuss the main objections in the general literature that will inform the framework outlined in this contribution.
First, to evaluate the bias of information, we should recognize the concept of truth as a regulatory idea, i.e. as a minimalist criterion to rank alternative discourses. This is not an issue of concern in the standard economic literature, but it does generate noteworthy discussion in the domain of policy evaluation. Causality is at the core of the standard approach, for it is framed in a positivist and critical rationalist epistemology.
Although we will not go into epistemological details given the limitations of space here, we maintain that there are consequences that can be causally related to a policy option, though these causal mechanisms are conditional on our theoretical hypotheses on the matter (Heckman, Reference Heckman2000, Reference Heckman2010). Cost–benefit analysis requires effort to evaluate and measure such consequences.
“Truth-based” normative criteria can also be criticized on different philosophical – but not strictly epistemological – grounds. Political climate and philosophical Zeitgeist influence the same debate regarding how collective deliberation should be achieved and how evidence for such deliberation should be provided. In other words, one may object that consequentialism is neither a necessary nor a sufficient criterion. On the one hand, an argument can be made in favor of deontological normative criterion, à la Kant, suggesting that certain public demands should be heeded regardless of the evaluation of consequences. On the other, one may dismiss the foundational discussion of knowledge and defend the political process in terms of its own validity.
In this respect, we simply recognize that causal analysis is fundamental to understanding the constraints associated with a certain course of action; however, politics can make choices and decide to face the implied costs.
A third question concerns the problem of quantification. It has been argued that a cost–benefit analysis masks value judgments; that is, concepts, measurement, and quantification depend on frames that are value laden or at least socially constructed (Cartwright and Hardie, Reference Cartwright and Hardie2012; Cartwright and Runhardt, Reference Cartwright, Runhardt, Cartwright and Montuschi2014; Munro, Reference Munro, Cartwright and Montuschi2014). According to Saltelli and Giampietro (Reference Saltelli and Giampietro2017), the implicit selection of a frame as the basis for quantification leads to oversimplification and “socially constructed ignorance” through the conflation of important aspects that are overlooked and that weaken the produced quantifications and inferences. Perhaps the party that best hides the value judgments implicit in its framing will win the rhetorical battle: the concepts and measurements used in empirical research may be chosen and framed by particular groups, especially by those in power. This may bias the research and lead to a disregard of the views of less powerful social groups. On this point, we simply follow Sunstein (Reference Sunstein2000): cost–benefit analysis should be complemented with proper qualitative information to make value judgments accountable, or, alternatively, the framing itself should be object of deliberation.
Finally, one may argue that costs and benefits may be valued by agents as a function of who will bear the former and who will reap the latter (because of inequity aversion), or certain cues: a death is valued differently if it includes pain or if the risk that precipitated it is assumed to have been voluntary (Sunstein, Reference Sunstein2000). Since this is a partial equilibrium framework, with the couple (c, K) assumed exogenous, the aforementioned is not a concern in the current analysis.
3. What can be expected when science and policy interact?
Unpacking our three dimensions: interest, value, and uncertainty
The environment in which experts, citizens, policy makers, and interest groups behave is shaped by three relevant parameters: the degree of uncertainty of the problem at hand, the presence of strategic or diffuse interests, and the presence or absence of value conflicts.
This concept of policy space is a fine-tuning of related taxonomies that have appeared in policy evaluation literature. Typically, they have either collapsed value and interest in one single dimension (Funtowicz and Ravetz, Reference Funtowicz, Ravetz, Castree, Hulme and Proctor2018), or they focus on interests and uncertainty only (Oreskes and Conway, Reference Oreskes and Conway2010), but they have never treated the three coordinates simultaneously.
The degree of uncertainty is related to the cost of providing valuable information, per the definition outlined in the previous section, as well as the likelihood of detecting selective information. An in-depth discussion of epistemic uncertainty surpasses the scope of the present paper (Codagnone et al., Reference Codagnone, Bogliacino and Veltri2018), yet a working definition can be formulated as follows: Scientists deal with situations of dense causality and holistic integration between the parts of the systems under analysis (Manzi, Reference Manzi2016). Suffice it to say that there are two main issues: (1) science, especially social science, is socially embedded; and (2) epistemologically, it is difficult to give a clear foundation for the scientific claim to truth. On the first point, it is important to understand that scientific facts are difficult to measure, and measurement can be seriously affected by social processes, which are value laden (Best, Reference Best2001; Tomlinson and Kelly, Reference Tomlinson and Kelly2013).
Regarding the second point, science aims at complying with both external consistency (with reality) and internal consistency between the propositions that are aimed at describing and explaining reality itself. In the domain of science and policy, Shadish et al. (Reference Shadish, Cook and Campbell2002) propose the use of the validity category, which is related to the robustness of the inference we are making inside a specific empirical exercise (internal validity) and the inference that can be made based on a specific trial (external validity). Different forms of validity are in conflict, and the scientist's trade consists of finding an appropriate balance.
Another relevant dimension is the geography and distribution of power and interests: how is the landscape of citizens, corporations, and interest groups characterized in this specific issue? Key elements include the magnitude of the stakes and the urgency with which a certain regulation or decision should be carried out.
If stakes are high and the decision urgent, it is more likely that citizens organize, firms and interest groups lobby, and the decision maker faces strategic actors with the power to influence. In an economic setting, this would be comparable to an oligopolistic situation. At the other end of the spectrum, we have low stakes and/or issues that are not pressing, for which collective action is costlier to organize, indifference may prevail, and the power distribution is more diffuse. This is what, metaphorically, we describe as a competitive environment; in such an environment, individual agents are zero mass and neglect decisions from other parties, only directly interacting with the decision maker. For the sake of clarity, the presence of strategic interests refers to private actors’ possibility of mobilizing resources to distort the incentives of experts, policy makers, or private citizens (Olson, Reference Olson1971 [1965]).
The distinction between specific (alternatively called concentrated) and diffuse interests is a classical one, following Mancur Olson's theory of collective action (Olson, Reference Olson1971 [1965], but see also Schattschneider, Reference Schattschneider1960).Footnote 2 Olson deems diffuse interests the “forgotten groups,” and portrays consumers as illustrative of a numerous but dispersed group with “no organization to countervail the power of organized or monopolistic producers” (Olson, Reference Olson1971 [1965]: 166). An extensive review of interest groups’ politics (Beyers et al., Reference Beyers, Eising and Maloney2008: 1109) revealed that the distinction between specific and diffuse interests remains quite influential among political economists and in the political science literature, whereas it is becoming less influential in the more specific literature on interest groups.
Olson's approach to the logic of collective action and concomitant hypothesis that concentrated interests dominate at the expense of diffuse ones has been criticized both on theoretical and empirical grounds (Hirschman, Reference Hirschman1982; Trumbull, Reference Trumbull2012). The most vehement critic has been Hirschman, who argues that participation in collective action cannot be considered more than an end in itself, deriving from the desired collective benefit and the resulting acquisition of a more general understanding of the common good beyond selfish interests.
On the contrary, public choice theory has developed both the theory of constitutional democracy and the theory of bureaucracy on the basic assumption that concentrated interests and diffuse interests have different likelihoods of organizing (Buchanan and Tullock, Reference Buchanan, Tullock and Buchanan1999; Niskanen, Reference Niskanen2008).
Our selective review of the specific literature on interest groups’ politics shows that the distinction between specific and diffuse interests is still relevant, though it has come to be considered an empirical issue.
Finally, our third axis of the topology is represented by the normative implications of the issue at hand and whether preferences can be aggregated in a consistent way or public debate can be shaped to reach compromise and consensus. These are the two poles of the third axis of our topology. Value conflicts may induce private citizens to mobilize because they may enjoy in-group bias. It can bring about selectivity in experts because whenever they are more likely to become pivotal in the final choice, they seek to leverage this advantage.Footnote 3
Discussing the implications for the role of science in policy and politics of the degree of value consensus in decision making, Pielke consider two scenarios: Tornado Politics and Abortion Politics (Reference Pielke2007, chap. 4). For the former scenario, there is an auditorium with 50 people attending an event who are then informed that a tornado is approaching; they all share the common objective of surviving and, consequently, they search for information to decide on the best course of action (i.e. staying inside or finding the best route to evacuate the auditorium). Hence, the systematic pursuit of knowledge is effective in helping reach a consensual decision. In Pielke's second scenario, the same 50 people are in the same auditorium but are tasked with deciding whether or not to allow abortion in their community, pitting religious values against free-choice values. Therefore, the pursuit of knowledge is ineffective. There is no shared commitment to a specific goal but rather conflicting commitments stemming from different values. While information matters in this situation, arguably no amount or type of scientific information about abortion can reconcile the differing values. But when Tornado and Abortion politics are conflated, various interests’ groups use science for their own battles, and science is forced into deliberation on issues dealing with values and politics (Pielke, Reference Pielke2007: 47).
Values are linked to emotions and affects that represent an important dimension in human judgment and choice and are relevant for all actors who may be involved in a debate on a given policy issue. Emotions affect opinion formation, attention, learning, and political behavior (Brader and Marcus, Reference Brader, Marcus, Huddy, Sears and Levy2013) in addition to attitudes on a wide range of issues related to world politics, such as nuclear proliferation, the logic of deterrence, the War on Terror, motives for war, alliances and defense policies, ethnic conflicts, and humanitarian intervention (Hutchison and Bleiker, Reference Hutchison and Bleiker2014). As such, emotions play a role also in the policy-making process (Thorngate, Reference Thorngate2001).
Emotion influences values, which are, in turn, intimately linked to identity formation and group belonging. This may explain why new information and evidence may be rejected by conflicting sides and have difficulty diminishing polarization as documented theoretically and empirically in contributions studying so-called ‘politically motivated reasoning’ (PMR; Jost et al., Reference Jost, Hennes, Lavine and Carlson2013; Kahan, Reference Kahan2013). From this perspective, which we only briefly review here, much of the persistent conflict and polarization over policy-relevant facts can be attributed to PMR. This occurs when positions on policy issues have a social meaning and are a marker of membership within identity-defining affinity groups. When this happens, group membership becomes more important than the rational appraisal of information and evidence. Group members will conform to the position of their group, regardless of the kind of additional information they may receive. The beliefs generated by this form of reasoning predictably galvanize the expression of attitudes and related affective states that convey a person's group identity. Such a disposition helps maintain one's connection to, and standing among, people with whom one shares important bonds. PMR refers to individuals’ tendency to unconsciously conform their assessment of information to some goal collateral to determine its truth (Kunda, Reference Kunda1990). Such goals may include maintaining a positive self-conception, for the truth-independent goal of PMR is identity protection, i.e. the formation of beliefs that maintain a person's connection to and status within an identity-defining affinity group whose members are united by shared values.
A model of information provision
In our model, both experts and citizens choose the level of information accuracy. We use the label “experts” to label professional providers of evidence in support of policy making. In other words, we define them as both having epistemic authority, social/institutional recognition, and a career based on being an evidence provider (e.g. an academic, a consultant, etc.). Organized citizens may have their own experts bearing some form of epistemic authority; however, their independence is obviously called into question, and as such, we will use the unifying label ‘citizen’ to include their spokespersons, or partisan experts.
Providing biased information is less costly and allows for taking a share of resources mobilized by private actors (and is parametrized by the degree of strategic interests, μ). However, lying is also costly because it violates social norms, though it is affected by the degree of value consensus (parametrized as the probability of agreement on a certain issue by two independent agents, α): Since group identity is usually valued, if there are value conflicts, citizens receive a private reward from groupthink. Moreover, lying can be punished (socially or institutionally) by an amount equal to the likelihood of being detected multiplied by the sanction (punishment) if detected. The likelihood of detection decreases in the degree of uncertainty (σ) and increases in the precision of the information provided by the counterpart (citizen or experts).
Formally, define x i = η i + εi, i = e, c as the degree of selectivity in the evidence provided. Experts and citizens maximize:
$$\mathop { max} \limits_{x^i} m^i\lpar {x^i;\; \mu} \rpar -p^i\lpar {x^i,\; x^j;\; \sigma} \rpar S^i + v^i\lpar {x^i;\alpha} \rpar -c^i\lpar {x^i} \rpar \; i,\; j = e,c\; i\ne j\;, \lsqb 1 \rsqb $$where m( · ) is the monetary reward for lying, μ is the parameter capturing strategic interests, p( · ) is the probability of detection, S is the sanction, v( · ) is the preference for compliance with social norms, c( · ) is the cost of gathering information. All the relevant functions are heterogeneous between experts and citizens, as shown by the superscript i.
The following assumptions hold.
$$Assumption \,1.\displaystyle{{\partial m^i\lpar {x^i;\; \mu} \rpar } \over {\partial x^i}} \gt 0;\;\displaystyle{{\partial p^i\lpar {x^i,x^j;\; \sigma} \rpar } \over {\partial x^i}} \lt 0;\;\displaystyle{{\partial p^i\lpar {x^i,x^j;\; \sigma} \rpar } \over {\partial x^j}} \lt 0;$$
$$\displaystyle{{\partial v^i\lpar {x^i;\; \alpha} \rpar } \over {\partial x^i}} \lt 0;\; \displaystyle{{dc^i\lpar {x^i} \rpar } \over {dx^i}} \lt 0$$
$$\eqalign{& Assumption \, 2.\;\displaystyle{{\partial ^2m^i\lpar {x^i;\; \mu} \rpar } \over {\partial {\lpar {x^i} \rpar }^2}} \lt 0;\;\displaystyle{{\partial ^2p^i\lpar {x^i,x^j;\; \sigma} \rpar } \over {\partial {\lpar {x^i} \rpar }^2}} \gt 0;\;\; \displaystyle{{\partial ^2p^i\lpar {x^i,x^j;\; \sigma} \rpar } \over {\partial x^i\partial x^j}} \lt 0;\;\displaystyle{{\partial ^2v^i\lpar {x^i;\; \alpha} \rpar } \over {\partial {\lpar {x^i} \rpar }^2}} \cr & \qquad\qquad\qquad\qquad\qquad\quad \lt {0;\; \displaystyle{{d^2c^i\lpar {x^i} \rpar } \over {d{\lpar {x^i} \rpar }^2}}} \gt 0}$$Assumption 1 is just the definition of m( · ), p( · ), v( · ), c( · ) as the monetary reward, probability of detection, value of conformity to social norms, and cost of acquiring information. Assumption 2 is a standard concavity requirement. Standard Inada conditions can be introduced to ensure an interior solution over the interval (0, 2).
As usual, we can use the following equilibrium definition.
Definition: An equilibrium for the society is a duple (x e*; x c*)such that (x i* i = e, c) is a solution to [1].
In choosing what kind of evidence to provide (their best response function, BRF), experts and citizens act strategically, i.e. taking into account their counterpart. In this concrete case, each part chooses the inaccuracy of their signals (ε i + η i), and we can define an equilibrium configuration as one in which both sides are unwilling to change the quality of the information provided given the behavior of the other. Our definition of equilibrium is standard and is simply the Nash equilibrium of the game. The equilibrium is also the predicted outcome of the interaction given that no party wants to deviate from it (by definition).
The following proposition holds.
Proposition 1.
Equilibrium for the citizens-expert interaction in the policy space exists under Assumptions 1 and 2. Moreover, Equilibrium is unique and stable.
Proof: see Appendix A.2.
Stability means that small deviations will push the system back to the equilibrium (i.e. this equilibrium represents a stable attractor).
Each side's selectivity of evidence reinforces its counterpart's selectivity, i.e. there is a positive feedback from one side's inaccuracy to the other side's inaccuracy until they reach equilibrium, and the opposite occurs when the initial level of inaccuracy is very high. The decision rules and the equilibrium are displayed in Figure 1, in which the unit line is also shown because, as discussed in the previous section, this is the threshold below which information remains accurate and valuable.
Best response functions and Nash equilibrium
An interesting question relates to the comparative statics: how the evidence provided in equilibrium reacts to a change in one of the parameters that define the policy space. In other words, what happens to the accuracy of evidence when uncertainty increases? What happens when the degree of concentration of interests increases? And, finally, what happens when we move toward value conflict?
Comparative statics can be derived under Assumption 3:
$$Assumption \, {\it 3}.\;\displaystyle{{\partial ^2m^i\lpar {x^i;\; \mu} \rpar } \over {\partial x^i\partial \mu}} \gt 0;\; \displaystyle{{\partial ^2p^i\lpar {x^i,x^j;\; \sigma} \rpar } \over {\partial x^i\partial \sigma}} \lt 0;\displaystyle{{\partial ^2v^i\lpar {x^i;\; \alpha} \rpar } \over {\partial x^i\partial \alpha}} \lt 0$$The meaning of Assumption 3 is straightforward. The marginal return of lying is increased strategic interest, because more resources are made available to experts and citizens willing to support private interests. The marginal effect on the likelihood of detection is negatively affected by uncertainty, and the value consensus increases the cost of violating a social norm, by lowering further the negative marginal effect of lying.
The following proposition holds.
Proposition 2.
Evidence selectivity increases when value conflict, strategic interest, or uncertainty increase.
Proof: see Appendix A.3.
When a source of tension in the policy space erupts, we end up with an equilibrium with a less informed society. BRFs shift upwards for both experts and citizens. Nevertheless, both sides may respond differently to each of the three parameters. For example, interest groups are obviously more incentivized to obtain biased results from epistemic authorities or to disseminate doubts and increase uncertainty (Oreskes and Conway, Reference Oreskes and Conway2010), for this approach increases their chances of success in the rhetorical arena. In contrast, groupthink may induce more biased responses in citizens, limiting the range of options considered (Turner and Pratkanis, Reference Turner and Pratkanis1998) and eliciting information selection that is easily understood as identity-protective behavior (Kahan, Reference Kahan2013). Experts are very sensitive to reputation loss because reputation is the bedrock of their epistemic authority. As a result, they are very sensitive to uncertainty, for it opens space for selectivity of the evidence produced, lowering the risk of being disproved. From a rational point of view, this explains why they are less sensitive to identity-driven behavior and respond less to groupthink.
In Table 1, we summarize these intuitions and explain the underlying rationale.
Elasticity of the Best Response Functions over the Policy Space
As an example of comparative statics, we show in Figure 2 the implications in terms of equilibrium behavior of moving from diffuse to strategic interests and from value consensus to value conflicts in terms of individual behavior and observed outcomes in the policy domain.
Comparative statics
On the left panel of Figure 2, we show what happens when we move from value consensus to value conflict. As can be seen, in the presence of value conflict, the behavior of experts is less affected (there is a less conspicuous shift of the BRF left and upwards) than that of citizens (whose BRF shifts right and downwards). Note that in the presence of value consensus, experts provide more precise information than citizens, but the equilibrium is characterized by x i <1, i = c, e, which means that in either case, information is accurate. When we move to value conflict, the new equilibrium is characterized by inaccurate information for both experts and citizens: although the elasticity to value conflict is higher for citizens than experts, the fact that citizens are more biased drives down the accuracy of experts because they are no longer controlled by an informed community (or are to a lesser extent). In other words, moving around the policy space generates two different effects. The first is the incentive effect of uncertainty, value, and interests, and the second is the strategic effect, via interaction of experts with citizens and vice versa.
When we consider the right-hand panel in Figure 2, in which we consider the change from diffuse to strategic interests, we see the same logic at work as in the left-hand panel; the difference, however, lies in the fact that the right panel shows experts as those who are more deeply affected by the change at the parameter.
We now introduce a numerical example that will be instrumental in the analysis performed in the following section, where we compare institutional arrangements.
We chose the following functional forms:
$$m^i\lpar {x^i;\; \mu} \rpar = x^{i{\beta _1^i}} \mu ^{\beta _2^i}, \; \beta _1^i \lt 1,\; \beta _2^i \lt 1,\; i = e,c$$
$$p^i\lpar {x^i,\; x^j;\; \sigma} \rpar S^i = \displaystyle{1 \over {1 + exp \, \lpar {x^i + x^j} \rpar }}\cdot \displaystyle{1 \over {1 + exp \,\lpar {k^i\sigma} \rpar }}S^i,\; k^i \gt 0,\; i = e,$$
$$v^i\lpar {x^i;\alpha} \rpar = -\lpar {x^i\alpha} \rpar ^{\beta _3^i}, \; \beta _3^i \gt 1,\; i = e,c$$
$$c\lpar {x^i} \rpar = \displaystyle{1 \over 2}\lpar {2-x^i} \rpar ,\; i = e,c$$For the simulations in Figure 3, we use the following calibration:
$$\beta _1^e = \beta _1^c = 0.5$$
$$\beta _2^e =. 9,\; \beta _2^c = 0.5$$
$$\beta _3^e = \beta _3^c = 2$$
$$\beta _4^e =. 1,\; \beta _4^c = 0.9$$
$$k^e = 1,\; k^c = 1.01$$
$$S^e = 5,\; S^c = 1$$Equilibrium provision of information under strategic interaction
Finally, the policy space is calibrated in the following way:
• Value consensus: α = 1
• Value conflict: α = 1/3
• Strategic interests: μ = 2
• Diffuse interests: μ = 12
• Low epistemic uncertainty: σ = 1
• High epistemic uncertainty: σ = 11
The advantage of a numerical example is that it allows us to simulate scenarios and compute equilibrium values for expert- and citizen-provided knowledge.Footnote 4
In Figure 3, we plot the simulation results. In each of the three panels, we compute the numerical solution of the equilibrium strategy by varying one parameter in the policy space: value consensus (α) in the upper-left panel, strategic interest (μ) in the upper right panel, and uncertainty (σ) in the lower panel.
The numerical example reproduces the stylized fact in Table 1. The effect of increasing value conflicts is shown in the NW panel. It has a significant effect on citizens but only an indirect effect on experts. In this case, the variation is almost negligible yet still exists for the reason explained above. The opposite occurs for strategic interests, depicted in the NE panel: the possibility of capturing private resources benefits experts more than citizens, inducing greater selectivity in equilibrium on the side of experts. This fact results in less accuracy in citizens as well. Finally, as shown in the S panel, increasing epistemic uncertainty impacts both experts and citizens: the latter are less sensitive, which explains why there is less accuracy for low-level uncertainty; nevertheless, when uncertainty increases, experts are less worried about being discredited and provide less valuable information. In the example, experts end up being less credible than citizens, but this, of course, is not a general prediction of the model, and strictly depends on how large the difference between experts and citizens’ sensitivity is. However, it points out a risk of any sort of technocratic environment: epistemic uncertainty will ultimately generate moral hazard on the part of the experts.
Mapping policy issues
The three-dimensional axes (uncertainty, interests, and values) allow policy issues to be located in the policy space, as a sort of coordinate system.
We provide three examples: atherosclerotic vascular diseases, vaccines, and immigration.
In the 1950s, the US saw an increasing prevalence in atherosclerotic vascular diseases such as coronary heart disease or hypertension. There was a perception that dietary elements associated with the US lifestyle may have been responsible; two main hypotheses were put forth. Namely, the culprit was either sugar or fat (total fat, saturated fat, or dietary cholesterol). This was viewed as a threat and an opportunity by the sectors involved. That is, on the one hand, in the sugar lobby's eyes, for example, a campaign against sugar as the root of disease would be harmful. On the other, assigning the responsibility for disease to fat meant thousands of weekly calories would be freed up, offering the sugar sector unprecedented areas of growth in the share of household budget.
Epidemiological research is a very uncertain domain, with purposeful behavior by the agents relevant and observational data prevalent. Thus, this branch of medicine is much closer to the social sciences than the hard ones. Of course, value consensus is high, since the possibility of access to improved living conditions is shared by citizens and experts.
Unsurprisingly, the main problem came about in the presence of concentrated interests fueled by epistemic uncertainty. Kearns et al. (Reference Kearns, Schmidt and Glantz2016) document the effort by the Sugar Research Foundation, a lobbying group, to finance studies (Project 226) and promote a framing in the court of public opinion to lay the blame squarely on the shoulders of total fat, saturated fat, and dietary cholesterol. Project 226 was a literature review, but it was published in a very prestigious journal; through the analysis of emails and other private communications, Kearns et al. (Reference Kearns, Schmidt and Glantz2016) show that there was a systematic effort by the lobby to manipulate the results and the conclusions. In our policy space, this issue would be placed under the following coordinates: strategic interests groups, high epistemic uncertainty, value consensus.
Vaccination is another interesting example that can be addressed inside our policy space. The coordinates of this problem are very different from those of the sugar case: Despite vaccines production by global players in the pharmaceutical industry, the share of total turnover is rather limited according to the World Health Organization. The scientific evidence is overwhelming. The link between vaccination and autism is proposed as a justification for refusing the former by some parents. Part of the explanation is that as vaccination increased among the population, the diagnosis of autism improved, meaning that more cases were detected. A reconstruction of the false argument against vaccination is in Gerbert and Offitt (Reference Gerbert and Offit2009) and Offitt (Reference Offit2009).
The fact that the scientific evidence is so overwhelming yet unable to represent the smoking gun in the debate is due to the fact that the value conflict related to the topic (freedom to choose, policies that constrain families, opposition to so-called Western medicine, etc.) is typical of what Pielke (Reference Pielke2007) calls Abortion Politics.
The last case, immigration, is the best example of a policy issue that includes tackling the three aforementioned problems at the same time (value conflict, high epistemic uncertainty, and strategic interest groups). The evidence is quite inconclusive because experimental evidence is completely out of the picture and because immigration includes general equilibrium effects that make counterfactual evaluations very complicated to follow up on. Interests are also an issue: as suggested by Borjas (Reference Borjas2013. Reference Borjas2015), there is significant redistribution among workers and from workers to capital implied by immigration flows from the Global South to the Global North, and this redistribution is precisely what is implied by those very same models suggesting that freedom of movement would be a massive push for the world's GDP (Collier, Reference Collier2013). Needless to say, the cost of integration, the desirability of a multicultural society, and, more recently, the fear of terrorism or consequences of a global crisis fill the value dimension with conflicts.
4. Efficiency analysis
The outcome of the policy process under three different institutional arrangements
To understand the logic behind the efficiency analysis of institutions, we can go back to Figure 1. As stated, when we move around the policy space, two effects are produced. The incentive effect is the direct effect of a change in one of the parameters of individual preferences, which affects behavior. The strategic effect is the adjustment in behavior as a result of the interaction between citizens and experts.
In general, the incentive effect overrides the strategic effect. Technically, this is a consequence of the stability of equilibrium (the stability requires that the BRF of the counterpart cuts cross its own BRF from below, but this implies this balance between strategic and incentive effect). If one party overreacts to the behavior of the other, the interaction will not converge back to the equilibrium. If this is the case, the decision maker should request information from the party that is less affected by the problem characterizing the issue at hand and should provide a system of incentives to counteract the effect of uncertainty, interests, or values. In a nutshell, if the decision maker could displace – even slightly – the BRF of one party, the decision maker should neglect the strategic effect and concentrate on the party less affected by the problem. Assume for a moment that in the left-hand panel of Figure 2 the policy maker could shift the BRFs of the experts to the bottom and to the right by even a tiny bit. In response, it would become better to operate with experts. Even though citizens’ lack of control drives experts’ inaccuracy upwards, the equilibrium value of experts’ information can be driven down, below the threshold of one, guaranteeing cost–benefit efficiency.
What institutional arrangements are studied?
In the 1997 electoral campaign, the Labour Party's manifesto stated: “We will be a radical government. … What counts is what works. The objectives are radical. The means will be modern.” This became the manifesto of evidence-based policy (EBP), which now constitutes the mainstream in policy design and evaluation.
To be clear, EBP is not a general quest for evidence. A desire to ground policy in a source of privileged knowledge is as old as humankind, yet the modern formulation of a science agenda in support of policy can be found in Campbell's (Reference Campbell1969) “Reforms as Experiments” manifesto, claiming that society is a sort of laboratory in which marginal change is to be pursued and an open society is to be defended. It was also defended as one of the guiding principles of institutionalism, in the sense that the instrumental criterion was supposed to judge not only the technical effectiveness but also the appropriateness of value judgments with respect to alternatives (Gordon, Reference Gordon1990; Tool, Reference Tool1979).
As a result, EBP is not a vague statement advocating the use of science in policy, which would certainly garner universal praise. Rather, EBP designates the expert as the ultimate source of knowledge with respect to a policy action and involves the expert as an actor within a proper institutional setting to provide policy support. EBP sets the benchmark of a decision as protected from political negotiations to ensure that relevant information flows from those who “speak truth to power,” to borrow Wildavsky's phrasing (Reference Wildavsky1979).
In our opinion, assuming that experts can provide the benchmark for knowledge implies that either they are fundamentally different from non-expert citizens or that institutions in the expert community shield them from biases affecting non-expert citizens.
In Codagnone et al. (Reference Codagnone, Bogliacino and Veltri2018), we reconstruct EBP's trajectory, providing a critical assessment of its historical contours and theoretical underpinning. In practice, EBP faces critique for neglecting the fundamental divide between the time horizon of research and that of policy: By conflating the two, it transforms research into policy-based evidence making (Strassheim and Kettunen, Reference Strassheim and Kettunen2014), or artificially produced ignorance (Saltelli and Giampietro, Reference Saltelli and Giampietro2017), because evidence merely becomes another rhetorical weapon.
One proposed alternative is the democratization of knowledge, which refers to citizen mobilization and civil society's potential provision of policy-relevant information. We label this solution deliberative governance. A version of this approach entails a reframing of the interactions between experts, policy makers, and citizens, used by Carrozza (Reference Carrozza2015) with particular emphasis on the domain of environmental policy. Carrozza traces the deliberative turn in the field of science and technology studies (STS), and further characterizes it as a call for the democratization of expertise in reaction to a “the tendency of experts to join the political fray and participate in public debates supporting specific policy options through the use of scientific discourse” (Reference Carrozza2015: 110).
Nevertheless, the deliberative turn has also been the object of criticism, especially in the wake of the post-truth hypothesis, holding that social media accentuate the propensity to look for confirmation of a priori conjectures and other forms of reinforcing individual identities, amplifying humans’ bias to selectively update their prior beliefs (confirmation bias, Sperber et al., Reference Sperber, Clément, Heintz, Mascaro, Mercier, Origgi and Wilson2010).
Additionally, we explore a third option. That is, we consider situations in which parties negotiate, becoming more selective in presenting relevant arguments for public debate. We refer to this alternative as negotiated conflict.
In fact, selecting citizens or experts because they more accurate could be a good solution for a one-dimensional problem, i.e. when we introduce strategic interests, epistemic uncertainty, and value conflict, but everything else is kept constant. Public policy, though, entails situations in which two or three of the problems occur simultaneously.
We think that in this case, public policy could also try to make both citizens and experts more sensitive to each other and less sensitive to epistemic uncertainty. This is a similar argument to the economic logic of an adversarial judicial system: By having to deal with a non-professional audience, prosecutorial and defense counsel are very sensitive to cross-interrogation and preselect witnesses by virtue of their potential to “survive” cross-interrogation (Posner, Reference Posner1999). In a similar fashion, we may think of a situation in which parties have to negotiate, becoming more accurate in selecting the relevant arguments in public debate. This sort of negotiated conflict is very similar to the standard corporatist logic of social democracies, and we think that this alternative warrants exploration.Footnote 5
In the literature, the concept of negotiated conflict has been developed in the reflection over the Scandinavian system. In particular, it is argued that instead of independent decision-making by private agents or hierarchical decision by bureaucratic or elected bodies, this system is based on negotiation by independent bodies, interacting without the use of legal sanctions and through a process where the shaping of preferences is a decisive factor (Nielsen and Pedersen, Reference Nielsen and Pedersen1988). In order words, it highlights the importance of coordination mechanisms such as communication and negotiation, as alternative to command and market (Nielsen, Reference Nielsen1992). In this perspective, negotiated conflict can be conceived as an implementation of the instrumental criterion into a framework of reasonable value and price (Commons, Reference Commons1934). In other words, an institutionalist interpretation argues that what works should be measured with regards to getting closer to the set of outcomes that would obtain “if everyone actually possessed equal power to wait for the other to give in, that is, equal bargaining power” (Ramstad, Reference Ramstad1991: 434).
In our stylized model, arrangements are introduced in the following way. Evidence-based policy consists of designing a system of incentives to make experts more cautious and request their opinion. Deliberative democracy occurs when citizens are directly consulted after being offered a proper system of incentives.
In the model, we consider each institutional arrangement a contract in which one of the equilibrium responses is picked (the experts and the citizens for, respectively, EBP and deliberative democracy) and a system of incentives is in place (a negative incentive, i.e. a fine, is applied if detected).
Given the model's stylized nature, it does not allow for the introduction of negotiated conflict through the modification of a policy parameter. Therefore, we change a baseline parameter of the model, making the citizens and the experts more sensitive to each other.Footnote 6
Prescriptive analysis under specific domains
To conduct the analysis we go back to the numerical simulation. The baseline calibration becomes:
$$k^e = 1,01,\; k^c = 1.05$$We then define the following policy experiments:
• Evidence-based policy: run the simulation with (S e = 7.5, S c = 1) and select the equilibrium value of information by experts;
• Deliberative governance: run the simulation with (S e = 5, S c = 3) and select the equilibrium value of citizens;
• Negotiated conflict: run the simulation with (S e = 6, S c = 2, k e = k c = 0.5) and select the average equilibrium value (0.5 x i* + 0.5 x j*).
The four panels presented in Figure 4 represent four scenarios in which the three governance/institutional frames (deliberative governance, EBP, and negotiated conflict) perform differently in terms of information accuracy across the policy space presented in section 3 (subsection entitled ‘Mapping policy issues’). These simulations support the following conclusions.
Comparative analysis of evidence-based policy, deliberative governance, and negotiated conflict
First, it is important to stress that as we move toward the most complicated areas of the policy space (i.e. where the sources of tension increase simultaneously), information tends to become inaccurate and all parties involved cease to be reliable.
The deliberative frame performs best in the context of Panel A, which is characterized by high strategic interests and their grip on experts. In such a context, when experts are at risk of being influenced by structured strategic interest groups, the deliberative frame moderates such risks.
The EBP frame is best suited to the scenario portrayed in Panel B, when in a condition of epistemic uncertainty and value conflict, the public can be misled. However, it should be kept in mind that, in this situation, EBP remains reliable, even if it may not represent the best solution for certain parameter values.
In conditions of both presence of value conflict and strategic interests (Panel C), the deliberative governance and EBP frames do not produce the best outcome; however, a negotiated conflict frame does. In the case of Panel D, there is a highly conflictual situation with low consensus and high strategic interests. In this context, while the general outcome is a low equilibrium of information accuracy, the deliberative governance approach performs better than the others, followed by the negotiated conflict and then EBP.
5. Concluding remarks
In this paper, we contribute to the current critical debate on the EBP paradigm and, more generally, to the debate on the relationship between science and policy by simultaneously considering three dimensions: uncertainty, interests, and values.
Using this framework, we assess different institutional arrangements (EBP, deliberative governance, and negotiated conflict) for policy makers’ use of evidence that is produced by either experts or citizens in order to pursue an efficiency criterion defined by means of a cost–benefit analysis. This model demonstrates the lack of a one-size-fits-all solution, instead revealing that institutional arrangements perform differently depending on the configuration of the problem under consideration.
From a policy perspective, our paper brings to the fore an important institutional arrangement overlooked due to the polarized dispute between supporters of a more technocratic EBP approach and supporters of deliberation and citizen science. The arrangements we have referred to herein as negotiated conflict may produce more policy-relevant and reliable evidence under certain configurations than the other two arrangements studied. We also show that this negotiated conflict arrangement has its own foundations in terms of institutionalism and can be traced back the reformistic approach of Scandinavian social democracy.
On a final note, this paper has some limitations, though these are the source of potentially fruitful inquiry. First, it is important to clarify that we look at the citizen–expert interaction, while leaving aside the political economy of the relationship between science and politics. As a result, although it may look similar, this is not close to the more classical approach such as that of Habermas (Reference Habermas and Habermas1976), which put forth three arrangements of decisionistic, technocratic, and pragmatic policy making, according to the balance of power between government and scientists.
Second, the behavioral assumptions concerning relevant actors may be further developed, endogenizing some of the primitives of the behavioral choices. For example, one can think that rents are actually searched for by agents and that polarization is endogenous to the choice of agents to pursue rents. Similarly, one could discuss entry and exit in the “expert” market (Koppl, Reference Koppl2018).
Author ORCIDs
Francesco Bogliacino, 0000-0001-8874-8846.
Acknowledgments
A previous version of this paper circulated under the title “Frames of governance: Comparing institutional arrangements over the policy space.” Some ideas behind this model have been presented in the sixth chapter of Codagnone et al. (Reference Codagnone, Bogliacino and Veltri2018; in Italian), we thank all those that commented on the book, in particular Andrea Saltelli, Pietro Ortoleva, and Leonardo Pejsachowitz. We would like to thank those who commented on previous versions of this contribution, and participants at the Homero Cuevas Lectures in 2016. This version has greatly benefitted from the comments of the editor and three anonymous referees. A special thanks to Joseph Wager for revising the final text.
Appendix A
A.1 The value of information
δ < γ implies 
$1 + \displaystyle{{1-p} \over p}\displaystyle{{\eta ^i} \over {1-\epsilon ^i}} \lt 1 + \displaystyle{{1-p} \over p}\displaystyle{{1-\eta ^i} \over {\epsilon ^i}}$ and thus 
$\displaystyle{\eta \over {1-\epsilon ^i}} \lt \displaystyle{{1-\eta ^i} \over {\epsilon ^i}}$, which by simple algebra implies ε i + η i < 1. To prove that the condition is also sufficient, just notice that ε i + η i <1 implies both η i <1 − ε i and 
$\displaystyle{1 \over {1-\eta ^i}} \lt \displaystyle{1 \over {\epsilon ^i}}$, which together imply 
$\displaystyle{\eta \over {1-\epsilon ^i}} \lt \displaystyle{{1-\eta ^i} \over {\epsilon ^i}}$. The rest follows, by running the previous argument backwards.
If ε i + η i >1 by simple algebra (1 − p)η i >(1 − p)(1 − ε i). Rearranging, we get (1 − p)η i + p(1 − ε i) >(1 − ε i), which implies 
$\displaystyle{{\lpar {1-\epsilon^i} \rpar } \over {\lpar {1-p} \rpar \eta ^i + p\lpar {1-\epsilon^i} \rpar }} \lt 1$ and thus γ < p. Using a similar argument (1 − p)εi >(1 − p)(1 − η i), and by simple algebra δ > p.
A.2 Proof of Proposition 1
Existence is straightforward. First of all, notice that an equilibrium is a fixed point of (x i*(x j); x j*(x i)). Since x i* ∈ [0, 2] by definition of the rates of false positive and negative, x i*(x j*) maps from a compact convex subset into itself, it is single valued by Assumption 2, and continuous by the Maximum Theorem. Existence follows by Brower Fixed Point Theorem.
Now look at the First Order Conditions:
$$\displaystyle{{\partial m^i\lpar {x^i;\; \mu} \rpar } \over {\partial x^i}}-\displaystyle{{\partial p^i\lpar {x^i,x^j;\; \sigma} \rpar } \over {\partial x^i}} + \displaystyle{{\partial v^i\lpar {x^i;\; \alpha} \rpar } \over {\partial x^i}}-\displaystyle{{dc^i\lpar {x^i} \rpar } \over {dx^i}} = 0\; \lsqb 2 \rsqb $$Under Assumption 2’ the solution is unique because the Second Order Conditions (SOCs) are negative, making the problem globally concave.
To prove uniqueness we also need to prove monotonicity of x i*(x j). We can use the Implicit Function Theorem around [2], getting:
$$\displaystyle{{\partial x^{i{^\ast}}\lpar {x^j} \rpar } \over {\partial x^j}} = \displaystyle{{-\displaystyle{{\partial ^2p^i\lpar {x^i,x^j;\; \sigma} \rpar } \over {\partial x^i\partial x^j}}S} \over {-SOC}},\; \lsqb 3 \rsqb $$
which is positive by Assumption 2. Formally, to guarantee stability of x i(x j(x i)), we need the first derivative to be lower than one, i.e. 
$\left\vert {\displaystyle{{dx^i} \over {dx^j}}\displaystyle{{dx^j} \over {dx^i}}} \right\vert \lt 1$, which by the Inverse Function Theorem means that in Figure 1, we need the BRF of the experts to cross the BRF of citizens from above. Given monotonicity and uniqueness, this will always happen unless the response function has a negative intercepts, but this holds because of x i* ∈ [0, 2].
A.3 Proof of Proposition 2
By applying the Implicit Function Theorem, and using Assumptions 2 and 3, we can derive:
$$\matrix{ {\displaystyle{{\partial x^{i{^\ast}}\lpar {x^j} \rpar } \over {\partial \alpha}} = \displaystyle{{\displaystyle{{\partial ^2v^i\lpar {x^i;\; \alpha} \rpar } \over {\partial x^i\partial \alpha}}} \over {-SOC}} \lt 0} \cr {\displaystyle{{\partial x^{i{^\ast}}\lpar {x^j} \rpar } \over {\partial \mu}} = \displaystyle{{\displaystyle{{\partial ^2m^i\lpar {x^i;\; \mu} \rpar } \over {\partial x^i\partial \mu}}} \over {-SOC}} \gt 0} \cr {\displaystyle{{\partial x^{i{^\ast}}\lpar {x^j} \rpar } \over {\partial \sigma}} = \displaystyle{{-\displaystyle{{\partial ^2p^i\lpar {x^i,x^j;\; \sigma} \rpar } \over {\partial x^i\partial \sigma}} S} \over {-SOC}} \gt 0} \cr} \; \lsqb 4 \rsqb $$
