The analysis of collective choice procedures—from small groups and local institutions, to democratic states—is one of the major preoccupations of political and social science. Often, there are more or less explicit assumptions that such procedures should be rational, informed, fair, equitable, or in general “better”. We think about what the interaction between rational representatives and citizens should look like, and build appropriate models. We analyze election systems and committee voting rules in terms of their fairness, representativeness, or with respect to the outcomes they produce; we worry about the skewed composition of many parliaments, about the quality of public political debate, or the lack of information citizens have when making political choices. Behind all these issues lie normative theories of collective decision-making, theories that tells us what goals we should aim for when designing such institutions: fair outcomes, for instance, or efficiency, or perhaps accuracy of judgment. Only on this basis, then, can we analyze and evaluate the causal effects of different rules, procedures, and decision mechanisms.

Jon Elster’s most recent book, “Securities Against Misrule: Juries, Assemblies, Elections” argues that successful institutional design—at whatever level of complexity—requires us to get both of these dimensions right. Taking his cue (and the title of the book) from Jeremy Bentham’s writings on political rules and procedures, Elster attempts to provide a relatively universal answer to the question of what we can and should aim for when designing collective decision-making procedures, and how to look at institutional mechanisms on this basis.

However, as the book shows, these questions are enormously difficult and do not admit of easy answers. The great achievement of this book is that it does not shy away from those problems but takes into account the cognitive and moral limitations of collective decision-makers. Perhaps even more importantly, the book also takes into account the limits of what the political and social sciences can actually reliably tell us about the causal and normative dimensions of collective choice. Hence, the message of the book may seem somewhat negative: what we need to do, Elster argues, is scale back our ambitions. What we can do is to try to prevent the worst abuses and distortions of the choice process through institutional design, but beyond that, as in John Hart Ely’s phrase quoted in the book, we must “let the chips fall where they may”.

The book therefore contains two parts: a rejection of more ambitious normative theories of institutional design, as well as an outline of Elster’s minimalist alternative and its concrete implications. These are illustrated with a staggering variety of examples drawn from many spheres of collective decision-making.

The major part of the critique is targeted at instrumental theories of collective choice that propose to design decision procedures that tend to produce the right outcomes. Elster argues that such a project must fail because of what he calls a “double indeterminacy—of plausible-sounding but unprovable normative views, and of plausible-sounding but unprovable causal theories” (4). The problem is this: in order to improve the outcomes of a decision procedure, we first need to know, in substantive terms, what a good outcome would be. Second, we need to know which institutional rules and arrangements would tend to produce those outcomes. A major part of the book is concerned with showing that our knowledge with respect to either question is radically limited. Social choice theory, of course, has long suggested that “the will of the people” cannot serve as a standard here. Collective preferences may be non-transitive, which means that no uniquely determinate way of aggregating individual preferences may exist. More objective standards, such as the “justice” of a decision similarly remain unavailable, since there is (at least for now) an irresolvable pluralism of theories of what constitutes a just outcome.

Even when it is entirely clear what the procedure should do, we still face a problem. An example that comes up several times in the book is this: how should we design the rules for criminal trials? (17) How should juries be composed, how should they deliberate, or should we have juries at all? It seems obvious that the procedure should be designed to maximize the number of correct verdicts. However, there is a logical problem here: we have no independent access to the guilt or innocence of the accused; and hence we cannot know which procedures will be best at determining it. In order to compare different judicial arrangements we would need an objective measure of correct and false verdicts––of course if we knew that, we would not need to hold any trials at all. Whenever we face such uncertainty about outcomes, we cannot in principle tell which institutional design would produce better outcomes.

Elster’s alternative is what he calls a theory of impure procedural justice. We should focus on improving the intrinsic qualities of the procedure itself. In contrast to Rawlsian pure procedural justice, however, we should not attempt to approximate a fully fair procedure. Such a project would mean that, once again, we face indeterminacy as to what arrangement would count as fair in each instance. Instead, what we can do is try to purge the procedure of the worst misbehaviors, irrationalities and biases. We can engage in negative institutional design, aiming not for justice or fairness, but merely implementing the “securities against misrule” of the title.

The author calls this method normative behavioural theory. It means that we start from the behavioural characteristics we find in voters, jurors, or politicians, and design our institutions on this basis in a way that minimizes the distorting influence of selfishness, bias, or irrationality on the decision process. More precisely, Elster argues, we should focus on “causal factors that can have a distorting influence on decisions reached by any procedure and judged by any criterion” (84). For example, if a procedure allows cognitive bias to influence decisions, then we can expect the decision to be bad on any understanding of what a good decision procedure should do.

In the central parts of the book, this basic theory is operationalized on the basis of Bentham’s idea that collective decision procedures ought to maximize the official aptitude of decision-makers. This consists of moral aptitude (their impartiality and incorruptibility), intellectual aptitude (their capacity to find effective means to their ends), and their active aptitude (the time and effort they are willing and able to expend on the decision). Bentham and Elster agree that these are desirable, whatever our goal may be. This means that, despite the indeterminacies of evaluating collective choice, it is always better that those taking the decision should remain impartial and rational, have access to the appropriate knowledge, and spend the bulk of their time and energy actually dealing with the problem. The meticulous and amazingly comprehensive analysis of how different institutional designs can promote or reduce this aptitude, across all sorts of settings, is where the book really shines. Elster considers ideas from the apparently mundane—such as whether representatives ought to sit on benches or behind desks—to the highly complex—such as how to find the adequate balance of secrecy and publicity for contexts of deliberation and voting.

The book contains a wealth of subtle insights for anyone interested in the actual practical issues of how to decide how to (collectively) decide. The unjustly neglected Benthamite project of removing through institutional means the obstacles to official aptitude of voters, jurors, constitution-makers, or legislators is as important as it is compelling.

Nevertheless, one may accept all that while wondering if the negative part of the theory––the rejection of more ambitious ideas about structuring collective choice situations––is equally compelling. In the following paragraphs I will focus, first, on the question of whether we really have no available independent standards to assess the outcomes of collective decision-making and, second, on the question whether the Benthamite project itself does not implicitly contain some outcome standard, albeit a minimal one.

The jury example cited by Elster is a strong one, and indeed it seems that indeed we have no way of reliably assessing the epistemic performance of juries or judges. There is no independent way of verifyng whether verdicts are correct or not (unless in those cases where previously unknown exonerating or incriminating evidence happens to be discovered). However, in many ways, the courtroom is a special case. After the verdict is rendered, there is no “revelation” of whether it was right or not. In other forms of inquiry, though, it may be possible that such an ex post facto assessment is available to us. For instance, we can verify the forecasting accuracy of weather channels, economics experts, political pundits, or even Congress. We could consider a study such as Philip Tetlock’s Expert Political Judgment, for instance.^{Footnote 1} Tetlock’s conclusion that experts tend to be relatively bad at forecasting is possible only because we can verify the accuracy of their predictions after the fact. Given that much of collective choice activity (outside of the courtroom) involves predicting the effects of certain decisions, perhaps we may, at least in principle, learn from their past success or failure which institutional mechanisms enable more reliable forecasts than others. For example, we may come to think that randomized controlled trials (RCTs), with all their limitations, enable us to make relatively reliable predictions about the likelihood of success of certain decisions.^{Footnote 2} Accordingly, we may think that collective decision-making should involve the use of RCTs wherever possible. Now of course the use of RCTs in decision-making may actually be an “all purpose” tool and as such be a part of what Elster calls intellectual aptitude, but in that case the lines between impure proceduralism and outcome-oriented criteria become blurred.

Even with respect to the normative goals of collective decision-making, the problem may be somewhat less severe than Elster argues. It is true that there is deep disagreement about what an ideally just society would look like, and it seems problematic to assess a procedure’s tendency to produce just outcomes even if we have a record of its past decisions. Like a defendant’s guilt or innocence, it seems we cannot verify a policy’s degree of justice, even after it has played out.

However, a lack of agreement on what justice means does not necessarily imply that we cannot agree on a partial ranking of states of the world. There may be outcomes, for example, that are considered unjust from any theory of justice people have and, in Amartya Sen’s terms, we may be able to agree on a relatively small maximal set, even though which element of the set is optimal is indeterminate.^{Footnote 3} Hence, while we may not be able to find a determinate complete ranking of possible outcomes, we may be able tell when a policy has removed or ameliorated a particular injustice. For example, the fact that children have to die from curable diseases may be considered a grave injustice on any plausible account of what we consider a just outcome. Hence, the avoidance of this, and of similar “universal bads” could be such an instrumental standard. Of course it is still difficult to determine which decision procedures reliably avoid such outcomes, but at least it is not in principle impossible.

There is therefore a fundamental difference between the courtroom and other collective choice situations: jurisprudence is not a diachronic endeavor, and we cannot try out verdicts or render them only hypothetically. But we can try out policies or other collective choices. While typically we cannot tell whether they were “right” in a more demanding sense, such as whether they were optimal all things considered, in some rather moderate sense we can often tell whether they have worked or not: have they alleviated a grave injustice? Have they resolved a particular political conflict? Or even more modestly: has the policy achieved what was intended?

This, however, leads me to the second point. The Benthamite project is actually not that far away from such a moderately outcome-focused view. I say this because it seems to me that the recommended strategy of blocking the inhibitions on moral, intellectual and active aptitude cannot actually do without an outcome standard. The reason is that, in concrete situations, often what counts as aptitude depends to some extent on what we want to achieve.

One essential part of intellectual aptitude for Elster is the absence of cognitive biases (86), such as anchoring bias, hindsight bias or the “planning fallacy”. The idea behind this is that whatever we want to achieve in our collective choice, we want decision-makers to form beliefs on a rational basis, and hence it will be advantageous if they are not too greatly affected by these fallacies of reasoning.

Now consider this simple example: should public health insurance cover homeopathic treatments? Many people believe, against overwhelming evidence, that homeopathic medicines are effective (beyond the placebo effect) in treating diseases. This seems like a prime instance of confirmation bias, in which confirming evidence (for example anecdotal evidence from friends) is accorded great weight while disconfirming evidence (from medical trials) is disregarded. Should our collective decision-making process shield against this confirmation bias? If our goal is to make the public health system cost-effective and balance the different interests involved as objectively as possible, then we should do so. On the other hand, many people appreciate homeopathic treatments even if there is no evidence for their effectiveness. If our goal is to find collective decisions people can feel good about, we may want to treat the widespread demand for homeopathy, irrational as it may be, as a preference and not as an instance of bias to be prevented. If on the other hand the goal is merely to do what meets with immediate public approval, why bother with improving official aptitude at all?

What this example is supposed to show is that the Benthamite project of bias prevention is implicitly tied to an understanding of public decision-making as efficient and rational problem solving, which is, it seems to me, an outcome standard that is compelling but not completely neutral. This does not take away from the usefulness of the project. However, it suggests that it is closer in spirit (and method) to other attempts at evaluating political and other collective decision procedures in instrumental terms—at least those that acknowledge the uncertainty and indeterminacy involved—than Elster seems to believe.