1 INTERESTS, KNOWLEDGE, EXPERTS

How ought a democratic process to make decisions, if it is to sustain democracy's core values and promote citizen interests? One influential answer is to aggregate the preferences (over representatives or policies) of free citizens by counting their equally weighted votes. Robert Dahl (1989, Reference Dahl1998), among others, has argued, against Plato and other epistemic regime theorists, that democratic values are preserved, and citizens’ interests advanced, when policy is set by a majority of voters whose preferences express their own opinions about their own best interests. Dahl's approach preserves liberty, political equality, and dignity by asserting that each individual voter is the best (even if necessarily imperfect) judge of his or her own interests, and that a majority of such individually-chosen interests, expressed as equal votes, deserves to be established as state policy. Yet the majority's preferences, even if they do track the real interests of the majority, will often fail to reflect the interests of all citizens. It is because the preferences of a majority might ignore or harm the most basic interests of individuals or of a minority that, in liberal democracies, certain fundamental interests are legally protected as rights.

Classical theorists approached the matter of preserving values and fostering interests somewhat differently. Aristotle regarded advancing the special interests of a majority in a democracy (or of an empowered minority in an oligarchy) as unjust, if it came at the expense of promoting common interests. He supposed that a just community would identify common (rather than partial/factional) interests and, by appropriate use of practical wisdom (phronēsis), would select policies most likely to advance those interests. Political decision-making, for Aristotle, was an epistemic endeavor in that it was meant to discover the best answers to questions of appropriately-shared concern. If we are willing to accept the common (although not universal) thoughts that (1) people do have real interests (and not merely preferences) and (2) that some interests are in fact shared by some members of a community, there is, by extension, no reason to reject a priori the possibility that certain interests (e.g. in security, fair rules, welfare) could be so widely shared as to be reasonably described as commonly held. Moreover, one need not embrace Aristotelian eudaemonism to regard pursuing certain common interests as a normatively choice-worthy political goal, or to suppose that practical wisdom is equal to the task of identifying certain interests as shared. For the purposes of this article, a superior common-interest choice is one that selects the available option that, all things considered, best advances an interest shared by the members of a community.Footnote ³

I do not propose that we follow Aristotle in assuming that politics can or ought to be reduced to a search for the best answers about common interests and their advancement: Many political matters involve hard choices among conflicting social values; many other matters concern the interests only of certain persons. Yet it seems implausible to say that politics never is, and never ought to be, concerned with interests that are reasonably held to be shared. Democratic politicians, in antiquity (Ober Reference Ober1989) and modernity alike, frame proposals as promoting the common interest; their rhetoric is not empty insofar as it acknowledges a shared conviction that addressing common interests is at least part of what politics ought to be about. Common interests might, hypothetically, be identified by consensus, but, for my purposes, consensus is unnecessary. Majoritarian mechanisms may be employed to identify and advance a common interest, so long as the majority decision actually represents a superior common-interest choice, as defined in the previous paragraph. It is implausible to suppose that majority votes always (or often) accurately identify or advance common interests. It is sufficient for my argument that certain interests are sometimes shared and, in such cases, there is such a thing as the better policy. Although common and partisan interests can never be fully disentangled, focusing on shared interests limits the subjectivity of decision-making and thereby allows the performance of decision-making groups to be evaluated (Yates and Tschirhart Reference Yates, Tschirhart and Ericsson2006).

To be plausible on the face of it, any approach to epistemic democracy must address the challenges of transitivity (and thus of cycling), collective rationality, and elite control. Options, in order to have stable outcomes, must be transitively ordered, such that if A > B and B > C then A > C.Footnote ⁴ Collective decisions made through the participation of many individuals, including decisions about common interests, can be regarded as the rational judgment of a collective agent without reference to metaphysically mysterious conceptions of agency only when decisions are made under the right conditions (List and Pettit Reference List and Pettit2011). Finally, by definition, epistemic democracy must decide matters democratically, avoiding capture by elites. I will hope to show that Relevant Expertise Aggregation, the “middle-way” decision-making process described here, addresses the challenges of transitivity, agency, and elite capture as well or better than deliberation and independent guess aggregation in their conventional forms.

Insofar as democratic politics is a means for choosing which available option best serves a shared interest, it involves the use of knowledge (accurate information, true beliefs) for discovering the best option. Given indeterminacy and contingency, this cannot mean “identifying the approach that infallibly achieves a common interest.” It must mean instead, “choosing the option among available alternatives that has the best chance, all things considered, of advancing a common interest.” Two fundamental premises of epistemic approaches to democracy are that (1) there are better options, in that the chances of a good outcome are better if that option is chosen, and (2) decision-makers can, under the right conditions, identify better options.Footnote ⁵

Options are likely to be better insofar as they take fuller account of relevant facts about the world. Reality-tracking is valued (in this context) because of its consequences: All other things being equal, options that take account of the relevant facts are more likely to lead to a better outcome, all things considered, than those that do not. In this article, I am not concerned with assessing the moral status of interests pursued by a democratic community (e.g. by weighing the shared interests of a community against global interests or universal human rights). Nor am I concerned with the normative value of epistemic democracy relative to, e.g., agonistic pluralism (Honig Reference Honig1993; Lacau and Mouffe Reference Laclau and Mouffe2001). Nor, finally, am I concerned to show that epistemic democracies will invariably out-perform autocracies. I am concerned with designing institutional mechanisms that enable citizens in a democracy to make relatively better decisions, and thereby advance common interests, while sustaining core values.

Epistemic decision-making is necessarily concerned with expertise. Experts in a given domain (say, chess masters) are more capable than others at producing a desired outcome (winning) and the probability of achieving the outcome is increased by better choices (good moves). The Callipolis of Plato's Republic is an example of an ideal epistemic regime in which rulers are experts. Their choices accurately track the Form of the Good and thereby produce a just society. Callipolis is, however, neither realistic nor democratic. It is undemocratic because a few experts rule without consulting the other citizens. It is unrealistic because it assumes the existence of general, architectonic political experts. The philosopher-kings are “architect-like” in their master knowledge of a blueprint that perfectly directs the actions of all those whose work (and subsidiary expertise) is necessary to create and sustain a just society. Absent a Form of the Good to which some individuals have privileged access, there is no reason to believe that such general political experts exist.Footnote ⁶

Politics is unlike domains of endeavor in which individuals achieve true expertise (say, chess or violin playing: Ericsson Reference Ericsson and Ericsson2006) in its level of complexity. There are, ex hypothesi, no general experts in politics because, lacking access to the Form of the Good (or some similar metaphysical resource), such experts would need to master a range of hard-to-acquire specialized expertise that exceeds the bounds of human cognitive capacity. It is, however, highly plausible to suppose that there are true experts in many domains relevant to political decision-making (Ericsson et al. Reference Ericsson2006). Domain-experts may come to believe, wrongly, that they are general experts. The catastrophic results that can follow when political authority is ceded to domain-experts are well documented (Scott Reference Scott1998). But this ought not be a reason for excluding expertise in relevant domains from democratic decision-making. The goal of Relevant Expertise Aggregation is to bring relevant domain-specific expertise into the process of decision-making without ceding political authority to experts.Footnote ⁷

REA captures some of the ways by which better options were chosen in antiquity and are chosen in modernity. While there are epistocratic versions of REA (REA I: fourth section, below), other versions (REA II and III: fifth and sixth sections, below) are democratic in retaining the principle of equal votes and (REA III) in resisting elite capture through agenda control. REA is Aristotelian in that it is inspired by and elaborates upon passages in Aristotle's texts. It is not fully specified in the texts but it is, I believe, a plausible extension of Aristotle's thinking on political science. An approach to collective judgment closely resembling REA II and III was, I will argue, the basis for Aristotle's celebrated claim that, under the right conditions, the many may be wise: a better judge of some matters than an excellent individual or small group.Footnote ⁸

The conditions necessary for REA are demanding, but realistic: (1) Decision-makers addressing an issue seek the best available option. (2) The issue is divisible into parts, each of which has a specifiable (relative to other parts) relevance to the issue at question. (3) Each of those relevant parts is explicable as a domain of knowledge that can be enlightened by identifiable domain-experts willing and able to disclose private information.Footnote ⁹ Certain forms of REA require a further assumption that (4) conditions 1–3 are common knowledge in a group that exists over time such that its decisions take the form of a repeated game, and that (5) its members update beliefs about experts in overall reality-tracking ways.Footnote ¹⁰

2 DELIBERATION, INDEPENDENT GUESS AGGREGATION, AND MIDDLE-WAY DECISION THEORY

The primary mechanisms discussed in the epistemic democracy literature are deliberation (DEL), understood as a form of reciprocal reason-giving among citizens (Cohen Reference Cohen and Benhabib1996; Gutmann and Thompson Reference Gutmann and Thompson2004), and independent-guess aggregation (IGA), understood as a method for mathematically aggregating votes with a specified probability of being correct (Condorcet Reference Condorcet1785; List and Goodin Reference List and Goodin2001). DEL and IGA have real-world applications. DEL has been put into practice as a method of polling and for making political decisions (Fishkin Reference Fishkin2009; Baiocchi Reference Baiocchi2005; Warren and Pearse Reference Warren and Pearse2008). IGA includes prediction markets (Sunstein Reference Sunstein2007), but the primary mechanism considered here is the Condorcet jury, in which (in its canonical form) jurors make independent judgments and the likelihood of each voter choosing correctly on a binary decision is assumed to be better than half. As the number of votes increases, the majority choice is ever more likely to be correct.

In a spectrum of approaches for organizing diverse information and dispersed knowledge for the purposes of democratic decision-making, DEL and IGA, in their strong conventional forms, stand at opposite poles: DEL assumes non-strategic exchange of information and reasons, and regards that interaction as necessary for achieving better answers and outcomes. DEL values updating by decision makers, on the basis of new information and reasons offered by others, both as a means to achieving better outcomes and as an expression of the values of equality and reciprocity. Canonical forms of IGA assume voter independence – there is no pre-decision information-sharing. Independence is valued as preserving freedom of individual choice, but also because it prevents the informational cascades (group-think) and polarization (extremism) that have been associated by Cass Sunstein, among others, as inherent anti-epistemic features of deliberation (Sunstein Reference Sunstein2000, Reference Sunstein2002; cf. Mendelberg Reference Mendelberg2002; Mutz and Martin Reference Mutz and Martin2001). Pre-decision communication among decision-makers, in ways that violate the independence of their individual choices may be taken as a source of corruption (List and Pettit Reference List, Pettit, Catton and Macdonald2004). Yet, in the real world, choices are rarely, if ever, truly independent: The presence of opinion leaders can compromise free speech, thereby making votes dependent on a limited number of schools of thought (Ladha Reference Ladha1992). Moreover, jurors in a courtroom vote for guilt or innocence on the basis of the evidence they have all heard, rather than on their independent knowledge of the state of the world. If the evidence is misleading, or if jurors are incapable of assessing relevant evidence, the classic Condorcet result, in which certainty of correctness is approached as the size of the jury increases, will be weakened (Dietrich and List Reference List, Pettit, Catton and Macdonald2004). These considerations point to the limitations of IGA, but do not obviate its value in enabling groups to make better-than-random choices among options. Real-world IGA does not offer infallibility, but democratic theorists often value fallibility, especially when the recognition of human fallibility is conjoined with experimentalism (Schwartzberg Reference Schwartzberg2008; Knight and Johnston Reference Knight and Johnson2011).

In some recent empirical work on decision-making (Sunstein Reference Sunstein2007), DEL and IGA are set up as incompatible alternatives, to the detriment of DEL. By contrast, John Dryzek and Christian List argue persuasively that deliberation, rather than being incompatible with aggregative approaches to democratic decision making, actually offers a solution to the challenge posed by social choice theorists (e.g. Riker Reference Riker1982), who claim that the democratic aggregation of views, interests, or preferences across individuals is “bedeviled by impossibility, instability, and arbitrariness” (Dryzek and List Reference Dryzek and List2003, quote: 2). Deliberation has subsequently been incorporated into work on Condorcet juries (Gerardi and Yariv Reference Gerardi and Yariv2007; Elster Reference Elster2011). This article contributes to the “middle way” literature of reconciliation between DEL and IGA by drawing from ancient Greek political thought and practice.

Reconciliation is especially desirable because neither DEL nor IGA, in their canonical forms, fully suits the needs of a democratic community that must make complex, highly consequential, time-sensitive choices. IGA suffers from a democratic deficit in that it is predicated on agendas (issues and options) that are set exogenously. Lacking the opportunity to deliberate, the voters constituting the group cannot themselves determine the issues on which choices must be made, or the options from among which they will choose. Nor can they, as a group, establish rules governing issue selection or the option menu. The issues and available options must ordinarily be presented to voters by some external authority.Footnote ¹¹ Insofar as it sticks by its own premise of voter independence, IGA therefore comes with external (presumptively elite) agenda control built in.

Under DEL, issues and options may be set by an external authority. An example is a deliberative poll in which the issue and options are set by the authorities commissioning and/or conducting the poll (Fishkin Reference Fishkin2009). But a group employing DEL also has the capacity to set its own agenda: It can decide what issues to take up, and what options to choose among. It can establish rules governing what issues will be taken up when, and what options will be available on a given issue. Strong forms of DEL require equality of deliberative opportunity and are committed to rules of neutrality. As such, even when it does not require complete consensus, the conventional form of DEL lacks a mechanism for closure (when have enough reasons been offered to holdouts?), and thus cannot offer a practical way forward in time-sensitive decision contexts without violating its own premises.

In real-world democracies, citizens must be able to set their own agendas (or democratically establish rules for doing so) and must make decisions under time constraints. Middle-way theory potentially answers those demands, by relaxing certain assumptions on which DEL and IGA are predicated. REA may be thought of as a variant of DEL since some versions of DEL do incorporate voting (Fishkin Reference Fishkin2009; Warren and Pearse Reference Warren and Pearse2008). Or it may be regarded as a variant of IGA, since some Condorcet jury models do allow communication among voters (Gerardi and Yariv Reference Gerardi and Yariv2007), and allow for division of issues into parts (Bovens and Rabinowicz Reference Bovens and Rabinowicz2006). The innovative feature of REA is establishing the functional relationship between multiple relevant criteria and the best choice among options through equal relevant-domain-weighted votes based on the reputations and testimony of domain-experts. This article argues that, by shifting the focus from judging complex issues “in the round” to judging experts in circumscribed domains, REA enables a collectivity to make superior common-interest choices, under time-constraint and without elite capture. Table 1 summarizes the relevant features of REA compared to conventional IGA and DEL.

TABLE 1.

Three approaches to epistemic-democratic decision-making

Notes: Endogenous = issues or options are determined by the decision-making group. Exogenous = issues and options are determined by external agency.

Under the right conditions, a middle-way epistemic approach to democracy can answer the challenges of sustaining transitivity and collective agency, while avoiding elite control.Footnote ¹² Yet it still faces procedural hurdles: Determining what issues ought to be addressed and when; defining the options among which a choice must be made; choosing well among those options under time constraint. The distinguishing features of REA, which enable it to clear those hurdles, are that (1) issues are parsed into a manageable number of domain-parts; (2) domains of expertise are weighted according to their relevance to the issue; (3) experts in each domain rank options according to the criteria relevant to that domain. Relevant domains may be established by rules or may be determined by a combination of deliberation and voting. In either case, the number of parts into which the issue is parsed is (ideally) an optimum that balances two aspects of transparency: Including more domains allows domains to be defined more narrowly, producing greater clarity about the expertise relevant to each domain. Yet restricting the number of domains clarifies the relative relevance of each domain to the issue being decided.

The discussion that follows the next section considers three versions of REA. In REA I issues, options, and domains are pre-established by institutional rules and an infallible expert in each domain ranks the options according to the criteria of judgment appropriate to that domain. REA II also assumes pre-established rules, but introduces mass voting by domain, on options, advised by multiple domain-experts, as a solution to the problem of the fallibility of individual experts. REA III assumes that issues, options, and relevant domains are set in a deliberative/voting stage that precedes and is advisory to the final vote. Table 2 sets out the primary features of each version.

TABLE 2.

Three versions of Relevant Expertise Aggregation

3 ARISTOTLE POLITICS 3.11: THE WISDOM OF THE MANY

In a celebrated passage of the Politics (3.11 = 1281a42-b10), recently and skillfully analyzed by James Linley Wilson (Reference Wilson2011) as an example of “deliberative integration,” Aristotle claims that, under the right conditions, “the many” judge certain matters better than any excellent individual or small group.Footnote ¹³ This section fills out Aristotle's bare-bones account of the wisdom of the many in order to show that the preconditions required for Aristotle's approach to epistemic democracy are not so demanding as to relegate the wisdom of the many to the realm of ideal theory alone. Aristotle's account of collective judgment suggests how diverse expertise might be aggregated by a group of democratic decision-makers confronted with a variety of possible answers.

The relevant passage is laid out schematically, below, with clarifying notes in brackets and key Greek terms in parentheses. The subdivision of the passage into eight sections is my own and is the basis of subsequent citations of this passage (flagged §):

1. 1. The many (hoi polloi), of whom none is individually an excellent (spoudaios) man, nevertheless can, when joined together, be better than those [the excellent few].

2. 2. Not [better] as individuals but all together (hōs sumpantas),

3. 3. just as potluck meals (sumphorēta deipna) can be better than those provided at one man's expense.Footnote ¹⁴

4. 4. For, there being many, each person possesses a constituent part (morion) of virtue (aretē) and practical wisdom (phronēsis),

5. 5. and when they have come together, the multitude (plēthos) is like a single person (hōsper hena anthrōpon), yet many-footed and many-handed and possessing many sense-capacities (aisthēseis),

6. 6. so it [the multitude] is likewise [like a single person with multiple capacities] as regards to its facets of character (ta ēthē) and its intellect (dianoia).

7. 7. This is why the many (hoi polloi) judge better (krinousin ameinon) in regard to musical works and those of the poets,

8. 8. for some [judge] a particular part [of the issue] (ti morion), while all of them [together judge] all [of the parts taken as a whole] (panta de pantes).

Politics 3.1281a42-b10. Trans. C. Lord, adapted.Footnote ¹⁵

Aristotle's account is highly compressed. If we are to understand how, in the concluding phrase (§8), “some judge a particular part of the issue, while all of them together judge all of the parts taken as a whole” we must do some unpacking, in light of information familiar to Aristotle's original readers (Wilson Reference Wilson2011: 263–67). Aristotle's point (§1) is that under the right conditions, a group of ordinary people judge better (i.e. choose more reality-tracking options with better expected outcomes) than a few excellent persons. The group achieves its correctness of judgment as a group (§2). Aristotle offers two homey examples (potluck meal, judging music/poetry) of a collectivity achieving a superior outcome. I assume that Aristotle expected each example to be familiar to his original readers, and that this presumed familiarity accounts for some of the compression of the passage.

The first example (§3) is an analogy: the “potluck meal,” to which prospective diners bring different contributions. The potluck meal is, potentially, an excellent whole. We must presume, based on Aristotle's core assumptions about justice as a joint-and-several common good, that the result of a successful potluck meal is a better experience for each contributor than would be the case if a “simple” meal (3.1286a29–30) were provided at one man's expense. The several contributions are the parts that constitute this potentially excellent whole. As such, in order to achieve the meal's potential, each of the parts must be of the right sort. The potluck, as a whole, will go wrong if the parts, the various contributions, are not at once diverse and good. Yet how will diversity and goodness of contributions be assured? If we assume independence of contribution choice (i.e. no diner knows ex ante what another will bring), then each diner may just happen to bring the same thing - if we have six diners, we may end up with six courses of pasta salad. Regardless of the goodness of the pasta salad, this will not be an excellent meal and certainly no better than one provided by an individual. Moreover, potluck dinners are susceptible to free-riding: A free-riding diner might choose strategically to bring something cheap and poor, anticipating that others will bring better fare. If each diner fears being stuck with a sucker's payoff by free-riders, there will be a race to the bottom and the common repast will be correspondingly poor.

If the meal is to be excellent, as Aristotle specifies it can be, the right conditions – the operating assumptions of the contributors to the dinner – must include some common knowledge and some rules in the form of social norms. First: even assuming that they do not deliberate about specific contributions in advance, the multiple contributors must have good reason to expect particular people to bring a particular sort of food or drink. That is to say, they must share common knowledge of one another so as to be able to predict what each is likely to bring to the table. Next, there must be a norm that ensures that each brings something good to the dinner. These are not excessively demanding conditions. Anecdotally: I have gone on picnics with the same group of friends for many years. We seldom prearrange who was to bring what, but we have a good idea of what sort of thing each is likely to bring. Moreover, there is a norm of contributing good things. Per Aristotle's specification, it is arguably a better experience – gastronomically as well as socially – than a dinner provided at one individual's expense. I assume that this outcome is not uncommon among long-lived associations with shared norms and stable memberships. Aristotle's original readers were certainly familiar with the practices of voluntary associations, which were prevalent in classical Greece (Ismard Reference Ismard2010); some will have had experience with potluck meals; others would know of them through reading.Footnote ¹⁶

Aristotle's potluck analogy gets us some way towards understanding what Aristotle was after in the “wisdom of the many” passage. The potluck analogy introduces the issue of parts (contributions) and wholes (the meal). The analogy suggests, moreover, that the optimal number of parts falls within a range: with too few dishes the meal will be excessively simple. Yet at some point it will not be improved, indeed may be made worse, by the addition of further dishes. The analogy also shows why the members of the group must share some relevant forms of knowledge, and why they must share social norms about quality.

Aristotle's second example (§7) concerns judgment of a musical/poetic production: He states as a matter of fact that hoi polloi do judge better (than the few) in regard to “musical works and those of the poets.” Aristotle may have various musical/poetic productions in mind here. But it seems certain that among the productions Aristotle and his original readers had in mind was the performance of drama, and especially tragedy. Tragic theater was, in Aristotle's view and that of modern scholars, the definitive “musical/poetic” venue in classical Athens, where Aristotle lived for most of his working life (Wilson Reference Wilson2000). It is a safe guess that Aristotle's original readers were reasonably familiar with the rules governing the judgment of dramatic performances.

In classical Athens, prior to a Dionysian festival, three tragic poets (and three comic poets) were chosen by a state official to present works for performance. Each tragic poet presented a group of three plays. After the performances, the three play-groups were judged; on the basis of that judgment the poets were awarded first, second, and third prizes. The judging was by a panel chosen by lottery from among the citizens of Athens. In practice, the mass audience of several thousand spectators, by its response to the performances, gave the judges their cue - as appeals for the audience's good will in Aristophanes’ extant comedies clearly demonstrate. It is, in sum, a likely hypothesis that Aristotle had the mass judgment of a theater audience in mind when he stated that hoi polloi judge better than any one individual in regard to musical and poetic works.Footnote ¹⁷

Aristotle specifies (§4) that each member of the decision-making group possesses a constituent part of virtue and practical wisdom. These parts can be effectively aggregated: Aristotle states (§5–6) that a mass can, under the right conditions, be like a single person – yet unlike a single person it possesses a multitude of sense-capacities, and likewise the facets of its character and its intellect (two components of practical wisdom): are multiplied. Aristotle seems to mean that the group possesses among its membership multiple and diverse conjunctions of sensibility, virtue, and practical wisdom, and that these conjunctions are relevant to collective judgment. Individuals manifesting these multiple conjunctions are, therefore, explicable as parts (§4) that, when properly aggregated (come together), produce a right single-person-like decision. Since the example (§7) is judgment of musical/poetic productions, and since (so I have argued), Aristotle had tragedy in mind, some of the multiple conjunctions may be understood as forms of expertise that bear on the judgment of the multiple parts that make up a good tragedy (§8).

Aristotle's point here seems to be that an individual may indeed have the sensibilities, virtue, and practical wisdom to be a very good at judgment of a given part of tragedy. But he will lack some conjunctions of sensibility, virtue, and practical wisdom – some forms of expertise – relevant to judging the whole. A group with the right sort of diversity will possesses all forms of expertise necessary to making a good decision. The group will judge well if it is able to bring the relevant forms of expertise to bear on the constituent parts of the tragedies being judged, while giving each form of expertise the right weight.Footnote ¹⁸ If the account is to avoid metaphysical mysteriousness, the aggregation process must be accomplished without losing sight of the fact that, while it is in some ways like a single person, the group actually consists of multiple individuals. The aggregation is explained unmysteriously if we suppose (as the context of the passage clearly implies) that Aristotle was referring to a decision-making institution. If we assume that Aristotle was in fact referring to the judgment of tragedy in §7, we can fill in his account of the “wisdom of the many” by reference to actual institutional rules, domains of expertise, and a hierarchy of relevance. Analyzing how a group of the right sort, acting as a quasi-person through procedural rules, could correctly rank tragedies, puts some flesh on the bones of Aristotle's account of the wisdom of the many.

4 REA I: COMPLETE RULES WITH INFALLIBLE EXPERTS

Athenian institutions stipulated the rules for ranking (first, second, third prize) three tragic poets based on the performance of three tragedies by each poet. In the Poetics (1450a6–14), Aristotle provides an account of the six parts of tragedy that (for our purposes) can be regarded as additional rules: “Necessarily then every tragedy has six constituent parts (merē), and on these its quality depends. These are plot, character (ēthē), diction, intellect (dianoia), spectacle, and song. . . This list is exhaustive, and practically all the poets employ these elements.” In listing the parts that make up the whole that is a proper tragedy, Aristotle specifies the relevant domains of expertise. The Poetics passage recalls the Politics passage. Both concern character and intellect as natural causes and co-determinants of the quality of an action.Footnote ¹⁹ Moreover, both are concerned with how subsidiary parts (merē, moria: terms used interchangeably by Aristotle) constitute a whole. With reference to the potluck analogy, we can think of each of the six parts of tragedy as analogous to dishes that make up a fine meal, although the analogy is inexact in that it is judgment of quality, not quality in itself, that is Aristotle's concern in §7 and §8 of the Politics passage.

In order for a tragedy group to be judged “best,” each of the six constituent parts must be taken into account and properly weighted for relevance; ignoring or improperly weighting any relevant part will result in a flawed overall judgment and thus in a bad outcome: the prizes will be awarded to the wrong poets. Aristotle specifies that the six parts are ranked in the following order of importance: plot, character, intellect, diction, song, spectacle (Poetics 1450a-b). Thus, plot is most relevant to quality and must be most heavily weighted; spectacle is least relevant and is weighted least heavily, with the others in between. In the model that follows (Table 3) the relevance-weighting of the six parts (Pw) is 8, 6, 5, 4, 3, 2; thus the most relevant part (plot = 8) is assumed to be four times as important as the least relevant (spectacle = 2). We do not know how close this is to Aristotle's own weighting, and of course changing the weights produces different outcomes. The following simple (no interaction terms) and stylized model is meant only to show how the tragedy ranking problem is solved, when we assign specific weights to the six parts, and (extrapolating from Aristotle) make some assumptions about rules and experts.

TABLE 3.

Hypothetical Aristotelian ranking of three options

Notes: Options are groups of tragedies (T1, T2, T3). Pw = weighted relevance of a given part (domain of expertise). Pw is multiplied by the expert ranking (REA I: 3 = highest), or mass vote (REA II), to achieve the weighted score. T(n)u = unweighted rank or vote for a given part for each option. T(n)w = weighted score of a given part. Aggregate T(n)w (the sum of weighted scores of all parts) determines the correct (Aristotelian) ranking among options: T1>T2>T3.

In this stylized epistocratic system, the assumed features of the tragedy choice are as follows: (1) the number of options (three tragedy-groups) and the actual options (T1, T2, T3) are set by rules; (2) the six relevant parts (per Aristotle's Poetics) are set by rules; (3) each part is a domain for which there is an infallible expert, who judges correctly by ranking the three options accurately, in terms of the criteria relevant to his domain;Footnote ²⁰ (4) weighting of the relevant parts (i.e. Aristotle's hierarchy, now assigned particular weights) is set by rules. Having watched the performances, each expert ranks the options based on his domain-expertise. For example, the Plot expert, impressed by T1's narrative development, ranks T1 highest, yielding an unweighted rank of 3, and a weighted score of 24. After each expert has ranked the options, the aggregate score for each option is tallied. The final ranking (and thus the outcome: distribution of prizes) is determined by the aggregate scores. The procedure ranks the three options transitively, in the correct (Aristotelian) order: T1 (score of 59) is awarded first prize; T2 (56) is second; T3 (53) third.Footnote ²¹

REA I models the decision-making processes of an individual faced with making choices on an issue that is divisible into domains of expertise. Suppose, for example, that an experienced opera critic must write a review that includes a rating (say, one to four stars) and a ranking (better or worse than other operas this season). Between acts of the performance she seeks the opinion of audience-members expert in the various aspects of the performance she will highlight in her review (singing, staging, acting and so on). Her experience enables her to identify experts and to weight the parts as she decides on rating and ranking. Or suppose that an attending physician must decide among several treatment options for a patient. The physician's experience with this sort of case leads her to recognize that the case has several distinct aspects, and that each aspect demands a certain expertise. The physician consults medical experts, each of whom makes a recommendation based on his or her specific domain of expertise (e.g. surgery, diet, psychology). Under the right conditions, the physician aggregates appropriately weighted expert opinions, and thereby decides on a best course of treatment.Footnote ²²

Neither the opera critic nor the attending physician is likely to assign numerical values to the expert opinions that contribute to the final decision, but the basic features of REA remain: a complex matter is broken down into parts, known experts judge each part, options are delimited by rules, weighting is by relative relevance among parts. In each case the decision must be timely (the critic has a deadline, the treatment must begin before the patient dies). The opera critic and attending physician examples show how the institutionalized rules of REA I enable several experts to be “like a single (rational and experienced) person” in coming to a decision. It is, however a non-democratic method in that the agenda is not set by the group, ranking in each domain is by a single dictator-like expert, and the experts’ votes are unequal (due to weighting). Moreover, this epistocratic “cloistered expert” approach (Ober Reference Ober2008: 1–3) is ill-suited to addressing complex situations that lie outside the group's prior experience.

5 REA II: COMPLETE RULES WITH MASS VOTING

In REA I the provision of experts in multiple domains is essential to achieving the superior outcome, but there is no mass vote. The rules determine the range of options; after the domain-experts have ranked the options, all that is needed is for a competent authority to apply the weighting formula and add up the scores. Yet Aristotle's “wisdom of the many” passage assumes that it is a multitude, acting like a single person (§5), that judges the whole; the superior judgment on musical and poetic works is made by “the many” (§7) – not simply by a properly diverse body of domain-experts. The judging tragedy case benefits from adding a mass of voters if we relax the expert infallibility assumption. In the real world the information held by an expert is sometimes wrong – data may be noisy, methods imperfect. But even an expert who knows the right answer might not disclose it. A self-interested expert may misrepresent her belief for strategic reasons. Absent the right incentives (section 7, below) strategic behavior could corrupt the process. If one or more of the experts is corrupt in Aristotle's sense (he values some private benefit above a common interest in right ranking), the best tragedy may not be ranked first. Aristotle was aware of the danger corruption posed to decision-making. He uses the metaphor of a large quantity of water that remains relatively unpolluted by a small admixture of impurity to illustrate why decisions made by a crowd are less subject to corruption-caused error (Politics 3.1286a3–35).

One solution to the problem of corruption (Aristotelian in spirit, although not spelled out in the text) is for a mass of citizens to vote on options, by domain, based on the reputations and testimony of multiple experts in each domain.Footnote ²³ After each domain-expert has explained the reasoning behind his ranking (“some judge a particular part”: §8), each citizen votes for one option in each relevant domain (“all of them together judge all of the parts”: §8), basing his vote on a judgment about the experts’ reputations and arguments. This approach is modeled in Table 3 (REA II), assuming 500 voters. This vote of “all together” (§2) in each domain stands in for judgment of a single infallible domain-expert and so the mass is non-mysteriously acting “like a single person” (§5). Insofar as the mass of voters judges the experts well in each domain, its collective decision is better than that of the excellent individual (§1), because the corrupt expert's mis-ranking is rejected and so does not corrupt the outcome. The votes on the domains are counted and weighted (based on the rule); the option with the highest total score (in this case T1, score of 6315) wins. Although the domain-votes are weighted, each voter has equal say in the overall outcome because each casts an equal vote in each domain.Footnote ²⁴

The REA process remains dependent on experts and their testimony. It assumes at least some experts who accurately report the state of the world, as it is relevant to their domain. As in the real-world cases of Condorcet juries with opinion leaders (Ladha Reference Ladha1992) and jurors hearing the same evidence (Dietrich and List Reference List, Pettit, Catton and Macdonald2004), the result is likely to fall short of the near-certainty that is theoretically achieved by a “classic” Condorcet jury, in which independent votes are based directly on a state of the world. The REA process will fail in the absence of non-colluding genuine experts, who value their reputations, and who are competent at explaining the reasoning behind their rankings to non-experts. It also requires that the mass be able to judge experts well.

Aristotle has specified that each of the ordinary individuals constituting the crowd (§1) possesses only a “part” (morion) of virtue and practical wisdom (§4). The group's capacity to judge well – i.e. to act as a qualified (if not infallible) quasi-person (§5) – depends on its collective possession of an adequate level of virtue (at a minimum: non-corruption) and practical wisdom (capacity to judge reputations and arguments). In the discussion that follows the “wisdom of the many” passage, Aristotle emphasizes that the decision-making group must be of the right sort if it is to make superior judgments (Politics 3.1281b15–20, 1286a36–38). I take it that this means that the group is collectively sensible to the loci of relevant kinds of expertise, and it pays proper attention to the reputations and testimony of those who are especially adept in each domain. Being of the right sort also means being collectively attentive to the common interest and being marginally competent at judging individual experts – which is different, and (ex hypothesi) less cognitively demanding, than competence in judging the substance of an issue as a complex whole. Some voters may be corrupt or incompetent at judging experts, but the mass is assumed to be large enough to dilute their influence; it is not systematically corrupted nor dominated by incompetents. If we assume average individual competence in the judgment of experts to be over 0.5, on a scale of 0–1 (see Table 4) the Condorcet jury effect will push in the direction of a correct choice. The single-person-like collectivity has, on this reconstruction, sufficient virtue and adequate practical wisdom (conjoined character and intellect: §6) to judge the reputations and reasoning of domain-experts, and thus to choose the best option.

Table 4.

Hypothetical distribution of competence among 25 voters

Notes: Competence is ranked on 0–1 scale of competence (likeliness to choose correctly) in each task-category.

Like Condorcet, Aristotle assumes a group of preference-sharing decision-makers, each of whom believes that there is a best choice and seeks that best choice. Like Condorcet, the choices of marginally competent voters are aggregated at scale. Aristotle did not do Condorcet's math, but his intuitions seem to point to something like a jury theorem. Yet unlike Condorcet, in Aristotle's approach (as reconstructed here) a central role is played by reason-giving and intra-group epistemic diversity. Diversity in relevant domains of expertise (Page Reference Page2007), along with the aggregation of marginally competent choices, is what grants a REA group its potential to choose well. Aristotle can assume that, were they to decide independently on the substance of the issue (as Condorcet voters unguided by domain-experts), most members of the decision-making group would be likely to choose wrongly, and so the aggregate decision would tend to be wrong. If all goes well, the collectivity comes to the right answer on an issue with several parts, even though most voters are unable to make individually correct judgments on the complex whole. REA II replaces Condorcet's assumption of individual marginal likelihood of correctness on the matter being decided with an assumption that some people are expert in any given domain and that a sizable majority of voters will be reasonably good at judging experts. The assumption of “marginal likelihood of correctness” remains essential to the process, but it is deferred: away from the substance of the matter at issue and towards competence in judgment of experts.

Since REA II depends on the capacity of ordinary individuals to assess reputations and arguments made by experts, we must ask whether it is plausible to assume that, even under the best conditions, a large, modern group of ordinary persons could be adequately competent in judging experts. Although small ancient communities enjoy advantages in terms of shared values and social knowledge, a contemporary REA II process could avail itself of statistical instruments for ranking the reputations of experts, based (for example) on standard methods for assessing the impact of academic scholarship, or on now-familiar peer-based systems for rating and ranking products and services. For some voters, reputation (the high rating of an individual expert or convergence of opinion among highly-rated experts) might be dispositive. Yet reason-giving by experts remains important, both as an alternative to numerical ranking and because citizens’ dignity demands that they be treated as adults with the opportunity to act on the basis of reasons offered by others. The imperative for reason-giving by experts is a constraint on the selection of domains, in that both premises and conclusions of arguments in highly specialized subfields will remain opaque to non-specialists. Each domain on which citizens are to vote must be capable of fielding genuine experts ready and able to make arguments that can be taken up by ordinary citizens. This is demanding – but not excessively so, in light of contemporary experience with expert elicitation (Renooij Reference Renooij2001) and expert testimony in “civic science” (Rhoads et al. Reference Rhoads, Wilson, Urben and Herricks1999; Bäckstrand Reference Bäckstrand2003), which suggests that citizens need not become experts to competently assess arguments made by experts.

Informal examples of REA II are readily available in the contemporary world. To take a familiar example, suppose that the faculty of a large and well functioning (thus best-choice seeking) academic department must decide among three candidates for a faculty appointment. The various domains relevant to the excellence of candidates (say: research, teaching, service), and the relative importance of each domain to the issue of candidate choice, are adequately specified by rules, formal and informal. There are certain individuals recognized as especially knowledgeable who will be consulted on each domain (e.g. specialists in the field of research; students with knowledge of a candidate's teaching). There may be doubt about the quality or impartiality of some of those consulted. Reputations, level of consistency among experts, and the quality of arguments will influence the decisions of the voters. In the end, the members of the department vote, and (if the process has worked as it ought), the vote chooses the best candidate based on the relevant criteria. The collective memory of the department records each step of the process and the relationship of process to outcome (Did the chosen candidate turn out to be as excellent as claimed by experts in the relevant domain?). Reputations are updated accordingly.

REA II respects democratic values: It employs equal votes of free citizens, acting as dignified adults in making judgments on significant matters of public concern. It allows for timely decisions. Yet insofar as the rules governing options and relevant domains are set by an external authority (e.g. an academic Dean), it remains subject to external agenda control.

6 REA III: INCOMPLETE RULES, MULTIPLE STAGES

REA III is a modified REA II process, in which the rules for determining domains and options remain unspecified, and to which a mass vote directly on options (not by domains) is appended. Upon completion of an REA II process by a citizen council, the issue is referred to a larger citizen assembly with final decision authority. The council's ranked options are the starting point for deliberations and voting in the assembly. Compared to REA I and II, REA III may not increase the accuracy of option choice, but, by allowing relevant domains and options to be determined by decision-makers rather than rules, it escapes the shadow of external agenda control. Moreover, the plenitude of issues and domains that might be considered leaves open the possibility that any citizen could, at some point, be regarded as an expert whose special knowledge (like that of students in the appointment case) might contribute to making important public choices. Finally, because REA III incorporates a final, decisive popular vote on the issue, it is attentive, in ways that REA I and II are not, to the practical impossibility of capturing all nuances of a complex issue through subdivision into several discrete domains of expertise. REA III allows greater scope for the practical wisdom and experience of ordinary citizens to be brought to bear on salient matters (cf. Wilson Reference Wilson2011: 265–66 on the “judgment of journeymen”).

As a result of these features, REA III more fully expresses the core democratic values of liberty as free speech and assembly, political equality as equal influence through voting, and dignity as being treated as an adult. Yet rules for establishing relevant domains and options need not remain unspecified. Rule-bounded REA decisions are democratic if the rules specifying relevant domains and options were democratically established and so long as they remain subject to democratic amendment. Less-stylized versions of REA I and II could be incorporated into a democratic constitutional order, if an incomplete-rule procedure is employed in constitution-making and amendment. I argue in the final section (7) that a democratic and epistemic regime can, over time, develop rules specifying relevant domains for some categories of decision.

In the case discussed below, a democratically appointed council prepares an agenda for a larger assembly empowered to make binding decisions on matters of common interest. Although pre-established rules govern some aspects of deliberation and voting, relevant domains and options are freely chosen in early stages of the decision process. The background REA conditions are presumed to hold: the decision-making group seeks the best option in addressing a matter of common interest; issues are divisible into parts, each with a specifiable relevance and explicable as a domain of expertise. Additionally, REA III assumes that these background conditions are common knowledge, that the group exists over time, and that group-members update their judgments based on new information.

The historical case considered here – the Athenian response to the threat of Persian invasion in 481 BCE, as described by the historian Herodotus (Histories 7.140–44) and recorded in the “Themistocles Decree” – was well known to Aristotle.Footnote ²⁵ As with Aristotle on the wisdom of the crowd, the compressed accounts of Herodotus and the Decree seem to describe a version of REA. Some aspects of the following narrative, meant to illustrate how a democratic decision-making process may be parsed into discrete REA stages, are hypothetical. The actual process employed in 481 BCE was certainly messier, but the process described here is intuitive and it accords with the known facts about decision-making procedures in the Athenian council and assembly (Rhodes Reference Rhodes1985; Hansen Reference Hansen1987; Missiou Reference Missiou2011).

In 481 BCE, as a massive Persian force prepared to advance west into Europe, the Athenian state was confronted by a clear and present danger. The issue of how to respond to the invasion would ultimately be decided by a vote in a citizen assembly attended by several thousand (adult male) citizens. The agenda for the assembly meeting was set by a council of 500 citizens, chosen by lot. The council's role was to define policy options. The Greek term for the council (boulē) refers directly to its deliberative function (bouleuein = “to deliberate”), and it is quite certain that votes were also taken. We have no record of the council's proceedings, but by the time the assembly met, three primary options – flee, fight on land, fight at sea – had emerged. Because the agenda for the assembly was announced in advance, ordinary citizens had the opportunity to deliberate among themselves before the decisive meeting, at which a vote of the citizenry would decide how the state would respond to an existential threat.

Although Athens was used to fighting its Greek neighbors, invasion by the Persian empire was an exceptional circumstance. Because decision rules cannot specify relevant domains for exceptional circumstances, and because the decisive assembly vote would be directly on the options (rather than by domains), citizens shared responsibility not only for judging experts, but also for establishing relevant domains and for aggregating domain-specific expertise into an option choice. The procedural task of identifying relevant domains and experts is, in REA III, accomplished in the first instance through deliberations and voting in the council, but all citizens must judge experts and calculate aggregates of domain-specific expertise. There is surely substantial variance among citizens in respect to capability in performing these tasks. Table 4 suggests a hypothetical distribution of capabilities among a sample population of 25 citizens. A minority of the citizens in this hypothetical sample falls below the presumed point (0.5 on a scale of 0–1) at which his vote will make a positive contribution in each task. Yet, because the mean competence level in each column is above 0.5, we can appeal to a variant of the Condorcet jury theorem (Gronfman, Owen, and Feld Reference Grofman, Owen and Feld1983; List and Goodin Reference List and Goodin2001) to suggest that the collectivity will do adequately well in each task. As the number of voters increases (from 500 councilors to thousands of assemblymen), the likelihood of a spread among vote counts adequate to yield a best choice among options likewise increases.

The council takes up the issue of the Persian invasion in meetings that are less time-constrained than the subsequent decisive meeting of the assembly. The deliberation begins by determining the relevant domains. Based on their previous experience in deliberating on other issues, the councilors have a good sense of the distribution of capabilities among their membership – that is, they have got something like Table 4 in their heads.Footnote ²⁶ Because they seek the best choice and the stakes are high, the councilmen allow Councilors A, B, C, D, and Y, who are rightly (Table 4 column 1) thought most capable at setting relevant domains, to take the lead in these deliberations. Councilman A gains majority support for the importance of the attitude of the gods and thus the morale of the populace.Footnote ²⁷ This therefore becomes one of the domains in which expert advice will be sought. Councilman B, C, D, and Y make cogent and successful arguments on behalf of other domains: Persia's strategic goals, Athenian mobilization capacity, attitudes and capacities of Athens’ allies, and potential threats to Athenian unity. The selection of relevant domains ends when no majority favors adding another domain.Footnote ²⁸

The council's deliberations and votes establish the relative importance of the domains and determine the domain-experts who will offer testimony. As the experts’ recommendations are discussed, the views of C, E, F, Q, and Y – who are known to be especially astute judges of experts (Table 4 column 2), are especially influential. In the course of iterated deliberation and voting, the three primary options (flee, fight on land, fight at sea) emerge and are tested against the views of domain-experts. The process, which was, other than in domain- and option-setting, functionally similar to REA II, led the council to rank the “fight at sea” option first.Footnote ²⁹

With the options identified and ranked, the assembly is called and the results of the council's deliberative/voting process are reported. The assembly then carried out its own deliberations, thus potentially allowing domains of expertise ignored by the council to be considered. Herodotus reports (7.142) that “many opinions” were offered, with prominent citizens supporting different options. Expert testimony was offered for and against each option – Herodotus reports that the chrēsmologoi (oracle interpreters) argued that the gods disfavored the option of fighting at sea. Although the Athenians were strongly committed to the value of free speech, if we are to judge by later (fourth-century BCE) practice, those regarded as non-expert in the relevant domain under discussion were not given a full hearing; the time constraints did not allow the luxury of attending to uninformed opinion.Footnote ³⁰

The winning option in the final stage of a successful REA III process is the one that best takes into account the most relevant domains and the most credible experts. Because the final vote is directly on the options, rather than (as in REA II) on options by domains, each individual citizen is ultimately responsible for calculating for himself the relative importance of the several domains (Table 4 column 3), as well as judging the experts in each domain (column 2). Some citizens will undoubtedly over-weight a relatively trivial domain. Yet the assumption that there is an adequate aggregate competence in the procedural task of calculation does not place an impossible cognitive burden upon the citizens. Their burden is, in, any event, lighter than that taken on by voters in non-REA democracies, for whom domain-relevance and domain-experts are not specified in advance by an experienced deliberative body.

In 481 BCE, the fight-at-sea option, put into the form of a proposal by Themistocles (recorded in the inscribed Decree), eventually carried the day. Themistocles’ own reputation as a trustworthy leader and as an expert in naval affairs certainly played a role in the choice. Yet Herodotus and the Decree suggest that those favoring the fight-at-sea option took up several domains of expertise that we may guess were heavily weighted: The attitude of the gods was, according to Herodotus (7.143), addressed by Themistocles’ convincing reinterpretation of the key oracle. Persian strategic goals, plans for Athenian naval mobilization, the attitudes and capacities of Greek allies, and provisions for securing unity by recalling exiled citizens were addressed in detail in the Decree. Because the final vote was conducted by estimating the number of raised hands, rather than by counting ballots, the vote was observed by the assembly as a “unified whole” (Schwartzberg Reference Schwartzberg2010). Herodotus and the Decree describe the decision as that of the people of Athens, not of a majority faction. The assembly's vote in favor of fighting at sea was understood as a direct expression of the demos’ collective will about a matter of common interest. The Decree was implemented on the following day, as generals began allotting commanders, marines, and rowers to Athenian warships.

Given the impossibility of specifying counter-factuals, we cannot know whether the assembly chose the best option, but fighting at sea certainly appears, ex post, better than the other options. The decision, as Herodotus emphasizes, changed the course of Greek history in ways that were overall positive for the Athenians. Herodotus states (7.139) that the Athenian decision to fight at sea determined the outcome of the war and that after winning the Battle of Salamis, democratic Athens went on to become the preeminent state of the Greek world. Herodotus elsewhere pointed out (5.97.2–3) that decisions of the Athenian assembly sometimes led to bad outcomes. Yet, over time, the Athenians’ capacity to organize useful knowledge through democratic processes, and thereby to make relatively good policy choices, helped to make Athens an exceptionally influential, secure, and prosperous city-state (Ober Reference Ober2008).

Ancient Athens, with a total population of perhaps a quarter-million, was tiny compared to most modern states. REA III might, however, be adapted to contemporary purposes, for example in a public referendum. After the ballot measure has been certified, a large (say 500 member), representative citizen council is selected by democratic process (say sortition). Through the process of deliberation and voting described above, the council establishes the relevant domains, hears expert testimony in each domain, and rank-orders options. The results of the council's work are published in advance of the popular vote on the issue. The published results provide voters with valuable information about the judgments of a cross-section of their fellow citizens (Hawthorne und.: 40–53). Some voters might review the council's narrative of its hearings, votes, and deliberations. Yet simply by checking the relevance-weighted domains by which the council voted, a voter can decide how well the council's choice and ranking of domains tracks her own sense of these matters. The vote in each domain (see Table 3: REA II) tells her how the various options fared domain-by-domain, based on the councilors’ judgment of experts. She knows which option would be chosen, and what the final scores for each option would be, if the decision were made on the basis of the aggregated domain-votes of the councilors. And she can take all of this into consideration in her own option choice, in light of her views on the competence of the councilors and any other information to which she cares to attend. This variant of REA III resembles experiments incorporating deliberation into real-world democratic decision-making (Warren and Pearse Reference Warren and Pearse2008), but it arguably provides more information directly useful to voters at a lower information-procurement cost to each voter.Footnote ³¹