1. INTRODUCTION
Imagine the following scenario. Smith murders Jones with his 9mm. After the body is discovered, the Commissioner decides to put three detectives, call them ‘One’, ‘Two’, and ‘Three’, jointly on the case. During the investigation, each of these detectives discovers her own unique piece of evidence: One discovers that Smith owns a 9mm pistol, Two discovers that Jones was shot with a 9mm pistol, and Three discovers that Smith had gunpowder residue on his hands shortly after the murder. Assuming that these pieces of evidence are the only facts that our detectives know that are relevant to the crime, it seems at this point that each detective ought to suspend judgment on the claim that Smith murdered Jones. While One knows that Smith owns a 9mm, she doesn't know whether Jones was killed with a gun of that caliber, and even if she did, someone else might have used the weapon. Two and Three are in a similar position. Now if they each compulsively wanted to impress the Commissioner by solving the case solo, and thus each kept her evidence secret, perhaps even destroying the trail so the competition couldn't catch up, each would be stuck suspending judgment on the crucial claim. This kind of behavior would clearly be epistemically detrimental to the group. However, let's assume the Commissioner wisely and successfully created an incentive structure to discourage this kind of behavior. The group meets and shares the individual pieces of evidence, and all three form the true belief that Smith was the murderer. Meeting and sharing evidence has clearly been epistemically beneficial to the group.
In Knowledge in a Social World (1999), Goldman suggests a framework to quantify the epistemic characteristics exhibited by epistemic systems like this group of detectives. While many ways of quantifying epistemic progress for solitary agents have been discussed in the formal epistemology literature, where they are known as ‘scoring rules’, Goldman is one of the first to present such a rule for groups of inquirers. In this essay, I will show that Goldman's framework contains some substantial limitations, because first it cannot be applied as broadly as we would like, and second the natural extension of the framework has some absurd consequences. Fortunately, the extension of the framework can be repaired, and the required modifications carry an interesting consequence. In particular, I will argue that the following is a consequence of any legitimate group scoring rule modeled along the lines of Goldman's framework: a group containing members who disagree can always improve its epistemic status if its members form a consensus by averaging their assessments. I will then explain how this result can be used to show that individual and group-level epistemic norms can come into conflict.
The plan of the essay is as follows. In section 2, I present Goldman's framework for scoring a group's epistemic progress. In section 3, I expose the limitations of Goldman's framework; in particular, when applied to agents with credences, the framework (a) assumes probabilistic coherence and (b) has some counterintuitive consequences once this assumption is lifted. In section 4, I present a requirement for any legitimate scoring rule, namely, that the measure of inaccuracy involved must be a convex function. I argue that the requirement of convexity isolates a class of group scoring rules that all entail that groups whose members disagree can always improve their epistemic status by forming a consensus through averaging their credences. In section 5, I sketch a way that we might use this consensus result to generate group-level norms of rationality and show how, on such an account, group-level norms can conflict with individual-level norms of rationality. I conclude in section 6 by discussing some ways in which the assumptions the consensus result relies upon might be weakened and some possible consequences of doing so.
2. GOLDMAN'S SCORING RULE
Groups of inquirers can have a variety of characteristics that might be thought to carry epistemic value. For example, we could judge a group according to how honest its members are, how cognitively diverse they are, how much they agree with each other, etc. But while each of these properties exhibited by the group might carry epistemic benefits, one value seems to underwrite whatever value these characteristics have, namely, the tendency to foster true beliefs. Goldman (Reference Goldman1999) calls this ‘veritistic value’. The basic idea is that the proper aim of belief is truth, and so we can judge the quality of epistemic behaviors by their capacity to fulfill this aim. Take, for example, a group that has a great deal of cognitive diversity, something that might seem to be of direct epistemic value. Let's assume the members of this group all attack problems differently and share very few beliefs. If the members of the group start out believing mostly false things and never come to dispel these falsehoods, we wouldn't think this group is doing well, epistemically speaking. Only under certain conditions is something like cognitive diversity epistemically beneficial, namely, when it leads to an increase in true beliefs. While this kind of argument deserves much more attention than I offer here,Footnote 1 let's accept for now that veritistic value, as Goldman understands it, is of fundamental epistemic value.
Goldman proposes a quantitative way of judging the veritistic value of social systems, epistemic policies, or changes in a group's beliefs. For the sake of simplicity, I will focus here on the veritistic evaluation of changes in a group's epistemic states. In such cases, changes in veritistic value amount to changes in the accuracy of the respective states. Take, for example, our group of detectives mentioned at the beginning of this paper. Each of the detectives initially suspends judgment on proposition S, that Smith murdered Jones. But after they share their evidence, each comes to believe the true proposition S. To judge this modification, we can first judge the epistemic improvement of each individual and then simply average those improvements across the individuals.
Here is how we figure out the changes in accuracy for an individual. Goldman proposes that if S is true, we give an agent 1 point if she believes S, 0.5 points if she suspends judgment on S, and 0 points if she disbelieves S. Goldman further assumes that if an agent believes S then she disbelieves not-S and vice versa. That way, for example, if we know that an agent disbelieves not-S, and S obtains, then we know she will receive 1 point. This way of going about the scoring strips out the task of scoring an agent's attitudes toward both S and not-S. Now, getting back to our detectives, we can see that at the initial time when they all suspend judgment concerning Smith's guilt, each detective would receive a score of 0.5 points, and so the group score would be the average of these scores, namely 0.5 points. But after sharing their evidence, the group would receive a score of 1 point, thus capturing the epistemic benefit of sharing evidence. In cases where we are scoring a group's attitudes toward more than one proposition, we simply score each individual proposition, sum these values for each agent, and then take the average.Footnote 2
For various reasons (which I won't pursue here), many epistemologists prefer to think of belief states in terms of credences. The idea is that, instead of thinking of agents as believing, disbelieving, or suspending judgment on a proposition, full stop, it is better to represent them as having degrees of confidence in the proposition. Goldman thinks that his framework applies equally well to credences. First, he assumes that each agent's credences over a set of propositions collectively form a probability function, meaning that they abide by the axioms of probability theory (1999: 88). So, just as we assumed that an agent who believes a proposition P also disbelieves its negation, the axioms of probability guarantee that an agent with a credence of, let's say, 0.75 in P also has a credence of 0.25 in not-P. We know this because the axioms insure, among other things, that the probability of a proposition and the probability of its negation must sum to 1. Thus, in the credence case, we simply see whether P or not-P is true and then give an agent the number of points equal to her credence in whichever one obtained. If P is true, our agent receives 0.75 points, since she had a credence of 0.75 in P which came out true. But if P is false, she receives 0.25 points, since she had a credence of 0.25 in not-P which came out true. Like before, when we have a group of agents, we simply average their scores. We can then track changes in these values over time to judge the benefits or drawbacks of various epistemic behaviors for the groups.
These are the basics of Goldman's scoring rule. I am most interested in the implications of the framework when it is applied to credences, and thus I'll focus on this version of the framework for the remainder of the paper.
3. LIMITATIONS OF GOLDMAN'S SCORING RULE
One limitation of Goldman's scoring rule is that it requires that the agents in a group are all probabilistically coherent; that is, the credences of each agent in the group abide by the axioms of probability. This is worrisome for a few reasons. First, many actual agents don't have probabilistically coherent credences, so any group that contains even one of these agents cannot be scored. Thus the framework doesn't have the resources to quantify any improvement that a group might gain from its members moving from incoherent credences to coherent ones. Similarly, there might be better or worse ways for a group's members to move to coherent assessments or to move between incoherent assessments. We want a group scoring rule that has the resources to assess these kinds of changes, which we cannot have if we assume from the start that all of the members in the groups to be scored are fully coherent.
This limitation of Goldman's scoring rule doesn't ultimately give us a reason to reject the rule, since the assumption could be lifted, thus yielding a natural extension of the original rule. The assumption of probabilistic coherence in the case of credences serves a similar role to Goldman's assumption of logical coherence in the case of coarse-grained belief states. Recall that he assumes that if an agent believes P then she disbelieves not-P and vice versa. One way this factors in the framework is that, if an agent disbelieves P in a case where P is false, then instead of assessing this agent's attitude toward P, we assess her attitude toward not-P. And since this agent would be guaranteed to believe not-P, she would receive 1 point. The reason Goldman makes the scorer go through this process is that he doesn't want an agent to receive extra points for simply doing what comes naturally. Notice that another option would be to score for P and then also score for not-P, but as Goldman insists, ‘this is “double counting,” which seems inappropriate’ (89). While this kind of ‘double counting’ may seem problematic in the case of coarse-grained belief states, it isn't problematic in the case of credences. Once we admit that agents often have credences that violate the axioms of probability, it becomes clear that we need both an agent's credence in P and also in not-P to make an impact on her overall score.
Making this modification to Goldman's scoring rule would essentially turn it into a group version of a scoring rule that has been discussed in the scoring rules literature, namely, the absolute value rule. This rule would work as follows. First, I'll offer some definitions.Footnote 3 Let P = {P1, P2, … Pn} be the set of propositionsFootnote 4 over which we will score our group of agents G = {a1, a2, …, am}. Let pi = 1 if the proposition Pi is true and 0 otherwise. We'll denote an agent ai's credence in Pj as bi(Pj),Footnote 5 and we'll allow this to take any value in the real number interval between 0 and 1 inclusive.Footnote 6 In the individual case, for any agent a, we have the following rule:
Then, when we want to score our group of agents, we could simply average the individual scores. We could put this more formally as the following:
This extension of Goldman's rule bypasses the limitation raised above, since it is able to yield scores for groups that contain incoherent members.
The difficulty is that the absolute value rule for scoring credences has some absurd consequences (see Savage Reference Savage1971; Seidenfeld Reference Seidenfeld1985; Joyce Reference Joyce, Huber and Schmidt-Petri2009). The easiest way to see this is through an example such as the following (adapted from Joyce (Reference Joyce, Huber and Schmidt-Petri2009) who offers it in reply to Maher (Reference Maher2002)). For simplicity, I'll just stick to the individual case. Take an individual who is about to roll a six-sided die that she knows to be fair. Before the roll, she considers what credences she ought to have in each particular side landing face up.Footnote 7 Let P1 be the proposition that side 1 lands up, and similarly for the rest. Intuitively, she ought to assign each proposition a credence of 1/6, since she knows there is a 1/6 chance of each face landing up. But after learning about the modified version of Goldman's scoring rule, it seems she ought to reconsider. According to this rule, assigning a credence of 0 in each proposition is actually better, as far as accuracy is concerned. To see this, let's assume face 1 lands up. Then if she gave her intuitive assignment, we would compute her score as follows:
whereas, according to her later assignment, we would compute her score like this:
Given that higher scores are better according to rules like the Group Absolute Value Rule, our agent actually does better by assigning a credence of 0 to each proposition if the die lands with the first face up. The argument goes the same for each of the other faces. This puts our agent in a very strange position. While obviously one of the sides must land up, the rule suggests that she ought to have zero credence that it be any particular one. This example thus shows that there will be cases where the rule seems to encourage agents to become badly incoherent, which suggests that it isn't the kind of rule we would believe to hold normative import. This is not good news for the rule.
4. A NECESSARY CONDITION FOR ANY LEGITIMATE GROUP SCORING RULE
Given the absurd result we saw in the previous section, clearly the extension of Goldman's framework, the Group Absolute Value Score, can't be legitimate. The problem with this extension could arise at one of two places – either at the level where we judge the individuals' accuracies or instead at the level where we use these individual accuracies to determine the group's overall accuracy. In this section, I will explain how the absurd result stems from an error at the level where individual accuracies are judged.
For some time, statisticians have been working on how best to judge a set of probability assessments, and economists have long been developing closely related reward structures for probabilistic forecasters. Philosophers have recently taken many of the insights from the resulting literature on scoring rules and applied them to a close relative of the problem we face here. For example, Joyce (Reference Joyce1998, Reference Joyce, Huber and Schmidt-Petri2009) presents some minimal conditions on any legitimate notion of gradational accuracy at the individual level and then uses these conditions to argue that an agent's credences ought to abide by the axioms of probability.Footnote 8 Given the complexity of the details, I won't recap all of Joyce's conditions here. Instead, I'll simply focus on one relevant condition, namely, that any scoring function for gradational accuracy must be a convex function. But before getting to the details, we need to slightly modify how we are discussing these rules. Since the literature on scoring rules is generally framed in terms of inaccuracy as opposed to accuracy, and since recent authors in the philosophical literature follow this tradition, it will be helpful for us to put things in these terms as well. However, the modification isn't difficult. The inaccuracy measure basically quantifies a penalty paid for missing the mark, and thus it is a quantity that we want to minimize (as opposed to accuracy, which we wanted to maximize).
The fact that any so-called ‘proper scoring rule’ will be a convex function is now well established (see Gneiting and Raftery Reference Gneiting and Raftery2007; Predd et al. Reference Predd, Seiringer, Lieb, Osherson, Poor and Kulkarni2009).Footnote 9 Note that ‘proper’ is being used in a technical sense. A scoring rule quantifies the penalty a forecaster pays for the inaccuracy of her reported credences in a set of events, and a proper scoring rule is any rule for scoring probabilistic inaccuracy such that an agent minimizes her expected score by reporting her true credences (so long as her credences are probabilistically coherent).Footnote 10 For our purposes, we should frame things slightly differently. We viewed the extension of Goldman's original rule as offering a normative suggestion to groups, and so it will be helpful to think of the various other scoring rules as possibly doing the same. Instead of envisioning the situation as one where the agent has a set of credences, reports a possibly different set of forecasts, and then is scored on the latter, we can imagine that the agent has a set of credences, but that this may not be the best set of credences as measured by the scoring rule at issue.
So, why was the consequence in the dice case absurd? It was because our agent knew for a fact that the objective chance of each face landing up was a certain value, but, according to the Absolute Value Score, she could know ahead of time that she would get a better score by ignoring this information entirely. Thus, the scoring rule encouraged her to have credences that didn't match what she knew about the objective chances of the events.Footnote 11 A proper scoring rule wouldn't have had this consequence. If an agent who will be scored by a proper scoring rule has credences in line with what she knows to be the objective chances in some situation, then her credences must be probabilistically coherent, and she therefore maximizes her expected score by holding onto these credences. This won't be the case for non-proper scoring rules. For any non-proper scoring rule, we know there will be at least one case in which a probabilistically coherent agent could improve her score by falsely reporting her credences. So shifting over to the normative interpretation, this means there is one such coherent agent who ought to have different credences than she has. Since this agent's credences at the initial time are coherent, we can then assume that each of her initial credences was in line with what she knew about the objective chances in her situation, and thus we can show that this non-proper rule suggests that she ought to have credences that aren't in line with what she knows about the objective chances. We could construct an absurd result for any non-proper scoring rule in this manner, so only proper scoring rules are legitimate for our normative purposes. Moreover, since proper scoring rules must be convex, any legitimate group scoring rule must also be convex.
Before stating exactly what convexity means, I will need to introduce some more definitions.Footnote 12 Recall that we have a set of propositions P = {P1, P2, …, Pn} over which the agents in the set G = {a1, a2, …, am} have credences, and we denote the credence that ai has in Pj as bi(Pj). We denote the truth-value of Pj at a particular world as pj, which will take the value of 1 if Pj is true and 0 otherwise. Since each agent has a credence in each proposition, we can think of the collection of credences held by agent ai over these propositions as an ordered n-tuple bi = <bi(P1), bi(P2), …, bi(Pn)>. Last, we have a set of possible worlds W = {w1, w2, …, wq}. Since we know that every Pj in P will be either true or false at a particular world, we can think of each world as an ordered n-tuple wk of 1's and 0's.Footnote 13 In what follows, if I need to discuss an arbitrary belief vector or an arbitrary world, I'll simply denote these as ‘b’ and ‘w’ respectively.
I will now give a formal statement of convexity, which was stated by Joyce (Reference Joyce1998, Reference Joyce, Huber and Schmidt-Petri2009) as follows:
Convexity: 1/2 I(b, w) + 1/2 I(b*, w) > I(1/2 b + 1/2 b*, w) for any distinct credence functions b, b*.
What this rule requires is that if we were to take the mean inaccuracy of two distinct credences, this value will be greater than if we had first found the average of the two credences and then scored this mean credence's inaccuracy. Note that this is true not only for two distinct credences in a single proposition but also for full credence functions. Also note that the world we are measuring inaccuracy against is arbitrary. Therefore, this inequality must hold in all possible worlds.
The Absolute Value Rule (both the group and individual version) fails this condition. To see why, let's use the dice example as a test case. We saw that according to the Individual Absolute Value Rule, the agent scores better when she has a credence of 0 in all six propositions concerning which face lands up than she scores when she has a credence of 1/6 in each. Convexity requires that the average inaccuracy of these two distinct credence functions be greater than if we had first averaged the two credence functions and then scored them. Thus, the average inaccuracy of the two functions must be greater than the inaccuracy the agent would have if she had a credence of 1/12 in each face landing up. This condition is not met. When she has credences of zero, our agent's inaccuracy score will be |1 − 0| + 5 · |0 − 0| = 1 since one of the heads will land up (costing her one point), while it costs her nothing for each of the other five heads that don't land up. In the case where she has credences of 1/6, her inaccuracy score will be |1 − 1/6| + 5 · |0 − 1/6| = 12/3, since the face that lands up will cost her 5/6 and those that don't will each cost her 1/6. Thus, the average inaccuracy score of these two functions is 11/3. If we were to average the credences first and then score them, we get an inaccuracy of |1 − 1/12| + 5 · |0 − 1/12| = 11/3. Since this value is equal to the average inaccuracy of the other two credence functions, as opposed to less than, we can see that the condition of convexity is not met.
This requirement that any legitimate group scoring rule must be a convex function carries with it an interesting consequence:
CONSENSUS: Any group that contains at least two members who disagree can decrease its collective inaccuracy by coming to a consensus through averaging credences.
To see why, I'll present the more inclusive statement of convexity given by Joyce (Reference Joyce, Huber and Schmidt-Petri2009). We could state it as follows:
Convexity (group version): For any world w, any credence functions b 1, b 2, …, b m, such that b i ≠ b j for some i, j, and μ1, μ2, …, μm ≥ 0 with 
Footnote 14
What this statement of convexity says, in essence, is that if we take a weighted average of the inaccuracies of a set of credences at a world, this value will be greater than if we had taken a weighted average of the credences themselves and then scored their inaccuracies (so long as the credence functions aren't entirely identical). In the case of our group scoring rule, we are assuming that each agent's inaccuracy is evenly weighted, that is, that each μi = 1/m. Once we plug in these values, we can see that the requirement of convexity will insure that any group whose members disagree on at least one credence evaluation would improve its inaccuracy score if its members instead had full agreement in their credences at the average of the former values. Just as before, this is true regardless of which possible world the group inhabits.
5. CONSENSUS AND CONFLICTING NORMS
It's worth taking a moment to consider why we've explored these technical details and what the result might mean for social epistemology. Goldman proposes a way to score various groups for their veritistic value which he believes to be of fundamental epistemic importance. The idea is that groups with policies that foster increases in veritistic value are epistemically better off than those with policies that are detrimental according to this metric. His proposed framework for measuring this value was supposed to extend to cases where agents have fine-grained doxastic attitudes, i.e. credences. But as we saw, the framework required some revisions, first to allow for broad applicability and second to avoid some absurd results. We have seen that the latter revision carries with it an interesting result CONSENSUS, and so we should consider what implications this result might have for social epistemology. In this section, I will first examine what role CONSENSUS could play inside of Goldman's account of social epistemology. I will argue that while CONSENSUS will suggest norms for many groups within Goldman's system, it will often be unclear to whom these norms are supposed to apply. I will then offer an alternative account, and show how, on this account, norms of individual rationality and norms of group rationality can come into conflict.
Goldman, in his essay ‘Systems-Oriented Social Epistemology’ (2010), lays out three possible subfields of social epistemology. The first examines what individual rationality requires of agents who obtain evidence that is social in nature, as when an individual receives a piece of testimony or discovers she disagrees with an epistemic peer. The second examines what sorts of norms apply to groups that count as genuine agents in their own right, such as juries or funding agencies. The last examines what kinds of policies, procedures, or behaviors are epistemically beneficial to groups that don't (or at least might not) count as singular agents, like groups working in such areas as science, education, and journalism. CONSENSUS fits most naturally within the third subfield, since the result mentions neither social evidence nor group agency. Within this third subfield, Goldman takes an ecumenical approach concerning what characteristics of doxastic states can carry normative weight (Goldman Reference Goldman, Gendler and Hawthorne2010: 197 n. 9), and so a characteristic like group accuracy in the sense sketched above would surely meet his standards. Therefore, if a group can improve its accuracy by coming to a consensus, then this would be something the group ought to do, so long as doing so doesn't harm its accuracy for other reasons.Footnote 15
While I'm fond of Goldman's systems approach to social epistemology, and I agree that CONSENSUS makes normative suggestions for some groups, I think that Goldman fails to address the following puzzle: to whom (or what) do such normative suggestions apply? There are a few possibilities. First, as I mentioned earlier, there are some groups that count as singular agents, and such a group agent perhaps ought to be formed in a way that encourages its members to abide by CONSENSUS. But as Goldman admits, the purpose of a systems-oriented social epistemology is to examine norms that apply to groups that aren't necessarily group agents. A second possibility is that the groups subject to the norms suggested by results like CONSENSUS are those that can be affected by rules or policies that are under the control of other agents. For example, the National Science Foundation (or NSF) in the United States has the capacity to change its funding structure in ways that might modify the behavior of various groups of scientists. So the normativity that stems from results like CONSENSUS may apply to individuals like the NSF, to agents (collective or otherwise) that are not themselves members of the groups at issue. The difficulty is that the normativity discussed in Goldman's systems approach will also apply to groups for whom there is no external individual that has a vested interest in the group's epistemic success.
I would argue, as I have at greater length elsewhere,Footnote 16 that we can get past these limitations by invoking a somewhat unconventional notion of epistemic rationality, one based on an instrumentalist conception of epistemology. The instrumentalist account of epistemic rationality can come in a number of flavors, all of which aim to view epistemic rationality as a species of practical or means-ends rationality. On such accounts, an agent is epistemically rational if she follows the belief-forming procedures that are the most effective means to accomplishing her epistemic ends, whatever those ends might be.Footnote 17 Although various versions of instrumentalist epistemology attracted supporters in the late 1980s and early 1990s (e.g. Foley Reference Foley1987; Kitcher Reference Kitcher1992; Kornblith Reference Kornblith1993; Laudan Reference Laudan1990), the view has fallen out of favor, perhaps due to some prominent attacks, such as Kelly (Reference Kelly2003). I believe the view can be defended from such attacks, although I won't present the defense here.
If we accept an instrumentalist account of epistemic norms, this opens up the possibility that the normativity suggested by results like CONSENSUS applies directly to the agents in the groups under considerations. Just as an agent can have individual epistemic goals, an agent that is a member of a group can have epistemic goals for the group. Consider the following example. A small group of cryptographers is set to work on an encoded message. Each of these members desires that the group as a whole cracks this code. We could think the goal of cracking the code as a group epistemic goal, since it is a goal for the group that each of the members of this group collectively shares. Notice that for such goals to exist, there need not be a group agent of any sort. For example, the group can have this collective goal even if the cryptographers never share information, never decide how to form the final group opinion, and don't accidentally come to a consensus or majority opinion about any matters relevant to their collective goal. Since each individual's group epistemic goal is itself an epistemic goal, we can see how such goals could generate epistemic norms on an instrumentalist account. If the agents in this group care most about their group epistemic goals, then they each ought to follow whatever belief-forming processes are most effective for achieving those ends.
Now imagine that, instead of cryptographers, we have a group whose members each desire to improve the group's collective accuracy in any cases where modifying their credences is guaranteed to do so. So long as this group goal is the dominant epistemic goal for the group's members, it will turn out that the group's members ought to come to consensus by averaging their credences when they find themselves in disagreement.Footnote 18 This is how we can utilize a result like CONSENSUS to generate epistemic norms that apply to the individual members of certain groups.
This instrumentalist take on group rationality raises some interesting possibilities. Let's call any epistemic norm that requires a group epistemic goal a ‘norm of group rationality’, and let's call any norm that only requires an individual epistemic goal a ‘norm of individual rationality’. A result like CONSENSUS allows us to show that norms of group rationality and norms of individual rationality can dictate different kinds of behavior for an agent. For example, take a randomly chosen individual from the group whose members wish to maximize their accuracy as discussed above. From the individual's perspective, she will actually think that coming to a consensus will make her assessment less accurate.Footnote 19 Therefore, if her dominant epistemic goal is to improve her expected individual epistemic accuracy whenever possible, she shouldn't come to a consensus with the group by averaging. On the other hand, she also knows that the group's accuracy will be improved if she comes to a consensus by averaging. Thus, if the agent's dominant epistemic goal is to improve the group's accuracy, then she should come to a consensus. This suggests that group norms of rationality and individual norms of rationality can come into conflict.Footnote 20 Agents that find themselves in such situations are thus forced to choose which epistemic goals are the ones they hold most dear.
6. CONCLUSION
Through his discussion of veritistic value, Goldman (Reference Goldman1999) became one of the first to present a scoring rule for the accuracy of a group of agents' credences, and little has been written on the topic since (at least in the philosophical literature). In this essay, I hope to have shown that the topic deserves our careful attention. First, Goldman's rule has serious limitations, and the natural extension of the rule has absurd consequences. Thus it shouldn't be seen as the final word on the topic. Second, the rule can be repaired, and repairing the rule carries with it an interesting consequence, namely, that groups in disagreement could improve their accuracy by coming to a consensus at the average of their credences. Last, this consequence can be used to show that norms of individual and group rationality can at times dictate conflicting kinds of behavior for an agent.
I'd like to conclude by discussing a few of the assumptions the consensus result relies upon, how these might be weakened, and what effect weakening them might have on the main result of the paper.
The first assumption the result relies upon, which traces back to Goldman's original rule, is that for us to judge a group's accuracy we ought to first measure the inaccuracies of the individuals and then take a flat average of these values. This assumption might seem problematic for a couple of reasons. For one, weighing the accuracies of certain members more than others might make sense for some groups. Take for example a group with a decision-making structure in place such that the views of certain members make a larger impact on the group's ultimate decisions. Within such a group, giving greater weight to the accuracies of these more important members may seem more reasonable than simple averaging. Fortunately, accounting for such groups takes very little modification. Recall that the condition of convexity in the group case originally contained a series of terms μ1, μ2, …, μm that must sum to 1 which amounted to the weights that we would give to the score of each individual. In the simple case, we set each of these to the same value, meaning that we took a flat average of the inaccuracy values of each individual's belief function. We need not do this. If certain individuals make a greater impact on the decisions of a group, and others make only a small impact, we can account for this when we determine how much to weigh each member's inaccuracy. What results is a weighted average, and the consensus result would still follow, though in this case the group does better if their credences are set to that differentially weighted average.
While the above worry can be addressed without much modification, one might wonder more generally why a group's inaccuracy ought to be an average of the individual inaccuracies of the members (weighted or otherwise) in the first place. One alternative would be to simply sum the inaccuracies of the group's members. We can show that such an alternative isn't preferable since, according to the rule, smaller groups would have an unfair advantage over larger groups. In fact, a group could improve its score by dropping a member who is strictly more accurate than any other member of the group. Another alternative would be to have a rule that works by first finding the average group credence and then scoring it. Any rule that operates like this wouldn't yield the consensus result, since the scores before and after consensus would be exactly the same. While I think such rules may seem like reasonable competitors when we focus solely on groups with some sort of belief aggregation structure (like the kinds of groups discussed by List and Pettit (Reference List and Pettit2011)), such rules clearly aren't appropriate for groups that don't undergo any kind of belief aggregation. Take, for example, a group of market forecasters at an investment firm that have the common purpose of determining whether a certain stock will rise in the next quarter. If they don't put in place any structure to aggregate their judgments, judging their collective accuracy based on the average of their credences would seem odd. Doing so would make groups that are wildly divergent just as accurate as groups in agreement, so long as the averages match. But if the former were to go out and bump around in the world (in this case making individual investments with the firm's funds), they would almost certainly be less successful in their goals than the latter.Footnote 21
Various other ways of measuring a group's accuracy will likely be legitimate for some kinds of groups. For example, teachers often make use of collective learning strategies—group work, for example—in order to help the students who generally perform the worst. If we wanted to judge how successful such strategies are, judging the accuracy of the group by the score of its least accurate member might make sense. Similarly, many groups are content with having some badly inaccurate members, so long as at least one member figures out the truth of the matter. The relevant sense of accuracy for such groups might reduce to the accuracy of the group's most accurate member.Footnote 22 There are likely many other types of groups for which group accuracy should take on some other form than the one sketched above. We would have to examine each to see if CONSENSUS would hold for such rules, since in some cases it won't.Footnote 23
