Hostname: page-component-7b7567c8c-tnzcf Total loading time: 0 Render date: 2026-11-13T22:48:12.603Z Has data issue: false hasContentIssue false
  • English
  • Français

Jurors are competent cue-takers: how institutions substitute for legal sophistication1

Published online by Cambridge University Press:  15 February 2007

Cheryl Boudreau
Cheryl Boudreau*
Affiliation:
University of California, San Diego, Department of Political Science
Rights & Permissions [Opens in a new window]

Abstract

What conditions are necessary for juries to work effectively? Legal scholars and social scientists have debated this question, and research in psychology and law demonstrates that jurors are easily confused by scientific evidence and readily swayed by the slick framing of argument. Rather than condemn juries as unworkable, however, I demonstrate experimentally that jurors need not possess legal or scientific sophistication to make reasoned choices during trials. Specifically, I demonstrate that various institutions embedded in our legal system (such as penalties for lying and the threat of verification) can substitute for sophistication and enable even unsophisticated individuals to learn what they need to know. Based on these findings, I argue that rather than advocate blue ribbon juries and bench trials as replacements for citizen juries, scholars should instead seek substitutes for jurors’ lack of sophistication in the institutions of our legal system.

Information

Type
Research Article
Information
International Journal of Law in Context , Volume 2 , Issue 3 , September 2006, pp. 293 - 304
Copyright
Copyright © Cambridge University Press 2006

‘In some types of modern trials – such as those involving complex scientific findings or esoteric economic or mathematical evidence – there probably is no adequate substitute for actually comprehending the evidence, the arguments of the parties, and the judge’s instructions.’

(Lilly Reference Lilly2001, p. 70)

Whether or not jurors are competent to perform their duties is a hotly debated topic among legal scholars, social scientists, jurists and others. While various scholars emphasise jurors’ ability to learn from the examinations, cross-examinations and competing expert witnesses presented at trial (Cooper, Bennett and Sukel, Reference Cooper, Bennett and Sukel1996; Lupia and McCubbins, Reference Lupia and McCubbins1998; Vidmar and Diamond, Reference Vidmar and diamond2001), many others lament jurors’ lack of basic knowledge about scientific evidence and legal procedures (Frank, Reference Frank1945; Elwork, Sales and Alfini, Reference Elwork, Sales and Alfini1977; Fisher, Reference Fisher2000–2001). Still other scholars note the increasing complexity of the modern day jury trial and the concomitant decline in the education levels of jurors (Cecil, Lind and Bermant, Reference Cecil, Lind and Bermant1987; Lilly, Reference Lilly2001). Based on this body of literature, many have questioned whether the jury system can possibly work, given that jurors seem to lack the sophistication required for the decisions they must make. Indeed, if there are no ready substitutes for jurors’ lack of sophistication, then trials may not be decided by jurors, but rather by slick lawyering, savvy expert testimony, and chance.

Rather than condemn the jury system as unworkable, however, I demonstrate experimentally that institutional features of our legal system can substitute for sophistication and may enable even unsophisticated individuals to learn what they need to know. Stated differently, regardless of jurors’ levels of legal or scientific expertise and regardless of the complexity of evidence presented at trial, there are various institutional heuristics embedded in our legal system that help jurors to assess the veracity of witnesses’ statements and to learn from them. For example, jurors know that all witnesses face penalties for perjury if they lie on the stand, and attorneys’ cross-examinations (a form of verification) often reveal when witnesses have made false statements.Footnote 2 Further, during both direct- and cross-examinations, attorneys typically highlight the interests and incentives of the witnesses (for example, a prosecutor might reveal that an expert witness is being paid by the defence). As I demonstrate in this paper, when conditions such as these are met, even unsophisticated jurors are able to learn from witnesses’ statements and make reasoned choices.

Because I cannot systematically manipulate institutions, witnesses’ statements, or jurors’ levels of sophistication in real world courtroom settings, I instead conduct a laboratory experiment (extending Lupia and McCubbins’s Reference Lupia and McCubbins1998 experimental design) in which I analyse whether and under what conditions institutions can substitute for sophistication. Because such an analysis requires a measure of subjects’ sophistication at a particular task and because there does not exist an agreed upon measure of legal or scientific sophistication, I analyse instead subjects’ mathematical sophistication, which I argue is analogous to sophistication in general and to legal and scientific sophistication, in particular. The important advantage associated with analysing mathematical sophistication, of course, is that there exists an agreed upon and straightforward measure of it – namely, SAT math scores. For this reason, I obtained a pretest measure of subjects’ mathematical sophistication (i.e. their SAT math score) prior to the experiment, and I asked subjects in my experiment to answer math problems about which they were uncertain (i.e., subjects did not necessarily know the correct answer to the math problems). I then revealed the correct answer to one of the subjects (dubbed ‘the Speaker’), who made a statement to the other subjects that could inform them about the correct answer to the math problem.

I then examined whether and under what conditions even unsophisticated individuals could learn from the Speaker and solve correctly the given math problems. To this end, I varied whether the Speaker had common or conflicting interests with subjects, and I also imposed two institutional conditions that have direct analogues in the legal process: penalties for lying and the threat of verification. In the penalties for lying condition, both the Speaker and the subjects knew that the Speaker would incur a penalty if he or she lied about the correct answer to the math problem. Similarly, in the verification condition, both the Speaker and the subjects knew that the experimenter would verify the Speaker’s statement with some probability to make sure that it revealed the correct answer to the math problem.

My results demonstrate that exposure to a Speaker who has common interests with subjects, who faces a sufficiently large penalty for lying, or who is verified with a sufficiently high probability enables even unsophisticated individuals to answer the math problems correctly. Based on these results, it appears that institutions that create conditions, such as penalties for lying or verification, can substitute for sophistication and enable even unsophisticated jurors to make reasoned choices during trials.

This paper proceeds as follows. I begin with a discussion of the literature on juror (in)competence, and I emphasise that there is much disagreement among legal scholars, social scientists and jurists as to whether jurors are capable of making reasoned choices during civil and criminal trials. I then suggest that even though jurors may not possess legal or scientific sophistication, there may be institutional substitutes for sophistication embedded in our legal system. Next, I describe in detail the experimental design that I use to test this proposition, and I suggest a number of hypotheses regarding subjects’ ability to solve the math problems under each of my experimental conditions. I then briefly describe the data source and statistical methods that I use, and I also comment on the generalisability of my results. I next present my experimental results. I conclude with a discussion of the implications that this research has for debates about juror competence and jury reform.

The debate: are jurors competent to carry out their duties?

Jurors are incompetent information-processors

Much of the legal and social science literature on juror competency suggests that jurors are not sufficiently sophisticated to process the information and arguments that are presented during trials. Indeed, many legal scholars lament jurors’ lack of factual knowledge about scientific evidence and legal procedures, while others demonstrate that jurors are easily confused by complicated issues and readily swayed by lawyers’ slick framing of argument (Elwork, Sales and Alfini, Reference Elwork, Sales and Alfini1977; Fisher, Reference Fisher2000–2001; Lilly, Reference Lilly2001).Footnote 3 Given the apparent naïveté of the individuals who are charged with determining the guilt or innocence of the accused and deciding the liability and punitive damages of parties in civil suits, many scholars have questioned whether juries should decide the outcomes of trials (Hastie and Viscusi, Reference Hastie and Viscusi1998; Mogin, Reference Mogin1998; Sunstein, Kahneman and Schkade, Reference Sunstein, Kahneman and Schkade1998; Fisher, Reference Fisher2000–2001; Lilly, Reference Lilly2001; Sunstein et al., Reference Sunstein, Hastie, Payne, Schkade and Viscusi2002a, Reference Sunstein, Kahneman, Schkade and Ritov2002b; Schkade, Reference Schkade2002).Footnote 4

Indeed, many scholars’ doubts about the competence of jurors have led them to advocate various reforms. Specifically, some scholars suggest that, under certain circumstances, we should replace citizen juries with juries composed of experts (otherwise known as ‘blue ribbon’ juries) or with judges (Lempert, Reference Lempert1981; Strier, Reference Strier1994, Reference Strier1997; Mogin, Reference Mogin1998; Fisher, Reference Fisher2000–2001). Thus, according to these scholars, lay jurors are not sufficiently sophisticated to make reasoned choices during trials and they should, therefore, be replaced with other, more competent decision-makers.

Jurors are competent cue-takers

In contrast to those who doubt the competence of jurors and who question the efficacy of the jury system, many other scholars suggest that jurors, despite their lack of legal or scientific sophistication, can nonetheless learn what they need to know during the course of a trial (Kalven and Zeisel, Reference Kalven and Zeisel1966; Hastie, Penrod and Pennington, Reference Hastie, Penrod and Pennington1983; Cecil, Lind and Bermant, Reference Cecil, Lind and Bermant1987; Cecil, Hans and Wiggins, Reference Cecil, Hans and Wiggins1991; Cooper, Bennett and Sukel, Reference Cooper, Bennett and Sukel1996; Lupia and McCubbins, Reference Lupia and McCubbins1998). These scholars (like those discussed in the preceding section) recognise that jurors may not possess the formal education or training that is needed for processing the complex, technical evidence presented during trials. However, rather than condemn the jury system as unworkable, scholars in this camp suggest that jurors are able to use heuristics and cues to assess the value of particular pieces of evidence and to arrive at reasoned choices (Hovland and Weiss, Reference Hovland and Weiss1951; Chaiken, Reference Chaiken1980; Hass, Reference Hass, Petty, Ostrom and Brock1981; Petty and Cacioppo, Reference Petty and Cacioppo1984; Cecil, Lind and Bermant, Reference Cecil, Lind and Bermant1987; Cooper, Bennett and Sukel, Reference Cooper, Bennett and Sukel1996; Shuman, Champagne and Whitaker, Reference Shuman, Champagne and Whitaker1996; Lupia and McCubbins, Reference Lupia and McCubbins1998).Footnote 5

It is this body of research on heuristics and cues that I seek to build upon in this study. Specifically, I focus on how institutions provide jurors with valuable heuristics, and I analyse experimentally whether and when these heuristics can substitute for sophistication and enable even unsophisticated jurors to learn what they need to know. It is to a discussion of my experiments that I now turn.

The experimental design

In order to assess whether the institutions of our legal system enable even unsophisticated individuals to learn what they need to know, I design a two-group randomised experiment that analyses one particular type of sophistication – namely, mathematical sophistication. Specifically, I obtain a pretest measure of subjects’ mathematical sophistication prior to the experiment (i.e. their SAT math score), and I then randomly select subjects into treatment and control groups. I next ask subjects to solve a series of binary choice math problems (i.e. the problems ask subjects to choose either ‘true’ or ‘false’, or ‘a’ or ‘b’ as the correct answer). The math problems that I use were drawn from an SAT II, level 2 math test and consist of many different types of problems (i.e. algebra, geometry, calculus) and several levels of difficulty. I told subjects in both the treatment and control groups that they would have 60 seconds to solve each math problem and that they would earn 50 cents for each problem that they answered correctly, that they would lose 50 cents for each problem that they answered incorrectly, and that they would neither earn nor lose 50 cents if they left a problem blank.

The main difference between the treatment and control groups has to do with the conditions under which subjects solved the math problems. In the control group, subjects solved the 24 math problems one at a time, with 60 seconds allotted for each problem. For each problem that subjects in the control group solved, I paid them according to whether they solved the problem correctly, incorrectly, or left the problem blank.

In the treatment group, subjects solved these very same 24 math problems one at a time, and they were also paid according to whether they solved the problem correctly, incorrectly, or left the problem blank. What differed between the treatment and control groups, however, was that subjects in the treatment group solved these 24 problems under very different conditions than did subjects in the control group. Indeed, before subjects in the treatment group solved any of the math problems, the experimenter randomly selected one subject to act as ‘the Speaker’ in the remainder of the experiment.

The Speaker’s role in the experiment was far different from that of the other subjects; that is, unlike the other subjects (whose role in the experiment was still to solve the 24 math problems one at a time), the Speaker was shown the correct answer to each math problem and was then allowed to make a statement to the other subjects about the answer to the math problem. After the Speaker made his or her statement, the other subjects were then given 60 seconds to solve that particular math problem and to mark their answers.

The key to this experiment is that both the Speaker and the subjects knew that the Speaker could make any statement that he or she wished. That is, the Speaker could lie about the correct answer to the math problem or tell the truth; it was entirely up to him or her. The Speaker’s ability to make whatever statement he or she wished was constant throughout this experiment and was designed to be an analogy to Crawford and Sobel’s (Reference Crawford and Sobel1982) and Lupia and McCubbins’ (Reference Lupia and McCubbins1998) models.

Although the Speaker could lie or tell the truth in all of my experimental conditions, I varied the conditions under which the Speaker made his or her statement. Specifically, I varied two types of things: the interests between the Speaker and subjects, as well as the institutional conditions that were imposed upon the Speaker. When varying the interests between the Speaker and subjects, I first analysed subjects’ ability to solve the math problems when the Speaker had common interests with them. I then made the Speaker and subjects have conflicting interests, at which point I imposed various institutional conditions, such as a penalty for lying or the threat of verification.

Because the strength of these institutional conditions is likely to vary in real world courtroom settings, I varied both the size of the penalty for lying and the probability of verification when running my experiments. Specifically, I tested the effects that different size penalties (namely, a $15 penalty, a $5 penalty and a $1 penalty) and different probabilities of verification (specifically, a 100%, 90%, 70%, 50%, or 30% chance of verification) had on subjects’ ability to solve the math problems correctly. Each of these experimental variations (i.e. common interests, conflicting interests, penalties for lying, and verification) was common knowledge at the outset of each part of the experiment.

So how did I vary the interests between the Speaker and subjects and impose the institutional conditions within the context of my experiments? In short, I varied both interests and institutions by manipulating the ways that the Speaker and the subjects got paid. For example, in the common interests condition, subjects knew that they would be paid 50 cents for each and every math problem that they answered correctly. Similarly, the Speaker knew that he or she would be paid 50 cents for each and every subject who got a particular math problem correct. So, if 11 subjects answered the math problem correctly, they would earn 50 cents each, and the Speaker would earn $5.50 (i.e. 50 cents for each of the 11 subjects who answered the problem correctly).

For my next experimental variation, I established conflicting interests between the Speaker and subjects, and I then imposed one of my institutional conditions. To induce conflicting interests between the Speaker and subjects, I merely altered the way that the Speaker got paid. That is, unlike the common interests condition, the Speaker in the conflicting interests condition earned 50 cents for each and every subject who got the math problem wrong. Subjects, on the other hand, still earned 50 cents for solving the math problems correctly. Simultaneously, I also implemented one of my institutional conditions – namely, a penalty for lying. So, in this segment of the experiment, the Speaker and subjects had conflicting interests, but I announced to both the Speaker and subjects that the Speaker would incur a penalty (of either $15, $5, or $1, depending on the experimental condition) if he or she lied about the correct answer to the math problem.

For my other institutional condition – namely, verification – I again maintained conflicting interests between the Speaker and the subjects. However, instead of imposing a penalty for lying upon the Speaker, this time, I verified the Speaker’s statement with some probability to make sure that it was a true statement before it was read to subjects. In the 100% chance of verification condition, if the Speaker chose to make a false statement about the correct answer to the math problem, then I charged the Speaker two dollars and announced the correct answer to subjects. If the Speaker chose to make a true statement, then I simply read the Speaker’s statement to the subjects. However, in the 90% chance of verification condition, I rolled a ten-sided die before the Speaker made his or her statement. If the die landed on 1, 2, 3, 4, 5, 6, 7, 8 or 9, then I verified the Speaker’s statement, charged him or her two dollars if he or she had chosen to make a false statement, and announced the correct answer to the math problem to the subjects. If the die landed on a 10, however, then I simply announced the answer that the Speaker chose to report, regardless of whether it was correct or incorrect. The 70%, 50% and 30% chance of verification conditions proceeded in a similar manner.

Hypotheses

Many of the experimental conditions described above yield straightforward predictions about subjects’ ability to solve the math problems. For example, in the presence of a Speaker who has common interests with subjects, I predict that subjects will have an improved ability to answer the math problems correctly, relative to subjects in the control group. This prediction stems directly from Lupia and McCubbins (Reference Lupia and McCubbins1998) and Crawford and Sobel’s (Reference Crawford and Sobel1982) models, which demonstrate that, in equilibrium, common interests between the Speaker and receivers (who in my experiments are the subjects) induce the Speaker to tell the truth and the receivers to base their choices upon what the Speaker says.

The reasoning behind this common interests prediction is perhaps best understood by considering the nature of my experiments. Recall that in the common interests condition, the Speaker earns money each time a subject solves a particular problem correctly. I therefore expect the Speaker to tell subjects the truth about the correct answer to the math problem, and I also expect subjects to trust the Speaker’s statement, to learn from it, and to base their answers to the math problem upon it. Indeed, because each experimental variation is common knowledge at the beginning of the experiment, subjects know that both they and the Speaker are better off financially if they solve the problems correctly. Thus, I offer the following hypothesis:

Common Interests Hypothesis: Subjects exposed to a Speaker who has common interests with them should have an improved ability to answer the math problems, relative to subjects in the control group.

In the $15 penalty for lying condition, I also expect subjects to have an improved ability to solve the math problems correctly. This prediction, again, stems from Lupia and McCubbins’s (Reference Lupia and McCubbins1998) model, which demonstrates that when the penalty for lying is sufficiently large, then, in equilibrium, the Speaker never lies and the receivers trust the Speaker’s statements.

In my experiments, the $15 penalty for lying is in fact ‘sufficiently large’ – that is, it is large enough to ensure that the Speaker always has an incentive to tell the truth. To see why this is the case, consider the way that the Speaker earns money under this condition when there are 11 subjects solving the math problems:Footnote 6 Given that there are conflicting interests between the Speaker and the subjects, the Speaker earns $5.50 if each and every subject answers a problem incorrectly. Although at first blush this might seem to give the Speaker an incentive to lie, note that the $15 penalty for lying would reduce the Speaker’s gain of $5.50 down to a loss of $9.50. Further, if the Speaker lies and all of the subjects happen to answer the problem correctly (perhaps realising that the Speaker is lying), then the Speaker would lose $20.50 (i.e. a $15 loss because of the penalty for lying and a $5.50 loss because 11 subjects answered the problem correctly).

If the Speaker tells the truth, however, then the worst he or she can do is to lose $5.50 (which would happen if each and every subject answered the problem correctly), and the best that he or she can do is to earn $5.50 (which would happen if each and every subject answered the problem incorrectly). As these payoffs make clear, the Speaker is always better off if he or she tells the truth about the correct answer to the math problem. Thus, I expect that the Speaker will make a truthful statement about the correct answer to the math problem, and I also expect subjects to trust and then learn from the Speaker’s statement. Hence, I advance the following hypothesis:

$15 Penalty for Lying Hypothesis: Subjects exposed to a Speaker who faces a $15 penalty for lying should have an improved ability to answer the math problems, relative to subjects in the control group.

In the 100% chance of verification condition, I also expect subjects to have an improved ability to answer the math problems correctly. As Lupia and McCubbins (Reference Lupia and McCubbins1998) note, increasing the probability of verification decreases the probability that the Speaker can benefit from making a false statement.Footnote 7 Thus, when the Speaker is verified with certainty (as is the case in this experimental condition), subjects should trust that the Speaker’s statement is correct and base their answers upon it. For this reason, I make the following prediction:

100% Verification Hypothesis: Subjects exposed to a Speaker who is verified with certainty should have an improved ability to answer the math problems, relative to subjects in the control group.

As for the other institutional conditions that I imposed in my experiment (i.e. a $5 penalty, a $1 penalty and a 90%, 70%, 50% or 30% chance of verification), I do not have any ex ante predictions about how they will affect subjects’ ability to solve the math problems. The reason for this lack of predictions is simply that, under each of these conditions, the Speaker may earn more money if he or she lies about the correct answer to the math problem. Knowing this, subjects solving the math problems may or may not trust and then learn from the Speaker’s statement.

Methodology

In order to test the above hypotheses, I conducted laboratory experiments at a large public university. When recruiting subjects, I posted flyers at various locations on campus (for example, in front of the library, in the cafeterias, in the dormitories and in academic buildings), and I also sent out campus-wide emails to advertise the experiments. A total of 263 adults who were enrolled in undergraduate classes participated.

Although I use these students as my source of data, note that my experimental results generalise to all humans. Indeed, because my experiments analyse the general processes of human learning and communication and because there is no reason to believe that these processes are different for college undergraduates than for the rest of the population, my results demonstrate the conditions under which all humans (be they college undergraduates or members of the general population) can learn from others.Footnote 8

When testing the above hypotheses, I simply analysed the differences among subjects in my various treatment groups and subjects in the control group (recall that subjects in the control group simply solved the problems on their own, i.e. without a Speaker). Specifically, I analysed the percentage of math problems that subjects in each condition answered correctly, and I then conducted difference of means tests to examine whether subjects who were exposed to the treatment in each condition performed significantly better on the math problems than did subjects in the control group.Footnote 9

Results

Confirming my predictions

Each of my predictions was in fact borne out in the data. That is, subjects who were exposed to a Speaker who had common interests with them, who faced a $15 penalty for lying, or who faced a 100% chance of verification exhibited an improved ability to answer the math problems, relative to subjects in the control group. Specifically, subjects in the control group answered only 40% of the math problems correctly (N = 66), while subjects who were exposed to a Speaker who had common interests with them answered 90% of the math problems correctly (N = 62). Subjects who were exposed to a Speaker who faced a $15 penalty for lying answered 86% of the math problems correctly (N = 46), while subjects who were exposed to a Speaker who faced a 100% chance of verification answered 83% of the math problems correctly (N = 54). Each of these differences between treatment and control group subjects is statistically significant (specifically, p = 0 for each of these comparisons between treatment and control groups).

The effects of smaller penalties

Having confirmed each of my three hypotheses, I now examine the effect that smaller penalties for lying have on subjects’ ability to solve the math problems. Indeed, because penalties for lying in a real world courtroom may not always be large enough to ensure that witnesses always tell the truth, I investigated experimentally the effects that smaller penalties (which sometimes make the Speaker better off if he or she lies about the correct answer to the math problem) have on subjects’ ability to answer the math problems correctly.

As my results demonstrate, there is a decrease in subjects’ ability to solve the math problems when the $15 penalty for lying is reduced to $5 or $1. Specifically, subjects who were exposed to a Speaker who faced a $5 penalty for lying answered only 46% of the math problems correctly (N = 39), while subjects who were exposed to a Speaker who faced a $1 penalty for lying answered 54% of the math problems correctly (N = 66). Both of these percentages are significantly lower than the percentage of problems that subjects answered correctly in the $15 penalty for lying condition (specifically, p = 0 when both the $5 penalty and $1 penalty conditions are compared with the $15 penalty condition).

Despite this decrease in the percentage of math problems that subjects answer correctly in the $5 and $1 penalty for lying conditions, subjects in one of these conditions still performed significantly better on the math problems than did subjects in the control group. Specifically, a difference of means test reveals that the 54% of problems that subjects answered correctly in the $1 penalty for lying condition is significantly greater than the 40% of problems that subjects in the control group answered correctly (p = 0.0001). Thus, it appears that these smaller penalties for lying, though not as effective as a $15 penalty, may enable subjects to perform better on the math problems.

The effects of smaller probabilities of verification

Just as I examined the effect that reducing the size of the penalty for lying had on subjects’ ability to solve the math problems, so too do I analyse the effects of reducing the probability of verification. Indeed, given that a 100% chance of verification is unlikely to occur in a courtroom, it seems particularly useful to examine the extent to which subjects are able to learn from the Speaker when the probability of verification is reduced. Thus, I assessed whether and to what extent 90%, 70%, 50% and 30% chances of verification reduced the percentage of math problems that subjects answered correctly.

As my results reveal, there is a decrease in subjects’ ability to solve the math problems when the 100% chance of verification is reduced to 90%, 70%, 50% or 30%. Specifically, subjects who were exposed to a Speaker who faced a 90% chance of verification answered 67% of the math problems correctly (N = 11), while subjects who were exposed to a Speaker who faced a 70% chance of verification answered 59% of the math problems correctly (N = 61). Subjects who solved the math problems in the presence of a Speaker who faced a 50% chance of verification answered 60% of the math problems correctly (N = 31), and subjects who were exposed to a Speaker who faced a 30% chance of verification answered only 46% of the problems correctly (N = 52). All of these percentages are significantly lower than the percentage of problems that subjects answered correctly in the 100% chance of verification condition (p = 0.04 for the 90% verification condition, p = 0 for the 70% and 30% verification conditions, and p = 0.0002 for the 50% verification condition).

Although there are decreases in subjects’ ability to solve the math problems correctly when smaller probabilities of verification are imposed, note that subjects in several of these conditions still performed significantly better on the math problems than did subjects in the control group. Specifically, a difference of means test revealed that the percentages of problems that subjects answered correctly in the 90%, 70% and 50% chance of verification conditions were significantly greater than the 40% of problems that subjects in the control group answered correctly (p = 0.0023 for the 90% verification condition, p = 0 for the 70% verification condition, and p = 0.0003 for the 50% verification condition). Thus, it appears that these smaller probabilities of verification (although not as effective as a 100% chance of verification) still enable subjects to improve their performance on the math problems.

How do institutions affect sophisticated vs. unsophisticated individuals?

Given that one of my central claims is that institutions enable even unsophisticated individuals to learn what they need to know, I now compare the effects that various institutional conditions have on subjects who are and who are not mathematically sophisticated. When classifying subjects as mathematically sophisticated or unsophisticated, I relied upon the SAT math scores that subjects provided prior to the experiment, as well as the nationwide SAT math percentile rankings that the Educational Testing Service (ETS) releases each year. Specifically, subjects whose SAT math scores fell in the 97th percentile or higher were considered to be mathematically sophisticated, while subjects whose SAT math scores fell in the 27th percentile through the 79th percentile were considered to be mathematically unsophisticated. Stated in terms of the scores associated with these percentile rankings, the mathematically sophisticated subjects’ SAT math scores ranged from 740 points to 800 points, while the mathematically unsophisticated subjects’ SAT math scores ranged from 450 points to 620 points.Footnote 10

I then examined the effects that common interests, a $15 penalty for lying, and a 100% chance of verification had on mathematically sophisticated and mathematically unsophisticated subjects’ ability to solve the math problems. As I discussed previously, each of these conditions dramatically increases the percentage of math problems that subjects answer correctly, relative to the percentage of math problems that subjects in the control group answer correctly. The question that those results leave open, however, is whether these conditions have different effects on subjects who are and who are not mathematically sophisticated. For example, it is possible that these institutional conditions are more effective for subjects who are mathematically sophisticated and who, therefore, already have some idea about how solve particular problems. On the other hand, it is also seems possible that these institutions would be more helpful to the mathematically unsophisticated, who might be more willing to listen to (and then learn from) the Speaker, as opposed to trying to solve the math problems on their own. Given these two conflicting conjectures, I broke down my aggregate results to examine the percentage of problems that mathematically sophisticated and mathematically unsophisticated subjects answered correctly in each experimental condition.

As the results in Table 1 reveal, the common interests, $15 penalty for lying, and 100% chance of verification conditions enabled both sophisticated and unsophisticated subjects to improve their performance on the math problems, relative to their sophisticated and unsophisticated counterparts in the control group. Further, in each of these experimental conditions, there was no statistically significant difference in the percentage of math problems that sophisticated and unsophisticated subjects answered correctly. Indeed, both sophisticated and unsophisticated subjects answered over 81% of the math problems correctly in each of these three experimental conditions. This result suggests the power of these institutional conditions, for regardless of subjects’ initial endowments of mathematical sophistication, the institutions enabled them to significantly improve their performance on the math problems.

Table 1

The effects that common interests, a $15 penalty for lying and a 100% chance of verification have on sophisticated and unsophisticated subjects∗

∗ Bold type indicates a statistically significant increase, relative to the control group, for each of the two levels of sophistication.

Note, however, that there was a statistically significant difference between the performance of mathematically sophisticated and mathematically unsophisticated subjects in the control group. Specifically, the mathematically sophisticated subjects in the control group answered 58% of the problems correctly, while mathematically unsophisticated subjects answered only 27% of the problems correctly (p = 0). Based on this finding, it appears that institutions can ‘level the playing field’ between sophisticated and unsophisticated individuals (see Kuklinski et al., Reference Kuklinski, Quirk, Jerit and Rich2001; Rahn, Aldrich and Borgida, Reference Rahn, Aldrich and Borgida1994 for a discussion of how other environmental conditions can level the playing field between sophisticates and novices). Indeed, my results show that these institutional conditions effected much larger improvements among unsophisticated subjects (relative to their unsophisticated counterparts in the control group) than among sophisticated subjects (relative to their sophisticated counterparts in the control group).

Conclusion

As the results of my experiments demonstrate, institutions can in fact provide individuals with heuristics that help them to learn what they need to know. Indeed, in many of the institutional conditions that I imposed in this experiment, subjects solved a significantly greater percentage of the math problems correctly than did subjects in the control group. Further, it appears that institutions enable even individuals who are mathematically unsophisticated to solve the math problems correctly. Taken together, these results demonstrate the powerful, positive effects that institutions can have on otherwise unsophisticated individuals.

These results have a number of implications for debates within law and the social sciences. As I noted earlier, within both the legal and social science literatures a debate has raged over whether jurors must possess legal or scientific sophistication to make reasoned choices (Frank, Reference Frank1945; Fisher, Reference Fisher2000–2001; Elwork, Sales and Alfini, Reference Elwork, Sales and Alfini1977) or whether heuristics and cues are sufficient for jurors to learn what they need to know (Vidmar and Diamond, Reference Vidmar and diamond2001; Lupia and McCubbins, Reference Lupia and McCubbins1998; Cooper, Bennett and Sukel, Reference Cooper, Bennett and Sukel1996). The results of my experiments largely support scholars in the heuristics camp, for they suggest that jurors can in fact learn what they need to know even when they lack sophistication. Specifically, in my experiments, many subjects lacked the ability to solve various types of math problems, but even without such ability, they were often able to learn from a knowledgeable and trustworthy Speaker’s statement and solve correctly the math problems. In this way, otherwise unsophisticated subjects were able to achieve results similar to those that I would expect from sophisticated subjects.

For this reason, my results suggest that calls for reforming (or doing away with) citizen juries are premature. Indeed, the experiments that I discussed in this paper suggest that it is possible to design institutions that promote learning even in the face of an unsophisticated citizenry. Although my experimental results suggest that the efficacy of such institutions is necessarily fragile (for example, recall the declines in the percentage of math problems that subjects answered correctly once the penalty for lying or the probability of verification was reduced), they also indicate that, at least under certain conditions, large improvements can be achieved. I will have more to say about the fragility of institutions and heuristics in future work (see, e.g., Boudreau, Reference Boudreau2006), but suffice it to say for now that sophistication does not appear to be a prerequisite for learning and making reasoned choices. Thus, rather than advocate blue ribbon juries and bench trials as substitutes for citizen juries, scholars should instead seek substitutes for jurors’ lack of sophistication in the institutions of our legal system.

Footnotes

2 For further discussion of this point, see Lupia and McCubbins (Reference Lupia and McCubbins1998, ch. 10).

3 For an interesting discussion of the limitations of framing effects, see Druckman (Reference Druckman2001).

4 Note that even prominent newspapers have suggested that juries should be replaced with judges. Indeed, as Blom-Cooper (Reference Blom-Cooper2003, p. 10) emphasises in his article in The Times, ‘The jury is the high point of amateurism, potentially a recipe for incompetence and bias. The mood of civilised systems of criminal justice increasingly demands professionalism. I am not contemptuous of the amateur’s ability to judge human conduct, only the task of evaluating evidence in the courtroom, which is a job for professionals.’

5 For an in-depth discussion of central versus heuristic information processing, see Petty and Cacioppo (Reference Petty and Cacioppo1986) and Chaiken (Reference Chaiken1980).

6 Indeed, the typical number of subjects in each experimental session was 12 (11 subjects who solved the math problems and one Speaker who made a statement to the other subjects).

7 Note that Austen-Smith (Reference Austen-Smith1994) also models verification, but he focuses on the speaker’s inability to verify his or her lack of knowledge to receivers.

8 That said, laboratory experiments always involve a tradeoff between internal validity and external validity (Trochim, Reference Trochim2001). Specifically, the main advantage of conducting my experiments in a controlled environment and randomly assigning subjects to treatment and control groups is that I am able to make internally valid causal inferences. For these very same reasons, however, my experiments are much weaker in external validity. In order to compensate for this, at a later date I plan to combine the results of these experiments with related quasi-experimental studies (which are weaker in internal validity but much stronger in external validity).

9 Note that I also analysed my data using nonparametric Wilcoxon Mann-Whitney tests. Unlike the standard difference of means test, the Wilcoxon Mann-Whitney test does not require the assumption that the differences between two samples are normally distributed. None of the results reported in this paper change if the Wilcoxon Mann-Whitney test is used instead of a difference of means test.

10 Note that my results are highly robust to different definitions of mathematical sophistication. Indeed, defining mathematical sophistication as the top 1/3 or top 1/4 of all SAT math scores reported and defining the lack of mathematical sophistication as the bottom 1/3 or bottom 1/4 of all SAT math scores reported does not significantly change the results that I report in Table 1. My results also do not change if I instead consider subjects to be mathematically sophisticated if they score in the 98th percentile or higher on their SAT math test (i.e. have scores that range from 750 points to 800 points) and consider subjects to be mathematically unsophisticated if they score in the 27th percentile through the 70th percentile (i.e. have scores that range from 450 points to 580 points). My results also do not change if I instead obtain a measure of subjects’ mathematical sophistication by predicting the percentage of questions that they will answer correctly and then rank subjects according to their predicted percentage of correct answers.

References

Austen-Smith, David (1994) ‘Strategic Transmission of Costly Information’, Econometrica 62: 955–63.CrossRefGoogle Scholar
Blom-Cooper, Louis Sir (2003) ‘A Judge Can Do the Work of 12 Amateurs, and Better’, The Times, 21 October, p. 10.Google Scholar
Boudreau, Cheryl (2006) ‘When do Heuristics Help Citizens? Assessing the Conditions under which Heuristics Improve Decisions’, Working paper, University of California, San Diego.Google Scholar
Cecil, Joe S., Hans, Valerie P. and Wiggins, Elizabeth C. (1991) ‘Citizen Comprehension of Difficult Issues: Lessons from Civil Jury Trials’, American University Law Review 40: 727–74.Google Scholar
Cecil, Joe S., Lind, E. Allan and Bermant, Gordon (1987) Jury Service in Lengthy Civil Trials. Washington DC: Federal Judicial Center.Google Scholar
Chaiken, Shelly (1980) ‘Heuristic versus Systematic Information Processing and the Use of Source versus Message Cues in Persuasion’, Journal of Personality and Social Psychology 39: 752–66.CrossRefGoogle Scholar
Cooper, Joel, Bennett, Elizabeth A. and Sukel, Holly L. (1996) ‘Complex Scientific Testimony: How Do Jurors Make Decisions?’ Law and Human Behavior 20: 379–94.CrossRefGoogle Scholar
Crawford, Vincent and Sobel, Joel (1982) ‘Strategic Information Transmission’, Econometrica 50: 1431–51.CrossRefGoogle Scholar
Druckman, James N. (2001) ‘On The Limits Of Framing Effects: Who Can Frame?’, Journal of Politics 63: 1041–66.CrossRefGoogle Scholar
Elwork, Amiram, Sales, Bruce D. and Alfini, James J. (1977) ‘Juridic Decisions: In Ignorance of the Law or in Light of it?’ Law and Human Behavior 1: 163–89.CrossRefGoogle Scholar
Fisher, Michael A. (2000–2001) ‘Going for the Blue Ribbon: The Legality of Expert Juries in Patent Litigation’, Columbia Science and Technical Law Review 2: 1–55.Google Scholar
Frank, Jerome (1945) Courts on Trial: Myth and Reality in American Justice. Princeton, NJ: Princeton University Press.Google Scholar
Hass, R. Glen (1981) ‘Effects of Source Characteristics on Cognitive Responses and Persuasion’ in Petty, R. E., Ostrom, T. M. and Brock, T. C. (eds.) Cognitive Responses in Persuasion. Hillsdale: Erlbaum, pp. 141–72.Google Scholar
Hastie, Reid and Viscusi, W. Kip (1998) ‘What Juries Can’t Do Well: The Jury’s Performance as a Risk Manager’, Arizona Law Review 40: 901–21.Google Scholar
Hastie, Reid, Penrod, Steven D. and Pennington, Nancy (1983) Inside the Jury. Cambridge, MA: Harvard University Press.CrossRefGoogle Scholar
Hovland, Carl I. and Weiss, Walter (1951) ‘The Influence of Source Credibility on Communication Effectiveness’, Public Opinion Quarterly 15: 635–50.CrossRefGoogle Scholar
Kalven, Harry Jr. and Zeisel, Hans (1966) The American Jury. Boston, MA: Little, Brown.Google Scholar
Kuklinski, James H., Quirk, Paul J., Jerit, Jennifer and Rich, Robert F. (2001) ‘The Political Environment and Citizen Competence’, American Journal of Political Science 45: 410–24.CrossRefGoogle Scholar
Lempert, Richard O. (1981) ‘Civil Juries and Complex Cases: Let’s Not Rush to Judgment’, Michigan Law Review, 80: 68–132.CrossRefGoogle Scholar
Lilly, Graham C. (2001) ‘The Decline of the American Jury’, University of Colorado Law Review 72: 53–91.Google Scholar
Lupia, Arthur and McCubbins, Mathew D. (1998) The Democratic Dilemma: Can Citizens Learn What They Need to Know? Cambridge: Cambridge University Press.Google Scholar
Mogin, Paul (1998) ‘Why Judges, Not Juries, Should Set Punitive Damages’, University of Chicago Law Review 65: 179–223.CrossRefGoogle Scholar
Petty, Richard E. and Cacioppo, John T. (1984) ‘The Effects of Involvement on Responses to Argument Quantity and Quality: Central and Peripheral Routes to Persuasion’, Journal of Personality and Social Psychology 46: 69–81.CrossRefGoogle Scholar
Petty, Richard E. and Cacioppo, John T. (1986) Communication and Persuasion: Central and Peripheral Routes to Attitude Change. New York, NY: Springer-Verlag.CrossRefGoogle Scholar
Rahn, Wendy M., Aldrich, John H. and Borgida, Eugene (1994) ‘Individual and Contextual Variations in Political Candidate Appraisal’, American Political Science Review 88: 193–9.CrossRefGoogle Scholar
Schkade, David A. (2002) ‘Erratic by Design: A Task Analysis of Punitive Damages Assessment’, Harvard Journal on Legislation 39: 121–137.Google Scholar
Shuman, Daniel W., Champagne, Anthony and Whitaker, Elizabeth (1996) ‘Assessing the Believability of Expert Witnesses: Science in the Jury Box’, Jurimetrics Journal 37: 23–34.Google Scholar
Strier, Franklin (1994) Reconstructing Justice: An Agenda for Trial Reform. Westport, CT: Quorum Books.Google Scholar
Strier, Franklin (1997) ‘The Educated Jury: A Proposal for Complex Litigation’, DePaul Law Review 47: 49–79.Google Scholar
Sunstein, Cass R., Kahneman, Daniel and Schkade, David (1998) ‘Assessing Punitive Damages (With Notes on Cognition and Valuation in Law)’, Yale Law Journal 107: 2071–152.CrossRefGoogle Scholar
Sunstein, Cass R., Hastie, Reid, Payne, John W., Schkade, David A. and Viscusi, W. Kip (2002a) Punitive Damages: How Juries Decide. Chicago, IL: University of Chicago Press.CrossRefGoogle Scholar
Sunstein, Cass R., Kahneman, Daniel, Schkade, David and Ritov, Ilana (2002b) ‘Predictably Incoherent Judgments’, Stanford Law Review 54: 1153–215.CrossRefGoogle Scholar
Trochim, William M. K. (2001) The Research Methods Knowledge Base (2nd edn). Cincinnati, OH: Atomic Dog Publishing.Google Scholar
Vidmar, Neil and diamond, Shari Seidman (2001) ‘Juries and Expert Evidence’, Brooklyn Law Review 66: 1121–80.Google Scholar
Figure 0

Table 1The effects that common interests, a $15 penalty for lying and a 100% chance of verification have on sophisticated and unsophisticated subjects∗