%20-%20https://sketch.com%20--%3e%3ctitle%3e3ECBBF50-FD08-412A-943D-2CFC5E025A16%3c/title%3e%3cdesc%3eCreated%20with%20sketchtool.%3c/desc%3e%3cg%20id='Stylesheet'%20stroke='none'%20stroke-width='1'%20fill='none'%20fill-rule='evenodd'%3e%3cg%20transform='translate(-926.000000,%20-668.000000)'%20id='BUTTONS'%3e%3cg%20transform='translate(100.000000,%20597.000000)'%3e%3cg%20id='BTN-/-RIGHTS'%20transform='translate(816.000000,%2065.000000)'%3e%3cg%20id='ICON-/-Rights'%20transform='translate(10.000000,%206.000000)'%3e%3cpath%20d='M13.5752841,0%20L1.34943182,0%20C0.59375,0%200,0.586666667%200,1.33333333%20L0,22.6666667%20C0,23.4133333%200.620738636,24%201.34943182,24%20L17.6505682,24%20C18.40625,24%2019,23.3866667%2019,22.6666667%20L19,5.36%20L13.5752841,0%20Z%20M13.5752841,1.89333333%20L17.0838068,5.36%20L13.5752841,5.36%20L13.5752841,1.89333333%20Z%20M1.34943182,22.6666667%20L1.34943182,1.33333333%20L12.1988636,1.33333333%20L12.1988636,5.36%20C12.1988636,6.10666667%2012.7926136,6.69333333%2013.5482955,6.69333333%20L17.6235795,6.69333333%20L17.6505682,22.6666667%20L1.34943182,22.6666667%20Z'%20id='Shape'%20fill='%23595959'%20fill-rule='nonzero'%3e%3c/path%3e%3ccircle%20id='Oval'%20stroke='%23595959'%20stroke-width='1.5'%20fill='%23595959'%20cx='9.5'%20cy='13.5355339'%20r='5.5'%3e%3c/circle%3e%3cpolyline%20id='Path-2'%20stroke='%23FFFFFF'%20stroke-width='1.5'%20transform='translate(9.500000,%2013.035534)%20rotate(45.000000)%20translate(-9.500000,%20-13.035534)%20'%20points='8%2016.0355339%2011%2016.0355339%2011%2010.0355339'%3e%3c/polyline%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e)
I. Introduction
Most studies of statutory interpretation concern themselves with how statutes should be read. Many fewer ask how statutes are read in fact, and the few studies of this second question have been based on what judges say about what theories of interpretation lead them to read statutes as they do. The project of this article is to reexamine the reading of statutes from the ground up by asking what cognitive challenges and hazards are associated with it. Our goal is not to explain what judges do, though the results found here might have some provocative implications on that score. Our goal is to better understand the baseline position from which a reader—any reader—comes to the task of interpreting a law; it is to know what baggage a human who reads a statute starts out with, so as to think more clearly about whether and how the baggage might be put down.
This study approaches those questions empirically. We report the results of a lab experiment of how law students interpret statutes before they have heard anything about how to do it. They are asked to perform simple cognitive tasks that surround the reading of a statute, such as separating their reading of its plain English from their policy preferences about how it should be read or separating those things from predictions about what other readers will think the statute means or about how a court might read it. The results offer two lessons. First, it is very difficult to separate judgments about the linguistic meaning of a statute from policy preferences about it. The same is true of predictions about what courts will do: readers tend to predict that courts will do what the readers themselves prefer. These entanglements suggest that the first and probably the hardest challenge in learning to read statutes is not the mastery of theories about how do it (textualism, purposivism, and so forth). The challenge is to suppress the influence of policy preferences on interpretation or to at least make it a matter of choice rather than infection. What makes the challenge especially hard to address is the invisibility of it. Nobody thinks their readings are contaminated, yet they usually are; the subjective experience of interpreting a statute is an unreliable guide to the influences that bear on it.
Second, however, the results also show that certain ways of framing the interpretive question can help with that process of separation: they reduce the entanglement of preferences about a text from judgments about the meaning of it. The key is to start with a nonidealized, external reference point for making the interpretive judgment—an approach that puts the question objectively rather than subjectively. Instead of simply asking readers what they think the plain text of a statute means or how the authors of it would likely want it applied, they are better asked how ordinary readers would interpret the text. This reframing of the question much reduces the impact that the preferences of readers have on their reading of a statute’s language. We will consider why this might be so in due course.
Our overall goal, again, is to reconsider the baseline from which the process of interpreting statutes is understood to proceed. We suggest that the best model is not one in which a reader climbs from understanding little about theories of interpretation to gaining a sophisticated knowledge of them. The more significant climb is from a state in which preferences greatly infect textual judgments to a state in which they are well separated, for then one can distinctly and responsibly consider matters of interpretive methodology and judgments about policy, giving each their due without confusion and unintended influence between them. (Some theories of interpretation may call for the reader to exercise judgment about policy, but no theory calls for a reader to be influenced by those judgments to an uncertain and unintended extent.) The education offered to lawyers about the first climb is extensive. Education about the second is rare, or at least rarely explicit. It may be that despite this inattention, experienced lawyers and judges generally overcome the tendencies found here and do not need help separating their preferences from their interpretations. But there are some reasons to be skeptical of that claim, and we will have a few words to say about them at the end.
This article is related to three connected strands of the psychology literature on communication. First, our findings are consistent with a top-down psychological model of reading in which experience and context matters to how readers interpret multiple words that constitute clauses or sentences (Goodman Reference Goodman and Ruddell1985; Smith Reference Smith1994). This contrasts with a bottom-up model in which words with (invariant) definitions are connected to produce meaning that is more robust to individual reader variation. Second, our findings are related to the cognitive psychology literature that models the process by which readers define specific words as one in which readers categorize words with other like words (not necessarily in a sentence) and use experiences common to the words in a category to define all the words in the that category. Because readers start from different experiences, they may generate different word associations and thus meanings (Solan Reference Solan1998; Mullins Reference Mullins2003–4).Footnote 1 Our findings suggest that context and framing may trigger different word associations and thus meanings. Finally, our work is consonant with experimental work on motivated reasoning and self-serving bias. Prior work has examined how a reader’s preferences can affect the assessment of judicial opinions (Simon and Scurich 2011)Footnote 2 and also judgments about fair settlements of cases and predictions about court decisions (Loewenstein et al. Reference Loewenstein1993; Babcock et al. Reference Babcock1995). In combination with a prior experiment (Farnsworth, Guzior, and Malani Reference Farnsworth2010), this article suggests that preferences can also influence both personal reading of statutory texts and predictions about courts’ interpretations.
This article is closest in design and findings to Braman (Reference Braman2006) and Farnsworth et al. (Reference Farnsworth2010). Braman gave law students a mock legal brief with identical arguments on both sides of a legal dispute about political speech by public employees. She randomized the content of that speech (pro-choice or pro-life) and found that students’ opinion on the content affected their judgments about the legality of the speech. Farnsworth et. al. used an experimental design similar to the one employed in this study but asked law students to make threshold judgments about whether statutory text was ambiguous. Students’ preference over outcomes were found to affect their judgment about whether text was unclear. In this article we extend the Braman study by examining how the framing of the inquiry alters the influence of preferences on legal judgments. Importantly, it finds that framing can limit the influence of preferences, that is, help in debiasing readers. We extend the Farnsworth et al. study by considering not just the influence of preferences on the level of judicial discretion allowed by text but also the ultimate legal judgments made about texts.
Section II of this article describes our survey instrument. Section III describes the results. Section IV discusses the implications for interpretative theory. Section V addresses some limitations, including the fact that we survey law students rather than lawyers or judges and that our survey is conducted in an experimental rather than a practice setting. Section VI offers concluding thoughts.
II. Methodology
We proceeded by creating survey instruments and administering them to over 1,500 law students, most of them in their first semester of study.Footnote 3 In each survey, the respondent was presented with ambiguous statutes and facts to which they might apply. The statutes and facts were taken from Supreme Court cases involving federal criminal law or civil disputes. The respondents were told what position each side to the case took.
Respondents were then asked three question, though not always in the same order.Footnote 4 First, the respondents were asked what outcome of the case they preferred as a matter of policy, setting aside the text of the statute. Second, they were asked to interpret the text of the statute, setting aside their policy preferences. Each respondent was asked this question in one of three different ways.Footnote 5 Some were asked which reading of the text they thought was the best fit to its ordinary meaning; others were asked which reading ordinary readers of English would think the best fit to its ordinary meaning; and yet others were asked which reading they thought was the best fit to the drafters’ intent.Footnote 6 Third, respondents were asked on occasion to predict which reading a court would prefer. The goal, of course, was to find any relationships between answers to the question about policy preferences and answers to the interpretive and predictive questions. As we shall see, policy preferences seem to affect them all—but not to the same extent.
A. Cases
Each of our surveys contained questions about a number of different statutory cases. The results from the surveys were highly repetitive from case to case, so we review in detail four of the cases from our most recent survey instruments.Footnote 7
The first case (“gun use case”) was based on Smith v. United States, 508 U.S. 223 (1993).Footnote 8
A federal statute, 18 U.S.C. § 924(c), provides an enhanced prison sentence for anyone who “uses” a firearm “in relation to … a drug trafficking offense.” Defendant, a drug dealer, owned a gun. He approached a confederate and offered to trade him the gun for some cocaine. His confederate turned out to be an undercover police officer, and defendant was arrested. He was charged with violating 924(c). Defendant did not brandish the gun or use it in a threatening manner, but he did offer it as an item of barter.
The question is whether offering the gun in trade was a “use” of it within the meaning of 924(c) (in which case the defendant gets the extra time on his prison sentence). Defendant’s reading is that offering a gun in trade is not a “use.” The government’s reading is that it is a “use.”
The second case (“child pornography case”), adapted from United States v. X-Citement Video, Inc., 513 U.S. 64 (1994),Footnote 9 was somewhat more complicated.
A federal statute, 18 U.S.C. § 2252, reads in part as follows:
-
(a) Any person who—
-
(1) knowingly transports or ships in interstate or foreign commerce by any means including by computer or mails, any visual depiction, if—
-
(A) the producing of such visual depiction involves the use of a minor engaging in sexually explicit conduct; and
-
(B) such visual depiction is of such conduct; shall be punished as provided in subsection (b) of this section.
-
-
The defendant was accused of violating the statute by selling pornographic videotapes that included footage of a woman who was under the age of 18, and thus was a minor. He defended on the ground that when he sold the tape, he did not know the person on the tape was a minor.
The question is whether the word “knowingly” in section (1) applies to the phrase “the use of a minor” in section (1)(a). The defendant’s reading is that “knowingly” does modify “the use of a minor.” The government’s reading is that “knowingly” does not modify “the use of a minor.”
The third case (“false statements case”) was based on United States v. Yermian, 468 U.S. 63 (1984).Footnote 10 “The federal ‘false statements’ statute, 18 U.S.C. § 1001, says:”
Whoever knowingly and willfully falsifies, conceals or covers up by any trick, scheme, or device a material fact, or makes any false, fictitious or fraudulent statements or representations, or makes or uses any false writing or document knowing the same to contain any false, fictitious or fraudulent statement or entry, if the matter lies within the jurisdiction of any department or agency of the United States, shall be fined not more than $10,000 or imprisoned not more than five years, or both.
The defendant worked for a company that had a contract with the Department of Defense. The company asked him to fill out a questionnaire to obtain a security clearance. He did so. His company mailed the questionnaire to the Department of Defense. The Department discovered that the defendant’s answers contained false statements. He was charged with violating the statute quoted above. His defense was that he had not realized that his questionnaire would be forwarded to the government.
The question is whether the statute requires proof that a defendant knew the matter in question was within the jurisdiction of a government agency. The defendant’s reading is that the statute does require such proof. The government’s reading is that it does not require such proof.
The final case (“attorney’s fees” case) was a civil case based on West Virginia Univ. Hospitals v. Casey, 499 U.S. 83 (1991).
A federal statute, 42 U.S.C. § 1988, provides in relevant part: “In any action or proceeding to enforce a provision of section 1983 of this title, the court, in its discretion, may allow the prevailing party [to recover from the losing party] a reasonable attorney’s fee as part of its costs.” Plaintiff brought a successful lawsuit against the government to enforce section 1983 and sought to recover fees paid to experts who advised his attorney.
The question is whether fees paid to experts by an attorney are covered by the part of § 1988 allowing recovery of “a reasonable attorney’s fee.” The defendant’s reading is that fees of experts who advise an attorney are not covered by § 1988. The plaintiff’s reading is that experts’ fees are covered.
B. Questions
After presenting facts from one of the cases above, the survey asked each respondent about her policy preference:
Setting aside the text of the statute, who do you think should win as a matter of policy preference?
-
(A) I strongly prefer that the defendant win.
-
(B) I mildly prefer that the defendant win.
-
(C) I mildly prefer that the government win.
-
(D) I strongly prefer that the government win.
Each respondent was also asked whether the defendant or the government’s reading of the statute was better. This question was asked in different ways. Some respondents were asked:
Setting your policy preference aside, which reading better fits the ordinary meaning of the statute’s text?
-
(A) The defendant’s reading
-
(B) Probably the defendant’s reading
-
(C) Probably the government’s reading
-
(D) The government’s reading
We will refer to the question just shown above as the “ordinary meaning” question. Some respondents were instead asked a different question that we will call “drafters’ intent”:
Setting your policy preference aside, which reading of the statute is a better fit to what the drafters of the statute intended?
-
(A) The defendant’s reading
-
(B) Probably the defendant’s reading
-
(C) Probably the government’s reading
-
(D) The government’s reading
Other respondents were asked an “ordinary readers” question:
Setting your policy preference aside, which reading of the statute would ordinary readers of English think is a better fit to the ordinary meaning of the statute’s text?
-
(A) The defendant’s reading
-
(B) Probably the defendant’s reading
-
(C) Probably the government’s reading
-
(D) The government’s reading
Finally, after the gun use case in particular, respondents were asked about how they predict a court would interpret the statute:
Which side’s reading do you predict that a court would agree with?
-
(A) The defendant’s reading
-
(B) Probably the defendant’s reading
-
(C) Probably the government’s reading
-
(D) The government’s reading
We thus recorded policy preferences for everyone and examined the relationship between those preferences and the answers given to the other questions.
III. Results
A. The Influence of Preferences on Interpretation
Figures 1 –4 describe for the four cases the correlation between respondents’ preferences and their answers to the different versions of the interpretive question. The horizontal axis lines up respondents according to their policy preferences about the outcome of the case (from those who strongly preferred that the defendant win to those who strongly preferred that the government win). The vertical axis shows which side’s reading the respondents thought was best in reply to the various interpretive questions we asked. We coded the reading from 1 to 4, where 4 indicates the most progovernment. Using this scale, each line presents the average answer to an interpretive question among respondents with a given policy preference. The whiskers present the 95% confidence interval for each average.
Correlation between policy preference and interpretation in the gun use case.
Correlation between policy preference and interpretation in the child pornography case.
Correlation between policy preference and interpretation in the false statement case.
Correlation between policy preference and interpretation in the attorney’s fees case.
The basic pattern is clear: respondents’ judgments about the ordinary meaning of the statute’s text and about the drafters’ intent are highly correlated with their policy preferences—even though the respondents were instructed to set those preferences aside. The one remarkable exception is the question about which side’s interpretation ordinary readers would think better fits the ordinary meaning of the statute’s text. The answers to that question are significantly less correlated with respondents’ policy preferences in every case. These results are confirmed in the regression analysis (unreported). Let us consider each of these findings in detail.
1. Ordinary Meaning
The results with respect to the ordinary meaning question are stark. Judgments about the ordinary meaning of a text are highly entwined with policy preferences about the outcome of the case the text is being used to decide. What makes the finding especially striking is that the respondents were explicitly told to separate the two considerations. They could not do it.
This failure has several implications. First and most simply, interpretations of the ordinary meaning of a text are highly prone to bias by the policy preferences of whoever is making the claim. Second, the makers of such claims are not likely to subjectively experience themselves as biased in this way. Their intentions were otherwise. The influence exerts itself invisibly.
2. Drafters’ Intent
The same pattern appears in answers to the drafters’ intent question.Footnote 11 We thought it possible that separating oneself a bit from the question of the statute’s meaning—being asked not what you think it means, but what you believe the author wanted—might reduce the influence of one’s own policy preferences. Unfortunately it does not. Estimates of the drafters’ intent, like judgments about ordinary meaning, closely track the respondents’ own wishes.
One reason for this result might be that respondents project their own preferences onto the legislators who they imagine drafted the bill. Here, as with the first question, the projection evidently is unconscious, for again the respondents were told to put their preferences aside when answering the question.
We wondered whether the answer to the drafters’ intent question might help explain the answers to the previous question on ordinary meaning. Maybe one reason policy preferences are bound up with replies to the ordinary meaning question is that people try to determine ordinary meaning by guessing at what the drafters of the statute must have meant, and they can only make headway on that question by asking what they themselves would have wanted if they had been the drafters. This may be part of the story, but it cannot be all of it, because the responses to the ordinary meaning question and the drafters’ intent sometimes were different in significant ways.
The differences between responses to the ordinary meaning question and the drafters’ intent question are nicely illustrated in figures 2 and 3. Looking at the left half of each chart—that is, to those respondents who preferred that the defendant win—we find that people answering the drafters’ intent question side with the government more confidently than people just saying which view of the text better fits its ordinary meaning. In the child pornography case (fig. 2), 89% of those respondents who strongly preferred that the defendant win thought the defendant’s reading better fit the statute’s ordinary meaning. By contrast, only 67% of that group thought the defendant’s reading better fit the drafters’ intent. That is a significant shift in the government’s favor. Respondents who mildly preferred that the defendant win exhibited the same shift: 79% of them thought the defendant’s reading better fit the statute’s ordinary meaning, while only 42% of them thought the defendant’s reading better fit the drafters’ intent.Footnote 12 Overall, 22% of respondents who strongly preferred that the defendant win and 37% of respondents who mildly preferred that the defendant win changed sides in their judgments about which side had the better reading when they were asked about the drafters’ intent rather than about ordinary meaning. There is no similar change in position for respondents on the right half of the graph—those respondents who preferred that the government win.
The result is easy enough to state. The drafters’ intent question makes prodefendant respondents—but not progovernment respondents—more likely to draw conclusions contrary to their policy preferences. The reason for that result is not so clear. Perhaps there is a tendency to imagine that legislators are more aggressive than oneself in wanting to put people in jail—that if legislators were asked which reading they preferred, they would err on the side of finding a violation when conduct arguably runs afoul of the statute. Whatever the explanation, it is interesting to see evidence that judgments about ordinary meaning, at least when viewed in large sets, are not quite the same as judgments about the drafters’ intent. These evidently are experienced as related but different questions.
3. Ordinary Readers
One of the most striking findings of this study is that policy preferences, as pervasive as they are, do not infect all interpretive judgments equally and often seem to have little or no effect on answers to one question in particular: which reading an ordinary reader would think best fits the ordinary meaning of the statute. We might call that the objective form of the question about ordinary meaning, as opposed to the subjective earlier version that asked the respondents for their own opinion about it. When asked for a judgment about what ordinary readers would think, respondents agreed a remarkably large share of the time. This question produces the black lines in figures 1–4 that sometimes are flat or nearly so, and that always have a lesser slope than the other lines—showing in either case a much reduced entanglement with policy preferences.
The child pornography case—the second case shown earlier—is the strongest illustration of what effect the ordinary readers question can have. When respondents were asked to put aside their policy preferences and say which reading they thought best fit the ordinary meaning of the statute, 85% of those who strongly preferred that the government win as a matter of policy also said that the government’s reading was better (or was “probably better”). But when respondents were asked which reading ordinary readers would think a better fit to the text, only 37% of those who strongly wanted the government to win chosen the government’s reading. On the other side of the spectrum, of those who strongly preferred that the defendant win as a matter of policy, only 11% preferred the government’s reading (or “probably” preferred it) on their own account. But when asked which interpretation ordinary readers would likely think correct, the number favoring the government’s reading rose to 40%. A final way to see the point compares the range between the answers that different respondents gave to different questions. Those considering the child pornography problem were asked the ordinary readers question; all groups of respondents, regardless of their policy preferences, favored the defendant’s reading from 57% to 63% of the time. When simply asked to judge for themselves which reading is better, the numbers choosing the defendant’s reading ranged from 16% to 89%, depending on the policy preference they reported. In short, asking what ordinary readers would think the statute meant made the respondents much more likely to give an answer that went against their rooting interests, whatever they were.
We should add two caveats. First, the ordinary readers question does not produce such a strong two-way shift in every case. In the gun use case, for example, most of the shifting is one-way. People who favor the government as a matter of policy are likely to shift to a judgment in favor of the defendant’s reading, but there is only a little movement the other way. Most of those who prefer the defendant as a matter of policy continue to say that the defendant’s reading is what ordinary readers would think correct. Nevertheless, all respondents do come to a general agreement in their answers, despite continued disagreement on the policy question. The gun use case seems to be an unusual one where the question about what ordinary readers would think produces an especially high level of agreement that the defendant is right and so calls for no movement by those who are rooting for defendant on policy grounds. The three other cases considered in this article all produce a two-way shift when the ordinary readers question is asked.
Second, asking what ordinary readers would think is not a cure-all for the influence of policy preference on interpretive judgments. In some cases that question does seem to wipe out the influence entirely, but in others it merely reduces the influence of preferences by comparison to its influence on other questions. Notice, for example, that in the false statement and attorney’s fees cases in figures 3 and 4, the “ordinary readers” line has a rather steep slope. Indeed, it is steeper than the slope of the line produced in the gun use case when respondents there are asked about the ordinary meaning of the text. In other words, asking about ordinary readers in the one case is worse (from the standpoint of contamination by policy preference) than asking any question in the other case. But that just shows that some cases produce policy preferences that are especially hard to contain. The fact remains that in any given case, asking what ordinary readers would think the text means always does a better job than any other question yet found of producing an answer that is independent of policy judgments.
Why does asking what ordinary readers would think do more to filter out bias than questions about the drafters’ intent or the likely views of a judge? It may be that thinking about what ordinary readers would say directs one’s attention to an external benchmark—the purely conventional meaning of the words—and that the attention is thus distracted from its concerns about outcomes. It may also be the case that answers become biased when the questions have any sort of aspirational quality. The ordinary meaning question asked which reading the respondent thought was better. The question about drafters’ intent invited the respondents to think about what someone else would have wanted—but the someone else was not just anyone. It was a legislator, a faceless but easily idealized author of the text who the respondents might easily imagine has about the same way of looking at things as they do. And the same could be said of the questions that asked what a court would likely do. This time the respondent is asked to imagine how a judge would read the language, and again the judge is a generic but idealized figure onto whom good sense—that is, policy preferences—can readily be projected.
When they are asked what ordinary readers of English would think the text means, it may be that something like the opposite movement occurs. The respondents are asked to imagine what would be thought by a population a little different than they are: mere ordinary readers, perhaps not as educated as the survey respondents (correctly) likely perceive themselves. So when they think about how ordinary readers would interpret the law, the respondents are looking due sideways or slightly down. “Ordinary readers? Well, I suppose they would just think X.” The inner experience, on this speculation, is that the reading is being “averaged” over the population or even “simplified” when one wonders what an ordinary reader would think. But the population sampling or dumbing down, if that’s what it is, is useful in an unexpected way, because it strains out a lot of the wishful thinking that spoils mental inquiries made with a more upward-looking angle.
These results are consistent with earlier work in which we examined judgments about whether a text is ambiguous (Farnsworth et al. 2010). In that study we found that respondents with strong policy preferences about a case were much less likely to find the statute at issue to be ambiguous—assuming they were simply asked for their opinion on that question. But when they were asked whether an ordinary reader of English would likely find the statute ambiguous, their judgments came loose from their preferences in much the same way we see here. The difference is that in the prior study we were talking just about threshold judgments of whether a statute fairly admits of two interpretations—an important question in statutory cases, but still separate from the final and substantive question of what the statute means. In this study we have extended the inquiry to that substantive question of statutory meaning, and we find the impact of asking an “external” or “objective” question even more profound here than it was with respect to ambiguity.
B. The Influence of Preferences on Predictions
After the gun use and attorney’s fees cases, we asked respondents to predict which reading a court would prefer. As figures 5 and 6 illustrate, their answers tracked their policy preferences.Footnote 13 The results were confirmed by regression analysis (unreported). We had speculated that asking what someone else—a judge—would think about the statute might help the respondents give an answer that was independent of their own preferences. It did not. In a way parallel to what we suggested when considering drafters’ intent, it may be that respondents project their preferences onto judges when they imagine them interpreting a text.
Correlation between policy preference and prediction in the gun use case.
Correlation between policy preference and prediction in the attorney’s fees case.
These results may shed a bit of light on why some lawsuits fail to settle. A typical settlement is based on overlapping predictions the two sides make about how a court will decide a case. To the extent those predictions are bound up with preferences about the outcome, they are likely to diverge and shrink the bargaining range between the parties.Footnote 14
IV. Implications
The principal goal of this article is to shed light on some basic cognitive challenges that lie in the way of anyone reading a statute. But once those challenges are revealed, along with what may be a partial solution to them, it is natural to wonder how those findings might intersect with the much more extensive scholarship about how statutes should be interpreted from a jurisprudential standpoint. Suppose, in short, that the findings shown above might continue to affect legal professionals who read statutes. Even if answers to the ordinary readers question are untainted by bias, how relevant are those answers as a legal matter? Second, all worries about bias to one side, how accurate are the answers that respondents give when they predict what ordinary readers would think? This section addresses those two issues.
To begin with the first question, of course there are well-developed schools of thought about the goals of statutory interpretation, and it might seem possible to link some of them to choices in our survey instruments. We could suppose that when we ask which reading better comports with the drafters’ intent, we are inviting the respondents to act like “intentionalists” or “purposivists” (Manning Reference Manning2005; Vermeule Reference Vermeule1998). And when we ask what ordinary readers of the statute would say it means, we are inviting the respondents to act like textualists—or one variety of textualist, anyway. But this picture does not do justice to those schools of interpretive thought. Most interpreters of statutes nowadays are likely to regard judges as agents of the legislature; they differ mostly in what evidence of legislative intent they think proper to consider (Vermeule Reference Vermeule2009; Easterbrook Reference Easterbrook2010; Gluck Reference Gluck2010). Obviously a good intentionalist and a good textualist will both want a lot more material to work with than anyone received in our surveys. The respondents had no basis for comment on the legislature’s purpose except their own speculations, and they did not have all the materials bearing on “semantic context” that a textualist would want them to have.
So nothing we have found or said here strikes a great blow for any one theory of interpretation against another. But the findings are suggestive and do allow some recommendations of emphasis. Asking what an ordinary reader would think a statute means is an important part of one kind of textualism. It is the type that puts an especially high priority on the public meaning of a law. Justice Antonin Scalia is a frequent advocate of this approach to interpretation, and he often resorts to arguments about statutes that are based on what an ordinary person might think a statute means (Scalia Reference Scalia1998, 17; Manning Reference Manning2005). The general theory behind the argument is that people are entitled to notice of what the law is, so a statute should be taken to mean just what it would mean to an ordinary reader. Letting it mean anything else sets a trap and offends the rule of law. This reasoning is especially powerful in criminal cases, where the defendant’s interest in notice—that is, in knowing before one acts what is criminal and what is not—seems especially important (Farnsworth et al. Reference Farnsworth2010, 23; “Textualism as Fair Notice” 2009). The gun use case is a good example.
This study allows us to suggest another point in favor of asking what ordinary readers would think a statute means, and giving weight to the answer. That question is better than other common questions about meaning at producing answers that are not contaminated by underlying policy preferences. It may or may not be the question one would most like to have answered about a statute, but a modest question that can be answered relatively well might be better than a perfect question that will tend to be answered badly.
Our second question was whether the respondents to our surveys, even without bias from their policy preferences, were correct in their statements about what an ordinary reader would think a statute means? This is surprisingly difficult to answer, even if we assume that those who took our surveys are themselves ordinary readers. It might seem then that we could then look at their views of what these statutes meant, use the results of that inquiry to decide what ordinary readers in fact think, and then compare those findings to what our respondents predicted ordinary readers would think. But not so fast. Which of their answers should be used to show what ordinary readers “really” think? We would not want to use everyone’s answers to the “ordinary readers” question, because that does not show what they thought the statute meant. It shows what they expected others to think it meant. We could just look at what our respondents said when they were asked which reading of the statute they thought was better. But then we get answers heavily biased by policy preferences—the red line in our graph (assuming we choose that color). That spoils the inquiry, for a good prediction of what ordinary readers would think of a text is not supposed to be a prediction of where their biases would lead them. We would end up with a paradox in which opinions about what ordinary readers are valuable because they are unbiased—but also wrong because they are unbiased.
The root problem is that when we ask what an ordinary reader would think a text means, we would like to check the answer against the views of ordinary readers who do not have policy preferences that get in the way of their judgments. It is doubtful whether any such readers are out there. That is one of the implications of this study. One could try setting a baseline by asking some random population of reader what they think a chunk of language means—“using a firearm,” perhaps—without any indication of why the question is being asked (in other words, without mentioning any legal case). But then the respondents are being forced to interpret the words without a context, and that is a different activity than interpreting them in the particular settings that appeared in our questions. In the end, we suggest that what ordinary readers would think only sounds like an empirical question. It really is not. The ordinary reader is an idealized creature, perhaps not unlike the reasonable person who juries are instructed to imagine in tort cases. Thinking about the ordinary reader is best understood just as a thought experiment, or heuristic. It is a useful device for getting oneself to think a certain way about a text—to focus on the conventional meaning of the words.
V. Limitations of the Study
A. Experimental Design
An initial limitation of our study is its design: it studies first-year law students in an experimental setting. It would be more informative to study practicing lawyers, even judges. The scarcity of prior studies that query lawyers and judges can attest to difficulty of studying those populations. We chose a lower cost sample, as is often the reason lab experiments use student subjects. In our case, however, we focused on students that will in a few short years be practicing lawyers, some even judges later in their careers.
Likewise, studying interpretive behavior in a laboratory setting may tell us little about how students would behave in practice with clients or in a courtroom. Setting certainly matters. It is uncertain whether real-world practice would cause students to rely more or less on their preferences. Our surveys were conducted in the classroom setting. Students may have felt that they were being tested by more knowledgeable teachers and thus worked harder to suppress their own biases. Yet our study is unlikely to be wholly uninformative. After all, hiring decisions by both law firms and judges is based on grades that are assigned based on classroom surveys—exams—of the sort we administered.
B. Maturation
We study not just students, but students in their first year of law school. It is possible that the effects we found have worn off by the time they later become lawyers, not to mention judges. But there are some reasons for skepticism about that. First, in prior rounds of this research we did administer our surveys to students at the start and end of their first year of law school. Those surveys did not include the ordinary readers question, but they did include other questions considered here—questions about policy preference, about which readings were more consistent with the statute’s purpose, and about which reading of the statute was better as a matter of text. We found the same results in both populations; there was no significant difference (Farnsworth et al. 2010, 34). If a year of law school has not made a dent in the tendency of preferences to influence interpretive judgments, then perhaps that tendency is likely stubborn enough to keep exerting some influence later in life. And even if the overall effects were reduced in strength, there is no reason to suppose that the relative effects of the different questions we asked would be changed. Asking what an ordinary reader would think of a text would still be a better question than others, even if the benefit in the reduction of bias is less among seasoned lawyers than it is among others.
There is also reason to hesitate before imagining that the effects shown here have been entirely rooted out by the time lawyers become judges. Consider the relationship between the findings shown here and in figures 7 and 8, adapted from an earlier empirical study of judicial behavior conducted by one of the authors of this article (Farnsworth 2005). Figure 7 is based on career data for all of Supreme Court justices from 1953 to 2004.Footnote 15 One line shows how often each justice voted for the government in nonunanimous criminal cases involving constitutional claims. The other line shows their votes in nonunanimous criminal cases depending on some nonconstitutional source of law—usually a statute or rule. The justices are ordered here according to the data (i.e., by the mean of the two lines) to show the alignment between the two trends.
How often US Supreme Court justices have voted for the government in nonunanimous criminal cases since 1953.
We can also view the comparison by removing the justices’ names from the graph and instead putting their votes in constitutional cases along the bottom and their votes in nonconstitutional cases along the side. This gives us figure 8, a scatter plot of the same data that correlates the proportion of the justices’ votes for the government in nonunanimous criminal cases of the two types—constitutional and not. An increase in the share of votes for the government along one of the dimensions is very likely to mean an increase along the other. A fitted line shows a strong linear relationship between decisions in favor of the government in either situation.Footnote 16
US Supreme Court justices’ votes for the government in constitutional cases by nonconstitutional cases, 1953–2002.
These charts show that any given justice votes for the government about as often in cases involving the Constitution as in cases that involve other sources of law. Why should that be? No known theory of interpretation would cause a judge to cast similar votes in cases that depend on entirely different sorts of legal texts. And while originalism, as a constitutional theory, might be expected to produce rulings friendly to the government (because defendants often want the protections of the bill of rights expanded beyond their original meaning), it is hard to see why textualism, intentionalism, or any other approach to interpreting statutes would have similar effects. These questions are explored more fully in the earlier work that produced the charts, but the study presented in this article makes a helpful new contribution to an understanding of them.
Nonunanimous criminal cases in the Supreme Court are precisely the ones where the legal materials are not conclusive on their face. They contain ambiguities and call for interpretation; most of the cases that served as the basis for questions in our surveys are represented in the set of nonconstitutional cases graphed above. It may well be that when confronted with ambiguous texts—statutes, of course, but probably also cases and constitutional provisions—judges, like other people, have trouble stopping their policy preferences from influencing their judgments. Those policy preferences cut across all sorts of criminal cases and are not sensitive to the particular type of legal material (statute, case law, etc.) on which the case seems to depend. In short, when judges vote for the government about as often in close criminal cases of every kind, we may well wonder if it is not partly because they acting much like the respondents to our surveys.
Obviously this is not a complete explanation of the data just shown. Some judicial votes are better explained in other ways that do not involve the bias exerted by policy preferences. But the purpose of this discussion is not to settle the reasons for judicial dispute. It is to add some suggestive data to the ways that the disputes can be explained. The charts just shown are offered here merely to cast doubt on the idea that judges are immune to the influences of policy preference that this study has illustrated. The evidence of judicial behavior does not suggest such immunity.
C. Causation
At times in this article we have spoken of mere “entanglement” between policy preferences and judgments about what a text means. That way of speaking implies no causation. At other points, though, we have talked of policy preferences “influencing” interpretive judgments or having an effect on them. Those claims do suggest causation, of course, so we should consider whether they are hasty. Instead of policy wishes influencing judgments about the text, could judgments about the text somehow be influencing policy wishes?
This is not likely. The policy preferences that respondents display remain the same regardless of what interpretive questions they are asked, but as we have seen, the answers to the interpretive questions sometimes closely follow those preferences and sometimes do not. The causal link we suggest is supported by similar findings in studies of wishful thinking (Gilovich Reference Gilovich1991, 75–87; Elster Reference Elster1999, 20–21) or “halo effects” (Nisbett and De Camp Wilson Reference Nisbett1977b, Reference Nisbett1977a) in nonlegal settings. These studies show how underlying preferences about outcomes or similar sources of biases frequently influence judgments about facts, and not the other way around. Our results can be viewed as a particular application of that same general observation.
D. Criminal Cases
Most of the case studies in our surveys involved federal criminal law. It is possible that the effects found here are special to criminal matters and would not carry over to civil cases—but again, it is not likely. To address this possibility, we included a noncriminal case that involved an award of attorney’s fees at the end of a civil action. As figure 4 illustrated, respondents displayed the same general pattern in their choices about those cases that they did in the criminal situations, though the effects were somewhat weaker and the progovernment effect surrounding “drafters’ intent” does not arise. That last point lends a bit of support to our earlier conjecture that respondents tend to think of the “drafters” as progovernment or more eager to put people in jail. In the civil context, the government is not trying to put someone in jail, and it would be surprising to find that respondents thought of the “drafters” as preferring the plaintiff over the defendant, or the other way around.
VI. Conclusion
The findings presented in this article suggest that the most important obstacle to intelligent statutory interpretation is not ignorance of theories about how to do it. The most important obstacle is cognitive; it is the tendency of policy preferences to influence the reading of a text without the reader’s awareness of it. A reader’s own difficulty in perceiving this influence tends to make it hard to root out. The influence is greatest when the interpretive question is made subjective, as by asking what the reader thinks is the best reading with all policy preferences set aside. Making the inquiry objective, as by asking how an ordinary reader would likely read the text, is a help toward debiasing the process of interpretation. Objective inquiries of that sort are just heuristics and are not a complete solution to the challenges suggested here, but they are a start. It may be that the best solution is time: perhaps professional experience reduces the influence of preferences on interpretation. But what we can observe about judicial behavior in close cases suggests that the tendencies found here may not be so easy to eradicate. In that case the effects presented in this article might become a help to understanding some behavior by legal actors, rather than just an agenda for the education of them.
