%20-%20https://sketch.com%20--%3e%3ctitle%3e3ECBBF50-FD08-412A-943D-2CFC5E025A16%3c/title%3e%3cdesc%3eCreated%20with%20sketchtool.%3c/desc%3e%3cg%20id='Stylesheet'%20stroke='none'%20stroke-width='1'%20fill='none'%20fill-rule='evenodd'%3e%3cg%20transform='translate(-926.000000,%20-668.000000)'%20id='BUTTONS'%3e%3cg%20transform='translate(100.000000,%20597.000000)'%3e%3cg%20id='BTN-/-RIGHTS'%20transform='translate(816.000000,%2065.000000)'%3e%3cg%20id='ICON-/-Rights'%20transform='translate(10.000000,%206.000000)'%3e%3cpath%20d='M13.5752841,0%20L1.34943182,0%20C0.59375,0%200,0.586666667%200,1.33333333%20L0,22.6666667%20C0,23.4133333%200.620738636,24%201.34943182,24%20L17.6505682,24%20C18.40625,24%2019,23.3866667%2019,22.6666667%20L19,5.36%20L13.5752841,0%20Z%20M13.5752841,1.89333333%20L17.0838068,5.36%20L13.5752841,5.36%20L13.5752841,1.89333333%20Z%20M1.34943182,22.6666667%20L1.34943182,1.33333333%20L12.1988636,1.33333333%20L12.1988636,5.36%20C12.1988636,6.10666667%2012.7926136,6.69333333%2013.5482955,6.69333333%20L17.6235795,6.69333333%20L17.6505682,22.6666667%20L1.34943182,22.6666667%20Z'%20id='Shape'%20fill='%23595959'%20fill-rule='nonzero'%3e%3c/path%3e%3ccircle%20id='Oval'%20stroke='%23595959'%20stroke-width='1.5'%20fill='%23595959'%20cx='9.5'%20cy='13.5355339'%20r='5.5'%3e%3c/circle%3e%3cpolyline%20id='Path-2'%20stroke='%23FFFFFF'%20stroke-width='1.5'%20transform='translate(9.500000,%2013.035534)%20rotate(45.000000)%20translate(-9.500000,%20-13.035534)%20'%20points='8%2016.0355339%2011%2016.0355339%2011%2010.0355339'%3e%3c/polyline%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e)
Sometimes, every vote really does count. In November 2008, the U.S. Senate race in Minnesota between incumbent Republican Norm Coleman and Democratic challenger Al Franken was so close that the state was forced to determine voter intent on thousands of ballots that had been filled out incorrectly (Falcone Reference Falcone2008). But voter intent is not always transparent, and election officials must make difficult judgment calls. How do people make such judgments? More specifically, are those people judging the ballots unconsciously influenced by their own political preferences?
Consider the ballot shown at the top left of figure 1, which is modeled on an actual disputed ballot. To correctly fill out this ballot, voters were asked to completely fill in the oval in front of their preferred candidate's name. But in this case, the voter filled in the oval in front of Franken's name and then drew an X through the oval in front of Coleman's name. Did this voter intend to vote for Franken or Coleman? Or is voter intent too ambiguous to award this vote to either candidate?
Ambiguous Ballot Designs (Franken-Targeted Versions) and Proportion of Votes Awarded to Target Candidate (Second Position), by Target Candidate and Respondents' Vote in November 2008 Minnesota U.S. Senate Election
People often have difficulty making unbiased judgments, because their baseline attitudes influence the way they perceive the matters they are trying to judge (Brownstein Reference Brownstein2003). Information that goes against people's underlying preferences is often perceived as a threat, and in response, people unconsciously interpret such information in ways that reduce this threat (Sherman and Cohen Reference Sherman and Cohen2002). For example, people's assimilation of political arguments, such as those put forward by candidates during political debates, is biased by their political affiliations (Cohen Reference Cohen2003; Munro et al. Reference Munro, Ditto, Lockhart, Fagerlin, Gready and Peterson2002).
In this study, we tested whether judgments of voter intent are influenced by underlying political preferences. We presented hypothetical ballots to adult Minnesota residents and asked them to judge voter intent before telling us for whom they had actually voted in the November 2008 U.S. Senate election. Using a randomized experimental design, we varied whether the second candidate listed on each ballot was Franken, Coleman, or an unspecified Candidate B (with the remaining candidates also unnamed). This design allowed us to compare unbiased judgments of voter intent (from participants who judged unnamed candidates with unnamed party affiliations) to judgments made with the knowledge of candidates' names and party affiliations.
METHODS
Participants
Study participants were residents of the state of Minnesota aged 18 and older who were drawn from a panel of Internet users administered by Survey Sampling International (SSI; Shelton, Connecticut, USA). We used a stratified random-sampling process that drew six separate samples (three age groups by two genders) to ensure age and gender diversity, and to offset differential response rates. The number of e-mail invitations in each demographic subsample was dynamically adjusted until each specific quota was achieved. Upon completion, participants were entered into both an instant contest and a monthly drawing administered by SSI for modest cash prizes. Data collection occurred between March 9 and March 19, 2009, a time period during which the outcome of the Minnesota Senate race was still undetermined. This sampling procedure, as well as all survey materials, received institutional review board exempt status approval prior to subject recruitment.
Survey Design
We presented each participant with four hypothetical ballots, each designed to resemble an actual disputed ballot (see figure 1). We pilot-tested these four ballots and a number of other ballots to identify those for which the average person would have difficulty determining voter intent.
We designed all four hypothetical ballots so that voter intent was ambiguous, but a generous interpretation of intent would favor Candidate B. For example, the four ballots reproduced in figure 1 arguably favor Franken over Coleman, and, therefore, participants judging these four ballots had to decide whether to award the vote to Franken or to no one. By testing only these four Franken-targeted ballots, we were able to assess whether voters who favored Franken were more likely to assign disputed votes to Franken than were voters who preferred Coleman. However, this research method left open the possibility that Franken voters simply required less proof of voter intent than Coleman supporters. To overcome this problem, we randomly assigned participants to one of three experimental groups. The Franken Group saw the four ballots reproduced in figure 1. The Coleman Group saw the same four ballots, but with Franken and Coleman's names and party affiliations reversed. Finally, the Unnamed Group received ballots that referred to candidates as Candidate A, B, C, D, and E from Parties A, B, C, D, and E.
The appendix shows the brief script that preceded people's exposure to these four ballots. After reviewing each ballot, we asked participants: “Who should this vote be awarded to?” We independently randomized the order of response options to ensure that these responses could not be influenced by primacy effects (Couper Reference Couper2008; Krosnick Reference Krosnick1999). Following their judgment of the four ballots, participants indicated their party affiliation, whether they voted in November's election, and who they voted for in the U.S. Senate and presidential elections, as well as demographic measures.
Analysis
For each ballot, we reported the percentage of ambiguous votes that were awarded to the target candidate. We then used chi-squared tests to assess whether these proportions varied significantly among those respondents who reported having actually voted for either Coleman or Franken. We also ran individual logistic regression analyses of whether participants awarded the vote to the target candidate (one for each ballot). Our model included dummy variables for target candidate (Coleman, Franken, or Candidate B), actual November 2008 vote (Coleman or Franken), and two-way interaction terms as covariates. Replications of these analyses using participants' reported party affiliations and November 2008 U.S. presidential votes resulted in very similar patterns, and thus, we report only the senatorial election results for brevity.
RESULTS
A total of 899 participants reached the study website and awarded a vote on at least one of the four ballots shown. Of these, 646 reported that they had voted for either Coleman or Franken in the November 2008 election and were therefore included in this analysis. Respondents' average age was 50.3 years (SD = 14.4; range: 18–87), and 51.9% were male. Reported levels of prior education varied substantially: while 32% of participants had completed a bachelor's or higher degree, 20% had only a high school education or less.
The right column of figure 1 shows the percentage of respondents awarding votes to “Candidate B,” the second name listed on each of the ballots. We break down these data based on participants' November 2008 vote for either Coleman or Franken (striped or solid bars). Judgments about “Candidate B” from the unnamed group participants, which are represented in the first pair of bars in each results graph, represent unbiased judgments about who should be awarded the vote. Franken voters were more likely than Coleman voters to award ballots to the unnamed “Candidate B”; this trend was significant for the last two ballots, χ2(1) = 4.98 and 8.63, respectively; p = .03 and .003, respectively.
A much more pronounced difference appears, however, when we examine responses to the Coleman-targeted and Franken-targeted ballots. In all four ballots, respondents who voted in November 2008 for Coleman were significantly less likely to award an ambiguous vote to Franken when voter intent favored that candidate than to award a vote to Coleman when voter intent favored him, χ2(1) = 23.45, 28.30, 32.86, and 9.62, respectively; all ps < .001 except the last ballot, p = .002. The same pattern was observed in reverse for Franken supporters: Franken voters were significantly less likely to award the vote to Coleman when voter intent favored Coleman than to award the vote to Franken when intent pointed to him, χ2(1) = 30.05, 23.83, 19.91, and 13.72, respectively; all ps < .001.
Logistic regression analyses confirmed the existence of the strong interaction effects between target candidate identity and respondents' party affiliation on awarding the ambiguous vote suggested by the graphs in figure 1. Overall tests of the target by party interaction terms were highly significant for all four ballots, χ2(2) = 50.60, 51.60, 45.12, and 23.64, respectively; all ps < .001.
DISCUSSION
Judgments about how to award ambiguous ballots are strongly influenced by political preferences, with both Coleman voters and Franken voters demonstrating reluctance to award ambiguous ballots to the opposing major candidate but showing no such reluctance for their own candidate.
This finding is consistent with literature on biased assimilation, which has shown that people's underlying biases—their preexisting attitudes and beliefs—influence how they interpret new information that is relevant to these beliefs (Lord, Ross, and Lepper Reference Lord, Ross and Lepper1979; Ross, Lepper, and Hubbard Reference Ross, Lepper and Hubbard1975). Sports fans will recognize this phenomenon—the referee's calls always seem to favor the opposing team, no matter the team for which one is rooting (Gilovich Reference Gilovich1991). This finding raises fundamental questions about the ability of people to evaluate ambiguous ballots in a neutral manner and goes a long way toward explaining why it is so difficult to resolve close elections in ways that satisfy all participants.
Our study has important limitations. First, we did not survey a representative sample of Minnesotans, nor of Minnesota voters. Thus, we cannot conclude that our results represent how all Minnesotans or Minnesota voters would have interpreted these ballots. Nevertheless, our sample was diverse, with substantial participation by both younger and older voters with varying levels of education. More important, our study was a randomized experiment. Therefore, we were able to conclusively demonstrate significant bias in the population we studied. The magnitude of our experimental result is too large to plausibly be a function of the non-representativeness of our participants.
Second, we did not study how election officials judged actual ballots. Our study was not a field study and was not designed to describe the real-world decision-making process of election officials. Instead, we aimed to test how the general public would interpret disputed ballots. To bolster the practical relevance of this study, we presented people with hypothetical ballots modeled closely after actual ballots under dispute in Minnesota, and we collected data while the Minnesota election was still under dispute.
In actual election disputes, of course, parties representing both candidates actively patrol the sidelines, watching for signs of bias among election officials. We do not attempt to show how such disputes are handled. Instead, our study shows some of the psychological forces that cause disputes to escalate—specifically, the finding that the judgment of voter intent on disputed ballots is influenced by partisan bias.
Fortunately, it is possible to design ballots in a way that overcomes these biases. First, in districts that choose to use paper ballots, ballot designers should create multiple versions of the ballot that vary the order of candidates' names. Some voters should see candidate A first, others candidate B, and so on. Variation of the name order in such a manner ought to be a routine part of ballot design, given research showing that name order can significantly influence voting behavior—all else being equal, candidates at the top of ballots receive more votes than other candidates (Brockington Reference Brockington2003; Ho Reference Ho2008).
Second, ballot designers should take care to make the process of judging ambiguous ballots in a blind manner easier for election officials in contested elections. For example, a properly designed ballot that randomizes the order of candidates could be slipped into a sleeve that obscures candidates' names, so that election officials would not know which circle was affiliated with which candidate. In this case, judges would decide voter intent without knowing who would benefit from their decision.
The election in Minnesota was essentially a tie, with the ultimate outcome depending on many decisions about which votes to count and to whom to award those votes. Our study demonstrates that many of these decisions are susceptible to unconscious partisan bias. Ballot design and the methods of awarding disputed ballots should be revised to account for such unavoidable biases.
APPENDIX: Survey Scenario (Unnamed Version)
As you may have heard in the news, the election for one of the U.S. Senate seats for the state of Minnesota was extremely close. Ballots had to be recounted, and an election board had to interpret some ballots that were filled out incorrectly. We would like to see how you would have made these decisions.
We have included some examples of ballots that were not filled out correctly. Imagine that you are responsible for deciding who will be awarded these sample votes. Please tell us to whom you would award each vote. Since the recount decided the race between the two candidates who received the most votes, Candidate B or Candidate C, these sample votes can only be awarded to one of those two candidates (or neither of them).
To help you decide to whom you would award these sample ballots, it might help you to know what the Minnesota law is for counting ballots that are not filled out correctly. Minnesota's law states that “A ballot shall not be rejected for a technical error that does not make it impossible to determine the voter's intent.”
