Published online by Cambridge University Press: 26 February 2004
Political scientists usually study elections. I tried to postpone one. Here's my story with some social science lessons.
The Ninth Circuit Court of Appeals postponed the California Gubernatorial Recall Election on September 15, 2003. The threejudge panel leaned upon the 2000 Supreme Court decision in Bush v. Gore to justify postponing the October 7th election until better voting systems could be installed in six California counties. The panel's reasoning echoes the 2000 Florida election controversy:
The choice between holding a hurried, constitutionally infirm election and one held a short time later that assures voters that the ‘rudimentary requirements of equal treatment and fundamental fairness are satisfied’ is clear. See Bush, 531 U.S. at 109. These issues are better resolved prophylactically than by bitter, post-election litigation over the legitimacy of the election, particularly where the margin of voting machine error may well exceed the margin of victory. The Supreme Court's admonition in Bush bears re-quoting: ‘The press of time does not diminish the constitutional concern. A desire for speed is not a general excuse for ignoring equal protection guarantees’; Id. at 108 (A3-Court, 9/15/03, 63–64).
Three sets of documents were produced during the court cases heard by the Federal District Court, the three-judge panel of the Ninth Circuit Court of Appeals, and the 11-judge panel of the Ninth Circuit. Documents submitted to these courts are indicated by D, A3, and A11 respectively. There were briefs from the American Civil Liberties Union (ACLU), the state of California (State), and the lawyers for Ted Costa (Costa). In addition there was a decision (Court). I reference documents by using the abbreviation for the venue, the author, the date of the document, and the page number. These documents can be found on FindLaw (http://www.findlaw.com/) by searching on “Southwest Voter Education Project v. Shelley.” My declarations can be found at: http://ucdata.berkeley.edu/new_web/recall.html.
The Ninth Circuit three-judge panel decision relied heavily upon declarations by Roy Saltman and me about the defects of Votomatic-style voting systems that use prescored punch cards to record votes. The first page of the decision cited my figure of 40,000 lost votes from the use of Votomatic style voting systems.2
This figure is the difference between the number of residual votes when pre-scored punch card systems are used and the smaller number when better voting systems are used.
The lawsuit brought by the plaintiffs named California Secretary of State Kevin Shelley as the defendant, but the initiator of the recall effort, Ted Costa, asked to intervene to defend the recall. The responses by the state and Costa's attorneys were remarkably different. In the first sentence of its brief, the state announced that “The Secretary of State concurs with plaintiffs that punchcard voting systems are antiquated” (D-State, 8/15/03, 1). Instead of arguing the facts about PPC systems, the state argued that postponing an election is an extraordinary remedy that should almost never be exercised and that the secretary of state was preparing an “aggressive effort to educate voters with respect to the methods of voting using all voting systems, with particular emphasis on PPC voting systems” (D-State, 8/15/03, 11).
The Costa brief took a much more aggressive approach that focused on PPC performance. Most of the brief was devoted to arguing that the net costs of using Votomaticstyle punch cards in October 7, 2003 versus new systems in March, 2004 were minimal because punch cards record votes accurately and replacement systems would do no better. The brief also argued that “there is no constitutional right to an error-free voting system or to a particular voting system—or even to the best available voting system” (11). The Fourteenth Amendment, the brief argued, requires only “substantial equality” which punch cards amply meet.
The brief made the following specific points. First, it questioned whether the statistical evidence actually demonstrates high residual vote rates for punch card systems. Second, it argued that even if there are higher residual vote rates for punch card systems, residual vote rates are not errors, they are often intentional undervotes. “Many voters simply choose to abstain from voting in particular contests, and others register vote protests by casting votes for competing candidates or positions” (17). Third, the brief argued that far from recording votes more accurately, “optical scanning systems and DREs are biased against overvotes and undervotes; they are designed to generate warnings that effectively force voters to cast a vote in each contest and not more than one, even if that is how the voter would otherwise prefer to vote” (18).3
The language in this statement is surprising, to say the least. Some voting systems notify people about possible errors, but they certainly do not “force” them to vote. Just as ATM's ask people “Are you sure?” some optical scan systems notify people that they have overvoted, and most DRE systems provide people with a summary of their votes and ask them if they want to change anything. DRE's (and traditional lever machines) do not allow overvotes, but they certainly allow people to forego voting by undervoting. Almost all election officials that I know think that these are good features of voting systems.
The first claim, that residual vote rates are not high for punch card systems, contradicted the results of every major study of voting systems.4
Three major national studies have concluded that punch card systems perform poorly according to the standards laid down by the Carter-Ford Electoral Reform Commission. These studies are: Cal-Tech/MIT, Voting: What Is, What Could Be? (Cal-Tech/MIT Voting Technology Project, 2001); Henry E. Brady, Justin Buchler, Matt Jarvis, and John McNulty, Counting All The Votes, (Survey Research Center and Institute of Governmental Studies, 2001); and General Accounting Office, Statistical Analysis of Factors that Affected Uncounted Votes in the 2000 Presidential Election, (Report GAO-02-122, 2001). The last two studies also find that punch card systems perform significantly more poorly than optical scan or DRE systems. The Cal-Tech/MIT study concluded that punch card systems perform more poorly than optical scan systems.
Much of my analysis was based upon data from the Statewide Data Base housed at the Institute of Governmental Studies at the University of California, Berkeley. These data provide tract and precinct level voting data and associated census information. Using these data, I found that the differences in Figure 1 persist even after controls are added at the tract level for education, age, racial and ethnic composition, poverty, and other factors.

Figure 2 presents a variant of this kind of analysis for the 20 largest counties in California comprising almost 90% of the total votes. The figure plots residual vote for the 2000 presidential election versus residual vote for the 1996 presidential election. Three counties (designated by solid triangles) changed from Votomatic-style punch cards to new systems (optical scan in all three instances). Seven used Votomatic-style punch card counties.

Except when field experiments are undertaken, this difference-in-difference analysis is about as close as we can get to establishing the counterfactual of what would happen if counties using Votomatic-style punch card systems changed to other systems. The results strongly support the notion that changing away from Votomatic-style punch cards leads to a significant reduction in residual votes. The results also demonstrate that it makes little sense to attribute the bulk of residual votes in PPC counties to intentional undervoting or overvoting6
The best survey evidence that we have (Stephen Knack and Martha Kropf, “Roll Off at the Top of the Ballot: Intentional Undervoting in American Presidential Elections,” Working Paper) suggests a figure of about 0.75% for intentional non-voting.
In 2000, for the eight PPC counties for which data are available, the highest overvote percentage was 0.48% and the average was 0.41%. The average undervote percentage was 1.82%. Hence, even if overvote notifi-cation eliminated all overvotes (which is false), this could at most account for about .4% of the improvement in Marin and San Francisco.
These results provided strong evidence for a violation of equal protection, but the Voting Rights Act required showing that punch cards disadvantaged minority voters as well. There were two ways to demonstrate this. One was to simply note that minorities were disproportionately located in PPC counties—Votomatic-style punch card counties had 46% minorities and other counties had only 32% minorities so minorities were more likely to encounter a PPC system. There still remained the possibility—amounting to a problem of ecological inference—that minorities actually did better with PPC systems than with other systems. Figure 3 countered this argument by showing what happened to residual vote rates in Census tracts with varying percentages of minorities in Fresno County when Fresno changed from Votomatic-style punch cards in 1996 to an optical scan system (without notice) in 2000. The picture shows a dramatic decline in residual votes for all tracts and especially for the virtually 100% minority tracts at the right of the picture. Clearly, minorities are affected by PPC systems.

The second way that PPC systems disadvantage minorities is that minorities are even more affected by Votomatic-style punch card systems than non-minorities. Not only do minorities have a greater likelihood of encountering a PPC system, but Figure 3 shows that they also have a greater likelihood of errors with these systems. In social science parlance, there is both a direct effect of PPC systems on residual vote rates and an interactive effect with the characteristics of the voters. Interactive effects, however, pose special problems for those who want to explain them, especially in the highly charged area of race. They invite questions about what it is about minorities that results in higher residual vote rates.
These questions, however, are poorly posed. The most important fact about Votomatic-style punch card systems is not that they lead to more errors for some people than others. The most important fact is that they lead to higher residual vote rates for everybody because they are hard to use. Even in Marin County, with one of the highest percentages of college graduates in the county, residual vote rates went down after the county switched away from PPCs. Votomatic-style punch cards are like badly designed automobiles that lead to accidents no matter who drives them.
Debate over the requirements of Section 2 of the Voting Rights Act, the complexity of interaction effects, and the difficulty of detecting discriminatory effects requires that an analyst walk a very fine line in interpreting these results. One interpretation of the requirements of the Voting Rights Act, argued by the plaintiffs, is that it requires showing that the challenged practice (the use of Votomatic-style punch cards in this case) results in or causes a disproportionate negative impact on minorities (A3-ACLU, 8/26/03, 40–52). The analysis described above certainly does this. It is true that if PPC systems are replaced by other systems, the residual vote rate among minorities will go down. Thus, there is very strong evidence that using Votomatic-style punch cards leads to disproportionately higher error rates for minorities, just as using poll taxes or literacy tests leads to disproportionately greater disenfranchisement for minorities.
Another interpretation, argued in the Costa briefs (A3-Costa, 9/4/03, 30–34), is that the Voting Rights Act requires a particular kind of causal connection, tantamount to showing intentional discrimination, between the challenged practice and the disproportionate denial of minority votes. My analysis neither proves nor disproves this kind of causal connection. If PPC systems perform badly for minorities because of discriminatory actions on the part of those operating the systems, then the proper test is to see if changing voters' racial status from Caucasian to some minority leads to higher residual rates.8
Some authors would reject the idea that it is even possible to talk about causal effects in this way when the “treatment” at issue, namely minority status, cannot be manipulated. See Paul Holland, “Statistics and Causal Inference,” Journal of the American Statistical Association, 1986, 81:954–955, 959.
But this test is too strong. Even if poll taxes and literacy tests were employed for some reason other than to deny blacks the vote, both devices would nevertheless violate the Voting Rights Act because both surely have a discriminatory impact on minorities and neither can plausibly be considered a legitimate requirement for voting.
Establishing connections between Votomatic-style punch card systems and high residual vote rates was hard enough, but I also had to demonstrate the significance of an error rate of about 1–11/2%. My approach was to calculate the number of votes that would be lost—about 40,000 assuming a statewide turnout of six million with 44% of the population in PPC counties—and to argue that this number was intrinsically significant and that it could also decide an election.
The initial District Court Decision of September 15, 2003 ruled against postponing the election. Although the Court agreed that the “alleged injury would be irreparable” (D-Court, 9/15/03, 23) because it could not envision an effective remedy after the election, the Court questioned whether the plaintiffs could prove that PPC systems produce a higher error rate and whether, even if there were proof of higher error rates, the use of PPC systems would “amount to illegal or unconstitutional treatment” (14) because “marginal deviations from precise vote equality, and minor burdens on the right to vote” (16) are justifiable, especially in the context of a specific election. Furthermore, the harm to the public interest in delaying an election was so great as to make the remedy worse than the problem.
The three-judge Court of Appeals panel disagreed with this analysis, but it was overruled by the 11-judge En Banc panel. The larger panel ruled that the district court “did not abuse its discretion in holding that the plaintiffs have not established a clear probability of success on the merits of their equal protection claim” (A-11, 9/23/03, 8), although the panel noted that the Voting Rights claim was stronger and that the plaintiffs were right in saying that they only had to demonstrate a causal relationship between the voting practice and the discriminatory result (and not an intention to discriminate). But the panel noted that “There is significant dispute in the record, however, as to the degree and significance of the disparity” (9). Finally, although the panel thought the plaintiffs “are legitimately concerned that the use of the punch-card system will deny the right to vote to some voters who must use that system” nevertheless “At this time, it is merely a speculative possibility, however, that any such denial will influence the result of the election” (11–12). As it turned out, each of the four questions on the October 7th ballot was decided by almost a million votes or more9
The closest was the recall question on which “Yes” garnered 4,972,524 votes and “No” obtained 4,006,021 for a difference of 966,503 votes. Arnold Schwartzenegger won the race for candidates to succeed Gray Davis with 4,203,596 votes—almost one and one-half million more than the second place finisher, Cruz Bustamente. The two propositions were each defeated by more than two million votes.
Postponing an election is a hard thing for a court to do, but perhaps the question is too narrowly posed. Should courts intervene in ongoing elections at all? In the recall case, the Ninth Circuit Court of Appeals ultimately refused to take action even though the facts of equal protection violations were much more strongly supported and less speculative than in Bush v. Gore where the Supreme Court did intervene.10
See Henry E. Brady, “Equal Protection for Votes” in The Longest Night, eds. Arthur J. Jacobson and Michel Rosenfeld (Berkeley: University of California Press, 2002).
What ultimately happened in 2003? Table 1 presents residual vote rates for the recall question and the voting system types in the 2003 recall race for the 20 largest California counties. The figures are stunning and surprising, even to me. All five large counties using PPC systems had residual rates above 5%, and Los Angeles had an astounding 8.9% residual vote rate. In a memorandum written just after the election, I noted that a rough and ready estimate of the number of votes that were lost on the recall question due to the use of PPC systems was 176,000—more than four times my pre-election estimate of 40,000.

Figure 4 reprises Figure 2 for the residual vote in 2003 versus 1996. Those counties that have consistently used Votomatic-style punch cards are in the upper right hand corner of the figure along the straight line. Those counties that have consistently used other systems are in the lower left hand corner. Those counties that changed away from Votomatic-style punch cards between 1996 and 2003 have residual vote rates that are significantly far from the straight line, indicating substantial improvement. In fact, the counties that had employed optical scan, DRE, or Datavote systems since 1996 had an average residual vote rate of 2.14% in 2003, and those counties that had changed from punch cards between 1996 and 2003 had almost exactly the same average residual vote rate of 2.17% in 2003.

Votomatic style punch card systems fail to record votes at an unacceptably high rate. Although many election officials ignore residual vote rates by arguing that they are almost entirely intentional undervotes and overvotes, the data overwhelmingly support the contention that residual votes provide a measure of voting system performance. Because the high residual vote rates of PPC systems come from a failure to even record votes, post-election recounts or other actions cannot solve the problem. With the passage of the Help America Vote Act and with the development of newer and better voting systems, it is time that we take a performance based approach to vote recording which makes sure that all the votes are properly recorded and counted.
My thanks to Paul Sniderman and Dan Tokaji for comments and to the ACLU lawyers, especially Mark Rosenbaum, who worked so hard on this case.




Residual Vote Rates from Lowest to Highest Among the 20 Largest California Counties for the Recall Question (All counties with over 100,000 Votes)
