Debates about voting technologies have riveted interest in recent years, especially in the United States (Alvarez and Hall Reference Alvarez and Hall2004; Reference Alvarez and Hall2008), and have spurred the growth of new voting technologies. This has been especially true in developing countries, where the use of e-voting has proliferated more quickly than in established democracies (Pomares Reference Pomares2012). E-voting has developed slowly in Europe; it has taken hold in some areas (for example, voting over the internet in Estonia, Norway and Switzerland), while in other nations e-voting initiatives have effectively been put on hold (the United Kingdom). By contrast, India, Brazil, the Philippines and Venezuela are the only countries that have extensively implemented e-voting outside the United States, which suggests that the use of these new technologies in emergent democracies should receive greater scrutiny.
Investigating the impact of voting technologies on the voting experience is especially important, since several developing countries are implementing e-voting in efforts to boost the legitimacy of the election process (Pomares Reference Pomares2012). One of the most recent studies of the trend toward e-voting in Latin America cited e-voting implementations in 11 Latin American nations (Alvarez, Katz and Pomares Reference Alvarez, Katz and Pomares2011), ranging from Brazil's early deployment in 1996 to more recent pilots in Mexico (Pomares Reference Pomares2012). Argentina began pilot testing e-voting in 2003. Although implementation has not been widespread, the use of quasi-experimental pilots has allowed a range of analyses of the effects of the new voting technologies on voters and outcomes (Alvarez, Katz and Pomares Reference Alvarez, Katz and Pomares2011; Katz et al. Reference Katz, Alvarez, Calvo, Escolar and Pomares2011; Calvo, Escolar and Pomares Reference Calvo, Escolar and Pomares2009; Pomares Reference Pomares2012). In this article we continue in that vein, by studying the gradual implementation of e-voting in the province of Salta, Argentina.
Previous analyses of e-voting experiments in Argentina focused on the behavioral impact of voting procedures in order to determine the extent to which alternative ways of presenting information to voters and registering voter choices affect the distribution of votes across parties and categories of candidates. Our study focuses instead on voter perceptions and evaluations of e-voting relative to traditional voting. The underlying intuition is that voting technologies frame the voting experience, thereby directly affecting the degree of satisfaction that people draw from that experience and indirectly influencing opinions about the transparency and trustworthiness of elections.
As acceptable as this intuition appears at first sight, it is not clear that voters have to use a new technology for it to affect their opinions and perceptions. Voters may form opinions about technologies based on the experience of other people or on their use of similar technologies for purposes other than voting. By comparing the opinions of citizens who voted using an electronic device with those of citizens who voted using a traditional paper ballot, our study sheds light on this issue.
Our analysis focuses on two sets of specific issues. The first relates to the ease of use of each voting procedure. The second relates to the confidence that different procedures give to voters. Though ease of use and confidence may depend on the details of the specific method used, a more general contrast may be drawn between electronic and non-electronic systems. Some voters may find electronic systems more complex and difficult to use than older technologies. A voting system that feels unfamiliar and difficult to use may also appear obscure and untrustworthy, and thus may negatively affect voter confidence.
Substantive concerns about the legitimacy of elections justify our focus. The unencumbered exercise of voting rights requires that casting a vote presents no particular difficulty and that votes are registered accurately and counted fairly. Whatever their substantive virtues, it is important that new voting technologies do not make voting more difficult or undermine voter confidence. Our exploration strives to determine whether this particular instance of electronic voting in Salta has any discernible, short-term impact on the legitimacy of elections. The increasing interest in e-voting technologies in developing countries is usually associated with the aim of inspiring confidence in the fairness of the election process. Studies of elections in Brazil (Hidalgo Reference Hidalgo2010; Rodrigues-Filho, Alexander and Batista Reference Rodrigues-Filho, Alexander and Batista2006) and Venezuela (McCoy Reference McCoy2005), as well as comparative analyses (Pomares Reference Pomares2012), show that the focus on boosting perceptions of trust in electoral processes is an important driver in the move toward using electronic voting technologies. Against this background, this article's findings are of key policy relevance since they shed light on the potential consequences of introducing e-voting in developing countries, many of which are already testing and deploying new voting procedures.
Methodologically, our article presents new approaches for studying how voting technologies affect voters. As we describe below, the e-voting pilot analyzed in this article did not involve randomly assigning voting technologies to polling places, which would be ideal for determining the causal effect of technology on voters. However, since voting systems were assigned to polling places on the basis of observable socio-demographic characteristics of the electorate, we argue that matching methods—a causal inference methodology that has been used in the past to correct for randomization problems in field and laboratory experiments (Imai Reference Imai2005; Herrnson, Hamner and Niemi 2012)—can alleviate the deleterious effects of the failure to randomize on inference. By closely matching voters who used the electronic voting system directly with similar voters who used the traditional voting system, we approximate a randomized field experiment and can draw stronger inferences about the effects on voters of using new technologies in Salta, Argentina. The contemporary tools of statistical causal inference have rarely been used to study the effects of new voting technologies on voter attitudes and behavior; we believe that our application in this article demonstrates the effectiveness of statistical methods like propensity score matching for evaluating e-voting initiatives throughout the world.Footnote 1
Our research approach differs from the one used by Herrnson et al. (Reference Herrnson, Niemi, Hanmer, Bederson, Conrad and Traugott2008; Reference Herrnson, Hanmer and Niemi2012) to evaluate the effects of voting technology on the voting experience. In our study, real voters used either e-voting or traditional voting to select candidates for all provincial and local contests at stake (for example, governor, members of state legislature, mayor and members of local legislature) in a binding election. In contrast, in Herrnson et al.'s experiments, subjects were recruited to participate in trial runs of the voting systems in a mock election. Additionally, while all voters participating in the Salta experiment used a single voting system on election day, Herrnson et al.'s study participants tried multiple voting systems—which contrasts noticeably with what happens in real elections, when voters use a single system. Thus we believe our research approach allows for more reliable inferences on the effect of e-voting on voting experiences in real electoral settings.
The methodology we use in this study is a cost-effective and scientifically powerful way to evaluate voting technology field experiments. It is costly (and often impractical) to undertake a field experiment of voting technologies with the randomization necessary to produce strong inferences about the effects of technologies on voters. But by collecting extensive data about those participating in the study, and by using appropriate causal inference techniques, we argue here that researchers and policy makers can regain some of the ability to make strong inference (and to thereby scientifically evaluate the effects of changes in the administration of elections), including the introduction of new voting technologies, on voter opinions and behaviors.
The next section of this article discusses the background of the Argentinean field experiment. We then discuss previous research and our hypotheses for this study. Subsequently, we describe our methodology and data collection effort, and finally we present our results. We conclude with a discussion of the implications of our results for the study of e-voting trends around the world.
An overview of e-voting trends in Argentina
Election administration in Argentina is highly decentralized. As a consequence of the process of constituting a federal unity, the first Argentine Constitution (1853–60) clearly established that each constituent unit enjoys the autonomy to dictate its own electoral and political institutions, as long as these respect republican and representative principles. Similarly, voting procedures for national elections have remained almost unchanged since the enactment of universal suffrage in 1912. Currently regulated by the 1972 National Electoral Code, voting procedures have not changed with the exception of altering the size and characteristics of the ballot in 2009.
There are two distinct features of the way in which voters cast a ballot in the country. First, Argentina is among the minority of countries that use the French system of ballot and envelope (one ballot per party or electoral alliance). Ballots for all races are printed on the same paper, and dotted lines on the paper show voters how to split their vote. Each party is responsible for printing the ballots and disseminating them to the polling stations. The second feature of Argentine voting procedures relates to the layout of the voting place: the voter goes into a ‘black room’ (cuarto oscuro)—usually a classroom—and puts the ballot into the envelope behind closed doors. Inside the room, ballots of each party list are displayed—usually on several tables, given the large number of parties presenting candidates for election.
Although election management is highly decentralized, provinces’ voting procedures are mostly identical. However, several provinces have begun to change their legislation in order to abandon the ballot-envelope system. Two provinces have introduced the Australian ballot, while six have passed legislation to roll out e-voting.Footnote 2 Buenos Aires, Argentina's most-populated province, has used e-voting for non-citizen residents since 2003.Footnote 3 The nation's smallest province (Tierra del Fuego) has implemented e-voting for municipal elections. Although the first pilots took place in 2003, the debate about voting procedures gained momentum in 2007 after a series of very competitive provincial elections. This was the case in Córdoba, where the incumbent vice-governor won the governorship by a margin of 1 per cent of the vote. After the preliminary tally of results was announced, the candidate in second place denounced the process as fraudulent and made several public appearances calling for a recount. This process eroded the legitimacy of the election process. Only three months after election day, the elected governor set up an Experts Committee, and a key recommendation of their final report was to abandon the ballot and envelope system.
In 2009, Salta became the first province to introduce an e-voting system for general provincial elections. A province on the nation's northwestern border with Bolivia, Salta has approximately 1.2 million inhabitants and an electoral roll of 850,000 voters. Its electoral administration is complex because much of the province is in the foothills of the Andes. Some of the locations are only accessible by mule, and some lack basic services like electricity. While illiteracy rates position Salta slightly worse than the country average (3.0 per cent compared to 1.9 per cent), illiteracy rates exceed 10 per cent of the population in the most rural areas of the province. Salta also has considerable ethnic diversity; 10 per cent of the population is descended from native peoples.
The electronic voting system was tested for the first time on a small scale in the 2009 provincial elections. On that occasion, 13,000 voters used it in the primaries of the Peronist party (the hegemonic party in the province) at 36 polling stations in the capital of the province and suburbs. In 2011, the e-voting system was tested again in January during the open primaries and implemented, for the first time in the country, for 33 per cent of the province's electoral roll (725 polling tables out of a total of 2,499) for governor and provincial legislature races as well as for municipal races.Footnote 4 The election resulted in the re-election of Peronist governor Urtubey (who obtained 60 per cent of the vote), and in the Peronist Party gaining control of a large majority of seats in both houses of the provincial legislature. The electoral alliance supported by the governor also won 54 out of 59 local municipalities.
In order to ensure a smooth roll-out in the first phase of e-voting implementation, the provincial government chose polling places with greater populations of technologically sophisticated, better-educated and higher-income voters. This explains why half of all registered e-voters were located in the provincial capital. According to interviews with Electoral Tribunal authorities, the bias in the choice of e-voting locations has two explanations. First, the higher education levels of e-voters were expected to make voting and election administration easier during the first phase of e-voting implementation. Secondly, locating half of the e-voting stations in the capital was expected to simplify the administration of the election process. This bias in the selection of e-voting sites should be taken into account when generalizing the results of the e-voting pilot to the whole population.
Similar to Brazil's gradual implementation, the roll-out of e-voting in Salta was planned to be incremental; it covered 33 per cent of the electoral roll in 2011, 66 per cent in 2013 and was originally planned to be implemented throughout the province in 2015. However, due to the highly positive response of the citizenry in 2011, the provincial government decided to move complete implementation forward to 2013. Most importantly for our purposes, however, the gradual implementation of e-voting in Salta represents a unique opportunity to learn about the impact of e-voting by comparing the voting experiences of e-voters and ‘traditional’ voters.
The e-voting system used in Salta was designed by an Argentine private vendor to resemble the country's traditional voting system. The new system consists of ballots with the electoral choices printed on them. To cast a vote, the user inserts this ballot into a computer and selects the options electronically on a touch screen. Afterwards, the machine prints the selection of electoral choices onto the ballot and the voter inserts it into a ballot box (identical to the one used in traditional voting). Each paper ballot is equipped with a Radio Frequency Identification (RFID) chip on which the choice is also registered. Once a ballot is printed, it cannot be re-used. The voter can verify that the options recorded on the embedded chip of the ballot correspond with the ones printed on it. Once the election is closed, the machine changes from ‘voting machine’ to ‘tally machine’. The poll authorities open the ballot box, take the paper ballots out one by one, and scan them by passing them through the machine's reader. Each successful read is indicated by a sound, and the progress of the vote count and preliminary vote totals are shown on the machine's screen. The voting software that contains the electoral options is stored on a CD-ROM. Although some characteristics of this system are similar to those exhibited by DRE machines—using a touch-screen interface to display and select electoral information—there is an important difference between this system and DRE machines: machines that serve only as voting machines do not keep track of votes cast; they are only used to present and select electoral choices and to print marked ballots.Footnote 5
Previous Research and Hypotheses
What does trust in electoral institutions mean? There is a well-established literature on trust that stems from different academic disciplines, but since the core trust relationship is interpersonal, theories of trust in political institutions are more sociological than political in nature (Warren Reference Warren1999). In this sense, ‘whatever it means to trust an institution is somehow scaled up from the domain of socially thick, face-to-face relations’ (Warren Reference Warren1999, 348). Political trust encompasses different types of institutions. Easton's seminal work differentiates between diffuse and specific support: the former is macro-level support for the political system, and the latter is micro-level support for specific system outcomes (Reference Easton1975). Based on this conceptualization, Norris (Reference Norris1999) establishes a continuum of five dimensions of trust (ranging from diffuse to specific support): political community, regime principles, regime performance, regime institutions and political actors. Trust in the election process can be defined as diffuse support to the extent that it is associated with a process rather than outcomes.
Capturing the specific traits of trust in the integrity of the election process requires a more nuanced understanding of the concept. In the research we discuss in detail below, we distinguish between two different dimensions of trust in the integrity of the election process: confidence that a vote will be counted as intended, and confidence that the ballot will be kept secret. Whereas the former assesses perceptions of accuracy of the voting system and fairness of the counting procedure, the latter captures the ability to preclude violations of privacy and voter intimidation.
The incipient literature on determinants of trust in elections considers three types of factors: type of voting technology used; electoral institutions; and voters’ partisan affiliations and socio-demographic characteristics. We focus here on the literature on the impact of voting technologies.
Most work analyzing interactions between citizens and electronic voting systems has focused on the United States; several computer and social scientists have compared the reliability and accuracy of different voting devices in the aftermath of the 2000 presidential election (Alvarez and Hall Reference Alvarez and Hall2004, Reference Alvarez and Hall2008; Everett et al. Reference Everett, Greene, Byrne, Wallach, Derr, Sandler and Torous2008; Herrnson et al. Reference Herrnson, Niemi, Hanmer, Bederson, Conrad and Traugott2008; Kimball and Kropf Reference Kimball and Kropf2008; Stein et al. Reference Stein, Vonnahme, Byrne and Wallach2008). Research in the political science literature has focused mainly on the effect of alternative voting devices on voting errors, with a particular emphasis on the ‘residual vote’ problem and its potential impact on the disenfranchisement of some segments of the electorate and on electoral outcomes (Sinclair and Alvarez Reference Sinclair and Alvarez2004; Card and Moretti Reference Card and Moretti2007; Stewart Reference Stewart2011).Footnote 6 However, the impact of alternative voting procedures—especially of new voting technologies—on citizens’ trust in elections and voters’ ease of use has been given scant attention so far, especially outside the United States.
In one of the few studies of the relationship between e-voting technologies and trust, Alvarez, Hall and Llewellyn (Reference Alvarez, Hall and Llewellyn2008) use data from two telephone surveys to examine voters’ trust in the electoral process. They find that voters who used e-voting machines to cast their vote in the 2004 US election were less confident than those using paper ballots that their ballots were counted as intended. The authors argue that skepticism toward e-voting may stem from the fact that voters simply do not trust the ‘black box’ nature of e-voting, and that their mistrust could also be influenced by negative media reports on the vulnerability of automated voting systems to tampering, failure or fraud in the aftermath of the 2000 election. Consistent with these findings, Stewart's (Reference Stewart2009) analysis of the 2008 US presidential election concludes that Americans are still skeptical of the reliability of e-voting systems; users of automated voting machines were found to be less confident in the integrity of the vote count than users of other voting technologies. Specifically, Stewart concludes that DRE machines had a stronger negative effect on voters’ trust in the vote count than optical scanners, highlighting the fact that specific characteristics and types of electronic voting systems may influence citizens’ trust in the electoral process. This finding is in line with the broader literature indicating that different voting technologies have different effects on voters’ attitudes and electoral behavior (Herrnson et al. Reference Herrnson, Niemi, Hanmer, Bederson, Conrad and Traugott2008; Everett et al. Reference Everett, Greene, Byrne, Wallach, Derr, Sandler and Torous2008; Reference Claassen, Magleby, Monson and PattersonClaassen et al. forthcoming).
Correspondingly, several authors maintain that, ceteris paribus, providing e-voting systems with a verifiable record of each vote—in particular a paper audit trail—can substantially increase citizens’ trust in elections and in the perceived transparency of electoral procedures (Riera and Brown Reference Riera and Brown2003). Voter-verifiable paper audit trails (VVPAT) would allow citizens to check whether ballots cast represent their true intent, which would increase the verifiability of the election outcome and improve confidence in computerized voting (Alvarez and Hall Reference Alvarez and Hall2008; Atkeson and Saunders 2007). However, there is heated debate in both policy and scholarly work about the actual influence of VVPAT on the perceived fairness of voting procedures. Some scholars argue that VVPAT could in fact undermine voters’ confidence in the electoral process in the case of inconsistency between electronic and paper records (Herrnson et al. Reference Herrnson, Niemi, Hanmer, Bederson, Conrad and Traugott2008). Empirical evidence from the United States shows no conclusive evidence. Whereas there is some evidence that paper audit systems do not improve voter satisfaction (Herrnson et al. Reference Herrnson, Niemi, Hanmer, Bederson, Conrad and Traugott2008), Llewellyn et al. (Reference Llewellyn, Hall and Alvarez2009) find that in the 2006 mid-term election, voters who cast ballots on an electronic voting machine with a VVPAT device exhibited higher rates of confidence than electronic voters without a VVPAT device.
While US studies emphasize the differences in voter confidence among types of e-voting machines, and especially the effect of voting technology on the perceived accuracy of the voting procedure, there has been little interest in comparing confidence in e-voting systems versus manual voting. The need to address this gap is particularly imperative in the case of developing countries where the implementation of new voting technologies is largely motivated by the aim of promoting greater confidence in the election process (Pomares Reference Pomares2012). Latinobarometer public opinion surveys show that in 2009, only 45 per cent of respondents believed that elections in their country were clean.Footnote 7 However, there is considerable variation among countries; while in Chile, Panama and Uruguay positive answers exceeded 70 per cent of respondents, the 2009 survey positioned Argentina below the regional average, with only 35 per cent of respondents reporting that elections are clean in the country. However, due to the wording of the survey question (asking whether elections are ‘clean or fraudulent’), responses might reflect feelings of electoral efficacy, and not just perceptions of the procedural fairness of the election process. According to data from the same Latinobarometer surveys, the proportion of respondents that trusts political parties in 18 Latin American countries is 24 per cent; in this context, several countries in Latin America decided to pilot e-voting.
There is little evidence on the degree of trust in electronic voting technologies in Latin America and whether the transition to e-voting has improved this trust. It could be hypothesized that electronic information technologies are better at increasing perceptions of the accuracy of the tally than the secrecy of the ballot, especially when the process involves transmitting information over the internet. Although there is exploratory evidence to support this intuition (as will be explained below), this evidence has only recently been subject to scrutiny in recent work on perceptions of ballot secrecy in the United States (Reference Gerber, Huber, Doherty, Dowling and Hill.Gerber et al. forthcoming) and in the context of internet voting (Birch and Watt Reference Birch and Watt2004). However the main concern about internet ballot secrecy involves the risk of impersonation, which differs from what is at stake with in-person e-voting.
Previous research on this issue from an e-voting pilot in Colombia points to a positive effect of electronic voting on confidence that the vote will be counted as intended (Alvarez et al. Reference Alvarez, Katz, Llamosa and Martinez2009). However, evidence from an e-voting pilot in Argentina supports the hypothesis that e-voting exerts distinct impacts on different dimensions of confidence: voters trust that e-voting machines register their choices as intended, but are reluctant to trust that e-voting systems will guarantee ballot secrecy (Pomares et al. Reference Pomares, Leiras, Tchintian and Ramos2011). Since the evidence on Argentina was collected during mock elections, more systematic analysis is needed of the impact of e-voting on voters’ confidence in the integrity of the electoral process in the context of binding elections.
Previous analyses of e-voting pilots in Argentina and Colombia also identified variations in confidence levels across socio-demographic groups. In line with past research on US and European e-voting experiences, a comparison of Argentine and Colombian pilots shows that older voters tend to be more confident that their vote was accurately registered than younger ones, even when the former—especially voters over 50—found e-voting more difficult than the latter (Alvarez, Katz and Pomares Reference Alvarez, Katz and Pomares2011). This research, however, does not provide evidence of the impact of e-voting on different dimensions of confidence.
Although there is scant evidence on the determinants of confidence in electronic voting in developing democracies, we hypothesize that changes in voting procedures exert distinct impacts on different dimensions of confidence in the fairness of the process. Specifically, we hypothesize that e-voting has a positive impact on voters’ confidence that their ballots will be counted as intended, but that e-voting systems raise concerns about the ability of new technologies to guarantee ballot secrecy.
Few comparative studies have analyzed voter perceptions of usability, and in particular how these perceptions might be affected by voting technologies. Exceptions include research from Belgium (Delwit, Kulahci and Pilet Reference Delwit, Kulahci and Pilet2004, Reference Delwit, Kulahci and Pilet2005) and the aforementioned work on e-voting pilots in Argentina and Colombia (Alvarez et al. Reference Alvarez, Katz, Llamosa and Martinez2009; Alvarez, Katz and Pomares Reference Alvarez, Katz and Pomares2011). The Colombian pilot showed that voters perceived new voting technologies as easier to use than manual voting systems in the context of a mock election; but it remains to be seen whether the experience of casting a ballot using an e-voting machine affects voters’ perceptions of usability in binding elections.
We use data from Salta's gradual implementation of e-voting to test these hypotheses and, more generally, to analyze the degree of confidence voters place in the election process and their opinions about various electronic voting technologies. This unique dataset allows us to assess the impact of electronic voting technologies on voters’ trust in the integrity of the election procedure and perceptions of usability. In the next section we discuss this unique dataset and our methodology for testing these hypotheses.
Data and Methodology
On election day in Salta, a long questionnaire was administered to 1,502 voters, 887 of whom used the e-voting method; the remaining voters used traditional paper ballots. A sample of 36 polling stations was created that paired voting stations using each method with similar socio-economic characteristics. Each pollster was expected to administer at least 40 surveys; voters were recruited based on gender and age quotas. Voters who used traditional ballots were asked primarily about their evaluation of that method, while those using e-voting devices were asked primarily about e-voting. The complete questionnaires are provided in Online Appendix 2. Other data about the e-voting elections were also collected as part of the larger project, including surveys of election workers in the polling places and detailed reports from independent observers in the polling locations.
Our evaluation focused on survey questions addressing confidence in the election process: confidence in the vote being counted accurately, confidence in the preservation of ballot secrecy and perceptions of the cleanness of elections in Salta. We also considered perceptions of ease of use and speed of the voting process. Additionally, we considered voters’ support for substituting traditional voting with electronic voting, qualification of poll workers, and voters’ preferred method of selecting candidates from different political parties. The latter is important in the Argentine context: the current paper-based voting procedure requires the voter to cut paper ballots along a dotted line to split the ticket. One of the reasons espoused by the provincial government to move to e-voting is to facilitate split-ticket voting.
Eight of our survey questions had ordinal responses with four categories; the two lower categories indicated negative voting experiences, and the two higher categories indicated positive voting experiences. To ease the interpretation of results, we created binary indicators based on ordinal responses: 1 indicates a positive voting experience and 0 indicates a negative voting experience.Footnote 8 For the last two questions (ease of use and overall evaluation), since more than 90 per cent of respondents reported a positive voting experience, we coded responses as 1 for a ‘very’ positive experience, and 0 for a ‘not-so-positive’ or negative experience. More specifically, the coding criterion used for each outcome variable was:
Overall assessment of voting experience
• Evaluation of voting experience: 1 if voting experience was ‘very good’; and 0 if ‘good’, ‘bad’ or ‘very bad’
• Qualification of poll workers: 1 if ‘very qualified’ or ‘somewhat qualified’; and 0 if ‘little qualified’ or ‘not at all qualified’
• Substitution of traditional voting by e-voting: 1 if ‘agree a lot’ or ‘agree’; and 0 if ‘disagree’ or ‘disagree a lot’
Ease of use and speed of voting process
• Speed of voting process: 1 if ‘very quick’ or ‘quick’; and 0 if ‘slow’ or ‘very slow’
• Ease of voting procedure: 1 if voting was ‘very easy’; and 0 if ‘easy’, ‘difficult’ or ‘very difficult’
• Preferred method for selecting candidates from different political parties: 1 if ‘electronically’, and 0 if ‘by hand’
Confidence in the fairness of the election process
• Confidence on vote being counted: 1 if ‘very sure’ or ‘sure’; and 0 if ‘unsure’ or ‘very unsure’
• Confidence on ballot secrecy: 1 if ‘very confident’ or ‘confident’; and 0 if ‘not confident’ or ‘not at all confident’
• Cleanness of elections in Salta: 1 if ‘very clean’ or ‘somewhat clean’; and 0 if ‘a little clean’ or ‘not at all clean’
These questions all assess the experience of Salta voters in the 2011 election in different ways, and in particular their interaction with either the e-voting or the traditional voting process.
Matching Procedure
Since each individual either votes electronically or votes using the traditional voting system, we cannot compute individual-level differences in voting experience under each system. Instead we use matching methods, which let us compare the average voting experience of e-voters with the average voting experience of traditional voters with the same relevant observable attributes that have a direct impact on both assignment to polling places and evaluations of the voting experience. The advantage of controlling for differences in observable attributes using matching methods, instead of regression adjustment, is that results are less sensitive to modeling assumptions (Ho et al. Reference Ho, Imai, King and Stuart2007; Morgan and Harding Reference Morgan and Harding2006).
To illustrate why it is important to consider samples of e-voters and traditional voters who have the same observable attributes, consider the following example. Suppose that voters assigned to polling places where voting was done electronically rely more heavily on new technologies in their everyday life compared to voters assigned to traditional polling locations. It might therefore be the case that e-voters report greater satisfaction with the voting equipment than traditional voters, but only because the latter are less familiar with technologies in general, and not as a result of the technology used to cast their ballots. To prevent this type of selection bias caused by the non-random assignment of voters to polling places, we conduct a matching procedure that produces look-alike samples of e-voters and traditional voters, and subsequently compute the impact of the electronic voting technology within the matched sample.
The matching methodology that we use is propensity score caliper matching (Cochran and Rubin Reference Cochran and Rubin1973). We first estimate propensity scores—defined as the probability that a voter is assigned to an e-voting location conditional based on observable attributes—and then match e-voters with traditional voters that have a sufficiently similar propensity score. To ensure that the distribution of individual attributes is similar across voting technologies (that is, to ensure that covariates are balanced across e-voting and traditional voting samples), we specify that the absolute difference between propensity scores for any pair of matched voters must be lower than 5 per cent of the standard deviation of the estimated propensity score (this constraint is usually termed a ‘caliper’). We conduct a series of statistical tests and repeat the matching procedure until covariates (as well as higher-order terms and combinations of covariates) are well balanced across samples.Footnote 9
After that, we estimate the causal effect of e-voting by computing the difference in voting experience between matched e-voters and traditional voters. We compute simple difference in proportions, as well as effects estimated based on regression adjustment in the matched sample, to control for any remaining differences in observable attributes between e-voters and traditional voters. If the matching procedure successfully balanced covariates, adjusted effects should not differ from simple differences in proportions computed in the matched sample (Ho et al. Reference Ho, Imai, King and Stuart2007). Finally, we conduct a series of Rosenbaum bounds sensitivity analyses to evaluate the robustness of estimated causal effects to unobservable differences between e-voters and traditional voters (Rosenbaum Reference Rosenbaum1995).
Results
We begin by presenting in Table 1 the overall proportion of total voters, e-voters and traditional voters that gave positive responses to each question prior to matching. The table also illustrates the differences in these proportions and provides a p-value for whether each difference is statistically significant.
Summary Statistics
*Test of difference in proportions.
Of the nine evaluative criteria reported in the table, most demonstrate sizeable and statistically significant differences between e-voting and traditional voting prior to matching. The one case with a small difference relates to the speed of the voting process: e-voters overall believed that this method was faster, but the difference is slight (3.5 per cent) and is barely statistically significant at conventional levels.
The remaining differences are sizeable and statistically significant; some of them are quite large. For example there is a 30-point difference between the overall preferences of e-voters for the method they used relative to traditional voters. And there are 20-point or better differences regarding the overall evaluation of the voting experience, ease of voting and whether e-voting should be used instead of traditional voting.
However, given the means by which the e-voting method was distributed across polling places in Salta, we cannot be confident that the differences seen in Table 1 are necessarily due to differences in the voting method rather than some form of selection bias introduced by the non-random allocation of e-voting devices to polling places. In particular, since e-voting was introduced in polling places where voters generally have a higher socio-economic status, and since demographic attributes may affect technology use (Norris Reference Norris2002) and aspects of the voting experience (Alvarez, Hall and Llewellyn Reference Alvarez, Hall and Llewellyn2008), ignoring systematic differences in variables such as age, education, gender and occupational status may lead to biased measures of the effect of e-voting on the voting experience. In an attempt to alleviate these potential biases, we use the matching approach described above. That is, each e-voter is matched to a corresponding traditional voter according to a series of covariates that we believe will best account for potential biases.
It is quite important to note at the outset, before we proceed with further detailed discussion of the results of our matching procedure, that when we compare differences in voting experience between e-voters and traditional voters among matched respondents, we see qualitatively and quantitatively similar results to the unmatched estimates. That is, the signs and relative sizes of the differences in proportions are qualitatively similar, which gives us some confidence that the samples are not severely unbalanced.
In Table 2 we present summary statistics for a series of covariates, in order to evaluate observable differences between e-voters and traditional voters. In the original sample, e-voters exhibit higher educational attainment and are more likely to report correct answers to a set of survey questions aimed at evaluating the level of political information. Traditional voters, in turn, are more likely to report not having a full-time job. We also compare individuals’ self-reported technology use and level of political information, as these factors may affect the perceived ease of the voting procedure and other aspects of the voting experience. Before the matching procedure, e-voters are more likely to be habitual internet users and to rely on technologies such as ATMs, cell phones and personal computers, and exhibit higher levels of familiarity with political information. After matching, differences between e-voters and traditional voters are greatly reduced and are no longer statistically significant. The percent balance improvement (computed as the reduction in observable differences between e-voters and traditional voters produced by the matching procedure) divided by the original difference exceeds 68 per cent for all covariates.Footnote 10
Balance Statistics
Note: For more details about balance improvement, see Online Appendix 3.
*p-values of Kolmogorov-Smirnov tests (ordinal variables), and difference in proportions tests (binary variables).
In Table 3 we present the overall proportion of e-voters and traditional voters that gave positive responses to each question prior to matching. The left-most panels provide these data prior to matching, while the right-most panels provide these data after matching. Even though observable differences between e-voters and traditional voters almost disappear as a result of the matching procedure, differences in reported satisfaction with the voting experience and attitudes toward voting technologies are still sizable and statistically significant for most outcome variables.
Causal Effect of e-voting
Note: sample sizes given in the first column (before matching) differ from those given in Table 1 because Table 3 omits respondents with missing values in covariates.
*Test of difference in proportions.
A first group of variables deals with the overall evaluation of the voting experience. E-voters are nearly 25 per cent more likely than traditional voters to characterize the voting experience as ‘very good’. When asked whether they approve of substituting e-voting for traditional voting, e-voters tend to answer affirmatively. When asked whether they prefer to select candidates electronically, instead of using the traditional method, e-voters are also more likely to answer affirmatively. As mentioned above, the traditional voting method in Argentina (the French ballot paper and envelope system) requires voters to physically cut ballots to cast a split ticket. Therefore it is reasonable to expect voters who wish to select different parties for each race to prefer the e-voting system. It is interesting to note that election results show a higher rate of split-ticket voting among e-voters (Calvo and Leiras Reference Calvo and Leiras2011).
A second group of variables assesses perceptions of the ease of use. When asked about the ease of the voting procedure, e-voters are more than 20 per cent more likely to report that voting was ‘very easy’. E-voters are also more likely to describe the voting process as ‘quick’ or ‘very quick’, although the difference is small and not statistically significant. This is an interesting finding, since one of the goals of e-voting implementation is speeding the election process. While this might still be a valuable goal for election administrators, voters do not perceive an important difference in speed (at least using this e-voting system) between traditional and e-voting systems.
Finally, a third group of variables deals with differences between e-voters and traditional voters with regard to confidence in the election process. Here we get the most interesting results. When asked about the cleanness of elections in Salta, e-voters are 16 per cent more likely to report that elections are more than ‘a little’ clean. As shown in Table 1, the overall perception of cleanness of elections is quite low (only half of the voters say elections are clean). Although this question might capture broader dissatisfaction with representative political institutions, introducing e-voting seems to lessen some of the negative perceptions. The effect of e-voting is also positive with regards to perceptions of whether the ballot will be counted. However, the one aspect of the voting experience that e-voters evaluate more negatively than traditional voters is perceptions of ballot secrecy: e-voters are at least 7 points less likely to say that they are ‘confident’ or ‘very confident’ that the secrecy of their votes was preserved. These negative opinions may arise from the specific layout of the voting booth used for traditional voting in Argentina (voters casting a ballot behind doors) and pose a caveat to the overall positive evaluation of the e-voting experience.
To control for any remaining differences in observable attributes that might explain the above-mentioned differences in reported voting experiences between e-voters and traditional voters, we conducted a post-matching regression adjustment, controlling for educational attainment, age, gender, technology use, occupation and level of political information. The ‘adjusted effect of e-voting’ reported in Table 4 gives the differences between e-voters and traditional voters computed based on the results of logistic regressions estimated in the matched sample. Adjusted differences are very similar to the ones described above, and the substantive implications remain the same: overall, e-voters report a considerably more positive evaluation of the voting experience and support replacing traditional voting with electronic voting.Footnote 11
Adjusted Effect of e-voting
In order to attribute these differences to the technology used for voting, it must be the case that e-voters resemble traditional voters not only in terms of observable attributes such as measures of education, political information and technology use, but also in terms of attributes that we do not observe yet which might affect both the voting experience and the assignment to either an electronic or traditional polling place. For instance, voters who support the incumbent party may report more positive evaluations of e-voting than opposition voters. If partisan attachments varied systematically across polling locations, omitting this relevant individual attribute would lead to biased estimates. However, we cannot control for this potential source of bias because we do not have information about partisan attachments.
To evaluate the sensitivity of matching estimates to unobservable differences between e-voters and traditional voters, we conducted Rosenbaum bounds tests (Rosenbaum Reference Rosenbaum1995).Footnote 12Table 5 shows the results of the sensitivity analysis. For each outcome variable, lower (L) and upper (U) bound columns give the p-value of a test of difference between responses reported by e-voters and traditional voters for a series of Gamma (Γ) values given in the first column. Γ values indicate the size of the omitted factor required for the p-value of the test statistic to stand at the values given in the corresponding row. To illustrate the interpretation of these results, consider the outcome variable titled ‘Agree substitute TV by EV’; for this outcome variable, the p-value exceeds 10 per cent for a level of Γ equal to 1.9 or higher. This means that in order for the difference between e-voters and traditional voters to become insignificant at conventional levels, the omitted factor must cause the odds of agreeing with the reform to increase by 1.9 times or more.
Sensitivity Analysis
Note: the table gives Rosenbaum bounds for p-values of the effect of EV on voter's satisfaction with the voting experience and support for e-voting. ‘L’ columns indicate lower bounds, and ‘U’ columns indicate upper bounds.
In the social sciences, estimated effects are often sensitive to hidden bias, and the levels of Γ required to do away with the significance of an explanatory variable of interest are usually low.Footnote 13 In this particular application, the levels of Γ required for an omitted factor to explain away the positive effects of e-voting discussed in the previous paragraphs are equal to or larger than 1.5 for five outcome variables: evaluation of the cleanness of the election (Γ = 1.6), agreeing to substitute e-voting for traditional voting (Γ = 1.9), overall evaluation of the voting experience (Γ = 2.6), ease of the voting procedure (Γ = 2.8) and preference for selecting candidates electronically (Γ = 3.1). These results indicate that most of our findings are robust to the presence of unobservable attributes, and reinforce the idea that e-voting is associated with a positive evaluation of the voting experience and higher support for replacing traditional voting with electronic voting.
Conclusion and Discussion
Our article makes both substantive and methodological contributions. On the methodological side, given the unique design of the Salta field experiment—in which some voters used electronic voting machines while others used traditional voting methods—we have a potentially powerful method of testing for the effects of voting technology on voter perceptions. Had voters been truly randomly assigned to one voting system or the other, we would be able to easily examine differences in voter perceptions across voting systems and without a great deal of statistical machinery. However, for practical reasons the voting system was allocated at the voting station (not voter) level; the allocation of electronic versus traditional voting systems was not randomized across voting stations either.
While randomization was not used in this field trial, we use statistical matching tools to test for (and alleviate the deleterious effects of) the failure to randomize on inference. That is, we directly compare voters who used the electronic voting system with similar voters who did not; by closely matching voters from each population in this way we approximate a randomized field trial, and can draw stronger inferences about the effects of using new technologies on voters in Salta, Argentina. The contemporary tools of statistical causal inference have rarely been used to study the effects of new voting technologies on voter attitudes and behavior; we believe that our application in this article demonstrates the effectiveness of statistical methods like propensity score matching for evaluating e-voting initiatives throughout the world.
The methodology we have discussed in this article is a particularly cost-effective, but scientifically powerful, method to evaluate these voting technology field trials. It is costly (and often impractical) to undertake a real field experiment of voting technologies with the sorts of randomization necessary to produce strong inferences about the effects of technologies on voters. But by using this quasi-experimental design, collecting extensive data about those participating in the study and using matching, we believe we can regain some of the ability to make strong inferences and to thereby scientifically evaluate the effects of this field experiment on voter opinions. We thus urge others who are designing future field trials to consider using this approach to test the utility of voting systems under consideration.
Our research also makes important contributions to the growing body of research on the use of voting technologies. There has been a number of important efforts in Latin America in the last decade to test and implement electronic voting, largely out of a desire to extend the franchise to citizens who might have trouble voting because of various accessibility problems (including geography and language), and to combat known forms of fraud—thereby increasing voter confidence that their votes will be counted as intended, as well as perceptions of electoral integrity (Hidalgo Reference Hidalgo2010; Pomares Reference Pomares2012).
We used a unique ‘laboratory’ in a recent implementation of e-voting using data from Salta, Argentina. There, efforts were made to help scientifically compare the utility of electronic voting and traditional voting in Salta. These efforts included a gradual implementation of e-voting, the use of comprehensive survey tools for evaluation, and the collection of other quantitative data from the election. Here we use one of those datasets—the voter evaluation surveys—to study how the quasi-experimental implementation of the two voting systems fared in the minds of those who will have to use one of these voting systems in the future.
We generally found that voters who used the electronic voting system in Salta evaluated it more favorably than voters who used the traditional voting system. Interestingly, while there is a large difference in perceptions of ease of use between voters using the two systems, we found little difference in perceptions of speed of the voting process. Whereas the latter might be a valuable aspect of voting systems from an election administration perspective, this attribute does not seem to be of utmost importance to voters. At least, similar perceptions of the speed of voting technologies do not affect the evaluation of the voting experience. Our findings also provide valuable insight into the study of confidence in elections. We found that although new voting technology positively affects opinions on the cleanliness of elections, there are some concerns about ballot secrecy. Although this negative effect is small, it should be subject to larger scrutiny. Since the type of voting technology used does not store information on the machine, these negative perceptions might be caused by broader concerns about the secrecy and privacy of new information technologies. Given that increasing perceptions of fairness of elections is one of the aims of this change in voting procedures in several developing countries, this article sought to make a contribution to the prospects of e-voting in its implementation beyond the Argentine case.
