Hostname: page-component-6898cfcb55-p2bvx Total loading time: 0 Render date: 2026-11-11T03:16:53.864Z Has data issue: false hasContentIssue false
  • English
  • Français

Increased Realism at Lower Cost: The Case for the HybridExperiment

Published online by Cambridge University Press:  28 June 2011

Delia Dumitrescu and
André Blais
Delia Dumitrescu
Affiliation:
Université de Montréal
André Blais
Affiliation:
Université de Montréal
Rights & Permissions [Opens in a new window]

Abstract

We propose an experimental design particularly adapted to the studyof individual behavior in collective action situations. Theexperimental protocol improves on the artificiality that is commonlypresent in lab and survey experiments to achieve a closerreplication of the real-life conditions of such decisions whileavoiding the high costs associated with field experiments. Weexemplify this design by means of a study on strategic voting inelections.

Information

Type
Features
Information
PS: Political Science & Politics , Volume 44 , Issue 3 , July 2011, pp. 521 - 523
Copyright
Copyright © American Political Science Association 2011

There has been a recent explosion of experimental studies in political science (Druckman et al. Reference Druckman, Green, Kuklinski and Lupia2006; Loewen Reference Loewen2008), using three dominant approaches: laboratory experiments, survey experiments, and experiments conducted in the field. We propose a flexible design that borrows features from all three to provide greater experimental realism than is available in common lab and survey studies, at lower costs and effort than is required by a field experiment. While the “hybrid” design might not suit all studies, we believe that it is particularly adapted to the study of collective action situations in which individual choices are consequential only in relation to the choices of others.

The “hybrid” experiment shares some convenient platform and procedural characteristics with lab and survey studies: participants are recruited (rather than observed, as in field experiments) and must make decisions in a specially designed setting. The study takes place online. Its novelty stems from two features that allow it to approach the realism of field experiments. The first is the incentive structure, which is designed to produce greater individual involvement with the experimental situation. Rather than using only material incentives, like most collective action lab experiments (Palfrey Reference Palfrey2009), we ask voters to decide on the real allocation of resources for a policy cause about which they care. Second, to better replicate real collective decision-making conditions, the voter is located in a wider community in which collectively tallied ballots decide the allocation outcome. While this community has factual value for the participants, researchers can experimentally manipulate parts of it.

These two features, we argue, allow for a better replication of real-life elections than has been achieved in previous lab or survey experiments in terms of both individuals' emotional investment and the realistic influence that an individual in a large crowd can have over an electoral outcome. The increase in realism that results from these features of this hybrid experiment carries a low cost compared to other existing experimental alternatives, and thus, this approach could be applied to a host of political behavior–related questions. In the following section, we exemplify these features in a study and discuss the merits and limitations of such a hybrid design.

THE STUDY

Two hundred undergraduate students from a large university participated in an online study in February 2010. The study required participants to vote in a series of first-past-the-post elections and respond to questions for a total of 15–20 minutes. They received an initial payment of $5 for their participation and could gain up to an additional $2.50, depending on the outcome of elections.

The experiment was designed to study vote choice in elections involving the distribution of public goods. Participants voted in five independent elections, each involving the three parties shown in figure 1. At stake was the allocation of $600 between Équiterre, an environmental nongovernmental organization (NGO), and the 200 participants. A victory by Party A would ensure the greatest gain for the environmental cause. A victory by Party C would see this gain reduced the most.

Figure 1

Experimental Payoff Structure

Note. The same parties competed in each election. Participants were informed that the final gains for them and Équiterre would be the average gain across all five elections.

The students were told that they were part of a community of 9,200 people, of which 9,000 had already voted. Once the 200 remaining persons, including themselves, had voted, the gains for each of the five elections would be tallied and their average distributed. The distribution of the previous 9,000 votes across the five elections was manipulated to induce varying strategic vote considerations. In all cases, Party C was ahead and Party A was trailing (see table 1). The gaps between parties were either small (10 votes) or large (100 votes). The expectation was that the larger these gaps were, the stronger the incentive would be for Party A supporters to strategically defect from their first choice. Note that the order in which these elections were held varied randomly across individual participants, with the exception of the fifth election, a contest that always came last and in which it was impossible for Party A to win.Footnote 1

Table 1

Distribution of the Initial 9,000 Votes, by Election

Note. The first four of these vote distributions were presented in random order. The fifth always came last.

Table 2 presents the evolution of vote choice in these elections as a function of choice in the open “Close AB, Close BC” race, which Party A comfortably won. We interpret this election as indicative of individual sincere preferences because of the small start-up distance between any two parties. Table 2 shows that among the 119 individuals who voted for the most pro-environment party in the “Close AB, Close BC” election (i.e., Party A), the great majority (102) maintained their loyalty to the party in the “Close AB, Far BC” election. This number decreased in the fifth election, with 39 previous Party A supporters switching to Party B and 9 switching to Party C, although the majority (71) did remain loyal to Party A.

Table 2

Election Results: Stability and Change in Vote Choice

Note. Cells represent number of participants who voted for each party. The open-race election was won by Party A, with a total of 119 votes among the 200 participants.

As expected, many Party A supporters deserted the party when its chances of winning decreased in other elections. However, a striking majority stuck with the party every time, even when it was impossible for it to win, as in the last election. This outcome is consistent with survey evidence suggesting that strategic desertion of weak parties in first-past-the-post elections is relatively limited (Blais Reference Blais2002). Such evidence proposes that elections are construed first and foremost as opportunities to express opinions or preferences (Brennan and Lomasky Reference Brennan and Lomasky1993), and that strategic considerations are weaker than sometimes assumed.

DISCUSSION

The hybrid experimental design allows researchers to blend real-life policy decisions that are made in a large community setting with controlled experimental manipulations. By giving people the possibility of contributing to a public good (represented here by the environmental NGO), we are able to harness all other aspects that accompany decision making, such as individuals' values and emotional attachments, instead of just relying on calculus. If elections are mostly about the provision of public goods, as decided by a collective body, and if these decisions are made on the basis of values and emotions as well as interests, then the two defining features of this hybrid experiment give it an edge in external validity over traditional lab studies that operate with monetary incentives in small-group elections.

The experiment need not be confined to an online platform and could be conducted in a laboratory. The primary advantage of using the Internet is that this approach circumvents important participation costs for the subjects and consequently allows for the easier recruitment of “ordinary” citizens. In this case, our sample consisted of university students, but participants could complete the study at their leisure, without amending their schedules to come to the experimental lab. Given the existence of Internet subject pools, it is possible to reach people who may be willing to participate in a small experiment on their computer but are unwilling to devote several hours to performing a task in a laboratory environment.Footnote 2

The hybrid experiment cannot replace field experiments, but we believe that it is an interesting alternative for researchers with limited resources. The hybrid allows scholars to increase the external validity of the manipulation without requiring an apparatus to “introduce” the experimental treatment in the field and measure its potential effects. Typically, a field experiment entails providing (e.g., through the mail) different groups of people with different pieces of information (and a control group with no information at all) and then conducting a survey to ascertain the impact of both the information and individual characteristics on people's opinions or behavior. A hybrid experiment is more time- and cost-effective to perform.

There are some limitations to this approach. Perhaps the most obvious is that hybrid experiments must take a little time to complete. Most complicated experiments require the use of a laboratory. The same observation may also apply to experiments that aim to test learning effects, as the same task may have to be repeated many times. Finally, hybrid experiments remain more “artificial” than field experiments.

Despite these limitations, hybrid experiments provide an effective, simple, and flexible opportunity to test ideas.

Footnotes

1 Deception was carefully avoided by telling respondents that “for this study, you are in a situation in which you are part of a group of 9,200 people who have to decide on how a sum of money will be allocated.” The outcome of the election—that is, which party won—was accordingly determined by adding the votes of the 200 participants to the 9,000 votes, about whose distribution participants had been (correctly) informed.

2 While the Internet is both cheaper and more convenient than the lab, we recognize its greater risks concerning data quality, as there is less control over the amount of attention that subjects pay to the task. We did record the time spent for the entire experiment, with the average time to completion being 30 minutes (95% of the sample took an average of 27 minutes to vote in the five elections and answer a number of question batteries, with a standard deviation of 12 minutes). This high percentage suggests that few people devoted either an unusually short or an unusually long time to this task. Moreover, to circumvent any identity problems resulting from the online nature of the study, we asked participants to provide their unique (university-based) e-mail address prior to the start of the experiment. However, it would be interesting to compare the results obtained under the two platforms.

References

Blais, André. 2002. “Why Is There So Little Strategic Voting in Canadian Plurality Rule Elections?” Political Studies 50: 445–54.Google Scholar
Brennan, Geoffrey, and Lomasky, Loren E., eds. 1993. Politics and Process: New Essays in Democratic Thought. Cambridge, UK: Cambridge University Press.Google Scholar
Druckman, James D., Green, Donald P., Kuklinski, James H., and Lupia, Arthur. 2006. “The Growth and Development of Experimental Research in Political Science.” American Political Science Review 100: 627–35.Google Scholar
Loewen, Peter J. 2008. “Experimentation and Political Science: Six Applications.” Ph.D. diss., Université de Montréal.Google Scholar
Palfrey, Thomas R. 2009. “Laboratory Experiments in Political Economy.” Annual Review of Political Science 12: 379–88.CrossRefGoogle Scholar
Figure 0

Figure 1 Experimental Payoff StructureNote. The same parties competed in each election. Participants were informed that the final gains for them and Équiterre would be the average gain across all five elections.

Figure 1

Table 1 Distribution of the Initial 9,000 Votes, by Election

Figure 2

Table 2 Election Results: Stability and Change in Vote Choice