No CrossRef data available.
Article contents
Multivariate Continuous Blocking to Improve Political Science Experiments
Published online by Cambridge University Press: 04 January 2017
Abstract
Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.
Political scientists use randomized treatment assignments to aid causal inference in field experiments, psychological laboratories, and survey research. Political research can do considerably better than completely randomized designs, but few political science experiments combine random treatment assignment with blocking on a rich set of background covariates. We describe high-dimensional multivariate blocking, including on continuous covariates, detail its statistical and political advantages over complete randomization, introduce a particular algorithm, and propose a procedure to mitigate unit interference in experiments. We demonstrate the performance of our algorithm in simulations and three field experiments from campaign politics and education.
Information
- Type
- Research Article
- Information
- Copyright
- Copyright © The Author 2012. Published by Oxford University Press on behalf of the Society for Political Methodology
References
Anderson, Michael L.
2008. Multiple inference and gender differences in the effects of early intervention: A reevaluation of the Abecedarian, Perry Preschool, and early training projects. Journal of the American Statistical Association
103: 1481–95.Google Scholar
Barnard, John, Frangakis, Constantine E., Hill, Jennifer L., and Rubin, Donald B.
2003. Principal stratification approach to broken randomized experiments. Journal of the American Statistical Association
98: 299–323.Google Scholar
Belfield, Clive R., Nores, Milagros, Barnett, Steve, and Schweinhart, Lawrence. 2006. The high/scope Perry Preschool Program. Journal of Human Resources
41: 162–90.Google Scholar
Boruch, Robert, May, Henry, Turner, Herbert, Lavenberg, Julia, Petrosino, Anthony, De Moya, Dorothy, Grimshaw, Jeremy, and Foley, Ellen. 2004. Estimating the effects of interventions that are deployed in many places. American Behavioral Scientist
47: 608–33.Google Scholar
Bowers, Jake. 2011. Making effects manifest in randomized experiments. In Cambridge handbook of experimental political science, eds. Druckman, James N., Green, Donald P., Kuklinski, James H., and Lupia, Arthur, 459–80. Cambridge, UK: Cambridge University Press.Google Scholar
Bowers, Jake, Fredrickson, Mark, and Hansen, Ben. 2010. RItools: Randomization inference tools. R package version 0.1–11, http//www.jakebowers.org/RItools.html (accessed August 16, 2012).Google Scholar
Bullock, John G.
2011. Elite influence on public opinion in an informed electorate. American Political Science Review
105: 496–515.Google Scholar
Cochran, William G., and Rubin, Donald B.
1973. Controlling bias in observational studies: A review. Sankhya: The Indian Journal of Statistics, Series A
35: 417–46.Google Scholar
Donner, Allan, and Klar, Neil. 2000. Design and analysis of cluster randomization trials in health research. London: Arnold Publishers.Google Scholar
Duflo, Esther, Glennerster, Rachel, and Kremer, Michael. 2008. Using randomization in development economics research: A toolkit. In Handbook of Development Economics, ed. Paul Schultz, T., Vol. 4, 3895–962. Amsterdam: Elsevier, B.V.
Google Scholar
Epstein, Lee, and King, Gary. 2002. The rules of inference. University of Chicago Law Review
69: 1–133.Google Scholar
Freedman, Laurence S., Gail, Mitchell H., Green, Sylvan B., and Corle, Donald K.
1997. The efficiency of the matched-pairs design of the Community Intervention Trial for Smoking Cessation (COMMIT). Controlled Clinical Trials
18: 131–9.Google Scholar
Gail, Mitchell H., Byar, David P., Pechacek, Terry F., and Corle, Donald K.
1992. Aspects of statistical design for the Community Intervention Trial for Smoking Cessation (COMMIT). Controlled Clinical Trials
13: 6–21.Google Scholar
Gerber, Alan S., Gimpel, James G., Green, Donald P., and Shaw, Daron R.
2011. How large and long-lasting are the persuasive effects of televised campaign ads? Results from a randomized field experiment. American Political Science Review
105: 135–50.Google Scholar
Goldstein, Daniel G., Imai, Kosuke, Göritz, Anja S., and Gollwitzer, Peter M.
2010. Nudging turnout: Mere measurement and implementation planning of intentions to vote. Manuscript.Google Scholar
Gosnell, Harold F.
1927. Getting out the vote: An experiment in the stimulation of voting. Chicago, IL: University of Chicago Press.Google Scholar
Green, Donald P., and Kern, Holger L. Forthcoming 2012. Modeling heterogeneous treatment effects in survey experiments with Bayesian additive regression trees. Public Opinion Quarterly.Google Scholar
Greevy, Robert, Lu, Bo, Silber, Jeffrey H., and Rosenbaum, Paul. 2004. Optimal multivariate matching before randomization. Biostatistics
5: 263–75.Google Scholar
Hangartner, Dominik, and Moore, Ryan T.
2011. Generalizing and stabilizing the augmented inverse propensity weighted estimator. Proceedings of the Midwest Political Science Association Annual Meeting.Google Scholar
Hansen, Ben B.
2004. Full matching in an observational study of coaching for the SAT. Journal of the American Statistical Association
99: 609–18.Google Scholar
Hansen, Ben B., and Bowers, Jake. 2008. Covariate balance in simple, stratified, and clustered comparative studies. Statistical Science
23: 219–36.CrossRefGoogle Scholar
Hansen, Ben B., and Klopfer, Stephanie Olsen. 2006. Optimal full matching and related designs via network flows. Journal of Computational and Graphical Statistics
15: 609–27.Google Scholar
Heckman, James, Moon, Seong Hyeok, Pinto, Rodrigo, Savelyev, Peter, and Yavitz, Adam. 2009. A reanalysis of the high-scope Perry Preschool Program, Unpublished manuscript.Google Scholar
Heckman, James, Moon, Seong Hyeok, Pinto, Rodrigo, Savelyev, Peter, and Yavitz, Adam. 2010. The rate of return to the high-scope Perry Preschool Program. Journal of Public Economics
94: 114–28.Google Scholar
Ho, Daniel, Imai, Kosuke, King, Gary, and Stuart, Elizabeth. 2007. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis
15: 199–236.Google Scholar
Holland, Paul. 1986. Statistics and causal inference. Journal of the American Statistical Association
81(396): 945–60.Google Scholar
Horiuchi, Yusaku, Imai, Kosuke, and Taniguchi, Naoko. 2007. Designing and analyzing randomized experiments: Application to a Japanese election survey experiment. American Journal of Political Science
51: 669–87.Google Scholar
Hudgens, Michael G., and Elizabeth Halloran, M.
2008. Toward causal inference with interference. Journal of the American Statistical Association
103: 832–42.Google Scholar
Hyde, Susan. 2010. Experimenting in democracy promotion: International observers and the 2004 Presidential elections in Indonesia. Perspectives on Politics
8: 511–27.Google Scholar
Iacus, Stefano M., King, Gary, and Porro, Giuseppe. 2011. Multivariate matching methods that are monotonic imbalance bounding. Journal of the American Statistical Association
106: 345–61.Google Scholar
Iacus, Stefano M., King, Gary, and Porro, Giuseppe. 2012. Causal inference without balance checking: Coarsened exact matching. Political Analysis
20: 1–24.Google Scholar
Ichino, Nahomi, and Schündeln, Matthias. 2012. Deterring or displacing electoral irregularities? Spillover effects of observers in a randomized field experiment in Ghana. Journal of Politics
74: 292–307.Google Scholar
Imai, Kosuke, and van Dyk, David A.
2004. Causal inference with general treatment regimes: Generalizing the propensity score. Journal of the American Statistical Association
99: 854–66.Google Scholar
Imai, Kosuke, King, Gary, and Nall, Clayton. 2009. The essential role of pair-matching in cluster-randomized experiments, with application to the Mexican universal health insurance evaluation. Statistical Science
24: 29–53.CrossRefGoogle Scholar
Imai, Kosuke, King, Gary, and Stuart, Elizabeth A.
2008. Misunderstandings between experimentalists and observationalists about causal inference. Journal of the Royal Statistical Society, Series A
171: 481–502.Google Scholar
Imai, Kosuke, Keele, Luke, Tingley, Dustin, and Yamamoto, Teppei. 2011. Unpacking the black box of causality: Learning about causal mechanisms from experimental and observational studies. American Political Science Review
105: 765–89.Google Scholar
Imbens, Guido W.
2011. Experimental design for unit and cluster randomized trials. Manuscript prepared for the International Initiative for Impact Evaluation.Google Scholar
Keele, Luke, and Morgan, Jason W.
2011. Stronger instruments by design. Poster presented at the 28th Annual Summer Meeting of the Society for Political Methodology.Google Scholar
King, Gary. 1995. Replication, replication. PS: Political Science and Politics
28: 444–52.Google Scholar
King, Gary, Gakidou, Emmanuela, Imai, Kosuke, Lakin, Jason, Moore, Ryan T., Nall, Clayton, Ravishankar, Nirmala, Vargas, Manett, María Téllez-Rojo, Martha, Eugenio Hernández Ávila, Juan, Ávila, Mauricio, and Hernández Hernández Llamas, Héctor. 2009. Public policy for the poor? A randomized assessment of the Mexican universal health insurance program. Lancet
373: 1447–54.Google Scholar
King, Gary, Gakidou, Emmanuela, Ravishankar, Nirmala, Moore, Ryan T., Lakin, Jason, Vargas, Manett, María Téllez-Rojo, Martha, Eugenio Hernández Ávila, Juan, Hernández Ávila, Mauricio, and Hernández Llamas, Héctor. 2007. A “politically robust” experimental design for public policy evaluation, with application to the Mexican universal health insurance program. Journal of Policy Analysis and Management
26: 479–509.Google Scholar
Lachin, John M.
1988. Properties of Simple Randomization in Clinical Trials. Controlled Clinical Trials
9: 312–26.Google ScholarPubMed
Linday, Linda A., Tsiouris, J. A., Cohen, Ira L., Shindledecker, Richard, and DeCresce, Robert. 2001. Famotidine treatment of children with autistic spectrum disorders: Pilot research using single-subject research design. Journal of Neural Transmission
108: 593–611.Google Scholar
Loewen, Peter John, and Rubenson, Daniel. 2011. For want of a nail: Negative persuasion in a party leadership race. Party Politics
17: 45–65.Google Scholar
Lu, Bo, Zanutto, Elaine, Hornik, Robert, and Rosenbaum, Paul R.
2001. Matching with doses in an observational study of a media campaign against drug abuse. Journal of the American Statistical Association
96: 1245–53.Google Scholar
Mason, Robert L., Gunst, Richard F., and Hess, James L.
1989. Statistical design and analysis of experiments: With applications to engineering and science. New York: Wiley.Google Scholar
Mebane, Walter R.J., and Sekhon, Jasjeet S.
1998. GENetic Optimization Using Derivatives (GENOUD).Google Scholar
Moore, Ryan T.
2012a. blockTools: Blocking, assignment, and diagnosing interference in randomized experiments. R package version 0.5–6, http://www.wustl.edu/software.blockTools.him (accessed August 16, 2012).Google Scholar
Moore, Ryan T.
2012b. Replication data for: Multivariate continuous blocking to improve political science experiments. http://hdl.handle.net/1902.1/18341, IQSS Dataverse Network [Distributor] V1 [Version].Google Scholar
Moore, Ryan T., and Moore, Sally A.
2012. Blocking for sequential political experiments. Manuscript in preparation.Google Scholar
Murray, David M.
1998. Design and analysis of group-randomized trials. New York: Oxford University Press.Google Scholar
National Cancer Institute. 1995. NCI Monograph #6: Community-based interventions for smokers: The COMMIT field experience. Technical Report 95-4028, National Institutes of Health.Google Scholar
Paluck, Elizabeth Levy, and Green, Donald P.
2009. Deference, dissent, and dispute resolution: An experimental intervention using mass media to change norms and behavior in Rwanda. American Political Science Review
103: 622–44.Google Scholar
Panagopoulos, Costas, and Green, Donald P.
2008. Field experiments testing the impact of radio advertisements on electoral competition. American Journal of Political Science
52: 156–68.Google Scholar
Pechacek, Terry F.
2006. Personal communication, Centers for Disease Control's Associate Director for Science for the Office on Smoking and Health, 5 December.Google Scholar
R Development Core Team. 2012. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.Google Scholar
Rosenbaum, Paul R.
2007. Interference between units in randomized experiments. Journal of the American Statistical Association
102: 191–200.Google Scholar
Rousseeuw, Peter J.
1985. Multivariate estimation with high breakdown point. Mathematical Statistics and Applications
8: 283–97.Google Scholar
Rousseeuw, Peter J., and van Zomeren, Bert C.
1990. Unmasking multivariate outliers and leverage points. Journal of the American Statistical Association
85(411): 633–9.Google Scholar
Rubin, Donald B.
1980. Bias reduction using Mahalanobis-metric matching. Biometrics
36: 293–8.Google Scholar
Rubin, Donald B.
1990. Formal modes of statistical inference for causal effects. Journal of Statistical Planning and Inference
25: 279–92.Google Scholar
Rubin, Donald B.
2001. Using propensity scores to help design observational studies: Application to the tobacco litigation. Health Services and Outcomes Research Methodology
2: 169–88.Google Scholar
Sekhon, Jasjeet S.
2011. Multivariate and propensity score matching software with automated balance optimization: The matching package for R. Journal of Statistical Software
42: 1–52.Google Scholar
Sobel, Michael E.
2006. What do randomized studies of housing mobility demonstrate? Causal inference in the face of interference. Journal of the American Statistical Association
101(476): 1398–407.Google Scholar
Tamhane, Ajit C.
2009. Statistical analysis of designed experiments: Theory and applications. Hoboken, NJ: John C. Wiley & Sons.Google Scholar
Wantchekon, Leonard. 2003. Clientelism and voting behavior: Evidence from a field experiment in Benin. World Politics
55(3): 399–422.CrossRefGoogle Scholar
Moore supplementary material
Supplementary Material
PDF
222.1 KB
You have
Access
- 41
- Cited by