HISTORY OF EXPERIMENTAL PSYCHOLOGYFootnote ¹

The first experimental laboratory in psychology was set up by Wilhelm Wundt at the University in Leipzig in 1879.Footnote ² Originally trained in medicine at the University of Heidelberg, Wundt separated psychology from philosophy by developing rigorous methods of manipulation and experimentation. Wundt wanted to explore the foundation of conscious thought through the systematic investigation of introspection. He was particularly interested in human physiology and wrote the first psychology textbook, Principles of Physiological Psychology, on this topic in Reference Wundt and Titchener1874. Wundt's foundational influence on psychology was felt not only through his incredibly prolific writings over the course of more than 60 years of scholarship, but also through the influence of his numerous prominent students. One of those students, G. Stanley Hall, the first president of Clark University, became the founder and first president of the American Psychological Association, which began in 1892, and started the American Journal of Psychology in 1887. Hall is also credited with setting up the first experimental laboratory in the United States at Johns Hopkins University in 1883. However, for understanding the history of psychological experimentation in America, perhaps the most important actor was Edward Titchener. Arguably the most devoted of Wundt's students, Titchener set up an experimental psychology laboratory at Cornell University that proved deeply influential for training generations of students. These students included Edwin Boring (who authored one of the first books documenting the use of experiments in psychology, History of Experimental Psychology (Reference Boring1929)) and, despite Titchener's noted sexism, Maragaret Floy Washburn, the first female psychologist, as well as such luminaries as Abraham Maslow. Following Wundt, Titchener sought to understand the nature of internal mental processes through a strategy he called “hard introspection,” designing and developing methods and measurements that attempted to standardize representations of internal experiences across individuals. His work became known as structuralism and vied for dominance in American psychology against William James' (Reference James1890) functionalist perspective, which primarily relied on observation. James had little patience for the structuralist perspective, claiming it had “plenty of school, but no thought” (James Reference James1904), and the influence of the paradigm essentially died with Titchener, who passed away in 1927. Although James won the early battle, he lost the war for the heart of psychological theory as both perspectives quickly lost influence to more systematic models offered by psychoanalysis and later behaviorism. Ironically, much like the way that the survey methodology developed by sociologists, such as Paul Lazarsfeld at Columbia University, allowed political scientists to break away from both psychologists and sociologists in their investigation of mass political behavior, the very experimental methods established by Wundt and Titchener, applied to behavior and not internal mental representation, allowed behaviorism to take hold and dominate academic psychology throughout the mid-twentieth century.

HISTORY OF EXPERIMENTAL ECONOMICSFootnote ³

The development and widespread use of experiments in economics has a more recent history. Early work was not experimental. Most histories attribute the birth of experimental economics to the classic Reference von Neumann and Morgenstern1944Theory of Games and Economic Behavior written by von Neumann and Morganstern and the later work in game theory it helped generate, including the foundational work by Thomas Schelling (Reference Schelling1960), Strategy of Conflict. Maurice Allais famous paradox, published in Econometrica in Reference Allais1953, provided the first experimental evidence of systematic violations of expected utility theory. Reinhard Selten, also working on this topic in Europe (Reference Selten1995), combined his work in mathematics and economics with courses he had taken in experimental psychology to explore the social implications of Von Neumann and Morganstern's work as well as Herbert Simon's notion of bounded rationality. Selten went on to develop the notion of sub-game perfection as an outgrowth of a larger experimental project, and he collaborated with Werner Guth, the first person to publish on the now ubiquitous Ultimatum game (Guth, Schmittberger, and Schwarz Reference Guth, Schmittberger and Schwarz1982). Around the time of Allais' early work, Vernon Smith (Reference Smith1962) began conducting some experiments in economics; the first work was published in the Journal of Political Economy. Smith (Reference Smith1976) is credited with creating experimental economics with his seminal piece on Induced Value Theory, in which he strongly advocated the use of stringent experimental methods to test economic theories and argued that such procedures constituted a rigorous empirical test of models developed using standard economic theory. Interestingly, he was soon joined by psychologists Amos Tversky and Daniel Kahneman (Reference Tversky and Kahneman1974; Kahneman and Tversky Reference Kahneman and Tversky1979) as well as others such as Paul Slovic, Sarah Litchenstein, and Robyn Dawes in conducting experiments on economic topics. Demonstrating that even the Nobel Prize Committee refused to privilege one discipline over the other, Smith and Kahneman both shared the Nobel Prize in Economics in 2002 for this early work in developing the field of behavioral economics that fundamentally rested on psychological experimental methods and procedures. It may appear that the development between psychological experimentation and experiments in behavior economics developed in circular ways. Throughout the early 1990s these fields proceeded in recursive interaction to refine experimental methods involving the use of simple games, although economists, not surprisingly, tended to focus on those methods exploring economic allocations whereas psychologists tended to concentrate on topics related to conflict and cooperation.

Note that each of these disciplines developed experimental methods for their own purposes to explore those topics most closely aligned with their own central theoretical concerns. In psychology, experimentalists wanted to understand the nature of internal human consciousness and experience and how those influences affected behavior. Experimental economists remained primarily concerned with the nature of human choice around preference, utility, and value and on understanding how those forces helped shape primarily economic decision making. Although both these concerns are central to the social choices political scientists study, neither concentrates specifically on the primary institutional and political concerns, particularly around dominance and the mechanisms of coercion and governance, that tend to preoccupy political scientists. These differences remain critical: if political scientists want to study the effect of governance across countries—whether that investigation involves exploring the influence of regime type, or humanitarian intervention, or any of the other myriad potential topics of examination—questions of deception and incentives may prove much less challenging than the logistical challenges faced by experimenters. Simple economic games in the field may reveal interesting cultural diverges across regions; however, investigators may be more interested in the effect of much larger changes on downstream societal consequences such as democratization, globalization, or economic liberalization. In these areas, experimenters may need to use natural experiments (Dunning Reference Dunning2012) that instigate changes exogenously, but nonetheless offer scholars the opportunity to examine the consequences of macro changes on micro practices and beliefs. As a result, although experiments remain the most powerful methodological tool to traction causation, political scientists often take sides in disciplinary differences that derive from disputes whose origins and meanings lie outside the purview of their interests; political scientists need not restrict themselves solely to the practices of economists or psychologists just because they developed particular styles best suited to their own needs and purposes.

TOWARD A POLITICAL SCIENCE CONSENSUS ON BEST PRACTICES IN EXPERIMENTAL METHODS

Several important explorations of the use of experiments in political science (Druckman et al. Reference Druckman, Green, Kuklinski and Lupia2011; Morton and Williams Reference Morton and Williams2010) provide useful information for political scientists to understand these issues in greater depth or to conduct experiments. However, as an unintentional result of divergent disciplinary mimicry, a lack of consensus about the proper use of experimentation in political science has emerged and hinders the inherent flexibility and power it offers. Rather than follow in lockstep with other disciplines, political science should adopt those experimental methods and techniques across disciplines of origin that best serves the purposes of the discipline and avenues of inquiry. As long as technical experimental procedure and protocol is followed, the particular plumage that surrounds these experiments can, and should, be dictated by what is most useful for answering the questions under investigation and not by what other disciplines dictate as proper protocol for investigations that serve different incentives, goals, and restraints.

Therefore, to begin a conversation within the field, if not to achieve such a consensus, I offer, with a nod to Robert Huckfeldt for suggesting the phrase, ten commandments to guide effective use of experiments for the purposes that define our disciplinary needs and purposes. I suggest these principles in true humor and humility, to generate more discussion and increased self-consciousness in the choices that we make. My purpose is to ensure practices that can simultaneously promote the health and safety of our subjects as well as our intellectual edification and professional advancement. I do not claim these commandments as the only way forward; I merely suggest that rendering such choices visible, rather than unconscious or automatic, serves our purposes better than simply following in paths laid down by others pursing different journals simply as a result of disciplinary envy.

1. 1. Thou shalt replicate and be plentiful upon the discipline. A single experiment tells something about the results of a specific test in a particular population. The ability to generalize results derives not so much from the external validity of that singular population, but from replication of the result across many populations. Robust findings will replicate across populations, environments, and even operationalizations. Experiments best help locate the limits, dimensions, and contingencies of particular behaviors through a strategy of conscious and aggregated replication.

2. 2. Thou shalt search far and wide for representative population samples. Sometimes student populations are more than adequate, especially to investigate phenomena that are expected to be universal. Sometimes other restricted populations prove sufficient for studying particular phenomena; for example, experiments would want soldiers in a study on the effects of combat on various psychological processes such as memory. Indeed, experimenters should seek to study the population that best represents the questions or problems posed by the investigator along with a relevant comparison control group.

3. 3. Thou shalt randomize. This is the crux of the experimental method, upon which all analysis of results depends. Randomization is the process that allows for the reasonable exclusion of alternative explanations for observed findings. Randomization allows experimenters to discount the causal influence of any preexisting differences between subjects because such variance can safely be assumed to occur in a manner that will not systematically affect observed results.

4. 4. Thou shalt publish null results. Of course, this admonition is directed more toward journal editors and field norms than toward individual scholars. But systematic publication bias wastes untold time, money, and energy, not to mention agony. Conscientious publication of null results would not only save numerous people countless wasted hours conducting an experiment unaware that others have tried and failed but also respects truth in seeking both positive and negative knowledge.

5. 5. Thou shalt honor both laboratory and field experiments. Political scientists can, and should, welcome all experimental comers. For some questions, controlled laboratory experiments increase internal validity and allow the quickest and most cost-effective investigation. For other questions, taking the experiment into a field setting allows the investigator to explore the influence of real-world factors on subject responses, increases external validity, and expands the population it is possible to study. Experiments embedded in nationally representative samples also offer a particularly powerful and effective tool for achieving high levels of control in representative populations.

6. 6. Thou shalt respect both internal and external validity. Psychologists privilege internal validity, whereas political scientists typically focus, some might say obsess, about external validity. Both are important, but an inherent trade-off exists between the two. Nonetheless, careful design can pay attention to both factors, understanding their mutual interdependence, and the temporal nature of their evolution. If there is no internal validity, external validity becomes a moot issue. If there is no external validity, internal validity is rendered airless and musty and devoid of real-life meaning and purpose.

7. 7. Thou shalt allow promiscuous incentive structures. Subjects can be rewarded and incentivized in many ways. Economists believe that money represents the only true incentive, and this may reflect a self-fulfilling prophecy for economists who only care about money. But political scientists need not publish exclusively in journals that require monetary remuneration of subject populations, from whence the real incentive structure for economists emerges. In reality, most people who are not economists care about other things besides money that can be difficult to purchase with money, belying the notion that money constitutes an infinitely fungible resource; these goals include things such as love and family and status and reputation. Even when proxies, such as sex, can be purchased, the monetary exchange by definition changes the meaning, and thus perhaps the experience. Similar to the way that adjusting the temperature in the room can affect behavior without conscious awareness of the subjects, things that might be exchanged for money even in the near term, such as food or drink, achieve powerful force in a moment of deprivation or discomfort and thus can be used to experimental advantage in short-term, mild ways. Money can be an incentive, but so can credit for grades for students or juice squirts for a thirsty person. Any use of incentives implicitly assumes that subjects only receive what is offered in return, yet subjects enter every experiment with their internal incentives, perhaps unknown, and certainly uncontrolled by the experimenter. Experimenters should remain humble and realize that subject incentives, like subject behavior, might reflect untested assumptions. And it can improve experimental outcomes for researchers to understand what internal incentives structures might exist for subjects inherent within any given design, including the desire for knowledge, appreciation, and approval. Sometimes these desires can help an experimenter's purpose, sometimes they can undermine it. But like leaders who often issue statements to other leaders to impress domestic audiences, experimenters are always well served to remember that subjects may prefer to influence another subject rather than doing what the experimenter requests.

8. 8. Thou shalt allow deception. Deception is not like kosher; it does not contaminate everything it touches. Leaving aside reflections about the conservative ideology that privileges concerns about purity over those associated with harm, just because an experiment that uses deception takes place in a particular room does not render said space forever unusable like some kind of viral radiation poisoning that inevitably infiltrates and violates any future experiment that takes place in that room. Again, political scientists need not restrict their publication venues to journals that prohibit deception. Like abortion, deception should be legal but rare. If it is not necessary, it should not be used. If it is the only, or best, way to explore important human phenomena, it should be used carefully and can be used legitimately. Some of the most important phenomena known about human psychology, from conformity to obedience to the capacity for violence and aggression, come from the use of deception. Few would argue that we would be better off without this knowledge, although most would agree that exquisite protection of subjects is absolutely demanded by such practices. Although some might argue it is better we not know the things we have learned through deception, such a debate rests on values outside this consideration. However, in my long experience, the majority of subjects care more about their time and their money or other incentives than they care about any duplicity being undertaken against them. Student populations, in particular, find these practices no more loathsome than lack of transparency in grading, and no economists seem to oppose the use of grades, lending some hypocrisy to their rejection of deception in all its forms. In my observation, only narcissism makes experimenters believe that subjects put the same kind of thought into their experimental purpose as they do. Experimenters may care about the use of deception, but subjects have better things to do with their time, and better things to think about than whether an experiment involves deception. If they want the incentive, they participate. If they do not, they will not. After all, human subjects require experimenters to tell subjects prior to participation that they can quit at any time. Note that the reason we know that such actions might prove difficult for subjects, and thus we inoculate them against this concern from the outset, is precisely because of the results we learned from experiments involving deception.

9. 9. Thou shalt be thy own first subject. Like a good doctor who takes his own medicine first, and leaving aside those doctors who abuse their own drugs, experimenters should be their own first subject. They should first run through the experiment they want others to experience to see if any aspect of it makes them confused, uncomfortable, or uncertain. Experimenters should try to do this with a blind mindset without the preconceptions they derive from knowing the experiment's design and its purpose. This activity allows the experimenter to discover things that the subject will see that might have been lost on the experimenter during the complications of arriving at the design. This strategy can prevent unexpected problems from cropping up early in an experiment that cost time and possibly lose subject data. The experimenter may also gain important new ideas about possible extensions.

10. 10. Thou shalt run and debrief at least some of one's own subjects. An experimenter never knows what happens in an experiment if he or she is not there to observe. And these observations can prove critical in subsequent understanding of unexpected findings or in generating ideas for follow-up studies. If an experiment is well designed, the experimenter should be able to run subjects blind as to condition in a way that will not allow him or her to run subjects without affecting demand characteristics in the experiment in a way that systematically influences subject response. Only by talking to subjects about their experience can experimenters really know whether subjects are actually interpreting the task or situation as the experimenter assumes. Such variance in interpretation helps identify the source of unexpected findings or failed manipulations. This knowledge alone sets the experimenter free to uncover new sources of influence and novel interpretations of results.

The beauty of experiments, as an art and a practice, is that they embody the conundrum that only discipline will set you free. Careful attention to fundamental aspects of random assignment, meticulous construction of the operationalization and measurement of variables, and precise treatment and control protocols will ensure that an experiment will achieve its greatest likelihood of success. In that precise attention to detail in the grounding of an experiment, creative and innovative ideas can take flight, and find rest on the branches of predictable human behavioral responses.