Pre-Reading

Students were assigned two readings in preparation for the lab. The first reading, a chapter from Howard’s (Reference Howard2017) Thinking Like a Political Scientist entitled “Using Documents as Evidence,” introduced students to strengths and potential pitfalls of working with archival documents. It emphasizes attention to authority, bias, and currency in written records, and it describes the process through which scholars can triangulate different sources of evidence. The second reading, the introduction to O’Rourke’s (Reference O’Rourke2018) book entitled Covert Regime Change: America’s Secret Cold War, provided substantive background for this lab.

Instruction on Working with Archival Evidence

Before receiving the assignment, students were given explicit instruction on conducting archival research (see the online appendix). We discussed advantages and limitations of archival research as well as how to become oriented to the organizational scheme of an archive, interpret individual documents, and use documents as evidence in their analysis. We provided practical tips for working with large amounts of archival material. The lesson also highlighted several collections of archival materials that have been digitized, and we provided a list of additional resources on archival research that were well suited for undergraduates.

Introducing the Lab

We assigned students into groups of four or five and distributed the Archival Methods Lab assignment (table 1). The lab asked students to draw on an assigned set of documents from the FRUS series to evaluate competing hypotheses about what motivated American regime-change efforts in the Dominican Republic in the 1960s. We also gave them a handout adapted from O’Rourke’s (Reference O’Rourke2018) book that explains each hypothesis and provides brief background information on the politics of the Dominican Republic (see the online appendix).

Table 1

Archival Methods Lab

The lab instructed students to proceed in three steps: (1) specify the observable implications of each potential hypothesis; (2) read and analyze the assigned set of documents; and (3) write a memo drawing on their documents to evaluate the plausibility of each hypothesis. In class discussion, we encouraged students to think about whether the material in their set of documents would “be consistent with” or “cast doubt on” potential explanations instead of “proving” that one was correct. We allowed students about 30 minutes of class time to begin working on the lab. This gave them an opportunity to ask questions of the instructors and to work out a plan with their group for tackling the assignment. Each group was assigned a specific set of documents from the FRUS series (see the online appendix) and had five days to complete the assignment.

Debriefing

In debriefing the lab in the following class session, we asked students to describe their approach and identify any challenges. We then shared common themes that we observed in their memos. We found that the groups that struggled the most were those who did not clearly specify observable implications in advance or were too broad in what they understood to constitute evidence in support of a particular hypothesis. For example, some groups counted any mention of American firms as evidence that economic motives drove US actions, whereas other groups made a similar leap with references to the Soviet Union and the hypothesis about superpower rivalry. As a possible solution, we discussed looking at an initial sample of documents to refine the observable implications before reviewing the remaining documents. This discussion led to (1) the insight that actions taken to “preserve regional hegemony” and “preempt a Soviet threat” might appear similar to one another; and (2) the conclusion that the sample hypotheses must be specified more clearly in advance to be distinguished.

Another challenge students faced was in providing sufficient context for archival evidence cited in the memos. It was common for memos to include a quotation without indicating the type of archival document from which it originated or the author. In the debriefing discussion, students recognized that some familiarity with the actors and institutions mentioned in a document was a prerequisite to understanding its meaning and purpose. For example, documents with “talking points” for telephone calls with counterparts in the Dominican Republic differed significantly from intelligence briefings intended for internal consumption. We emphasized to students that providing this type of context was crucial when using quotations in support of their claims. We also discussed what students did with conflicting evidence; asked them to reflect on which perspectives were missing from the material they reviewed; and asked them to brainstorm other potential sources that they might use to triangulate evidence if they were to continue this research. After the conclusion of the lab, students reported greater comfort working with archival materials; several incorporated archival sources into their theses; and we observed an improvement in their ability to specify observable implications.

Pre-Reading

In preparation for this lab, students were required to read the Tomz and Weeks (Reference Tomz and Jessica2013) article, “Public Opinion and the Democratic Peace,” which used survey data to explore how individuals in the United States and the United Kingdom view military action against democracies. They also were required to read the associated appendix, which included survey questions and how they were coded for analysis. We also asked students to read the introduction and appendix from Mosley’s (Reference Mosley2013) edited volume, Interview Research in Political Science. Mosley’s introduction sets up the usefulness of interviews in political science research, describes examples of their use in the discipline, and discusses decisions and challenges related to using interviews. The appendix includes examples of multiple scholars’ interview materials, such as sample consent scripts and questionnaires.

Instruction on Interviews and Surveys

As we did with the archival lab, we began the survey and interview research lab with concrete instruction on each method (see the online appendix). We discussed research ethics and explained the process required for our institution’s IRB approval to conduct research with human subjects. We then discussed surveys, explaining their advantages and disadvantages, with a primary focus on the practical aspect of survey design. Finally, we discussed the benefits and drawbacks of interviews, highlighting how the two methods require the researcher to craft different types of questions. This lab illustrated how scholars can approach the same research question with different research methods.

Introducing the Lab

In this lab, students first crafted a research question to build on the existing scholarship on attitudes toward immigration summarized in Hainmueller and Hopkins’s (Reference Hainmueller and Hopkins2014) article, “Public Attitudes Toward Immigration.” Second, they developed survey questions that could help them answer their question. Third, they brainstormed a relevant group of interview subjects who were accessible to them and crafted a set of interview questions that would shed light on their question. Fourth, students prepared answers to several questions on our institution’s IRB form and drafted an informed consent letter (table 2).

Table 2

Survey and Interview Methods Lab

Debriefing

When we graded the interview and survey labs, we noted strengths in particular groups’ write-ups as well as common problems. We found that our students were able to develop a research question to address a gap in the literature based on the information that they were given.Footnote ³ However, they struggled to frame questions in ways that would give them useful survey data and fruitful interview material. They had trouble remembering that they should not ask their participants to directly answer their research question but rather should ask questions that help them—as researchers—to answer the question. In the surveys, this meant that students often were drafting questions that asked respondents to directly assess cause and effect (e.g., “On a scale of 1 to 5, how much do you think the media you consume has impacted your policy views?”) as opposed to crafting questions that capture a single concept. They needed to be reminded that survey analysis allows them to explore whether different questions have correlated responses.

For the interview questions, students had trouble crafting open-ended questions and often phrased them in ways that might cause a participant to feel as if they were being quizzed (e.g., “What did you learn about immigration in school?”). They needed to be reminded that interviews are best for generating in-depth explanations about motivations, reasoning, and processes and that the best way to elicit this type of information is to craft questions that encourage participants to tell stories. We were able to use the debriefing to brainstorm ways to refine the questions the students had developed. At the end of the discussion, students displayed a stronger understanding of the different strengths and limitations of interviews and survey work. Some students subsequently incorporated these methods into their thesis projects.