This article explores the relationship between the method of process tracing and the data collection technique of elite interviewing. The process tracing method has become an increasingly used and cited tool in qualitative research, a trend that has recently accelerated with the publication of Alexander George and Andrew Bennett's text (2005), Case Studies and Theory Development in the Social Sciences. That book outlines and explores the process tracing method in detail, highlighting its advantages for exploring causal processes and analyzing complex decision-making. Yet while the book presents a rigorous and compelling account of the process tracing method and its critical importance to case study research, the value of the method itself remains contested in some quarters, and there are aspects of George and Bennett's treatment of it that require further exploration.1
For recent critical discussions of George and Bennett (2005), see Qualitative Methods: Newsletter of the APSA Qualitative Methods Section; and of process tracing specifically, see Checkel (2006).
This article identifies one issue that has considerable relevance for process tracing, but that is under-explored in George and Bennett (2005)—the research tool of elite interviewing. Process tracing requires collecting large amounts of data, ideally from a wide range of sources. Yet George and Bennett concentrate largely on documentary research and generally refer to interviewing only in passing. This article draws together the disparate discussions of interviewing in George and Bennett (2005), and supplements them with insights from the wider literature on interview methods, and on elite interviewing in particular. The article focuses on three primary issues: first, the uses of interview data for the process tracing method; second, considerations of how to establish the evidentiary value of interview data; and finally, the implications that the process tracing method has for how researchers should approach sampling their interview subjects. In particular, the article argues for the use of non-probability sampling approaches to elite interviewing when utilizing the process tracing method.
As will be discussed further below, the goal of process tracing is to obtain information about well-defined and specific events and processes, and the most appropriate sampling procedures are thus those that identify the key political actors—those who have had the most involvement with the processes of interest. The aim is not to draw a representative sample of a larger population of political actors that can be used as the basis to make generalizations about the full population, but to draw a sample that includes the most important political players who have participated in the political events being studied. Consequently, random sampling runs against the logic of the process tracing method, as it risks excluding important respondents from the sample purely by chance. When sampling interviewees in a process tracing study, the ultimate goal is to reduce randomness as much as possible, and thus non-probability sampling approaches are the most appropriate.
These considerations in turn speak to recent research that explores the different criteria that can be used to evaluate certain methodological practices. As Collier, Brady, and Seawright (2005a) recently observed, a core concern of research design is managing the trade-offs that come with using different methodological approaches. Even with the common overarching goals of developing theory and making causal inferences, divergent research priorities can lead to different emphases on the appropriateness of particular methodological tools (2005a, 195–227). Consequently, what might be right for one study, for example random sampling, will be inappropriate for another study with a different set of research objectives. The sections below explore these issues in greater depth, beginning with the nature of the process tracing method itself before turning to issues of interviewing and sampling.
Process Tracing: Definition and Method
Recent years have seen a significant growth in the study of qualitative methods in political science, with a wide range of new studies addressing core methodological issues such as concept formation, case selection, and causal analysis.2
For a review of this recent literature, see Bennett and Elman (2006).
The process tracing method was first developed over two decades ago,3
but has been most comprehensively outlined and developed in George and Bennett's (2005) Case Studies and Theory Development in the Social Sciences. That book represents a robust defense of qualitative methodology, and case study research in particular. Touching on a wide range of theoretical, methodological, and philosophical issues, Case Studies and Theory Development in the Social Sciences provides guidance for case-study research and delineates the ways in which case studies can contribute both to theory development and to theory testing. At the heart of the George and Bennett approach is a concern with developing and testing theory in ways that incorporate attention to the causal processes at work in political life—to the causal mechanisms that link causes to effects. The authors argue that causal mechanisms are central to causal explanation, and that case studies and within-case analysis are the methods best able to examine the operation of causal mechanisms in detail (George and Bennett 2005, 12, 21). In turn, the authors present process tracing as the most appropriate method for uncovering such causal mechanisms. As the authors write: “In process tracing, the researcher examines histories, archival documents, interview transcripts, and other sources to see whether the causal process a theory hypothesizes or implies in a case is in fact evident in the sequence and values of the intervening variables in that case” (6). They continue: “The process tracing method attempts to identify the intervening causal process—the causal chain and causal mechanism—between an independent variable (or variables) and the outcome of the dependent variable” (206).As the first quotation makes clear, George and Bennett envisage interviews as one of the central tools of research that can be used to apply the process tracing method. Yet throughout their book, interviews are mentioned infrequently and are only rarely considered in isolation from other data sources, especially archival material and historical secondary sources. As John S. Odell (2006, 38) has noted, George and Bennett tend to equate the case study method with the historian's method, and in particular with archival and document-based research. As a result, while the role of archival work, and its inherent strengths and weaknesses, is discussed in detail, other forms of data collection, including interviewing, are left largely under-explored.
I argue, however, that interviewing, and especially elite interviewing, is highly relevant for process tracing approaches to case study research. Particularly in political science, process tracing frequently involves the analysis of political developments at the highest level of government, and elite actors will often be critical sources of information about the political processes of interest. Also, as George and Bennett (2005, 223) point out, the process tracing method requires enormous amounts of data in order to identify causal mechanisms at every step of the process of interest. It is thus important that we consider all data collection options, and that we explore their strengths, weaknesses, and uses in detail.
Elite Interviewing for Process Tracing: Uses, Strengths, and Weaknesses
In their more general discussion of process tracing and case study research, George and Bennett outline a range of uses for process tracing data, including interview data. These range from gathering basic information about a case (89) and filling in gaps in existing historical accounts (96), to more theoretical concerns such as finding omitted variables and hypotheses (fn32, 20) and uncovering causal processes (6). In this section I focus in particular on elite interviews, and examine the uses of elite interviewing when applying the process tracing method, and also the issues of evaluation that are raised when using this data collection technique to uncover causal processes (see Table 1). The subsequent section of the article then turns to examine issues of sampling that arise when conducting elite interviews in the context of the process tracing method.
Uses of Elite Interviews
The wider literature on elite interviews has identified a broad number of uses for this particular form of data collection:
1) To Corroborate What Has Been Established from Other Sources
Elite interview data are rarely considered in isolation, and the goal of collecting such data is often to confirm information that has already been collected from other sources. When documents, memoirs, and secondary sources provide an initial overview of the events or issues under examination, interviews with key players can be used to corroborate the early findings. In this way, interviews contribute toward the research goal of triangulation, where collected data are cross-checked through multiple sources to increase the findings' robustness. By ensuring that data are not collected from only one source, or one type of source, the triangulation strategy can increase the credibility of findings that are supported across multiples sources, and can reveal the weakness of some sources that might otherwise have been viewed as reliable.4
For a discussion of triangulation and elite interviewing, see Davies (2001).
2) To Establish What a Set of People Think
As well as serving a corroborative purpose, elite interviews can also be used for additive purposes—to provide new information that will advance the research process (Davies 2001). One such additive function is to establish what people think—what their “attitudes, values, and beliefs” are (Aberbach and Rockman 2002, 673).5
While it may be possible to obtain this information from other sources, the interview format allows analysts to probe respondents at length regarding their thoughts on key issues relevant to the research project. As opposed to surveys, interviewing allows researchers to ask open-ended questions and allows respondents to talk freely, without the constraint of having to answer according to fixed categories. Researchers can thus gather rich detail about key elites' thoughts and attitudes on central issues.3) To Make Inferences about a Larger Population's Characteristics and Decisions
Interviews can be used not only to gather new data about the beliefs or actions of specific individuals, but also to make inferences about the beliefs or actions of a wider group without interviewing everyone. When analysts choose interview subjects through random selection, and when the broader population of interview subjects is sufficiently large, analysts can obtain a representative sample and generalize from the findings of that sample to the wider group (Babbie 1995). In the context of elite interviews, this can be particularly important when seeking to research the beliefs or activities of key political groups, such as politicians or civil servants, that are too large in number to interview individually. Examples of such generalizations include the work of Joel Aberbach and his colleagues, who sampled both politicians and top civil servants in the United States and generalized their findings to the wider population of Congress members and government administrators. Characteristics, traits, and actions found through the sample group were inferred to exist also in the population from which the sample was drawn, and the authors made general conclusions concerning the wider political elite (Aberbach, Putnam, and Rockman 1981; Aberbach and Rockman 2000).
4) To Help Reconstruct an Event or Set of Events
Finally, the usage that is arguably most relevant to process tracing entails conducting elite interviews to establish the decisions and actions that lay behind an event or series of events. Through direct and focused questioning, researchers can reconstruct political episodes on the basis of the respondents' testimonies, stitching together various accounts to form a broader picture of a complex phenomenon. Elite interviews can shed light on the hidden elements of political action that are not clear from an analysis of political outcomes or other primary sources. By interviewing key participants in the political process, analysts can gain data about the political debates and deliberations that preceded decision making and action taking, and supplement official accounts with first-hand testimony.
Elite interviews are thus a potent source of data for political researchers, and can contribute to empirical research in important ways. To a great extent, researchers can pursue each of the uses listed above using other forms of data collection, but elite interviews have specific advantages that can compensate for weaknesses in alternative techniques.
One of the strongest advantages of elite interviews is that researchers can interview first-hand participants of the processes they are investigating and obtain accounts from direct witnesses to the events in question. While documents and other sources may provide detailed accounts, there is often no substitute for talking directly with those involved and gaining insights from key participants. The nature of interviewing also allows interviewers to probe their subjects, thus moving beyond written accounts that may often only represent an official version of events to gather information about the underlying context and build up to the actions that took place.6
While this paper is not directly concerned with interview techniques, the above point suggests that a form of semi-structured interview techniques would be appropriate for process tracing related interviews. Set questions can ensure the interview is focused on the theoretical concerns of the research project, and the ability to ask probing follow-up questions will be necessary to ensure as much relevant information as possible is gained from the respondent. For more on the nature of semi-structured interviewing, see Hammer and Wildavsky (1989).
Other advantages contrast the particular weaknesses of archival documents, as interviews can compensate for both the lack and limitations of documentary evidence (Davies 2001). Important political processes often lack an accompanying body of documentation, for a range of reasons. Not all participants document important processes with written material, as they either feel their actions are not important enough to merit recording them, or instead feel they are too sensitive to document in written form. Over time important documents may also be lost, as they are unintentionally discarded or as archives are destroyed. Governmental secrecy rules can also ensure that key documents are withheld from public analysis (Davies 2001; George and Bennett 2005, 99).
Furthermore, even when relevant archival material is available, documents can still entail some inherent weaknesses of which the researcher must remain aware. In particular, documents can often be incomplete and present a misleading account. By presenting the official version of events, documents often conceal the informal processes and considerations that preceded decision making (George and Bennett 2005, 103). They may also imply consensus and agreement with a decision, when in reality disagreements may have been widespread and that other, undocumented, decisions may have been considered extensively (Davies 2001). Finally, in cases where there is an abundance of documentation, the wealth of primary data can become a liability, as the difficulty of sorting through the evidence and prioritizing the most important documents increases. Interviewees can often help the researcher cut through this surplus of data since respondents can distinguish the most significant or accurate documents from those that may be marginal or may present a selective account of events (Seldon and Pappworth 1983).
When considered together, these observations highlight the particular role that elite interviewing can have in facilitating the process tracing method and in providing the kind of data that can be critical in uncovering the causal processes and mechanisms that are central to comprehensive causal explanations. Process tracing requires data collection on key political decision-making and activity, often at the highest political level, and elite interviews will frequently be a critical strategy for obtaining this required information. While their corroborative function should not be under-played, it is their additive role that is most relevant in association with process tracing. Such interviews can allow the researcher to collect first-hand testimony from direct participants and witnesses regarding critical events and processes. They provide the researcher with a means to probe beyond official accounts and narratives and ask theoretically guided questions about issues that are highly specific to the research objectives. When interviewees have been significant players, when their memories are strong, and when they are willing to disclose their knowledge of events in an impartial manner, elite interviews will arguably be the most important instrument in the process tracer's data collection toolkit.
Yet elite interviews are not a panacea, and have their own limitations and weaknesses as well as strengths. As George and Bennett (2005, 99) highlight in relation to evidence more generally, researchers need to critically assess and weigh the value of collected data, recognizing the pitfalls that may limit its usefulness along with its benefits. While in many circumstances interviews can compensate for the distortions that exist in written sources, it is also sometimes the case that interviewees misrepresent their own positions in ways that raise questions over the reliability of their statements. In particular, politicians may attempt to slant their accounts and inflate or minimize their own role in an event or process depending on whether there is political capital to be gained or lost (Kramer 1990).7
For a more comprehensive survey of the limitations of elite interviewing, see Seldon and Pappworth (1983, Chapter 2).
As a result of these potential limitations, it becomes incumbent on the researcher to be aware of the possible drawbacks, and to critically evaluate any interviews that are carried out. In order to evaluate such interviews, however, it is necessary to have criteria against which such interview data can be assessed. George and Bennett (2005, 99) themselves provide one framework for assessing the evidentiary value of primary sources and counsel that researchers should ask four questions when considering particular documents or interviews: (1) who is speaking; (2) to whom are they speaking; (3) for what purpose are they speaking; (4) under what circumstances? Another set of evaluation principles has been proposed by Davies (2001), who suggests three more specific criteria that need to be met before elite interviews can be considered reliable. These are: (1) that the information obtained should be from a first-hand witness, and not based on hearsay; (2) that the level of access of the interviewee to the events in question should be known, with senior-level elites to be viewed as more reliable;8
Note that this is in contrast to a point raised by George and Bennett (2005, 103)—that often lower-level officials may be better interview sources given their day-to-day involvement with political processes.
Ultimately, the final criteria used should be a function of the research objectives, and researchers will have to decide which criteria are the most relevant given the type of person they are interviewing and the kind of issues that are under discussion. What is generally applicable, however, is that researchers should consider these issues consciously and clearly specify the criteria being used. Furthermore, as Davies (2001) counsels, even when researchers consider the interviewees reliable, they should consult multiple sources for all significant data points and report levels of uncertainty concerning the reliability and validity of such data.
Sampling: Probability and Non-probability Approaches
Elite interviewing is thus well-suited to the process tracing method, and although the technique should be used cautiously, its particular advantages can enhance the search for causal mechanisms. It is not only that case, however, that the potential of elite interviewing has implications for the way process tracing should be carried out. It is also the case that the nature of the process tracing method itself has implications for the way researchers should pursue elite interviews. In particular, using elite interviews for data collection in a process tracing study has implications for the manner in which elite interview subjects should be selected, and researchers must therefore take sampling considerations into account when pursuing this form of research design.
In much of the work that involves elite interviews, elites are selected through random sampling so that findings from the sample can be generalized to the wider population. As Berry (2002) observes, most elite interviewing depends on a few well-established templates, especially ones where elites are chosen at random and are then subjected to a common interview protocol. For research studies that seek to make causal inferences about large groups of elites, such as politicians and civil servants, limited time and resources tend to preclude the possibility that researchers can conduct detailed interviews with all the subjects of interest. Instead, they choose a sample of the group to make studies feasible and retain the ability to make statements that concern the wider group. Through probability sampling, the rules of selection ensure that the researcher will be able to estimate the relationship of the sample to the population of subjects from which it was drawn. As Kidder, Judd, and Smith (1991, 132) state, certain sampling strategies “can guarantee that if we were to repeat a study on a number of different samples selected from a given population, our findings would not differ from the true population figures by more than a specified amount in more than a specified proportion of the samples.”
The key to achieving this link between the sample and the population is to draw a representative sample from the wider population of subjects by using random selection, where the probability that each unit of the population will be selected for the sample can be specified. The researcher can then estimate the extent to which findings based on the sample are likely to differ from findings that would have been obtained from an analysis of the full population (Kidder, Judd, and Smith 1991, 132). Random sampling, however, does not equate to a disorganized or ill-considered selection of the sample, but rather to a selection of the sample according to a set of rules that ensures each unit of the population has a known probability of being selected. Examples of probability sampling include simple random sampling, where all members of the population are listed and subjects are chosen from that list in random order. Stratified random sampling is an alternative method of probability sampling, where random selection is balanced with the intentional manipulation of the population list to ensure that certain important categories of subjects are not excluded from the sample through chance. For example, a population of voters in the U.S. may be divided into two strata, one including Democrats and one including Republicans, and half of the sample is then selected at random from each sub-group.9
See the discussion of sampling in Epstein and King (2002).
When the goal of a study is to generalize from a sample to the wider group from which the sample is drawn, some form of probability sampling is essential for the robustness of such generalizations. Without the randomness that probability sampling entails, it would be impossible to be certain that the sample was not selected in a biased manner, and that the selection rule is not in some way related to the variables being used in the study. As Epstein and King (2002) state, “random selection is the only selection mechanism in large-n studies that automatically guarantees the absence of selection bias.”
Sampling can also take alternative forms, however, and non-probability sampling techniques involve researchers drawing samples from a larger population without requiring random selection. The distinguishing character of non-probability sampling is that subjective judgements play a role in the selection of the sample because the researcher decides which units of the population to include (Henry 1990). While this means the researcher has greater control of the selection process, the trade-off is that such sampling techniques severely limit the potential to generalize from the findings of the sample to the wider population. Given the role that subjective selection plays in drawing the sample, selection bias can easily be introduced, compromising the possibility of arriving at robust findings and generalizations. For example, a researcher may tend toward selecting a certain type of respondent and certain sectors of the population can be systematically omitted from the sample. Similarly, there is no clear way for the researcher to estimate the extent to which the sample resembles the population of interest once the sample is drawn. As a result, many argue that non-probability sampling is inherently inferior to probability sampling, and that it should only be used under limited circumstances, such as when resources are limited (see Table 2). Henry (1990, 32), for example, states unequivocally that “only in the cases where probability samples cannot be used are non-probability samples viable.”
Advantages and Disadvantages of Probability and Non-probability Sampling
Yet is it also the case that non-probability sampling has its own advantages, and that the strength of each approach depends heavily on the aim of the particular study. When the aim is to generalize about a large group of elites, probability sampling has clear advantages that cannot be matched without random selection. However, if the aim is not to make generalizations from the sample to the population, as with the process tracing method, then non-probability sampling can be assessed using a separate set of criteria, and considerations of the relationship between the sample and population become much less important (Kidder, Judd, and Smith 1991).
This is precisely the situation that exists in the context of process tracing, when such generalizations are not an aim, and the goal is rather to obtain information about highly specific events and processes. To recap, the aim of process tracing is to uncover the causal mechanisms that link independent and dependent variables to one another in a particular context. Often, the causal processes of interest are very specific episodes of decision-making at the elite level, where a limited set of actors are involved in deliberations, decisions, and actions regarding a particular political outcome. In some cases, the number of actors will be so small that sampling of any kind will not be necessary, and it may be possible to interview the total population of relevant elites. More often than not, however, many actors will work on a particular political process, often in different contexts and at different levels of seniority. Limitations of time and resources will make it difficult, or impossible, to interview all those who played a role in the process, and it will thus be necessary to draw some type of sample.
The goal with the process tracing method, however, will not be to draw a representative sample in order to use interviews to make generalizations about the characteristics, beliefs, or actions of the full population of relevant actors, but rather to obtain the testimony of individuals who were most closely involved in the process of interest. While the sample should be representative of the wider population to the extent that it does not systematically exclude a set of actors who played an important role, it does not need to be drawn from the population through random selection. This does not mean that process tracing is not concerned with generalization of any type—on the contrary, once an underlying causal mechanism is established, the goal will often be to generalize about this mechanism to other contexts. Rather, the method requires focused attention to very specific actors, events, and processes to determine the mechanisms at work to begin with.
Thus, when researchers use process tracing, the key issues to consider when drawing the sample are to ensure that the most important and influential actors are included, and that testimony concerning the key process is collected from the central players involved. In such circumstances, random sampling would be a hindrance rather than a help, as the most important actors of interest may be excluded by chance. Instead, the principal goal with process tracing is to reduce randomness as much as possible, which in turn requires that researchers establish the identities of the most important actors and approach them directly for interviews.
The suitability of any research tool is a function of the specific research goals being pursued. In the case of sampling procedures for elite interviews, the choice of either probability or non-probability approaches will not depend only on practical issues such as available time and resources, but also on methodological considerations derived from the research objectives. While many argue that probability sampling is inherently superior to non-probability sampling, that position may only hold if the aim is to extrapolate broader generalizations from the sample to a wider range of respondents. In much qualitative research, however, including process tracing, generalizing from a sample to a wider population is not the aim, and non-probability sampling can be judged on the basis of a different set of criteria. These considerations also hold for other forms of methodological practice and the standards we use to evaluate them. In a recent comparison of quantitative and qualitative traditions in political research, Mahoney and Goertz (2006, 246) highlighted that the different norms and assumptions underlying the two traditions will suggest different methodological practices, and that “what is good advice and good practice in statistical research might be bad advice and bad practice in qualitative research and vice versa.” As discussed above, methodological choices involve trade-offs, and how we evaluate those trade-offs and develop research designs on the basis of those evaluations will depend strongly on the assumptions we hold and the research aims we are seeking to pursue.
Types of Non-probability Sampling
The considerations above suggest that when using the process tracing method there are also strong imperatives for using non-probability sampling approaches with elite interviewing. This in turn raises the issue of which form of non-probability sampling is most appropriate when applying the process tracing method. As with probability sampling, there are a number of types of non-probability sampling, each with varying rules for selecting the final sample.
Convenience Sampling (also known as availability or accidental sampling)
This form of sampling involves the researcher selecting the most readily available respondents, regardless of characteristics, until the required sample size has been achieved. As its title suggests, its primary advantage lies in its convenience—there are no strict selection rules and the sample can be drawn in whatever way is easiest for the researcher. Yet its primary drawback derives from exactly the same feature—without any selection rules, there is no way to tell what wider population the sample group represents or how the sample might differ from other potential samples. An example of this form of sampling is the use of the students in a professor's class as a sample of the wider student body—while interviews with members of the class may reveal a range of student opinions, without knowledge of the relationship of the sample group to the wider student population the scope for robust generalizations will be severely limited.
Quota Sampling
Another method overcomes some of convenience sampling's problems by seeking to ensure that certain characteristics are present in the sample in proportion to their distribution in the wider population. For example, a researcher might ensure that the chosen sample includes ethnic groups in the same proportion as they exist in the wider population so that no groups of interest are excluded through chance. This method provides the researcher with a greater degree of certainty regarding the sample's makeup and its relationship to the broader population of interest.
As with most sampling methods, however, there are certain drawbacks to this selection method. First, the researcher must know the population's characteristics beforehand, which is not always possible. Second, while the sample is representative of the population on the characteristics of interest, there is no way for the researcher to be sure that it is also representative of other characteristics that may be important, for example class and ethnicity. Even though certain proportions are guaranteed, the selection rules or the interviewers' inadvertent biases may lead to the over-representation of subjects with other particular characteristics (Kidder, Judd, and Smith 1991).
Purposive Sampling
Purposive sampling is a selection method where the study's purpose and the researcher's knowledge of the population guide the process. If the study entails interviewing a pre-defined and visible set of actors, the researcher may be in a position to identify the particular respondents of interest and sample those deemed most appropriate. As Kidder et al. (1991) suggest, the basic assumption is that with good judgement and an appropriate strategy, researchers can select the cases to be included and thus develop samples that suit their needs. An example here is the strategy of market researchers who stand on busy streets and seek to question passers-by—often they have a specific target group in mind and will purposefully try to interview people of a certain age, race, or gender, ignoring those who do not fit the criteria.
Snowball/Chain-referral Sampling
One of the most well-known forms of non-probability sampling is the snowball sampling method, which is particularly suitable when the population of interest is not fully visible and when compiling a list of the population poses difficulties for the researcher (Babbie 1995). This approach is commonly used in sociological studies on hidden populations that may be involved in sensitive issues or illegal activities, such as drug use and prostitution (Biernacki and Waldord 1981). Yet the method is also used in political science and the study of elites, where the most influential political actors are not always those whose identities are publicly known.
The snowball, or chain-referral, sampling method involves identifying an initial set of relevant respondents, and then requesting that they suggest other potential subjects who share similar characteristics or who have relevance in some way to the object of study. The researcher then interviews the second set of subjects, and also requests that they supply names of other potential interview subjects. The process continues until the researcher feels the sample is large enough for the purposes of the study, or until respondents begin repeating names to the extent that further rounds of nominations are unlikely to yield significant new information.
As with random sampling, the snowballing method is not as uncontrolled as its name might suggest. The researcher is heavily involved in developing and managing the initiation and progress of the sample, and seeks to ensure that the chain of referrals remains within boundaries that are relevant to the study at all times (Biernacki and Waldord 1981). One of the dangers with snowball sampling is that respondents often suggest others who share similar characteristics, or the same outlook (Seldon and Pappworth 1983), so the researcher needs to ensure that the initial set of respondents is sufficiently diverse so that the sample is not skewed excessively in any one particular direction.
Sampling for Process Tracing
As discussed above, non-probability sampling is particularly well-suited to the process tracing method, where the aim is not to generalize to a wider population from a smaller sample. Yet the question, therefore, is whether there is a particular form of non-probability sampling that is best for process tracing projects. Answering this question requires considering the kind of criteria that researchers may wish to use to select their respondents, which in turn may vary from project to project. This section discusses two criteria in particular that will have relevance for many process tracing studies, and explores which forms of non-probability sampling might be suitable to apply once researchers select their criteria of choice.
In the first approach, researchers use positional criteria to identify desirable respondents; that is, analysts specify a set of positions, or occupations, of key elites that are the focus of the study. The central characteristic of this approach is that researchers have sufficient knowledge of their area of interest, and sufficient knowledge of the political structures of relevance, to identify ex ante the type of actor that will be of interest. For example, in a study that seeks to examine the involvement of senior civil servants in specific governmental processes, it may be possible to specify in advance the type of respondent that will be required, and the positional criterion might simply be that all respondents are top-level bureaucrats in a particular government department. In some cases, these individuals may be publicly known or easily identifiable, and a form of purposive sampling will be most appropriate for selecting the specific subjects for interview. If information on the identity of top-level bureaucrats is lacking, however, researchers may need to initiate a snowball sampling process to identify the key individuals who hold these positions, starting with the known civil servants and using a chain of referrals to identify the unknown actors who occupy the specific positions of interest.10
For examples of the use of positional criteria, see Denitch (1972) and Rivera, Kozyreva, and Sarovskii (2002).
An alternative approach to positional criteria is one that references reputational criteria in the selection of respondents. This approach does not involve defining the desired set of respondents according to the particular positions they hold, but rather according to the extent to which they are deemed influential in a particular political arena by their own peers. Instead of having a fixed category of actors, researchers are open to including respondents from any political arena or position who may have played an important role in the political process of interest. With this method, the snowball sampling method is particularly appropriate, as it is impossible for the researcher to know, in advance of consulting an initial set of elites, which individuals will be viewed as having the greatest influence in the area of interest. Consequently, the most appropriate method of establishing a sample based on reputational criteria is to identify an initial subset of respondents based on their known relevance to the research topic, and then initiate a chain-referral process whereby these respondents are then asked to provide a list of people they feel are influential in the same field. This procedure is then repeated with the new nominees once initial respondents propose fresh names. Farquharson (2005) has argued that this reputational snowball method is successful at identifying influential actors who might otherwise have been ignored, as elites can often suggest influential players who researchers may not initially have presumed relevant to the study. Similarly, this method has advantages in assessing the level of influence of key elites, as the number of nominations that each person receives provides an indication of their stature within a policy or issue arena. When individuals receive several nominations from peers, it suggests they may be particularly influential in their fields, and thus be critical interviewee subjects.
Deciding between these alternative approaches to sampling elites for interviews will often be a function of the research goals, and in some cases it may be clearly preferable to choose one approach over another. However, when using elite interviews as part of a process tracing method, this article contends that a combination of the two approaches is an optimal method. As process tracing involves obtaining information about a very specific set of events or processes, it will usually be the case that it is possible for the researcher to identify a key set of relevant actors according to their positions, for example, by virtue of their membership in a government committee, or their holding a senior cabinet position in a relevant department. Process tracing will thus generally involve some form of purposive sampling, especially in the early stages of a project, as researchers select respondents according to the position they have held and their known involvement in a political process.
However, as process tracing seeks to uncover as much information as possible about political processes, and establish the fine-grained associations that link independent and dependent variables, it is imperative that researchers strive to obtain all the information that could contribute to reconstructing the processes of interest. They need to be open to the possibility that the process in question was influenced by unknown or unexpected political actors who may have held political positions that were initially not considered relevant or important. In order to avoid missing such individuals, and thus omitting key political figures from the sample of interview respondents, researchers need to select respondents according to reputational as well as positional criteria. Initiating a snowball sample with known participants can identify not only the individuals that occupy key positions that researchers have identified in advance as being important, but can also shine light on the identity of important actors who may have played a role from unlikely or unpredictable positions. The sampling method can thus contribute to the process tracing method, as information is collected from a complete set of relevant players and no significant actors are omitted from the study as a result of the selection rules.
Conclusion
Process tracing provides a crucial method for the analysis of complex political phenomena, and rightly places an emphasis on uncovering the causal mechanisms that connect independent and dependent variables. By prioritizing fine-grained research that seeks to identify the critical steps and stages of political processes, the process tracing method allows scholars to both generate and assess critical data, and thus enhance their efforts both at theory development and at theory testing. Yet discussions of process tracing to date have been somewhat selective in their treatment of the kinds of data collection strategies that are appropriate for this method. In particular, there has been a tendency to emphasize the historical method and archival research over other forms of data collection. This article has argued that elite interviewing should also be viewed as an appropriate, and at times critical, form of data collection in the context of a process tracing project. Although interviewing produces data that face many of the same challenges of evaluation and reliability as other forms of data, it enables researchers to move beyond written sources, and ask probing, theoretically-driven questions of key participants in the events and processes of interest. By allowing researchers to communicate with key players directly, and by enabling them to frame that communication according to theoretical interests and priorities, interviews can facilitate the collection of data that is highly relevant and specific to the research objectives being pursued.
This article also highlights the implications that the process tracing method has for approaching the elite interview process, and especially for selecting the interview sample. Political scientists regularly suggest that the ideal selection method is probability sampling, and that the advantages of non-probability sampling are limited. Yet such suggestions rest on the assumption that the sample will be used as a source from which to generalize findings to the population, and this is not the case when researchers pursue process tracing. In this situation, the aim is profoundly different, as scholars seek highly specific information about highly specific events and processes. Once the researcher reconstructs such processes and makes findings about the relationships between the variables and causal mechanism, it may then be desirable to generalize those processes to other cases. But in reconstructing that initial process, the aim of collecting interview data is to obtain specific data to fill in a specific gap, and not to abstract to a wider population of interviewees.
In order to pursue non-probability sampling, however, researchers need to consciously consider the criteria they will use to select their interview respondents. Two approaches have been discussed, based respectively on positional and reputational criteria, and both have relevance for process tracers. It is this article's contention that neither approach should be pursued in isolation of the other, and that the ideal strategy when using the process tracing method is to combine the positional and reputational approaches when sampling potential interview subjects. In this way, researchers can not only interview a set of political actors that their research objectives suggest will be highly relevant, but will also open their research to the possible inclusion of other influential players who may not be obvious ex ante—that is, political actors who will only be identified by their peers through a process of sampling based on reputational criteria and snowball sampling. Whichever method qualitative researchers ultimately select, it is imperative that they be aware of both the advantages and disadvantages of elite interviewing, especially when used in conjunction with process tracing, and that the decision to use this form of data collection and the manner in which they collect interview respondents are based on informed methodological reasoning.
Author Bio
Oisín Tansey is a lecturer in international relations at the University of Reading. His research focuses on state-building, democratization, and qualitative methodology.
