1. “Case Studies” in the Social Sciences: Definitional Issues

Many different ‘styles’ or modes of scientific reasoning can be found among the social sciences, including statistical, experimental, taxonomic, even mathematical, but what may be particularly symptomatic of the social sciences is a strong tradition of case study work. Case studies appear regularly in sociology, political science, and management fields based on sociology, anthropology, and psychology, but of course they are also found in medical sciences such as neurology and may be understood to have cousins in the field studies of ecology, the model organisms of biology, and the exemplary narratives of the humanities (see Creager, Lunbeck, and Wise Reference Creager, Lunbeck and Wise2007).

Case studies as an epistemic genre—as a way of doing science—did not appear in the well-known list of such genres advanced by Alistair Crombie (Reference Crombie1988) and Ian Hacking (Reference Hacking and McMullin1992b) but were added by a seminal paper of John Forrester (Reference Flyvbjerg and Sampson1996): “If p, Then What? Thinking in Cases.” His paper did not make any marked distinction between ‘a case’, one of a series of roughly similar objects, and ‘a case study’, an in-depth study of a single whole.Footnote ¹ And though his paper raised an important question, exactly what reasoning in (‘thinking in’ or perhaps ‘with’) cases consists of remains difficult to characterize.

The first problem lies in providing a good definition of a case study in social scientific fields.Footnote ² What follows is not a practitioner definition but draws on the practitioner literatures to fit their sense of what is involved. A case study is defined here in terms of the following characteristics:

1. 1. A case study investigates a bounded whole object of analysis. The unit of analysis points to a level of wholeness, not singleness in the material (i.e., not a single but a whole industry, firm, town, or ritual); within that whole there may be many single elements (i.e., many firms in an industry, people in the town, etc.).

2. 2. Case study research maintains a considerable degree of open-endedness, and the boundary between subject of analysis and context is not clear at the start of the research and may remain fluid during the study. The topic or problematic of the research question is chosen in broad terms, but the extent of the work undertaken, what studied, and the divide into content and environment or context emerge only during the process of research.

3. 3. A case study involves researching directly a ‘real-life’ whole, which creates a considerable depth of engagement with the subject and dense evidential materials across a range of aspects of the topic. This contrasts with the relatively thinner (but internally comparable) materials produced by the sole use of surveys or statistical work or the more artificial materials produced from the isolated and controlled world of the laboratory experiment or modeling of various kinds.

4. 4. Many potential research methods may be used within the case study. A case study in the social sciences may involve many different techniques: survey work, statistical work, ethnographic work, and historical work.

5. 5. The outcome is a complex, often narrated, account that typically contains some of the raw evidence as well as its analysis and that ties together the many different bits of evidence in the study.

The second problem lies in figuring out how such a research genre—case studies—can be used to develop scientific knowledge and so asking how it is to be defended. While there are few analyses from philosophers of science, practitioners in the social sciences have provided a considerable, and methodologically sophisticated, literature analyzing case studies as a mode of reasoning in their fields.Footnote ³ This literature—both critical and positive, defensive and supportive—is written in a language and offered in forms that are grist for the mill of philosophy of science (see Ragin and Becker Reference Ragin and Becker1992; King, Keohane, and Verba Reference King, Keohane and Verba1994; George and Bennett Reference George and Bennett2005; Byrne and Ragin Reference Byrne and Ragin2009).

2. In Defense of Case Studies as an Epistemic Genre

The defenses social scientists offer for case studies are conducted by comparing them with, and understanding them as a version of, either laboratory experiments or statistics.Footnote ⁴

On the former comparison, case studies are presented as a form of experiment—not a controlled laboratory experiment but an experiment in the world. Rarely is this defense constructed in terms of a ‘natural experiment’, though some studies clearly could fall into this class. For example, the sociologist Robert Merton (Reference Merton, Fiske and Curtis1946) introduces his case study of a radio fund-raiser for war bonds during World War II in just such terms, for this was a 1-day event in which he claimed the cultural and political context was rather stable and so the stimulus-response of the real-life event could be treated as if it were a controlled experiment. More usually, researching the complex whole does not lend itself to such a defense but to an alternative argument that because the study goes into such detail of the whole event (point 1 in the definition), all the relevant factors that affect the event can be taken into account. They are not removed, shielded, or held ceteris paribus but are present, and because their variations become known during the case study, the critical variables can be parsed out and their influence can be assessed.Footnote ⁵ For example, Robert Burgelman's (Reference Burgelman1994) study of a computer chip–making company's decision to alter its product range can be interpreted in this light. The study looked at a range of causal factors in different sections of the firm and showed how the product switch emerged not from a top-down strategic decision but from a series of individual or group moves in the industrial (competitive) context of the day. In this ‘case study as an uncontrolled experiment’ defense, N = 1: there is only one whole event, and so one thing observed, but it is a multiply dimensioned observation on the complex whole over time, not a single-dimensioned observation on a single element within that whole.

On the latter comparison with statistics, case studies are presented by social scientists not as a form of statistical study but as having some of the equivalent features. Most especially, N = many: but instead of many comparable, numerically expressed, observations on a rather small number of aspects of the object as in most statistical work (e.g., on the size of firms, turnover, and profit of 100 firms, so N = 100), there are multiple observations on very many different elements and aspects of the whole object in the case study (as in the Burgelman case, all the information that can be gathered in any form on one firm and a major strategic decision). The variety and multiply dimensioned nature of the observations create a difficulty: because the pieces of information are complementary rather than comparable, there is no route to statistical validation of the case study. But Donald Campbell (Reference Campbell1975), a statistician and previously a fierce critic of case studies, found a way to turn this evidential richness to advantage in the context of justification. He was converted by examining a work in psychoanalytic anthropology of the Yurok Indians of Northern California by Erik Erikson in the 1940s. From this, Campbell argued that case studies had considerable power to “infirm” (weaken) as opposed to “confirm” (strengthen) degrees of acceptance for a theory under consideration. This power to infirm came from the multiple implications of the theory in such a case study, each of which could be infirmed by one bit of evidence.

Campbell (Reference Campbell1975) portrayed his new understanding of case studies in terms of the concept of “degrees of freedom,” an important notion in statistical reasoning: namely, the more information or data points relative to the parameters of the theory, the more degrees of freedom there are in fitting a hypothesized relationship to the many data points. The more degrees of freedom, the more possible relationships can be fitted, and so, when one statistical relationship coalesces, a certain confidence is gained in that relation. Carrying the idea over, case study research produces observations on many different things and creates the possibility of multiple interpretations, but few of those interpretations will likely be consistent with all the details of evidence: most accounts would be infirmed by some of the evidence. But if all the bits of information were to be found consistent with one interpretation, more confidence would adhere to that account.

So, in contrast to the two critiques that a case study provides only one uncontrolled experiment, or one statistical observation, the social scientists’ defense of the virtues and validity of case studies rests on the evidential density (point 3 in the definition) on a whole unit (point 1), which enables a full analysis of all the influencing factors and uses the multiplicity of different observations to justify its account.

Two general points arise from this analysis of these practitioner defenses of case studies. The first point is, why should the judgment of the worth of case studies be made by comparison with these two, or any other, epistemic genres? The answer here seems to have much to do with the fact that statistical and experimental work have accepted status among philosophers of science, a status that is reflected in the social scientists’ own focus for their defensive arguments. It is perhaps worth remembering Ian Hacking's (Reference Hacking1992a, Reference Hacking and McMullin1992b) writings about epistemic genres in which he argued that there was no appeal from within any of these genres to any higher, broader, philosophical argument that validated all the different ways of doing science as versions of one general method of science. It was a matter of history that each epistemic genre developed its own generic way of finding and validating knowledge, so that work within that genre came to be judged within that epistemic genre and by its community of practitioners, not according to the rules, or in the terms, of any other genre. This was not a radical relativism; rather there are many valid ways of doing science, and while comparisons may be insightful, it does not make sense to judge, for example, the epistemic genre of taxonomy and classification by the experimental mode or vice versa.

The second point is that the main thrust of these comparisons and arguments by social scientists is typically concerned with the context of justification, about the difficulty of testing specific theories or causal hypotheses within case studies and the difficulty of providing any valid justification for such findings. Yet, as philosophers of science well know, scientists using other epistemic genres—such as experiments or modeling—also have difficulties both in establishing the internal validity of their results and in grounding the transportability of those findings beyond the specific example. But this is not the only space within which social scientists argue for the usefulness of case studies, nor should it be for a philosopher of science.

3. Case Studies: Vehicles of Discovery?

Social scientists, at various times and in various fields, have argued that case studies are not primarily vehicles for theory testing, where this is usually taken to mean testing hypothesized relationships between variables. And this is not because case studies are approached theory free. Rather, so it is claimed, case studies are research in the context and service of discovery, not justification: they are for the formation of evidence-based concepts, for the development of measurement structures, the places where types are defined and kinds isolated, where phenomena might be revealed and theory developed (see Eisenhardt Reference Eisenhardt1989). Here I focus on the last of these elements: the role of case studies in revealing phenomena and developing accounts of them, and again I draw on the characteristics of case studies given in the definition earlier.

One of the oft-quoted practitioner texts, by Robert Yin (Reference Yin1994), argues that one of the true benefits of the case study method is that objects are studied in their context. This can be seen as a virtue in comparison with other genres in which the objects of case study research are divorced from context—as in the abstract modeling of economics, or have their context heavily controlled—as in the laboratory experiments of psychology and sociology. Yet, Yin goes on to suggest, it is often difficult for the researcher to draw lines between the object of interest and its context because of the open-endedness of the research question and because the object emerges to be fully distinguished from its context only during the course of the research (point 2 in the definition). While this might be reassuring to the young social science practitioner, it does not sound like an epistemic virtue to a philosopher of science. Yet his account points us to an important role case studies may play in the context of discovery. Surely for any science, locating a phenomenon involves being able to distinguish that object from its background and filter out the context in order to study the object more clearly. It is not likely that a social phenomenon will emerge as a distinct self-contained object like an egg from a chicken. Rather, distinguishing a phenomenon from its context involves identifying an element in the social field that is of particular interest, defining the important aspects of it, describing its characteristics, and suggesting its function or importance in the social field. Discovering new phenomena in the social field will involve both description and conceptualization.

A good example can be found in the case study that defined the characteristics of ‘street corner society’ and coined the label for that phenomenon. Both as a conceptual notion for a potentially widespread phenomenon and as a specific example, this was ‘discovered’ in the book of that name by William Foote Whyte (Reference Whyte1943).Footnote ⁶ There was no one moment in the study when one could say that the phenomenon of street corner society had been identified; rather it emerged as the outcome of a case study of a group of young men living in an urban US (North Boston) community. The detailed case work of this (largely) ethnographic study was originally undertaken to study the interactions within a small group. But the open-endedness of the case study enabled Whyte to branch out into new areas from that initial base, that is, from interactions within the one group into relationships between such groups and with the wider society in the form of the police and political activity. The lack of boundary between subject of interest and context freed Whyte to develop an account of a slum community as an organized society that failed “to mesh with the structure of the society around it” (273). Against a theoretical background in sociology of the day in which slum communities were understood not to be communities but rather to be a disorganized set of inhabitants lacking social organization, the study revealed something very different.

Another ‘discovery’ example, from the field of industrial economics, is found in Charles Baden-Fuller's (Reference Baden-Fuller1989) case study on the steel castings industry in the United Kingdom. Here, initially, context seemed all-important: it was an industry in terminal decline, a context that might seem to swamp any other considerations, and the focus of the study was the performance of a government scheme to help save the industry. This too was a study undertaken against the backdrop of a strong theoretical assumption, namely, the economists’ absolute belief that firms left an industry in the order of their profitability, with the least profitable exiting first (that exit was ‘efficient’), though there were few empirical studies of the matter. Although this assumption was not the focus of the research, when the case study found that the order of exit was contrary to that assumption, it documented a phenomenon of potentially much wider relevance. Despite the general decline of the industry, from which one might have expected all firms to close, some stayed open much longer than seemed reasonable given their profitability. The order of exit was found to depend on the size and diversity of the firm as well as on its resources (for it costs money to close) and its ownership structure (for single-unit owner-managers tended to be less willing to close their foundries than large diversified, non-owner-managed, companies). Here we find a mix of methods: statistical, mathematical modeling, and ethnographic and the development of middle-level theories to explore the various reasons why this unexpected phenomenon occurred.

The scientific community responses in both these examples—of street corner society and inefficient industry exit—were similar. For them, the case studies revealed phenomena that were unexpected against the standard assumptions and beliefs of the time and produced evidence that could not be explained within those original terms but could be explained in other terms. (And in both cases, those phenomena were not even the original focus of interest.) So these cases were not understood as examples of something already known and established.Footnote ⁷ Nor were they seen as hypotheses-testing case studies, though both studies developed explanations of the social structures or causal elements in the process of exploring their phenomena. Rather, the community's response was to understand the phenomena revealed as potentially generic, likely to be found beyond the individual case studies. These two case studies proved a starting point for research activities each in their own social scientific communities: searching for other similar cases of the phenomena, exploring the nature of the phenomena they revealed, and testing out the accounts they gave of the phenomena across a range of other cases and sometimes with other methods.

4. Case Studies and Internal Validity

Just what makes these phenomena-revealing case accounts valid to the community of social scientists? That is, what are the positive criteria that operate as the flip side for the notion of infirming? I suggest that case study accounts rely on the same kind of criteria in producing their explanations as MacCormick (Reference MacCormick2005) argues establish the validity of narrative accounts in legal cases. That is, the accounts exhibit (i) consistency with all the evidence found, (ii) coherence within the account (the bits of evidence fit together), and (iii) credibility of the explanation in social scientific terms. As in legal cases, these criteria produce a form of internal validity in relation to the materials of the case study at hand, but not to ones beyond it.

Different disciplines using different methods in their case studies (point 4 in the definition) seek this internal consistency and create their accounts in different ways to attain coherence and social scientific credibility. Whyte (as other social anthropology ethnographers) created an internally valid account of Street Corner Society in a way that relied on dense reporting of detailed evidence in a narrative process that drew the reader into direct engagement with the evidence and then stood back a little to draw threads of the evidence together to provide an account and explanation of the phenomenon of that society. Other fields seek more complementarity between the case study account and their experiments, models, or theories. So Baden-Fuller's account of exit iterated in quite formal, overt ways between the data that he gathered on firms and his different technical methods of analysis to provide the separate elements of the account. These were then knitted together to provide an overall explanation of the pattern of firm exit and the reasons for that pattern. Both these accounts—and regardless of the particular methods used—showed how the phenomena were revealed and how the various bits of different evidence that supported the account were joined up and integrated into an account that explained that particular case.

This process of creating internally valid accounts (consistent, coherent, and credible) within the case study points to characteristic features of the way case study evidence is combined. Perhaps all this is obvious, but it is in marked contrast with how internal validity is established in statistical accounts. Recall that statistics defends itself with the argument that it works with lots of comparable, but independent, observations on a small number (relative to case study evidence) of characteristics on the object of interest. These are treated, cumulated, and analyzed in order to establish a pattern that has a very much smaller dimension of elements compared to the number of observations collected. These data processes are subject to rules laid down in statistical theory, which also provide the criteria for assessing the validity of the account given.

The coherence and credibility of case studies are built on something else: not on independence of comparable and similar kinds of observations but on the interdependence of different kinds and bits of evidence, on the fact that these different elements must be connected bits of evidence. Here is where wholeness (point 1 of the definition) matters, for the case study assumption is that life is inherently multidimensional and multiply connected, and that is why the case account or explanation has to integrate the evidence into an account of the whole, and not just concentrate on some bits of evidence that fit together and ignore outlier observations (as might happen in statistical work). Case study work and accounts concentrate on fitting together a jigsaw puzzle of pieces to reveal a picture of one particular social world in its natural environment rather than in a controlled environment (as in a laboratory experiment) or in searching for patterns that hold in similar elements across different environments (as in statistical work). Case study findings rely on connecting the different bits and pieces of evidence together to create a coherent picture in the account (point 5 in the definition), one that must be credible to the social scientific community.

These processes of integrating elements of evidence into an account are also found in theory-testing case studies: those that seek to account for a particular example of an already-known phenomenon and are concerned with the justification of that account. Thus, the ‘analytical narratives’ of social science history construct accounts sequentially in an iterative process between evidence and theories; they may end up throwing out some of the explanatory structure as a result but not the bits of evidence.Footnote ⁸ ‘Process tracing’, a generic term used in political science, seeks an internally valid account of the historical record of political events, which may involve counterfactual reasoning or even the use of some comparable cases, but again these are primarily justificatory.Footnote ⁹ Consistency is a strong criterion in these justificatory exercises, consistency to create a coherent account, an account that is not infirmed by any piece of the evidence.

However, the integration of bits of evidence with explanations, in ways that colligate the processes of observation, analysis, and inference typically found in phenomena-revealing social science case studies, seems to call for something more than an account that relates failure to infirm to justification or that merely points to the criteria for internal validity in a case study explanation. What kind of explanation do the joint criteria of consistency, coherence, and credibility produce? The answer may be phrased in terms of Carl Hempel's discussion of ‘why-questions’, a discussion he thought as relevant for the social as the natural sciences. He distinguished between ‘explanation-seeking why-questions’ and ‘reason-seeking, or epistemic why-questions’ (Reference Hempel1965, 333–35). The latter ask for grounds (epistemic reasons) for believing something is true; the former take something to be true and ask for the explanation (substantive reasons) why it is so. Case study research and their reports attack both questions together. Is it true that firms in the steel castings industry exited in an inefficient order, and if so, what were the reasons why this was so? Is it true that small corner gangs have a well-ordered social system, and if so, what are the features of that system that make it so? In both cases, despite standard theory, an unexpected phenomenon was observed to be true (according to social scientific research that looked to epistemic criteria of consistency and coherence between evidence and account). But in each case, the phenomenon was explained in ways that took care to offer coherent and credible explanations to their own social scientific community. These involved epistemic criteria relating to the details of explanation but also to broader criteria of subject matter coherence and credibility. Such phenomena-revealing case studies answer the epistemic why-question as to the existence of the phenomena at the same time as giving substantive explanations of those phenomena without separating out the two kinds of questions. Such a feature of answering both kinds of why-questions together may well be a feature of other epistemic genres when they operate in the domain of discovery, but perhaps the answers do not appear in quite such an integrated form.