‘What the devil is going on around here?’
Abraham KaplanFootnote 1
In recent times, the quest for scientific understanding in the social sciences has come to be equated with a causal understanding of the world. The task of description, by contrast, is often identified with idiographic storytelling or with messy, observational data that is insufficient to reach causal inference. Indeed, the term has come to be employed as a euphemism for a failed, or not yet proven, causal inference. Studies that do not engage causal or predictive questions, or do not do so successfully, are judged ‘merely’ descriptive. The implication is that description is a mundane task – necessary, to be sure, and occasionally quite complex, but of little intrinsic scientific value.Footnote 2
To be sure, research continues on a variety of well-established topics and advances in several areas are impressive.Footnote 3 Yet, despite these achievements, description has never been recognized as a methodological topic per se, as has causation. There are, for example, virtually no journals, articles, books or courses devoted to the subject. Instead, one finds a highly fragmented universe of work devoted to specific topics (often under special disciplinary nomenclatures) – concepts, measurement, descriptive statistics, inference from sample to population, ethnography, and so forth – with no recognition that these might add up to a coherent whole (despite their evident interconnections). Description awaits a convincing synthetic account. Indeed, there may be doubts about whether one can say anything at all that pertains to this broad and seemingly incoherent subject.
I begin by addressing the problem of definition. What does description mean, or what should it mean? Secondly, I offer a taxonomy of descriptive arguments intended to delineate the diversity contained within this genre of empirical analysis. Thirdly, I address the status of description in political science, charting its decline over the past century and arguing that a shift of emphasis from description to causation marks a dramatic reorientation of the field, outstripping parallel developments in other scientific disciplines. Fourthly, I make the case for approaching the task of description independently, rather than simply as a handmaiden of causal theories. Fifthly, I address the methodological difficulties of descriptive inference relative to causal inference. The article concludes with a brief summary of research areas within the rubric of description that seem especially fruitful.
Implicit in this article is the notion that description can be integrated into the social-scientific method (understood, for present purposes, as the attempt to study human action in a systematic, rigorous, empirical fashion).Footnote 4 Far from being critical of the scientific ideal, this article will take the view that political science has misunderstood that ideal.
Defining Description
Causality and description are intimately related; one cannot be understood without the other. Consequently, throughout this article I shall compare and contrast these two ways of comprehending the world. Unfortunately, although everyone agrees that description should be distinguished from causation, there is great disagreement over what it is, precisely, that distinguishes the two realms. Evidently, a good deal rides on the way in which this set of polar concepts is defined.Footnote 5 So the definitional ground will be covered slowly, and with great care.
I shall approach description and causation as types of arguments, i.e., assertions, models, propositions, statements, or theories about the world. Arguments may be advanced in prose and/or in formulas; the qualitative/quantitative distinction is not relevant here.
A descriptive argument describes some aspect of the world. In doing so, it aims to answer what questions (e.g., when, whom, out of what, in what manner) about a phenomenon or a set of phenomena. Descriptive arguments are about what is/was. For example: ‘Over the course of the past two centuries there have been three major waves of democratization.’
By contrast, causal arguments attempt to answer why questions. Specifically, they assert that one or more factors generate change in some outcome, or generated change on some particular outcome.Footnote 6 They imply a counterfactual.Footnote 7 For example: ‘The third wave of democratization was caused, in part, by the end of the Cold War.’Footnote 8 It will be seen that descriptive arguments are nested within causal arguments. Both X and Y are descriptive statements about the world upon which the causal argument rests.
This provides a rough-and-ready way of distinguishing between these dominant forms of argumentation. Below, I offer a taxonomy of descriptive arguments, which will serve to further refine and define our topic. But, before moving on, it is essential to point out a few contrasts with other definitions of ‘description’ that have been salient at one time or another in the social sciences.
An older use of descriptive – still common in ethics, philosophy of science, and decision theory – is as a synonym for non-evaluative, empirical, positive or constative.Footnote 9 Thus, arguments are sometimes classified as (a) descriptive or (b) normative. This is not the meaning adopted here. Indeed, many descriptions (according to my own definition of the term) have both evaluative and non-evaluative connotations.
It is sometimes averred that description is factual in nature, arising from directly observable features of a case, while causal arguments are inferential since they involve a counterfactual.Footnote 10 My wide-ranging definition of description encompasses statements about phenomena that are directly observable, allowing for arguments that arise unproblematically from the facts of a case. However, most descriptive arguments of concern to social science involve a degree of inference from what is known to what is unknown. The inferential quality of a description may derive from the latent (unmeasurable) quality of a concept, from problematic sources of data, from problematic measurement instruments or coding procedures, from missing data, from sample-to-population extrapolations, and so forth. There are many sources of uncertainty. The lack of a counterfactual does not mean that descriptive arguments in the social sciences involve fewer and less problematic assumptions about the world, a theme that we will shortly take up.
It is commonly said that causal arguments provide explanation while descriptive arguments provide understanding.Footnote 11 However, the distinction is sometimes difficult to sustain. Note that some causal arguments are more focused on causal effects than on causal mechanisms, and thus do not provide an explicit explanation for X's relationship to Y.Footnote 12 Likewise, many descriptive arguments (as I have defined the subject) may be viewed as performing an explanatory function.Footnote 13 Other descriptive arguments may provide little understanding. So, the distinction between explanation and understanding is not always helpful in clarifying the distinction between causation and description, as I have defined these terms.
Finally, we must wrestle with current usage of ‘descriptive’ and ‘causal’ within mainstream methodological circles. Here, the principal objective is to distinguish carefully between evidence that permits strong causal inference (without a lot of questionable assumptions about the data-generating process) and evidence that does not. The term causal is often reserved for the former (i.e., for experimental or quasi-experimental evidence), while everything else is relegated to the category descriptive. Andrew Gelman advises: ‘When describing comparisons and regressions, try to avoid “effect” and other causal terms (except in clearly causal scenarios) and instead write or speak in descriptive terms.’Footnote 14 Some researchers go further, regarding all evidence as descriptive so as to emphasize even more strongly the interpretive leap that causal inference requires.Footnote 15
The problem with this way of dividing up the lexical terrain is that it deprives us of a way to distinguish between different analytic goals – descriptive, causal or some other – or, worse, it assumes that all analysis aims ultimately for causal inference. Note that any attempt to appraise the truth-value of an empirical proposition must begin by resolving what the goals of that proposition are. If a truth-claim is unclear it is impossible to falsify. My perspective, therefore, is that the terms causal and descriptive should be understood as forms of argumentation, not (or only secondarily) as characterizations of the sort of evidence available for causal inference. A causal argument is an argument about a (putatively) causal relationship; it may or may not be true and may or may not be grounded in quasi-experimental evidence. Likewise, a descriptive argument is an argument about a descriptive relationship, which may or may not have causal implications.
In using these terms in this way I am keenly aware that every lexical choice elucidates some features of the world and obscures others. Depending upon one's methodological goals and priorities, one may wish to define description in a variety of ways. Ultimately, the justification for these definitional decisions rests on their productivity for the ongoing work of social science.
A Taxonomy of Descriptive Arguments
It will already be apparent that description hides a diverse range of empirical assertions. We perform very different actions when we describe. Yet, these activities are not limitless, nor are they shapeless. Rather, each presupposes a somewhat different type of structure and associated methodological criteria.
Descriptive arguments, I will argue, assume five archetypal forms: accounts, indicators, associations, syntheses and typologies, as summarized in Figure 1. This is how social science carves up nature at the descriptive level. These are the patterns that we look for when attempting to describe events in the social world. This is what description means at the level of argumentation.
Taxonomy of descriptive arguments
Accounts refer to any analysis of an event or set of events with no explicit attempt to generalize beyond the specifics of a particular case. When T. H. White remarks that John F. Kennedy won the 1960 presidential election with a bare majority of votes (as reported by the states to Congress), he is describing a specific event, with no intent to generalize.Footnote 16 This style of description is common in historical and ethnographic narratives, including some interpretivist work.Footnote 17 Of course, a particularizing account may form a part of a larger (causal or descriptive) argument; indeed, different types of claims are always nested within each other. So, in labelling something as an account it is important to specify whether one is speaking of a passage, a chapter or an entire work. Note also that particularizing accounts may be inferential.Footnote 18 Indeed, individual ‘facts’ are often obscure and must, therefore, be inferred from a congeries of evidence, from our general knowledge of the world, or from a formal theoretical model.
Other categories of description are generalizing; that is, they refer explicitly to a class of events (which may or may not be larger than the sample under study). Of course, generalizing is a matter of degree. There are no truly universal propositions (every proposition has explicit or implicit scope conditions), just as there are no truly particular facts (facts understood without any reference to context or to other known facts). Nonetheless, it is helpful to distinguish between generalizing and particularizing statements about the world. It is also important to recognize that embracing the goal of generalization does not imply a neglect of specific cases and events. What it means, rather, is that the analytic goal of a case is to shed light on a larger class of cases, i.e., a population.Footnote 19 So viewed, a descriptive generalization, whether it takes a qualitative or quantitative form, provides a ‘formula’Footnote 20 or ‘theory’Footnote 21 with which to describe some part of the world.
An indicator aims to describe one feature (i.e., one dimension) of a population based on the empirical manifestation of some phenomenon. It is univariate description. Granted, the identified dimension may be the product of multiple measures which do not perfectly co-vary – an index. Even so, an index claims to reduce this complexity to a single dimension and, therefore, carries the empirical features of an indicator. So defined, the concept of an indicator encompasses near-synonyms such as attribute, dimension, factor, measure, property and variable. Note that the construction of an indicator is equivalent to the task of measurement, as usually understood – assigning cases to some sort of scale, and, in so doing, establishing a metric of equivalence so that diverse observations can be directly and explicitly compared. Whatever the terminology, an indicator is the primitive empirical proposition underlying all other general propositions, descriptive or causal. Prominent indicators in political science include those intended to measure democracy such as the PolityFootnote 22 and Freedom HouseFootnote 23 indices.
Descriptive arguments about multidimensional components of a phenomenon, or the properties of various units, are associational. Research has focused, for example, on the degree to which involvement in politics is skewed towards the middle and upper classes in the United States, i.e., on the association between social class and political engagement.Footnote 24 Trend analysis seeks to discover a relationship between a phenomenon and time. For example, Putnam argues that social capital in the United States has declined precipitously since the 1930s and 1940s.Footnote 25 Network analysis focuses on interrelationships between units (which may be understood in spatial, temporal or functional ways). Frequently, political scientists are concerned with networks among political and economic elites.Footnote 26 Each of these associational arguments has causal implications, to be sure. However, the descriptive patterns are important, in and of themselves.
Some multidimensional arguments attempt to group together diverse dimensions into distinct categories. If there is only one category of interest (others are not well defined), this style of argument may be called synthetic. The claim here is that diverse attributes of a topic revolve around a central theme which unifies the attributes, lending coherence to an otherwise disparate set of phenomena. This is a common form of argumentation with respect to national political cultures. Writers dispute whether American political culture is best characterized as liberal/egalitarian,Footnote 27 republican,Footnote 28 or a mixture of both, along with various ascriptive identities.Footnote 29 Each offers a synthetic generalization of a broad and diverse subject matter.
Where multiple categories are defined, the result is a typology. Here, the goal is to sort phenomena into discrete categories that are mutually exclusive and exhaustive on the basis of a uniform categorization principle or principles. Note that while the categorization principle may be unidimensional the purpose of the typology is not achieved unless it performs a grouping function. Apples and oranges are rightfully separated into different categories of fruit because they are different from each other along multiple dimensions (colour, taste, etc.).Footnote 30
Typologies come in various forms, which may be briefly illustrated. A simple typology has no further characteristics beyond the defining characteristics of a typology. Thus, Finer divides up extant polities into four categories: Palace, Church, Nobility and Forum.Footnote 31 More complex typologies have additional attributes. A temporal typology, or periodization, constructs categories according to discrete time-periods. For example, it is sometimes argued that democratic transitions arrive in waves, each with distinctive characteristics.Footnote 32 A matrix (or multidimensional) typology derives categories from the intersection of several factors.Footnote 33 Thus, Dahl identifies four regime-types through the intersection of two core factors, participation and contestation.Footnote 34 A taxonomy arranges neighbouring categories within a genus et differentium hierarchy in which each subtype possesses all of the attributes of the type, plus one (as in Figure 1). Thus, it might be argued that a polity (a form of government) has two subtypes, autocracy (rule by the few) and democracy (rule by the many); democracy is a polity composed of two subtypes, direct democracy (unmediated rule by the people) and indirect democracy (rule by representatives of the people); indirect (representative) democracy may, in turn, be disaggregated by subtypes; and so on. Configurational typologies, like taxonomies, form sub-types out of a single superordinate category. However, subtypes are created from a superordinate category by subtracting, rather than adding, attributes. This generates diminished sub-types (sometimes called radial categories) rather than augmented sub-types. These sub-types radiate outward from the superordinate category, which takes the form of an ideal-type.Footnote 35 In this fashion, it is sometimes argued that democracy is best understood as a set of relatively distinct models – electoral democracy, liberal democracy, majoritarian democracy, participatory democracy, deliberative democracy, and egalitarian (or social) democracy – each emphasizing a different aspect of the key term.Footnote 36 As an ideal-type, the superordinate category contains all the attributes of the sub-types. The subtypes, however, possess only one (or some) of the attributes of the ideal-type.
Naturally, this brief classification of descriptive arguments could be greatly extended and elaborated.Footnote 37 However, the purpose of Figure 1 is primarily illustrative, i.e., to demonstrate the diverse nature of arguments that qualify as descriptive – separate and apart from any causal significance they might have. Any bid to valorize description must begin by describing what describing means, and this entails coming to grips with the disparate array of descriptive tools that inform the discipline.
It should be emphasized that each of the foregoing genres of descriptive inference may be carried out with the assistance of multiple techniques. One can describe with prose, with numbers, with pictures, or with pictures of numbers (a visual display of data). Frequently, these techniques are combined. Our topic therefore embraces mathematically-based techniques for large-N analysis as well as non-mathematical approaches referred to variously as ethnographic, interpretive, hermeneutic, narrative or qualitative.
The Status of Description
Having defined our topic, we turn to the relative status of causal and descriptive arguments within the discipline of political science. Of course, most work combines both genres. It is difficult to characterize any study as purely descriptive or purely causal, particularly when one considers that causal propositions build on descriptive propositions. Nonetheless, it is usually possible to characterize the most important contribution of a study as descriptive or causal. Thus, when speaking of these modes of analysis I shall be inferring differences of degree: some studies are more descriptive in orientation, others are more causal.
I will argue that causality has been granted preference over description in recent decades, at least in mainstream American political science. In order to test this question in a systematic fashion, several counting exercises are conducted. Each content analysis focuses on a small set of top journals (presumed to be representative of the best mainstream work conducted in that discipline) that are general in purview (rather than subfield specific), have a long and continuous publication history, and are housed within the JSTOR database.
The first analysis focuses on three political science journals: American Political Science Review (1906–99), American Journal of Political Science/Midwest Journal of Political Science (1957–99), and World Politics (1948–99). For each two-decade period over the past century an automated search is conducted for the following terms: cause, causal, causality and causation. The number of articles including at least one of these search terms is then calculated as a ratio of all articles published within each two-decade period. Results, depicted in Figure 2, show a doubling of articles focused (at least peripherally) on causal questions – from 30 per cent in the first period to roughly 60 per cent in the final period.
Top Political Science journals
Note: The share of all articles (not reviews, editorials, or other matter) within top political science journals mentioning at least one of the following search terms: causation, causality, cause, or causal. Top journals include American Political Science Review (1906–99), American Journal of Political Science/Midwest Journal of Political Science (1957–99), and World Politics (1948–95). The denominator of the ratio – total articles within each stipulated period – was provided by JSTOR's data support team and is available upon request. Searches conducted using the JSTOR on-line search function in November 2008.
The second analysis is more fine-grained. Here, articles within American Political Science Review are hand-coded by a research assistant to discover whether the main argument of each piece (as revealed in the abstract, introduction and/or conclusion) is predominantly descriptive or causal. Coding includes all articles published within a given year, at decadal intervals – a total of forty-two issues and 330 articles. Results, displayed in Figure 3, show a relatively consistent decrease in descriptive articles, coincident with an increase in causal articles, beginning about 1920. By the late twentieth century the two genres have traded places as dominant motifs in the discipline's leading journal. Why questions have displaced what/how possible/when/whom, or in-what-manner questions.
American Political Science Review, 1910–2000
Note: The share of all articles whose principal argument is descriptive (generalizing or non-generalizing) or causal within the American Political Science Review. Coding includes all articles published within a given year, at decadal intervals – a total of 42 issues and 330 articles. Excludes book reviews, editorials, presidential addresses, personal notes and correspondences, short research notes, symposia of non-regular articles, and notes on current legislation.
Of course, many other disciplines may have registered a parallel change in orientation over the same period. Arguably, the paradigmatic shift from description to causation qualifies as a mark of disciplinary maturity within a natural or social science. Have changes in political science mirrored those in other disciplines? In order to investigate this question a third content analysis is conducted. Here, the percentage of articles mentioning causality and its cognates in top journals is compared across five social-science disciplines (political science, sociology, economics, psychology and anthropology) and in a polyglot group of natural sciences over the past two decades. Results, displayed in Figure 4, seem to confirm that preoccupations with causality are considerably stronger in political science than in any other social science or in the natural sciences.Footnote 38
The disciplines compared, 1980–99
Notes: The share of all articles (not reviews, editorials, or other matter) within top journals in various disciplines mentioning at least one of the following search terms: causation, causality, or causal. The denominator of the ratio – total articles within each stipulated period – was provided by JSTOR's data support team and is available upon request. Searches conducted using the JSTOR on-line search function in November, 2008.
Political Science (PO) journals include: American Political Science Review, American Journal of Political Science, and World Politics. Sociology (SO) journals include: American Journal of Sociology, American Sociological Review, and Social Forces. Economics (EC) journals include: American Economic Review, Quarterly Journal of Economics, and Journal of Political Economy. Psychology (PSY) journals include the American Journal of Psychology. (The eight other psychology journals in the JSTOR collection are subfield journals, and thus inappropriate for our purpose.) Anthropology (ANT) journals include: American Anthropologist, American Ethnologist, and Journal of the Royal Anthropological Institute/Man. Natural science (NAT) journals include all journals listed in JSTOR under the following disciplinary categories: Biological Sciences, Botany and Plant Sciences, Developmental and Cell Biology, Ecology and Evolutionary Biology, General Science, Health Sciences, Mathematics, and Zoology (N = 432).
One might question whether mentions of causality and its cognates accurately represent the methodological focus of journal articles. For example, authors may mention causality with the aim of debunking the objective; likewise, they may pursue causal questions without mentioning the term itself. In order to cross-validate this coding procedure a number of other searches are applied across these same disciplines. Keywords include exogeneity/endogeneity (important terms for any statistical analysis whose focus is causal assessment) and cluster analysis/factor analysis/discriminant analysis/multidimensional scaling (important terms for statistical analyses whose focus is not explicitly causal). These additional content analyses (available on request) reveal the same general picture. In political science, causal studies are more common, and descriptive studies less common, than in other social sciences or the natural sciences. Note also that our second content analysis involved hand-coding of the American Political Science Review (Figure 3) and is thus not subject to the foibles of automatic coding by keyword. It does not seem likely that this set of results is a product of arbitrary coding rules.
Let us try to make sense of these long-term patterns. Note that the demise of description and the attendant rise of causation coincides with the embrace of scientific methods within political science.Footnote 39 While perceptions of the scientific model have changed over the decades – from behavioralism, to rational choice, to potential-outcomes and Bayesian inference – each of these approaches privileges causation over description. Within political science, doing ‘hard’ science means analysing causal relationships. Likewise, methods courses tend to give short shrift to descriptive techniques of data gathering and data analysis (e.g., ethnography, field research, coding, content analysis, measurement, factor analysis) in favour of causal topics (e.g., experimental and quasi-experimental research design, case studies, regression, matching, instrumental variables).
By contrast, in most of the natural sciences – including anatomy, archaeology, astronomy, biology, botany, chemistry, ecology and environmental science, geology, linguistics, medicine, paleontology and zoology – advances are accompanied by the discovery of new things ‘out there’ which must be named, measured, characterized and classified, or re-classified.Footnote 40 This is a descriptive task, and an eminently important one.
Similarly, within other social-science disciplines – including anthropology, economics, psychology and sociology – the ancient task of describing the social world continues to inspire interest and to receive scholarly acclaim. Even in economics, which has played a pioneering role in the analysis of causal relations, a central role has been preserved for descriptive tasks. These are often closely tied to national governments (e.g., finance, commerce, agriculture and labour departments, along with national banks) and international financial institutions (e.g., the World Bank, the International Monetary Fund, the World Trade Organization, the Food and Agricultura; Organization and the Organization for Economic Co-operation and Development) that supervise the collection of economic data. Daily, an army of economists pores over surveys and national accounts in an attempt to describe specific features of the world's economies. The results of these labours inform the content of journals, where debates over conceptualization and measurement form an important category of work.
A signal of these general patterns can be seen in the prominence of measurement as a methodological topic across the social sciences. The development of a self-conscious field devoted to problems of social measurement is grounded in work by scholars of education (Edward Thorndike), psychology (Donald Campbell, Lee Cronbach, David Krantz, Georg Rasch, Charles Spearman, Louis Thurstone, Amos Tversky),Footnote 41 sociology (Ken Bollen, Otis Duncan,Footnote 42 Louis Guttman, Paul Lazarsfeld,Footnote 43 Samuel Stouffer), and – more recently – economics.Footnote 44 Surveys intended to cover the topic of measurement across the social sciencesFootnote 45 reflect this disciplinary grounding, as do journals that deal centrally with measurement issues (e.g., Psychometrika, Quality and Quantity, Social Indicators Research). By way of contrast, in political science no journal focuses on issues of measurement and no recent handbook or book-length treatment of the subject exists.Footnote 46 Indeed, the topic of measurement is treated gingerly by most methods texts, as noted by Brady with reference to King, Keohane and Verba.Footnote 47
Thus, the story depicted in Figures 1–3 has some prima facie plausibility, both from the perspective of the trajectory of political science and from what we know about the trajectory of other fields.
The Independent Status of Description
I have demonstrated that causal questions have higher status within political science than within other social and natural sciences, at least in the latter twentieth century, and that the task of description has fallen into relative desuetude. I turn now to the normative implications of this development. To what extent might this be considered a problem?
To clarify, the current view of description within the discipline does not deny the importance of the task. However, its importance is thought to derive from its role in causal explanation. Political scientists describe in order to explain. All phenomena eventually take their place within a causal schema – as X (the causal factor of theoretical interest), Y (the outcome of interest), M (the mechanism connecting X to Y), or Z (a covariate). Moreover, this schema guides the process of conceptualization and measurement, regardless of whether the research is qualitative or quantitative, experimental or observational.
I shall argue that while this asymmetry may be justified in some circumstances, difficulties arise when description is systematically subordinated to causation within a discipline. Specifically, if description occurs only in the quest for causal inference the causal motivation of researchers may mitigate both the quality and the quantity of descriptive inferences. We will know less about the world (descriptively) and what we know will be less precise, less reliable and perhaps subject to systematic bias – generated by scholars’ motivation to uncover a causal relationship. My argument is, therefore, that description should sometimes proceed independently of causal propositions – that it has, or ought to have, an independent status within the discipline.
Consider, to begin with, that description of a topic usually precedes causal analysis of that topic. King, Keohane and Verba point out that ‘it is hard to develop [causal] explanations before we know something about the world and what needs to be explained on the basis of what characteristics.’Footnote 48 It follows that in cases where description precedes causation, i.e., in those circumstances where knowledge of a topic is minimal, description must proceed independently of causal propositions.
Secondly, a large class of topics in political science is intrinsically important, regardless of any causal effects they might possess. This applies to subjects like democracy, human rights, war, revolution, standards of living, mortality, ethnic conflict, happiness/utility and inequality. Indeed, every causal analysis presupposes that something – at least Y, but sometimes also X – is important enough, substantively, to warrant investigation. For such topics, our interest is not contingent merely upon the roles they might play in causal propositions.Footnote 49
A final reason for liberating description from specific causal hypotheses is practical in nature. Often, it is more efficient to collect evidence when the objective of the investigation is descriptive rather than causal. Consider that data is collected from persons, governments, archives and other organizations. Collecting evidence from these sources in a systematic fashion usually requires a great deal of well-coordinated energy and resources, sustained over many years. When a data collection effort is constructed around a single causal hypothesis or theory the scholar's purview is naturally quite limited; only those factors having direct bearing on the hypothesis will be collected. This may be efficient in the short run, but it is not likely to be efficient in the long run.Footnote 50 Narrowly focused data expeditions entail scaling high cliffs and returning to base camp with a small and unrepresentative sample of what one finds at the peak. Later expeditions, focused on different hypotheses, will require re-scaling the same peak, a time-consuming and wasteful enterprise. By contrast, if an evidence-gathering mission is conceptualized as descriptive rather than causal (which is to say, no single causal theory guides the research), it is more likely to produce a broad range of evidence that will be applicable to a broad range of questions, both descriptive and causal.
Before concluding this discussion it should be pointed out that the subordination of description to causation would not be problematic if political science possessed a single causal-theoretical framework around which a coherent description of the world could be constructed – something on the order of evolution within the biological sciences. Lacking such a unifying paradigm it is difficult to say how a causally-ordered description of the political world might be organized or what it would look like (in concrete terms).
One might counter that in the multi-paradigmatic universe of social science one should look to smaller-scale causal hypotheses to organize the work of the discipline (along the ‘behaviouralist’ model). But here one stumbles upon another problem of indeterminacy. Because causal attribution is difficult to establish for most non-trivial questions in political science, it is problematic to assert that X matters as a subject of investigation only in so far as it causes Y (or Y matters only in so far as it is caused by X). Uncertainty about whether X really causes Y means that it may be safer to approach X and Y first as descriptive phenomena – important in their own right – rather than as potential independent and dependent variables.
As an example, let us consider the question of ‘party strength’. Presumably, this feature has many causal properties. However, we do not know for sure what these are; certainly, we do not know precisely what they are. Consequently, I would argue that the subject is better approached, at least initially, as a descriptive issue. This means, in practical terms, that the investigation is structured by the concept and its definition, by the understood or intuited boundaries of the concept (places where party strength is meaningful, or where it means roughly the same things), and by an anticipation of its possible causal relevance.
Description should never be carried out in ignorance of all causal potentialities. Rather, in circumstances where causal truth is open-ended – presumably the vast majority of cases in political science – descriptive inference ought to be carried out independent of any particular causal hypothesis. This helps to avoid a highly prejudiced (i.e., particularistic, idiosyncratic) definition of a subject matter. All plausible causal hypotheses are relevant – those in which a subject serves as an independent variable, those in which it serves as a dependent variable, and those in which it serves as a causal pathway in some larger subject. When considered in this open-ended fashion the subject of interest (e.g., party strength) is approached descriptively rather than simply as a preface to causal analysis.
The Challenges of Description
If description is intrinsically valuable, and (at least in some circumstances) independent of causal analysis, how might successful descriptive inferences be achieved? I turn now to the methodological challenges posed by descriptive analysis.
Note that all descriptive analysis involves the twin goals of conceptualization and measurement. There are, of course, additional methodological tasks associated with each genre of descriptive argument, as summarized in Figure 1. But conceptualization and measurement are fundamental to all and it is at this basic level that we shall approach the subject.
Note also that in this section I shall be concerned primarily with generalizing descriptive inferences – indicators, associations, syntheses and typologies – not with particularizing accounts. Likewise, in discussion of causal analysis I shall be concerned with inferences that cover a class of phenomena rather than those pertaining to a single unit (individual treatment effects).
Conventional wisdom presumes that causal inference is harder, methodologically speaking. ‘What questions are generally easier to answer than why questions’, states Glenn Firebaugh.Footnote 51 ‘Empirical data can tell us what is happening far more readily than they can tell us why it is happening’, affirms Stanley Lieberson.Footnote 52 Reading the methodological literature, one might infer that description is a relatively simple, largely intuitive act of apperception.
And yet, many descriptive questions circulating through the disciplines of social science are recalcitrant. Indeed, there is consternation in many quarters over the poor quality and measly quantity of evidence by which we attempt to make sense of the political world.Footnote 53 Descriptive accounts of phenomena such as corruption, campaign finance, civil service protection, judicial independence, and party strength are often problematic, or are restricted in purview to very specific contexts (and hence resistant to generalization). Likewise, the master concepts of political science – e.g., civil society, democracy, governance, politics, power, state – have no standard and precise meaning or measurement.Footnote 54 Meanwhile, whole tracts of political activity remain virtually terra incognita.Footnote 55 As a result, empirical phenomena on the left and right sides of our causal models are highly uncertain.
As a way of getting our minds around this topic, it may be helpful to consider several examples of descriptive questions that have agitated the discipline in recent years:
1. Do voters conceptualize politics ideologicallyFootnote 56 or non-ideologically?Footnote 57
2. Is global inequality increasingFootnote 58 or remaining about the same?Footnote 59
3. Is American political culture liberal/egalitarian,Footnote 60 republicanFootnote 61 or a mixture of both, along with various ascriptive identities?Footnote 62
These are all essentially descriptive questions about the social world (though, to be sure, they contain causal implications). They have also proved to be hotly, and enduringly, contested. And they are not unusual in this regard. A random sample of (non-trivial) descriptive arguments would be likely to reveal a high level of uncertainty. Why, then, are these sorts of questions so resistant? Why are (good) descriptive generalizations difficult?
I shall argue that these methodological problems may be summarized under the rubric of falsifiability.Footnote 63 With descriptive arguments in the social sciences it is often unclear what criteria might be invoked to disprove a proposition.Footnote 64
In certain respects, the challenges facing successful descriptive and causal accounts are similar. Both often attempt to infer from observed to unobserved features of reality. Both strive for breadth and precision while often failing to achieve one or the other – law-like propositions are rare. Both endeavours become more difficult as one abstracts from the concrete to the abstract.Footnote 65
Even so, an authoritative causal explanation of a subject may be easier to obtain than an authoritative description of that same subject. To see why this might be so, consider the following two questions:
1. What is democracy, and how might it be operationalized?
2. Does democracy enhance the prospect of peaceful coexistence?
Note that the second, causal, question presumes an answer to the first, descriptive, question. In order to estimate democracy's causal effect one must first establish the definition and measurement of this vexing concept. Logic suggests that if Proposition 2 builds on Proposition 1 it must be at least as difficult to prove as Proposition 1. And yet, by all appearances, there is greater scholarly consensus on the answer to the second question than on the answer to the first question. Scholars of international relations generally agree that regime-status has a causal effect on peace and war such that democracies are less likely to fight wars with one another, all other things being equal. Whether or not democracy is a sufficient condition for peace may never be determined, and scholars continue to debate the causal mechanisms at work in this relationship. However, there is still a large measure of agreement on the democratic peace as – at the very least – a probabilistic causal regularity.Footnote 66 All things being equal, two democratic countries are less likely to go to war with one another than two countries, one or both of which are non-democratic. By contrast, no such consensus exists on how to define and measure democracy (see citations above). The causal proposition is fairly certain, while the descriptive proposition that underlies it is highly uncertain.
How can this be so? Let us suppose that in some instances the conceptualization and measurement of key causal variables (X and Y) have marginal impact on the stated causal relationship. Setting a low definitional threshold for democracy, for example, means that cases like Wilhelmine Germany will be understood as democratic, a coding that will show the First World War as a violation of the democratic peace hypothesis. However, democratic dyads (so defined) will still be less likely to engage in violent conflict with one another than non-democratic dyads, reaffirming the theory in its probabilistic formulation. Similarly, different thresholds for what constitutes ‘war’ will impact one's findings but probably not so much as to reverse the general causal relationship. In this circumstance one may regard decisions about conceptualization and measurement as secondary. They are important, and certainly affect the precision of the analysis, but may be bracketed so long as alternative interpretations of the key concepts would not undermine the main empirical finding.
From the perspective of regression analysis, measurement error in the independent variable of interest can sometimes be successfully managed with an estimator that models the assumed pattern of error – an errors-in-variables model.Footnote 67 Likewise, error in the dependent variable suggests another sort of model.Footnote 68 Naturally, any such exercise requires a number of assumptions about the sort of measurement problem one is faced with. Results are only as strong as the assumptions necessary to establish the viability of an estimator. The point remains that error in measurement does not doom causal analysis. In some instances, measurement errors may be great but have no impact on causal estimates, as when X is systematically biased upward but its relationship to Y remains the same.
Let us now suppose that a different conceptualization and measurement of democracy does substantially alter the relationship between some chosen X and Y and that it cannot easily be corrected through a statistical model. Here, causation founders over description, and there are clearly many instances of this in the library of social science.Footnote 69 Indeed, it is a standard rejoinder to virtually any causal argument that ‘it depends on how you define inputs and outputs’. Even so, choices of conceptualization and measurement are apt to be regarded with forbearance so long as they are not patently idiosyncratic. More specifically, findings that turn out to be contingent upon a particular interpretation of a key term are understood as valid, contingent upon the author's choices of conceptualization and measurement. The causal finding holds, with a definitional/operational caveat. By contrast, work that is primarily descriptive in focus faces a higher hurdle. Such a study is focused centrally on how L, the latent concept of interest, should be described – on what it is. As such, issues of conceptualization and measurement cannot be ‘caveated’ away, for they lie at the centre of the argument. Meanings play a constitutive role in description, which is all about semantics.
Why is it, then, that these questions are so bedevilling? The existential question of what L is turns out to be difficult to answer in a definitive fashion because it stumbles against the superabundance of reality, as well as the ambiguities of language. As Weber notes, ‘a description of even the smallest slice of reality can never be exhaustive’,Footnote 70 and for that reason, can never be authoritative. Then there is the problem of labelling. Any phenomenon of significance to social science is likely to call up multiple words, and multiple definitions for each of those words. Consequently, and because much is at stake in an author's choice of terms and definitions, descriptive arguments are apt to remain essentially contested.Footnote 71 While causal inferences are anchored by two points (or sets of points) in time and space and by the delimited empirical question of whether X generates Y, descriptive inferences suffer from considerable leeway.
This leeway might be labelled the fundamental problem of descriptive inference: for any given subject there are often multiple perspectives, each more or less valid. As a consequence, there is usually more than one plausible answer to the innocent question: What is that? ‘Rashomon’ effects are endemic.Footnote 72 To be sure, some answers may be rejected flat out, just as some causal propositions may be falsified with a high degree of certainty. However, many other plausible answers are likely to remain, and among these it is difficult to discern which one provides the best account of a phenomenon. This means that description is at once simple to perform (indeed, impossible not to perform) and yet often difficult, if not impossible, to achieve in a definitive fashion as the example of ‘democracy’ attests.
One way of thinking about this is to say that descriptions are subjective to an extent that causal inferences are not. In the latter, a good deal of structure is provided by the nature of causal argument, and the focus is on the truth of the causal relationship rather than an author's choice of subjects and concepts. Of course, the degree of structure provided by causality is to some extent a product of how one chooses to define the key term. Following Holland, some would argue that causality should apply only to arguments about the effects of causes (not causes of effects) and only to arguments in which the causal factor of interest is directly manipulable.Footnote 73 This eliminates a lot of ambiguity and allows for a clear and highly structured analysis. At the same time, this narrow definition of the potential-outcomes framework also disallows a good deal of work conventionally defined as causal.Footnote 74
The point I wish to emphasize is that regardless of how causality is defined, causal inference is still a more highly structured – more ‘objective’ – enterprise than descriptive inference. One can investigate the causal relationship between trade openness and sweated labour without revealing one's attitude towards the subject, while it would be virtually impossible to do so in the course of a descriptive study focused on the same subjects. In this vein, Bourguignon, Levin and Rosenblatt conclude that the ongoing debate over the contours of global inequality rest largely on value judgements about how to operationalize key parameters.Footnote 75 It appears that writers with a more critical view of the current distribution of income (and the mechanisms underlying this distribution) are apt to make measurement decisions that emphasize inequalities, while those with a more hopeful view of the status quo may emphasize equalities. Neither view can be decisively proven or disproven, for they rest on interpretations of the facts.
At a more fundamental level, the problem of descriptive inference rests upon a philosophical distinction between instrumental and substantive rationality.Footnote 76 Causal inference is properly regarded as a species of instrumental rationality since it concerns the empirical relationship between two factors, while involving no explicit judgement about the normative significance of the resulting relationship. Descriptive inference, by contrast, is centred on a judgement about what is important, substantively speaking, and how to describe it. To describe something is to assert its ultimate value.Footnote 77 Not surprisingly, judgements about matters of substantive rationality are usually more contested than judgements about matters of instrumental rationality, and this offers an important clue to the predicament of descriptive inference.
Another point of contrast between descriptive and causal inference concerns the difficulty of achieving a genuine – recognizable – scientific contribution. Consider briefly the multitude of ways in which a single study may contribute to our causal understanding of an outcome, Y. That study may introduce a new causal factor (variable X), a new specification (vector X), or a fundamentally new theoretical framework to our understanding of Y. It may introduce a new way of conceptualizing or operationalizing Y, and hence a (slightly) new causal account for Y. It may introduce a new argument about the causal mechanisms lying between X and Y. It may test old models on new data, or transformed data (corrected for previous errors). It may employ a new sort of statistical technique or research design. It may show that a simple replication of previous work is not possible, and thus call into question previous findings. There are, in short, myriad ways in which research may contribute to a body of work devoted to a well-established causal hypothesis.
By contrast, inferences of a descriptive nature are often at pains to demonstrate a scientific advance over what has been done before. New work devoted to a well-tilled subject is likely to be judged derivative – new wine in old bottles – rather than innovative. The problem is that it is very difficult to say when true innovation has occurred. While multiple causal analyses of the same general subject may illuminate different corners of that subject and thus cumulate with extant knowledge, multiple descriptive analyses do not tend to divide up a subject in a neat and tidy fashion. Rather, each imposes a unique holistic order. This by itself would be unproblematic if the new order had greater claims to truth; but this seldom seems to be the case. Rather, we learn from a new study what an old subject looks like from a different (but not necessarily superior) perspective. Old wine is packaged under new labels.
While some might argue that a diversification of perspectives constitutes progress, it is not the sort of progress normally associated with the criterion of cumulation, for each perspective conflicts (rather than cumulates) with the rest. This returns us to the fundamental problem of descriptive inference. Because such inferences are inseparable from the ambiguities of reality and of language, multiple studies of the same subject rarely build upon one another. A simple re-jiggering of the conceptual scaffolding is unlikely to offer a marked improvement in the resulting inference, relative to the previous inference it is intended to displace. Authors apply a terminological gerrymander, but the overall semantic territory remains much the same. It is this sort of pseudo-innovation that contributes to description's bad name.
Thus, one finds a raft of studies focused on the causal relationship between democracy and peace, many of which can rightly claim to be making an important scholarly contribution. But there are fewer studies focused exclusively on the meaning, measurement or characteristics of democracy. Moreover, those studies that do adopt a descriptive focus are seemingly unable to resolve extant ambiguities. Instead, they tend to focus on the problem itself – demonstrating the lack of agreement over what democracy means and the difficulties of measuring this ambient concept (see citations above). There is no cumulation, except in the preliminary sense of agreeing on a set of possible options for conceptualization and measurement – a topic that must be revisited with each passing decade, as new meanings are adumbrated.
Disaggregation of an abstract concept like democracy offers some hope for disambiguation and for solving problems of measurement, but only at a lower level of abstraction.Footnote 78 One might hope to agree on how to define and operationalize ‘free and fair elections’. But even if we can achieve this feat and replicate it for other components of democracy the result will not inform us about how to define and operationalize ‘democracy’. The aggregation problem looms large. Likewise, abstract terms like democracy, corruption, accountability, globalization and public goods continue to animate the discipline. Talcott Parsons has been put to bed but our universe of hazy terms continues to evolve. And there is good reason to suppose that such terms are critical to any attempt to generalize about social phenomena – which is to say, to theorize. Consequently, problems of conceptualization are likely to remain central to the discipline for the foreseeable future.
Let us return to the problem of innovation. Arguably, true innovation in descriptive inference is established only by delineating a fundamentally novel empirical terrain, or by thoroughly revising our sense of an established terrain. For example, to say something new about democracy will probably require an extensive exploration of many regimes or a very intensive exploration of a single regime. One is unlikely to make a descriptive contribution simply by playing with extant facts and data. This is not an armchair occupation; ‘shoe leather’ is required. By contrast, lots of causal analyses involve re-workings of extant data. Description, in this respect as well, is more difficult than causal analysis because it imposes a higher threshold on innovation, requiring a greater expenditure of time, energy and funds. Frequently, the downstream benefits of this sort of innovation are greater than the corresponding benefits of causal analysis. But the fact remains that the descriptively oriented researcher faces a higher hurdle in the race to publication than the causally oriented researcher.Footnote 79
For those who might be inclined to view this genre of methodological difficulty as characteristic of concept formation, but not of measurement, let us explore the latter in greater detail. The question addressed by causal analysis is whether X causes Y, and if so what the causal effect might be, as shown in panel (a) of Figure 5. The question addressed by measurement is the location and quantity of L, the latent concept of interest, as shown in panel (b). Note that the phenomenon of interest in any measurement task is not directly measurable but rather latent. We do not consider directly measurable phenomena such as falling dominoes to present problems of measurement. They become so only when they are not directly measurable. Moreover, most intransigent problems of measurement in the social science are inherently unmeasurable. They are not like dominoes, which can be measured in many settings. They are like democracy, which cannot be directly observed in any setting.
Causal and measurement tasks contrasted
X = causal factor of interest; Y = outcome; [L] = latent concept of interest; I = indicator; → = causal relationship; –– = correlative or causal relationship.
Measuring a latent concept is necessarily based on assumptions about that concept, assumptions that are not liable to empirical testing. This may be demonstrated in a prima facie manner by consideration of several commonly proposed strategies. Face validity refers to an obvious or intuitive appeal, an approach that is evidently unsystematic and unavailing (by definition) in tough cases. Convergent and discriminant strategies attempt to validate a measure by comparing it with other measures that are deemed to be valid measures of the same concept (convergent validity) or different concepts (discriminant validity). Of course, both strategies depend upon the assumption that the comparator concepts are correctly measured and have a specified relationship to L, assumptions that cannot be tested. Causal strategies attempt to validate a measure by looking at its causal relationship to an input or output. Naturally, this causal relationship is inferred, not tested, since L cannot be directly measured.
This is not the place to enter into a discussion of sophisticated measurement techniques such as item response models and structural equation models.Footnote 80 Suffice to say that successful employment of these methods depends upon the data that is available – i.e., on those elements of a concept that are measurable and have been measured, and on the validity and reliability of these indicators. If garbage goes into the measurement model, garbage is likely to flow out (with more realistic uncertainty estimates). The employment of these methods also depends crucially upon assumptions (generally untestable) about the concept being measured: for example, which indicators ought to be chosen to measure a latent concept, what the structure of that concept should be (are there necessary or sufficient attributes?), whether there are natural boundaries (upper and/or lower limits) to the concept, and what an appropriate aggregation procedure might be. Thus, although advanced strategies of measurement are immensely useful, they do not offer a definitive solution to problems of measurement.
Here, again, it may be useful to contrast the corresponding dilemma of causal inference. Although the counterfactual for a single unit cannot be estimated with much confidence, the counterfactual for a class of units may be estimated with considerable confidence using experimental and quasi-experimental methods. This provides a benchmarking technique for other, less rigorous methods using observational data.Footnote 81 No analogous method of validition is available to help resolve the dilemma of measurement, a procedure that remains – and must remain – heavily inferential.
Before concluding this section let me return to an earlier point, lest it get lost in the scrum. Because causal arguments build on descriptive arguments, any causal argument where X and/or Y are hard to conceptualize and measure is subject to problems endemic to descriptive inference, as discussed. However, these problems tend to be overlooked. Because a causal argument is focused on X's relationship to Y, readers tend to forgive potential problems that concern the nature of X and Y if these problems are properly noted and if the author's choices in conceptualization and measurement are within the bounds of reason. They will credit the author for doing the best s/he can, given the flawed tools s/he is forced to work with. Or they will say, perhaps if X or Y were defined or measured differently they might not have the same relationship to each other. But at least in this case – with this particular conceptualization and measurement of X and Y – they do, and that is sufficient. In other words, writers whose goal is causal are granted a good deal of indulgence in dealing with problems of descriptive inference. The same indulgence is not granted to writers whose goal is primarily descriptive. Where the objective of a study is descriptive, problems of conceptualization and measurement are not so easily dismissed.
Whether this state of affairs is warranted or not strikes me as a difficult matter to resolve. Some might argue that raising the descriptive bar for causal analysis would have the effect of improving the quality of both descriptive and causal analysis. There should be a single standard of descriptive inference, one that is applicable to work that is oriented towards descriptive or causal arguments. Others might argue that these are difficult and time-consuming tasks whose missions are not easily accomplished within the scope of a single study. Moreover, causal inferences are not always dependent upon descriptive inferences since errors in measurement of X or Y do not always impair our ability to estimate X's impact on Y (as noted).
Discussion
Progress in the discipline of political science rests not simply on the development of new techniques of research design and analysis intended to solve recalcitrant problems of identification in causal inference. Equally important is the seemingly prosaic act of description. As it stands, we simply do not know with any degree of precision or confidence what is going on out there, as Abraham Kaplan suggests (see epigraph). To paraphrase Sartori, the more we advance in causal modelling, the more we leave a vast, uncharted territory at our backs.Footnote 82 The purpose of this article has been to map this relatively uncharted territory.
I argued, first, that in so far as description has come to be defined in social science venues in a residual fashion (relative to causal inference) much is lost. Instead, I propose to return to an older understanding of the topic, one rooted in the goals of an inference rather than the quality of the evidence provided to make that inference. Specifically, any empirical proposition that attempts to answer a what, when, whom, out of what, or in what manner question is classified as descriptive (and it may or may not be connected to a causal proposition).
I argued, secondly, that the rubric of description encompasses diverse styles of argumentation. A taxonomy, summarized in Figure 1, distinguishes five types of description: accounts, indicators, associations, syntheses and typologies. Each responds to somewhat different methodological criteria. But all aim to describe portions of the world in a manner that is useful to social science research.
I argued, thirdly, that the task of description has been neglected within the discipline of political science and that this neglect constitutes a significant paradigm shift over the course of the past century. Although innovative techniques have been developed to treat specific descriptive problems, little thought has gone into the generic features and methodological properties of descriptive inference – a way of knowing that is distinct from causal knowledge and not reducible to problems of measurement.
I argued, fourthly, that the task of description is often best approached independently, rather than as an adjunct to causal hypotheses or causal frameworks. Many descriptive inferences are important in their own right. Moreover, liberating description from causation will in some instances (though not all) lead to better – more valid, more precise, more complete – descriptions of reality, and also will help to overcome inefficiencies in the data-collection process.
I argued, finally, that the complexities of description are belied by the apparent simplicity of the descriptive act: to say what is, and what is not, i.e., questions of identity and non-identity. In certain respects, the description of a phenomenon proves harder to achieve than an explanation of that same phenomenon. It does not follow from this that one ought to consign description to the realm of conjecture. Description can be written out of the discipline only at great cost to our understandings of the world.Footnote 83 The relevant question is how to bring greater methodological rigour to this neglected corner of the discipline.
This article is offered as a ground-clearing exercise, intended to stimulate practitioners and methodologists to take up the challenge of description. Should this challenge be accepted, and should it muster the same zeal that has characterized research into causal inference over the past half-century, we can anticipate considerable progress in this vexed area over the coming years and decades.
Signs of this are evident in several areas, which may be briefly listed: techniques for estimating missing data,Footnote 84 item-response models,Footnote 85 fuzzy-set measurement,Footnote 86 the conceptualization and measurement of social identities,Footnote 87 including name-based techniques,Footnote 88 experimental techniques for measuring values and beliefs,Footnote 89 the automated content analysis of political texts,Footnote 90 geographic information systems (GIS) and other techniques for spatial representation of social phenomena,Footnote 91 innovations in data storage and retrieval,Footnote 92 and the construction of new datasets.Footnote 93 Meanwhile, important research continues on older subjects such as concept formation,Footnote 94 typologiesFootnote 95 and measurement validity.Footnote 96 A general call to arms has been issued by the American Political Science Association's Task Force on Indicators of Democracy and Governance, commissioned by Henry Brady in 2010 and directed by Michael Coppedge.
Whether this resurgence of interest in description will be more than episodic remains to be seen. Much depends, needless to say, on the reception this work receives from the political science community. It is to be hoped that description will no longer be prefaced by the derogatory adverb, merely – unless, that is, the goals of the researcher are causal and the evidence does not warrant the inference (in which case the epithet is amply warranted). In other instances, where the goal of the researcher is descriptive, our judgement of the inference should rest on its validity and reliability, and its contribution to knowledge. These issues, in turn, hinge crucially on the sort of description that is being offered, for which a rough-and-ready categorization is offered in Figure 1.
By way of conclusion, let me return to a point made in passing in the previous section. Good description is closely hinged to normative judgements about the world – to what we think is important and what we think is right or wrong, desirable or undesirable. It is difficult to separate the concepts that govern theoretically driven description – concepts like democracy, governance or stability – from these normative concerns. It follows that a re-engagement with description may also involve a re-engagement with the normative underpinnings of political science,Footnote 97 a topic often swept under the rug in causal analyses.
