Published online by Cambridge University Press: 06 August 2003
“[F]ield work refers to observation of people in situ ”…to examine behavior in naturally occurring groups—human beings “on the hoof”—as opposed to studies of ad hoc groups conducted in the laboratory.…It is the nature of the phenomena under investigation and the objectives of the study which must determine what approaches are taken and what materials are gathered by what methods.
In the first pharse Scott refers to Everett C. Hughes' (1960, v) definition of field research, quoted more fully on p. 261.
In teaching interpretive research methods, I call my students' attention to the commonplace discrepancy between word and deed, noting that observation of people and their acts (with whatever degree of participation) can be a useful check on what it is they claim to do or to believe when interviewed. This tension is well captured in the parental directive “Do what I say, don't do what I do” that parallels the imperative “Practice what you preach” underlying Andrew Bennett, Aharon Barth, and Ken Rutherford's essay title. So it is that the authors of these two Symposium essays direct our attention to possible discrepancies between the claims we make as political scientists and what we actually do in our editorial and curricular practices. Contemporary work and management studies, influenced by phenomenology and pragmatism, argue that reflection on (and in) action can lead to making such discrepancies explicit, under the assumption that humans desire “authenticity”—a greater accord between one's words and one's deeds. A centennial celebration is perhaps an appropriate occasion for a discipline to reflect on its practices and ask, to paraphrase Peregrine Schwartz-Shea's title, “Is this the political science we want?”
APSR has been criticized for many years for its narrow focus—for the predominance of behaviorist or statistical analyses, with relatively little political theory or “qualitative” work. Until now, this widespread feeling has not been grounded in systematic analysis. David Pion-Berlin and Dan Cleary (2002) provide such grounding, reporting on the last decade's worth of APSR articles in four methods categories: qualitative, statistical, mathematical or formal modeling, and theory. From 1991–2000, research based on statistics and modeling accounted for 74% of all published articles (53% and 21% respectively), political theory garnered 25% of journal space, and qualitative research captured 1% (one article each in 1992, 1993, 1995, 1996, 1997). Using the same taxonomy in a study of the American Journal of Political Science (the Midwest Political Science Association's journal) from 1960–2001, Greg Kasza (2003) reports that qualitative research dropped from 75% in 1960–61 to 26% in 1970–71 to 12.3% in 1980–81, to 2.6% in 2000–01. Philosophy (theory), at 15.6% in 1960–61, dropped to 8.2% ten years later and has ranged from 1% (1995–96) to 7.1% (2000–01) of page space since. Modeling grew from 0 in 1960–61 to 18.6% in 2000–01 (with a high of 29.7% in 1990–91). And statistics mushroomed from 9.4% in 1960–61 to 62.3% ten years later, with 71.7% of journal space in 2000–01 (and a high of 89.1% in 1995–96).
Bennet, Barth, and Rutherford's essay, in this context, provides us with an even broader range of data on editorial practices, looking as they did at 10 journals and reaching as far back as 1965 (although, restricting their study to empirical research, they do not tabulate philosophy-theory articles). Their study shows how editorial practices vary across fields as represented by publications in seven journals: case studies dominate overall in IR journals, take second place overall in Comparative, and all but disappear in American; statistics dominates in American and Comparative and takes second place overall in IR, although it appears to be challenging case studies for first place in IR and it was itself challenged by case studies in Comparative in 1998; modeling is in second place in American and third place in IR and Comparative, although in 1985 it challenged both statistics in IR and case studies in Comparative for second place (see Bennett et al. 383, Tables 3–5).
What are the implications of their study for disciplinary practices? Tables 6–8 (Bennett et al. 384), for example, suggest that two political scientists—one a modeler, the other a case study researcher—who have time to follow only four journals regularly and fully and who choose their readings on the basis of methods used (whether because they do not find methods other than their own trustworthy or because of familiarity tout cour) would never read the same articles: the case study researcher's top four journals are the modeler's bottom four. The modeler and a statistician would share 75% of the same reading (the former's top two and fourth are the latter's second, third, and fourth place journals); the statistician and the case study researcher would also share no common readings. In this light, we appear to be strangers occupying space under the same disciplinary roof: with theorists added to the mix, we make four boxers in a ring, each in his own corner. Imagine, then, the implications of such reading habits for course design: assuming that each of these four political scientists could plausibly be asked to design and teach an introductory overview course at either the undergraduate or the graduate level, what, substantively, of the latest research from journal articles would reach their students?
It is, I think, less difficult to imagine the implications of these editorial practices for the career needs of junior faculty (and I think the same could be said for doctoral students and senior faculty, albeit with different focuses). A “qualitative” researcher writing in some area of American government and wanting to maximize the chance of acceptance could submit papers to but one of these 10 journals (PSQ). If he wanted to develop his research at the intersection of his methodological and substantive concerns, he would have little to read in this journal collection: the journals publishing American work are dominated by statistics and modeling, and those publishing “qualitative” or case study work are dominated by Comparative and IR.
In sum, political scientists apparently come together at APSA meetings, but only in spatial terms. And even that shared space is not reflected in the pages of the longstanding journal of the Association: if these data are accurate reflections of disciplinary practices, we do not speak to (or, by implication, read) each other across scholarship divides. Is this the political science that we want?
Disciplinary practices are established and maintained not only through editorial decisions. Curricular decisions are, arguably, novice members' first points of encounter with the discipline, and Bennett et al.'s course data reflect the same allocation of space they found in the journals: statistics comes out on top (55% of the total number of topic-specific courses), being offered in all 30 departments studied, then modeling (29%), then “qualitative” (16%), with each of the latter offered in about the same number of departments (21 and 20, respectively). However, quantitative methods is required by 20 departments, but modeling and qualitative methods are each required by only two (and made available as options by five and eight, respectively).
Schwartz-Shea's study of doctoral program curricula provides more extensive data, covering nearly twice the number of programs (57 vs. 30), and contextualizes course listings in course and program descriptions. Whereas it might seem “selfevident” that scholars would gather, literally or virtually, in corridors and on journal pages with others pursuing similar interests, when a discipline is being constituted—that is, when newcomers are being socialized or acculturated to a practice that has some substantive coherence beyond its nominal designation —one might expect a weightier centrality or coherence in a commons than would be found in a collection of neighborhoods, some shared discourse that would support a claim to “disciplinarity.” Schwartz-Shea's study shows that as it is practiced, political science has no substantive discursive core: the only experience shared by doctoral students, the initiates into this practice, across all fields within a department and across all departments, is a course in quantitative methods. That is the only course taught in all 57 doctoral programs. (By contrast, according to Table 3, fewer than half offer a “qualitative” methods course, but 72% offer a game/formal theory course and 50% offer three of them.)
Interpretive analysis would read this practice of the quantitative commons as a text (or “text analogue”; Taylor 1971) embodying, in symbolic representation, our collective statement about what we value as a discipline. The commons has shifted. Qualitative research by such authors as Selznick, Blau, Kaufman, Crozier, Whyte—even Allison—once constituted central works in American government and its subfields of public policy and public administration, and Allison's work is, or was, read in Comparative and IR. Those and other similar works enable a shared discourse about political and governmental issues in sociopolitical contexts with their attendant values and cultural meanings. If statistics is all we can agree on as a common core, then our conversation is greatly circumscribed —I would say diminished —and it is no wonder that it is historians, rather than political scientists, who step to the fore to remind Congress of its Constitutional responsibility for declarations of war (as happened this past fall). We seem to be more EU than U.S., with APSA meetings our Brussels. Perhaps, given the burgeoning size of the annual conference and the proliferation of organized sections and conference-related groups, having marked our centennial we should disband, each discursive field constituting its own association.
I have said little to argue for the vantage point that is my raison d'etre in this group of commentators—that is, as an interpretive researcher—because I am more troubled by the implications of these essays for the state of the union. Let me speak now with this voice. Bennett et al. reach the conclusion that “qualitative” methods courses need to be offered more. But this statement constitutes a leap of faith (Schön 1985) in light of their preceding conclusion which is based on the observation that the journal data report a “relative stabilization” of methods use (Bennett et al. 385). Stability can be its own justification; it does not, of itself, call for change. One has to appeal to ancient disciplinary history (the 1960s and earlier) and the lost presence of the qualitative work that was in evidence then, or to some philosophical or ideological principle of pedagogical diversity and disciplinary identity, to make an argument for upsetting this stasis by teaching—or publishing— more “qualitative” research. That is, these two essays reflect things as they are—here is observational research presenting data accessed from various “agency” documents, describing the state of the discipline as practiced in the field—positioned against calls for some arguably more desirable state. But the latter does not emerge from the data—unless we consider these to be participant-observer studies conducted by researchers of a community in which they themselves are members, and we see in the call for change that community's local knowledge as articulated by these scholar-members.
Should we make that leap of faith along with these authors, we must then confront the problem with the “qualitative” category noted by Schwartz-Shea: it doesn't tell what kind of research is actually being published. “Case study” is not isomorphic as a category with “qualitative methods.” Traditional qualitative methods—in situ on-the-hoof participant observer or ethnographic field studies—which take off from constructivist ontological presuppositions and interpretive epistemological ones proceed according and adhere to different standards and criteria for trustworthiness than qualitative studies adopting “large Ôn'” criteria. Qualitative methods researchers of all sorts need to be much more explicit in describing what it is that they do, not only for taxonomic purposes but even more for educational ones: to elucidate the differences between positivist-informed qualitative research and constructivistinterpretive qualitative research, and to show why and how the latter is scientific (systematic and attitudinally testable), although in a broader sense of “science” than that defined by the positivist philosophical principles that undergird statistical and modeling analyses.
Moreover, if this is the political science we want, we will have to prosecute these changes not only in graduate and undergraduate curricular and editorial practices, but also in hiring, conference section, textbook, and other practices through which disciplines are constituted and renewed across generations. We will have to pay more attention to the language we use—to our explicit rhetoric, to be sure, but also to such seemingly innocuous things as course, position, section, journal, and textbook titles, so that “Political Methodology” encompasses the breadth that it denotes, rather than its present narrow focus on statistics and modeling. Statistics may be widely used in political science, but it is not the only method, nor is it “the best” overall. Its hegemonic status, documented in these two essays, may well have driven out other work and other researchers, who instead publish and give papers, and perhaps even teach, elsewhere than in the “top Ôn'” journals and departments. If what makes those “top” is their fit with an image in which statistics is the core, then their election has been part of a self-sealing circle. To call for more “qualitative” methods courses, then, in the end is not just to make a leap of faith under present conditions or to “add more and stir.” It is to seek to alter the disciplinary image itself fundamentally, including rethinking and redrawing the commons.