Language and Mathematics

First, I would argue that the raw data are misleading when one is comparing quantitative to qualitative course offerings because the units are not directly comparable. Schwartz-Shea argues that the claim that “Ômath is hard'…constitutes insufficient grounds for the imbalance in requirements and offerings of the curricular status quo” (Schwartz-Shea 378). This complaint conflates the substance of a quantitative methodology course (econometrics) with the language in which it is taught (math). The difficulty of math is not the issue; the extra time required to teach both a method and the language of instruction is.

To appreciate this point, imagine that all instruction and research in qualitative methods had to be conducted in Latin. No one would countenance a separate Latin course, since Latin has little to do with political science per se and would therefore alienate students, so Latin would have to be taught alongside methods. Methods that could be taught in one semester would require two or three due to the burden of an inherent language requirement. To those who desire more superficial instruction in English, the committed qualitative methodologist would reply, “Nil sine magno vita labore dedit mortalibus.”

How much of a quantitative methods class is math and how much is method? Answers vary widely. It is perhaps indicative that ordinary least squares regression typically constitutes the culmination of a semester's worth of hard work on a variety of topics (data distributions, probability distributions, statistical sampling, point and range estimation, etc.) that serve only to make econometrics possible. If a semester of math is required before students are competent to derive “the formula for a straight line,” (Schwartz-Shea 378) as Schwartz-Shea succinctly puts it, the ratio of math to method must be very high.

If the numbers of qualitative and quantitative courses are not directly comparable, how then should the merits of additional classes in either be evaluated? Probably by the number of students whose work would improve and the degree to which their work would benefit.

These quantities are more difficult to measure, of course—but as my qualitative colleagues would no doubt agree, quantities of interest should take precedence over quantities of convenience.

The Dearth of Pluralism

Bennett, Barth, and Rutherford may actually provide a rosier picture of methodological pluralism than the data warrant, though conclusions here must be somewhat tentative. Their conclusion, drawn from the Decade Survey and the Recent Survey, is that the proportion of multi-method articles was “very steady” (Bennett et al. 385) from 1975 to the present. Although those two surveys contain the broadest sample of journals, their coverage over time is rather sparse. It is instructive to examine the APSR data, kindly provided by the authors, to get a clearer sense of trends since 1965. The APSR is, as the authors point out, not the most representative source available, but the APSR data provide a longer and more thorough time-series than the basic survey. Because the issue is changes over time in the prevalence of multi-method work, the latter consideration seems more important, though the fact that we are trading both breadth and unbiasedness for depth should not be forgotten.

Figure 1 summarizes the number of articles in the APSR that combine two of the three methodologies by year, organized by combination (row) and subfield (column); the data are smoothed to emphasize central tendency. Two generalizations are immediately evident. The first is that the combination of formal and statistical methods is, as Bennett, Barth, and Rutherford noted, the most popular of the three. The second is that, in the APSR at least, very few encouraging trends for fans of multi-method research are in evidence. Most trends, if they are changing at all, are in decline. Even in American politics, a decline in formal/statistical work seems to have taken place in the 1990s (though the variance is high enough that inference is rather uncertain). Only formal/statistical work in international relations appears to be steadily, if slowly, on the rise. The fact that apparent trends are occasionally based on two or three articles (statistical/case in IR, for example, or formal/case in American) underscores both the caution with which conclusions should be drawn and the need for further investigation—including, ideally, a broader timeseries of a more representative sample of journals.

Still, the kinds of clear upward trends that would ideally be in evidence if the field were growing increasingly pluralistic are simply not there. What we do find, for most subfields and most combinations of methods, are zeroes, and lots of them.

What Price Pluralism?

Bennett, Barth, and Rutherford argue that scholars should be well-versed in all three methods, especially given that a significant percentage of the research done in each subfield cannot be understood without some training in each (Bennett et al. 385). I concur heartily. Agreeing that pluralism is good is one thing; implementing it, however, raises a host of questions that are far more difficult to answer.

• What do we mean by methodological pluralism? Bennett, Barth, and Rutherford seem to desire a greater balance among the three methods they describe. Schwartz-Shea advocates a much broader form of pluralism: “no a priori methodological commitments” (Schwartz-Shea 379). The latter bears more of a resemblance to nihilism than to pluralism; it would seem to leave the door open not just to interpretivist methods but to omphaloskepsis and phrenology as guides to truth.
• What is the goal of pluralism? Bennett, Barth, and Rutherford argue for “at least a reading competency in all three methods” (Bennett et al. 386)— but what does that entail? Indeed, if a method must be understood at an operational level before research that uses it can be competently evaluated, isn't the distinction between a “reading competency” and a “working competency” a chimera?
• Why don't we already have more of it? If pluralism is a good thing, valued by a substantial segment of the discipline, why don't we see more examples? In order to change an effect we must first understand its causes. Is the absence of pluralism the result of the increasing costs of adequate specialization in a single method, of structural factors inherent in graduate education, or of the quasi-tribal group mentality that Schwartz-Shea describes? Or is there just a “silent majority” out there that believes that training in anything other than the One True Method (whichever one that is) is simply a waste of time? Evidence about the cause is less clear than evidence about the effect. As specialists in theory-testing, though, we should be able to answer this question.
• What are we willing to sacrifice for it? There is an unavoidable tradeoff between substantive knowledge, methodological pluralism, in-depth knowledge of a particular method, and time: one can only come at the expense of at least one of the others. To obtain a more pluralistic outlook, should students take fewer substantive classes, cut back on advanced methods classes, or spend more time in courses of instruction?

There are no easy answers to these questions. I have already hinted at my own thoughts on the first two, and the third is an empirical question on which I am not qualified to comment for lack of data. The fourth is the most critical and perhaps the most complex, so a more extended commentary seems warranted. In short, I would argue that trading time for pluralism is, for most students, a no-brainer; trading substantive classes for methods classes is less clear-cut; and trading depth for breadth within methods sequences is usually a bad idea.

In practice, students often trade time for either expertise or pluralism, as interest in the annual ICPSR Summer Program in Ann Arbor, the newly-founded Qualitative Methods Institute in Arizona, the NSF-funded Empirical Implications of Theoretical Models (EITM) program, the Merriam Laboratory at the University of Illinois, the Santa Fe Institute, and other training institutions clearly demonstrates. Departments are loathe to institutionalize such a tradeoff, however, usually citing time-to-degree concerns—and, perhaps, the more humanitarian urge to allow underpaid, overworked graduate students to obtain a degree in a timely manner. On balance, it seems to me that the students have the better part of this argument: the marginal benefit of exposure to an additional summer or semester of methods for most students far outweighs the cost in time. (I would even argue that an additional year spent studying methods would be worth the lost time, in terms of both breadth and depth. Will graduating in six years versus graduating in five really matter in the end?)

The trade-off between substance and method is less straightforward. There is already a sense that methods-savvy students are dangerously lacking in substantive knowledge, as Schwartz-Shea points out. The source of this perception is unclear, however. The usual claim is that students take too few substantive classes in graduate school and therefore produce substantively uninteresting work, but it seems equally, if not more, likely that their research activities (collecting data sets, perhaps, or concocting proofs) are not recognized by practitioners outside their methodological tradition as reflecting or possessing substantive content. I can imagine one hypothetical test, at least, that would help clarify the issue: Are we more prone to argue that a methods requirement would detract from students' substantive education when the method in question is not our preferred one? If so, the objection to trading substance for method sounds more like prejudice than principle.

Finally, trading methodological expertise for methodological pluralism rarely makes sense. Students considering sacrificing, say, a second semester of statistics for a first semester of game theory will probably find themselves unable to make substantial contributions using either method. Those considering sacrificing much more advanced methods classes—say, a fifth statistics class on spatial methods or Markov chain Monte Carlo techniques — are probably destined to contribute primarily to the political methodology literature and need as much training in their chosen method as possible in order to compete effectively. Only at intermediate levels, when a student is considering acquiring depth beyond what research will likely require but short of what a professional methodologist would need, would sacrificing depth for breadth be an option worth exploring.