Graduate Training in Methodology

Training to use formal models takes time and practice. Even with a background in mathematics, a first course only enables one to solve simple models. Multiple courses are needed because there is a lot to learn. Peregrine Schwartz-Shea's argument that difficulty of material drives the need for multiple courses in formal models or statistical methods strikes me as odd. Mathematics is about the precise use of language and logical argument. Numbers themselves are just a precise way to discuss quantities; “42” is more precise than “many” or “a whole bunch.” This precision of language and argument requires careful study to gain its benefits, even for those like myself with training in mathematics. The effective use of models in research demands the ability to solve the model, write a formal proof, and interpret the results. These skills can only be developed through practice. Although some of the literature using formal models can and is taught in substantive courses, a general seminar discussion rarely delves into the details a specialist needs to master the use of formal models.

The demands of in-depth training to master formal models and statistics pose a problem for the creation of multi-method experts. Each of those methods requires several courses for the student to develop expertise in techniques and practical issues in their use. Although I received no formal training in qualitative methodologies, I assume that they also require study and practice to employ them skillfully. When can students take substantive courses if they must take two to four courses in each of two or more methodologies? This is not an idle question; those of us who teach modeling in my department face this issue directly as we develop a program to teach formal modeling. Do we create a sequence of courses solely about modeling or a single course on modeling followed by a selection of substantive courses focused on the use of models in different sub-fields of political science? Because students interested in modeling often also wish to learn statistical methods, the former approach could consume as many as six of the 14 graduate courses students take. The latter approach must choose between technical courses of little interest to those who do not wish to model and broad courses that may fail to teach technical skills to modeling students.

The alternative is a general course that seeks to cover key issues in philosophy of science and the range of methodologies used in political science. I can only speak with authority about formal models and statistics, but I doubt such a general course can teach the ability to read papers critically using those methodologies, much less teach them in any depth. Such a course can teach the broad philosophical issues underlying the range of methods used in political science; the scope and methods class I took from the late William Riker at Rochester certainly did. I favor such a course, but we must realize that it alone will not train students in multiple methods.

If students are to learn the full range of methods, they will not be able to take a depth of courses in any. The field will have less methodological diversity because all students will be forced towards a shallow understanding of all methods. I doubt such a curriculum could produce a student able to use formal models carefully and thoughtfully. I believe a commitment to diversity of methods is why my own department, at the University of Michigan, is one of seven out of the top ten departments that has no required courses at all. We trust our students to judge for themselves which methods they need to learn to pursue their substantive interests.²

A standard graduate curriculum across all departments that emphasized training students in a diversity of methods would reduce the diversity of methods used in the field. Students should be able to choose programs that specialize in particular methods and gain the depth of training needed to master those methods. I chose Rochester for graduate school because I wanted to use formal models in my research. Specialization in training within particular departments increases diversity in methods training across the field as a whole.

Diversity in Journals

Bennett et al. provide statistics that different journals specialize in papers that use particular methodologies and worry about two consequences of this specialization. First, they fear we are “dining at separate tables,” by only reading papers that use the methods that we know best. Second, they are concerned that papers that use more than one methodology may be more difficult to publish than those using a single method because the top journals specialize in only one of those methodologies. I share their concern about both of these possibilities, but do not believe that methodological specialization of journals is the root of the problem. Ending this specialization would be worse than what we have now.

If we look across the full set of journals, we see that the field publishes articles of all methodologies. The 10 journals Bennett et al. survey are available in most university libraries, and nine of the 10 are available through JSTOR (Comparative Political Studies is the exception). Additionally, JSTOR offers another 15 political science journals, including such methodologically narrow journals as International Security. The opportunity to read broadly across methodologies is readily available. If we do not take that opportunity, we have no one to blame but ourselves.

One could respond that even with all these journals available, we all tend to subscribe to just the journals that share our methodological orientation, and we are most likely to read papers in the journals we receive. I do not doubt this, but would the situation improve if all journals covered the full range of methods? Only if we read all the articles in every journal we receive. The key to methodological openness is the willingness to read outside one's methodological training. Those without that willingness will not read outside their method; those with it seek out pieces using other methods.

There are advantages to journals specialized by methodology. Specialization allows readers to find articles using a certain method and authors to reach particular audiences. I have published pieces in journals that do not specialize in either formal models or statistics because those journals give me access to an audience that otherwise might not see my work. I can tailor those articles to reach that audience on their own terms. Speaking across audiences requires the willingness to meet others at least halfway. I could not be certain that such crossover pieces would reach the intended audience if all journals covered all methods. Specialization of journals and their audiences allows authors to target new audiences.

Journals specialized by methodology might also cause those papers that use multiple methodologies to “fall through the cracks” as they would have no obvious outlet. Referees might judge such papers unacceptable for a top journal because they focus solely on the standard for the preferred methodology of that journal at the expense of the added value of multiple methods. I can speak from personal experience on this matter; it was difficult at first to publish papers that combined formal models and statistics because some referees were suspicious of one or the other method or felt that their “half of the paper” was not “up to snuff.” In practice, I suspect this is less of a concern. It is hard to write a good paper using multiple methods and stay within the page limits of our journals. Good multimethod papers are two good 15-page papers within one 30-page paper. Often, it is better to separate the two and develop each of the parts sufficiently to meet the standards of each methodology.³

The range of literature across our journals reflects the diversity of methods in political science, even if individual journals do not. Specialization in journals can aid diversity in the literature. The key to crossing methodological boundaries is the willingness to read and write for audiences that use other methods.

The Special Case of the APSR

The APSR holds a special place in our field; it is the research journal of our national association. As such, it should publish the best research in political science in order to give that work the widest possible audience. However, as Bennett et al.'s summary statistics show, APSR articles rarely utilize some widely-used methodologies (e.g., case study). Why is this be so? There are many possible explanations, and I do not pretend to have the sole and complete one. I merely wish to present some possibilities.

First and foremost, a journal can only publish articles that are submitted to it. Bennett et al. cite the Editor's Report for 1999–2000 (Finifter 2000) showing that the APSR received very few submissions that used certain methodologies during the period 1996–2000. The APSR cannot publish papers using certain methodologies if authors do not submit such papers. Speaking as a member of the Editorial Board of the APSR, I urge all political scientists to submit their best papers to the Review. Methodological diversity in the APSR will not occur if papers using certain methods are never submitted.

Even given a full spectrum of papers submitted, I do not believe that we should expect the proportion of articles in the Review using each methodology to exactly match the frequency that each method is used in the field as a whole. Political science is a diverse field, and different research communities focus their research output on different forms of publication. Some communities emphasize books as the most important vehicle for communicating the results of their research; others focus on journal articles. Formal theorists rely almost entirely on journals to communicate their research and so direct their best ideas and efforts into journal submissions rather than book manuscripts. Understandably, those in communities that emphasize books direct their best efforts toward projects best developed and communicated in books. Even if all communities produce equal amounts of quality research, different emphases on books versus journals will lead to different levels of quality in the pools of papers submitted to journals.

It is also possible that acceptance rates vary across methodologies. The review process of the APSR is extremely demanding. Referees insist on a higher level of quality than they do for other journals, and acceptance requires agreement among three or more referees that the paper is of the highest quality. Consensus among the referees is more likely when they can agree on exactly what the paper argues. Proponents of formal methods argue that models impose transparency of argument on the author (e.g. Fiorina 1975, 136–39). Transparency aids cumulation because technically trained readers will agree about the argument's assumptions, logic, and conclusions and can see how it fits with earlier research. This is not to say that informal arguments are necessarily imprecise, only that models impose a certain discipline upon the author. Consequently, technically trained readers of a paper with a formal model are more likely to agree on the soundness of the argument and its importance within the broader literature. They are more likely to agree on which papers meet the high standard of the Review and which do not. The transparency of formal models may increase the agreement among referees, making the editor's decision clearer.

Again, I am not suggesting that these factors are the only possible explanation for the underrepresentation of certain methods in the APSR. However, speculation that editors may have favored some methods over others (see Bennett et al., fn. 4, 386) seems premature. It would be a sad day if the high standards of the Review were compromised to force a particular mix of methods in the articles published.

Methodological tolerance and diversity falls on our own shoulders, not on those of journals or graduate programs. Diversity in methods can be found across our journals and graduate programs. Forcing uniformity in methodological diversity in both would reduce the diversity of methods in the field as a whole. It takes effort to read across methods and to write for audiences that use other methods. Communication across research communities calls for a willingness of all sides to meet in the middle, rather than insisting on debate on their own terms.4 It also calls for a willingness to accept substantive criticism of ideas from those who use other methods as something other than methodological “trash talk” or “lack of respect.” If we wish to avoid “dining at separate tables,” each of us needs to make the effort to push away from our own table.