%20-%20https://sketch.com%20--%3e%3ctitle%3e3ECBBF50-FD08-412A-943D-2CFC5E025A16%3c/title%3e%3cdesc%3eCreated%20with%20sketchtool.%3c/desc%3e%3cg%20id='Stylesheet'%20stroke='none'%20stroke-width='1'%20fill='none'%20fill-rule='evenodd'%3e%3cg%20transform='translate(-926.000000,%20-668.000000)'%20id='BUTTONS'%3e%3cg%20transform='translate(100.000000,%20597.000000)'%3e%3cg%20id='BTN-/-RIGHTS'%20transform='translate(816.000000,%2065.000000)'%3e%3cg%20id='ICON-/-Rights'%20transform='translate(10.000000,%206.000000)'%3e%3cpath%20d='M13.5752841,0%20L1.34943182,0%20C0.59375,0%200,0.586666667%200,1.33333333%20L0,22.6666667%20C0,23.4133333%200.620738636,24%201.34943182,24%20L17.6505682,24%20C18.40625,24%2019,23.3866667%2019,22.6666667%20L19,5.36%20L13.5752841,0%20Z%20M13.5752841,1.89333333%20L17.0838068,5.36%20L13.5752841,5.36%20L13.5752841,1.89333333%20Z%20M1.34943182,22.6666667%20L1.34943182,1.33333333%20L12.1988636,1.33333333%20L12.1988636,5.36%20C12.1988636,6.10666667%2012.7926136,6.69333333%2013.5482955,6.69333333%20L17.6235795,6.69333333%20L17.6505682,22.6666667%20L1.34943182,22.6666667%20Z'%20id='Shape'%20fill='%23595959'%20fill-rule='nonzero'%3e%3c/path%3e%3ccircle%20id='Oval'%20stroke='%23595959'%20stroke-width='1.5'%20fill='%23595959'%20cx='9.5'%20cy='13.5355339'%20r='5.5'%3e%3c/circle%3e%3cpolyline%20id='Path-2'%20stroke='%23FFFFFF'%20stroke-width='1.5'%20transform='translate(9.500000,%2013.035534)%20rotate(45.000000)%20translate(-9.500000,%20-13.035534)%20'%20points='8%2016.0355339%2011%2016.0355339%2011%2010.0355339'%3e%3c/polyline%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e)
“For the first time, I felt that I was part of a community, rather than someone doing interpretive work in relative isolation…. I learned that in fact there is a thriving community of interpretive researchers … with a history, with senior scholars from whom I can learn and fellow students with whom I can work.”
“I really got the sense that there are other people in the discipline trying to organize a space for the discussion of interpretation and interpretive methods/methodology in political science. For that, I am deeply grateful. It is good to feel like you are a part of some sort of disciplinary movement.”
—Evaluations (excerpted), 2009 NSF Workshop on Interpretive Methodologies in Political ScienceOne of the primary concerns driving Perestroika was the hegemony of quantitative methods in American political science research, curricula, journals, and positions, to the exclusion of qualitative and interpretive approaches. In this article, we assess the contemporary methodological diversity of U.S. political science, at the APSA in particular, to see what, if anything, has changed over the last 10 years. This is an admittedly rough assessment, as the deadline for this symposium did not allow time to repeat the research projects that started Perestroika's and our own solo and joint efforts, the latter preceding and then intersecting with the former. We therefore give a broad overview of methods-directed activities, although we cannot help but see events through the lens of our own involvement in them, and that view is perforce partial.
In short, our back of the envelope “calculations” suggest some substantial changes in the discipline for the better. However, whether these changes can be sustained in the face of contemporary trends affecting higher education—specifically, the growing research regulation climate that is already impinging on field research—is in doubt. In the pages that follow, we review our evidence and explain our concerns about the future. Given time and space constraints, we must be schematic and brief. We therefore begin with “raw data” in tabular form: a timeline of events and activities circling around methodological matters tied to Perestroika's beginnings. (Our timeline ignores longstanding qualitative-interpretive traditions inside and outside of political science.)
A METHODS-ORIENTED TIMELINE
• 1999
◦ March: Western Political Science Association Annual Meeting (WPSA), Seattle. Data collection started for study of methods textbooks.
◦ Spring: Research design crafted for curricular study.
• 2000
◦ March 24–26: Peregrine Schwartz-Shea and Dvora Yanow, “Is There Breadth in Methodological Training for Graduate Students in Political Science?” WPSA Annual Meeting, San Jose.
◦ October 15: Founding Perestroika e-mail sent.
• 2000–01
◦ Yale doctoral student agitation for qualitative methods instruction within the political science department.
• 2001
◦ April: APSA Council approves proposal to create Perspectives on Politics, intended to better reflect the discipline's methodological diversity.
◦ September: First Perestroika organizational meeting and two Perestroika roundtables—“Political Science Methodology and Perestroika,” Peregrine Schwartz-Shea (chair), Hayward Alker, Alexandra Cole, Kathy Ferguson, Melissa Harris-Lacewell, and Meredith Weiss; and “Shaking Things Up: Future Directions in Political Science,” Kristen Monroe (chair), Russell Hardin, Robert Jervis, Elinor Ostrom, Susanne Rudolph, Marion Smiley, and Rogers M. Smith, APSA Annual Meeting, San Francisco.
• 2002
◦ January: First session of the Training Institute on Qualitative Methods (today, Institute for Qualitative and Multi-Method Research).
◦ March: “Methods, Methodology, and Perestroika: Qualitative versus Interpretive Methods” roundtable, Peregrine Schwartz-Shea (chair), Hayward Alker, Kathy Ferguson, Kamal Sadiq, Meredith Weiss, and Dvora Yanow, WPSA Annual Meeting, Long Beach, CA.
◦ April: “Is American Politics Fixated on Quantitative Methods? The View from Perestroika” roundtable, Peregrine Schwartz-Shea (chair), Georgia Duerst-Lahti, Mark Graber, and Dorian Warren, Midwest Political Science Association (MPSA) Annual Meeting, Chicago.
◦ June: Peregrine Schwartz-Shea and Dvora Yanow, “‘Reading’ ‘Methods’ ‘Texts’: How Research Methods Texts Construct Political Science,” Political Research Quarterly 55 (2): 457–86.
• 2003
◦ March 29: “Interpretive Research Methods in Empirical Political Science Workshop,” Dvora Yanow and Peregrine Schwartz-Shea, co-organizers, WPSA Annual Meeting, Denver.
◦ Spring: Qualitative Methods Newsletter begins (http://maxwell.syr.edu/moynihan/cqrm/Newsletters/).
◦ July: Peregrine Schwartz-Shea and Andrew Bennett, “Symposium Introduction—Methodological Pluralism in Journals and Graduate Education? Commentaries on New Evidence,” PS: Political Science and Politics 36 (3): 371–72; Peregrine Schwartz-Shea, “Is This the Curriculum We Want? Doctoral Requirements and Offerings in Methods and Methodology,” PS: Political Science and Politics 36 (3): 379–86.
◦ September: First panels of Qualitative Methods (later Qualitative and Multi-Method Research) organized section; also, “Assessing the Perestroika Movement in Political Science” roundtable, Kristen Monroe (chair), Leslie E. Anderson, Samuel H. Beer, Martin O. Heisler, Timothy Luke, Kamal Sadiq, Sanford Schram, Peregrine Schwartz-Shea, and Joanna Scott, APSA Annual Meeting, Philadelphia.
◦ Fall: Interpretation and Methods (I&M) listserv starts (http://listserv.cddc.vt.edu/mailman/listinfo/interpretationandmethods).
• 2004
◦ Henry E. Brady and David Collier, eds, Rethinking Social Inquiry: Diverse Tools, Shared Standards (Lanham, MD: Rowman & Littlefield).
◦ September: “Making Political Science Matter” roundtable, Dvora Yanow (chair), Mary Hawkesworth, Timothy Luke, Ido Oren, and Peregrine Schwartz-Shea, APSA Annual Meeting, Chicago.
• 2005
◦ Interpretation and Methods section founded, WPSA.
◦ First Methods Café held at the WPSA Annual Meeting, Oakland, CA, Dvora Yanow and Peregrine Schwartz-Shea, co-organizers.
◦ First “Graduate Student Workshop on Interpretive and Relational Research Methods and Methodologies,” Patrick Jackson, organizer, International Studies Association–Northeast, Philadelphia.
◦ Alexander L. George and Andrew Bennett, Case Studies and Theory Development in the Social Sciences (Cambridge, MA: MIT Press).
◦ Kristen Renwick Monroe, ed, Perestroika! The Raucous Rebellion in Political Science (New Haven: Yale University Press).
• 2006
◦ First Methods Café held at the APSA Annual Meeting, Philadelphia, Dvora Yanow and Peregrine Schwartz-Shea, co-organizers.
◦ Gary Goertz, Social Science Concepts: A User's Guide (Princeton: Princeton University Press).
◦ Mary E. Hawkesworth, Feminist Inquiry: From Political Conviction to Methodological Innovation (New Brunswick, NJ: Rutgers University Press).
◦ Sanford Schram and Brian Caterino, eds, Making Political Science Matter (New York: New York University Press).
◦ Dvora Yanow and Peregrine Schwartz-Shea, eds, Interpretation and Method: Empirical Research Methods and the Interpretive Turn (Armonk, NY: M.E. Sharpe).
• 2007
◦ John Gerring, Case Study Research: Principles and Practices (Cambridge: Cambridge University Press).
◦ Audie Klotz and Cecelia Lynch, Strategies for Research in Constructivist International Relations (Armonk, NY: M.E. Sharpe).
• 2009
◦ National Science Foundation Workshop on Interpretive Methodologies in Political Science (http://www.ipia.utah.edu/imps).
◦ First panels, Interpretive Methods and Methodology Conference Group, APSA Annual Meeting, Toronto.
◦ Edward Schatz, ed, Political Ethnography: What Immersion Contributes to the Study of Power (Chicago: University of Chicago Press).
Much of Perestroika's energy was focused on getting disciplinary institutions—curricula, journals, association sections, and faculty appointments—to adopt a more pluralist approach to political science. The findings of the July 2003 PS symposium (Schwartz-Shea and Bennett Reference Schwartz-Shea and Bennett2003) provided systematic evidence in support of Perestroika's claims that the discipline, as seen through its teaching and journals, was in large measure not open to research questions answerable through other than quantitative methods. We do not know the extent to which the Qualitative Methods (now QMMR) section's creation was sparked or nurtured by Perestroika, but it clearly speaks to Perestroika's concerns. The section doubled from six panels/roundtables in 2003 to 12 in 2004, growing to 30 at its highest point in 2007 and maintaining 26 or 27 in other years. The lists of section events and Perestroika panels at APSA and other meetings between 2001 and 2004—the last year that we find designated Perestroika panels on conference programs—demonstrate both Perestroika's methods focus and the extent to which QMMR took over that energy with its debut in 2003. The section has clearly sought to carve out space within the discipline for “qualitative” methods—successfully, in our view, especially within the comparative politics subfield.
QMMR has been supportive of interpretive panels and other efforts, although its panels and newsletter articles have also at times succeeded—unintentionally, we believe—in highlighting the ways in which the section's dominant understanding of “qualitative” methods diverges from both more traditional Chicago School work and more recently developed varieties of such methods (on the distinctions, see Yanow Reference Yanow2003). The 2009 National Science Foundation (NSF) Workshop on Interpretive Methods in Political Science, which we codirected, sought to engage interpretive forms of research—meaning-focused archival, interview-based, and/or participant-observer/ethnographic, informed by a pragmatist-abductive, sometimes critical-theoretical, phenomenological hermeneuticsFootnote 1—some form of which has been carried on by many within Perestroika. The funding of the workshop by NSF and the response it generated—135 applications for 35 spots—constitute, together with other activities listed, concrete indicators of the greater presence of methodological diversity across the discipline since 2001, although the extent of that increase is difficult to assess with any precision.Footnote 2
These would seem like successes. Wherein, then, lie our concerns?
DARK CLOUDS OVERHEAD
Journals
The timeline's selective list of book publications demonstrates the depth of work in qualitative and interpretive research, a strength that is not being matched by the poor record of methodological inclusivity, diversity or pluralism in the APSA's flagship and other journals. The APSR group at UCLA held promise, but despite more pluralistic editorial board memberships (e.g., Jim Scott) and the work of two key editors, the track record has not much changed (see Kasza Reference Kasza2010, this issue). Perspectives on Politics, the creation of which Hochschild (Reference Hochschild and Monroe2005, 336), its first editor, attributes in part to Perestroika, seems to be enjoying success as an outlet for work that speaks broadly to the political, although it is not clear how publication in its pages is evaluated by tenuring committees.
Events at other journals do not bode well for qualitative or interpretive research. There is a renewed push toward replication in ways that are not accommodating of interpretive and much qualitative research. In a January 6, 2010, e-mail to the listserv of the Political Methodology section's membership, American Journal of Political Science (AJPS) editor Rick Wilson introduced several changes to the manuscript submission process, including the requirement that manuscripts accepted for publication must include in the first footnote an explicit statement concerning “where the data used in the study can be obtained for purposes of replication.” Although the e-mail also stated that studies not based on data would not require such a statement, interpretive and qualitative projects—while data-based—often rely on field-generated data that are not amenable to being stored in a way that would facilitate another researcher's access, even if replicability were not problematized—which it often is.Footnote 3
Wilson's further statement shows clearly that AJPS is not thinking about publishing field research:
I am encouraging authors to submit Supporting Information (SI) with the manuscript. Material included in the SI … might be a set of proofs or extensions to a formal theory, it might include computer syntax for coding variables, building simulations or generating estimates and it might include additional estimates and robustness checks.
While we appreciate the intended effort “to provide reviewers with additional material in order to make an informed judgment,” we are saddened, in a world of supposedly general-purpose political science outlets, that AJPS continues to exclude the full range of research conducted throughout the discipline—a state of affairs that does not characterize its sponsoring association, the MPSA, which has added sections on political anthropology, political sociology, and the like.
Research Regulation
Because of their implicit experimental research design model, institutional review boards (IRBs) in many places have had deleterious effects on much qualitative and interpretive research, as many scholars have noted. Even with the radical variability of IRBs by institution, we can still say that many board members are often ignorant of the methodological demands of such research, requiring researchers instead to conform to experimental procedures (e.g., Katz Reference Katz2004; Yanow and Schwartz-Shea Reference Yanow and Schwartz-Shea2008). These demands become another way of restricting political research to its more hegemonic aspects.
New Public Management and Productivity Measures
In the United Kingdom, European Union, Canada, and Australia, various output measures have been developed for ranking departments, faculties, universities, and professors. In combining the orientations of “evidence-based management” and the “new public management,” these tools are often hostile to interpretive and qualitative research. These measures promise an end to the book-length dissertations that are key to such research, being more favorable to the submission of three journal-length manuscripts (an application of a longstanding mathematics custom to the social sciences) and potentially exerting pressure against book publishing itself (see also Scott and Light Reference Scott and Light2003).
Program and university rankings are a familiar part of higher education, but whatever one's perspective on rankings, Academic Analytics (AA), a for-profit U.S. firm founded in 2005, constitutes a new, disturbing step in the direction of measuring and monitoring scholarly activity. Increasingly used by deans and other administrators on U.S. campuses, AA claims to offer the “only objective measure of faculty productivity” (emphasis in original; http://academicanalytics.com/). In contrast to the transparency of the National Research Council (NRC), which provides, online without charge, a 200-page monograph on its methods for producing rankings, only those universities that pay AA can obtain access to the detailed methods by which it produces its own program rankings. These rankings are based on “national standards within the particular discipline” (http://academicanalytics.com/About/AboutUs.aspx). A Chronicle of Higher Education (2007) Web page describes AA's general methods, but a description of specific standards, such as the journals used in producing program rankings by discipline, is nowhere to be found.
AA proclaims itself to be the “only completely quantitative measure of academic quality available to universities and other academic institutions,” as if that were a self-evidently desirable quality. The taken-for-granted nature of rankings, impact factors, and other such measures makes it difficult for faculty to ask fundamental questions about AA-approved journals, particularly concerning whether they accommodate qualitative and interpretive research. This lack of information might well impact researchers' decisions about taking on long-term research projects—such as those entailing extended fieldwork—when such research would mean publishing less in the short term, impacting merit raises or risking being assigned higher teaching loads if “productivity” is impaired. Decisions about which research projects to pursue are increasingly subject to such evaluations, something reported by colleagues in the United Kingdom who are “enjoying” the RAE (Research Academic Exercise) and its successor, the REF (Research Excellence Framework), as well as in the EU, where publication credit is restricted to A-list journals as determined by each faculty (or, at times, the national discipline group). Such measures (and the reporting requirements that accompany them) do not serve social science knowledge and are particularly inimical to qualitative and interpretive research, with its longer timelines for both research and writing. Such standardization at increasingly higher levels of aggregation brings to mind the banking industry's recent history of “credit default swaps”—instruments unmoored from the daily realities of human actors.
The story of a heterodox department of economics at the University of Notre Dame serves as an example of the ways in which “national standards,” such as those created by AA, can be used to discipline researchers who choose to pursue particular topics or methodologies and to publish in journals that do not contribute to the measures used to calculate program rankings (Donovan Reference Donovan2004). Administrators intent on improving the department's NRC rankings split it in two, between Economics and Econometrics, and Economics and Policy Studies. Only the former was allowed to train graduate students (Glenn Reference Glenn2009). By March 2010, Notre Dame had voted to dissolve the latter department, home to the heterodox faculty.
Methodological Legitimacy and Disciplinary Anxieties
Despite the Qualitative Methods section's notable successes, some members apparently felt that non-quantitative methods had not garnered sufficient legitimacy within the discipline, to the point that the section's leadership organized a name change in 2007, adding “Mixed Methods” to the title. This shift fits Brady and Collier's argument (Reference Brady and Collier2004) that qualitative and quantitative methods share the same evaluative standards. In this light, the name change made methodological and pragmatic political sense. It has not resolved the legitimacy question, however: as our earlier reference to AJPS requirements shows, qualitative (and interpretive) researchers continue to be challenged by those prosecuting research based on experimentalist criteria.
IN SUM
Has Perestroika impacted methodological pluralism in U.S. political science, as seen in its journals and departmental curricular requirements? On the positive side, new qualitative methods courses are being offered, and one department has even added a field exam in qualitative methods (although interpretive research is marginalized in it, at best). Yet Kasza's analysis suggests a more negative outlook, and we continue to hear from graduate students who are not able to find sufficient support in their home departments for their qualitative-interpretive projects.
By providing a forum where like-minded “pluralists” could find one another, Perestroika facilitated momentum on the methods/methodological front, along with interesting discussions and debates. But the discipline has not gone far enough, and Perestroika's voice is still needed. A movement with great verve, Perestroika hinges on both technology and the humans operating it. Without the virtual community glue provided by the listserv while its organizer regrouped over the last five years, both communication and organizing efforts foundered. As a consequence, the movement has lost out on a generation of graduate students who, like those in the epigraphs, may also have thought that they were working alone. We would support the creation of Perestroika II in a fashion that is more democratic, following Perestroika's own calls for such governance in the discipline, rather than continuing a regime subject to the decision making of a single, unrepresentational individual. The methodological status of the discipline needs a strong Perestroika still!
