Hostname: page-component-7b7567c8c-tnzcf Total loading time: 0 Render date: 2026-11-12T09:22:01.658Z Has data issue: false hasContentIssue false
  • English
  • Français

The State of Undergraduate Research Methods Training in Political Science

Published online by Cambridge University Press:  10 May 2005

Cameron G. Thies and
Robert E. Hogan
Cameron G. Thies
Affiliation:
Louisiana State University
Robert E. Hogan
Affiliation:
Louisiana State University
Rights & Permissions [Opens in a new window]

Extract

Debates over methodology have long occupied a prominent role in political science and its various empirical sub-fields. Recently, these debates and occasional dialogues seem to have intensified. The Perestroika movement within APSA protested the perceived hegemony of rational choice and quantitative methods in journal publications and graduate training (Kasza 2001). Renewed attention has focused on the types of methodologies employed by studies published in the discipline's leading journals (Garand and Giles 2003; Bennett, Barth, and Rutherford 2003; Braumoeller 2003). The kinds of concerns over methodological diversity that motivate these studies also inform discussions about graduate training (Alvarez 1992; Dyer 1992; Schwartz-Shea 2003; Morrow 2003; Smith 2003).

Information

Type
THE TEACHER
Information
PS: Political Science & Politics , Volume 38 , Issue 2 , April 2005, pp. 293 - 297
Copyright
© 2005 by the American Political Science Association

Introduction

Debates over methodology have long occupied a prominent role in political science and its various empirical sub-fields. Recently, these debates and occasional dialogues seem to have intensified. The Perestroika movement within APSA protested the perceived hegemony of rational choice and quantitative methods in journal publications and graduate training (Kasza 2001). Renewed attention has focused on the types of methodologies employed by studies published in the discipline's leading journals (Garand and Giles 2003; Bennett, Barth, and Rutherford 2003; Braumoeller 2003). The kinds of concerns over methodological diversity that motivate these studies also inform discussions about graduate training (Alvarez 1992; Dyer 1992; Schwartz-Shea 2003; Morrow 2003; Smith 2003).

Advocates of particular methodological approaches have even developed institutes designed to train graduate students and faculty members in methods that are thought to be under-represented in graduate curricula. The Consortium for Qualitative Research Methods (CQRM) offers an annual Training Institute at Arizona State University, as does the Institution for Social and Policy Studies at Yale University on field experiments. Both were preceded by the Inter-University Consortium for Political and Social Research's (ICPSR) summer program at the University of Michigan, launched as a result of the impact of the behavioral revolution on the social sciences. Methodologies associated with rational choice have also had their institutes, such as the now defunct Hoover Institution Summer Program in World Politics and the recently launched Empirical Implications of Theoretical Models (EITM) Summer Institutes. Qualitative Methods has now followed Political Methodology's lead to become the second APSA organized section specifically devoted to advancing methodology. Clearly, a great deal of intellectual effort is spent on ascertaining the relative status and merit of methodologies in the discipline and on attempting to alter the extent of their use through graduate training.

However, absent from this activity is any apparent concern for the impact of methodological debates on undergraduate education. There has always been discussion about the appropriate ways to teach undergraduate methods courses (Rodgers and Manrique 1992; McBride 1994, 1996), with The Political Methodologist serving as a reliable resource for advice about content and teaching strategies (Adams 2001; Clawson, Hoffman, and McCann 2001; Hojnacki 2001; Janda 2001; Lewis-Beck 2001; Stone 2001; Poggione 2001). There are also discussions of the use of the scientific method in substantive courses (Hewitt 2001; Segal 2002; Bennett 2002) and, to a lesser extent, essays explaining particular methods designed for undergraduate consumption (Odell 2001; Thies 2002). However, the high level of attention focused on the desirability of teaching different methods in graduate training and their presence in the discipline's journals has rarely been directed at our undergraduate programs.

Kim Quaile Hill's (2002) reflective essay “lamenting” the state of science education is an exception to the silence on undergraduate methods training. His essay reasons why the scientific method should be taught to political science undergraduates, and argues that undergraduates should learn what science is, how it compares to other forms of knowledge generation, what questions it can and cannot answer, and its procedures of inquiry. He also suggests that students become familiar with the state of theoretical and empirical knowledge in political science, the evolution of the scientific study of politics, the creative aspects of the research process, and how political science can be applied to the real world. We too believe that these are laudable goals for undergraduate education in political science. As Wahlke (1991, 48–49) notes in his report concerning the activities of APSA's Task Force on the Political Science Major, a liberal education requires a mastery of inquiry, including abstract, logical thinking and critical analysis, the understanding of numerical data, and science. Wahlke (1991, 51) reports that the Task Force recommends “that every political science major gain familiarity with the different assumptions, methods, and analytical approaches used by political scientists and by cognate disciplines (e.g., economics, history, psychology, law, and others).” In a recent Chronicle of Higher Education survey, 84% of respondents thought it was “important” or “very important” for colleges to teach students how to “discover more about the world through research” (Selingo 2003). An understanding of the variety of methodologies employed to make sense of the social and political world would serve all of our students well as they pursue a liberal education and move on to professional careers.

However, as Hill (2002, 116) notes, very few political science departments require systematic training in methodology, and none comes close to approaching his aforementioned goals for a curriculum. However, Hill's evidence is anecdotal in this regard. His conclusions are based on published degree requirements in the top 20 political science departments, along with those in “selected” lower-ranked departments, a review of syllabi in APSA's Course Syllabi Collection, conversations with “leading” political scientists, and the recollections of first-year graduate students in his home department concerning their own undergraduate training. While Hill's informal methodology matches our own intuition about the status of undergraduate training in research methods, we decided to undertake a more rigorous, systematic survey of the discipline.

Data and Methods

Data for this analysis come from a survey of departments identified by APSA as offering some type of undergraduate curriculum in political science. There are approximately 1,226 such programs, which APSA breaks down into categories based upon the highest degree offered: Ph.D., M.A., and B.A. programs in political science, B.A. programs in Social Science, and B.A. Combined programs (e.g., political science and history or geography). The survey was administered in late June and early July 2003, to 717 departments that had an email address listed in APSA's Directory of Political Science Faculty: 2002–2004 (2002). After sending an initial email along with a followup message a few weeks later, we received 303 completed surveys for a response rate of 42.3%.

Recent discussions of email survey research suggest a wide variation in response rates. As Sills and Song (2002, 25) note, reported response rates vary between 0 and 70% across the literature in the social sciences. In a direct comparison between email, web, and postal mail, Smee and Brennan (2000) find that email had the lowest response rate at 12.7% compared to 50% for postal mail and 61% percent for web-based questionnaires. Schaefer and Dillman (1998) found no significant difference between postal and email response rates, which in their study both came in at around 58%. As is the case with postal mail surveys, the characteristics of the sample and the subject of the study probably have significant effects on response rates. A recent APSA (2000) departmental survey using postal mail achieved a 41% response rate. Therefore, we believe that 42.3% is a respectable response rate given our choice of media, our potential respondents, and the subject of the study. The response rates are broken down by degree program in Table 1.

In addition to the general problem of response rates, the data produced by our current sample may not adequately represent the entire population of cases. Stanton (1998) and Couper (2000) point out that there are two categories of mismatches between the sample and the population to consider regarding representativeness. The first category is the proportion of the target population that can be reached via email. This should not pose a problem on its own as we can reach 58% of departments via email. However, the second category, differences in key characteristics between the proportion of the target population reached via email versus those who cannot be reached, may pose a problem for our study. There may be significant differences between the 509 departments that do not publish an email address with APSA and the 717 that do. The 509 non-email departments are likely to be very small in terms of faculty members, may be more likely to be part of combined programs, and the lack of Internet savvy may reflect an underlying predisposition against the teaching of social science. Table 2 suggests that the B.A. Combined and B.A. Social Science programs are less likely to have email addresses. Our current sample more closely represents B.A., M.A., and Ph.D. programs in political science.

In order to avoid the problem of response inconsistency induced by technical issues regarding Internet-based surveys (Stanton 1998; Couper 2000), we included the text of the survey in the body of the email and asked respondents to simply “reply with history.” We considered more complicated Internet survey techniques, but decided on a basic format that nearly all Internet users could access. The survey instrument itself is relatively short (see Appendix) and asked mostly for descriptive information about required courses on research design and research methods.

Findings

Table 3 reports the findings from the first question on the survey instrument regarding the presence of a required undergraduate course in research design or methodology. We are immediately struck by an interesting paradox: those institutions which have at their core the goal of producing original research and graduate students capable of such research are also the least likely to require any training in research methods at the undergraduate level. Only 46.8% of Ph.D.-granting institutions require such training for undergraduates. That number rises to the highest percentage when one considers M.A.-granting institutions, of which 78.7% require a methods course of their undergraduates. B.A. programs in political science fall in between, with 60.8% having a methods requirement. Most interesting are the B.A. Combined and Social Science programs, which have methods requirement rates near that of the M.A. institutions at 77.8 and 73.7%, respectively.

What are we to make of these findings? First, it is clear that while Ph.D. institutions are in the business of producing knowledge, they are not necessarily in the business of teaching undergraduates their trade. This reinforces Hill's (2002) anecdotal findings and his general belief that research methodology is not uniformly taught across the discipline. However, we were startled to see how much more likely M.A.- and B.A.- granting institutions were to require a methods course. It appears that institutions that focus on the goals of a liberal arts education may be more likely to see research methodology as a useful addition to the undergraduate curriculum. The differences between requirements in a B.A. in Political Science versus a Combined or Social Science program are stark as well. Interdisciplinary programs are much more likely to emphasize the unifying factor of methodology than traditional B.A. programs in political science. Hill's (2002) lament about social science teaching is particularly appropriate for political science B.A. programs. Finally, M.A. programmatic goals to prepare students for admission to Ph.D.-granting institutions may spill over into their undergraduate course requirements as indicated by their higher rate of methods course requirement.

Second, we thought that the decision to require an undergraduate research methods course might be a resource issue. We augmented our survey data with descriptive data from APSA on faculty size and the faculty to student ratio at each college or university for a quick comparison. Surprisingly, by either measure, schools with less faculty resources are more likely to require a research methods course, as reported in Table 4. Clearly, this is a resource issue, but not one of scarcity. Smaller programs (the B.A. and M.A. programs) are willing to devote faculty to teaching required research methods courses while larger programs (Ph.D. programs) are not. This may explain why much of the debate in the literature over research methodology coursework in political science is confined to graduate courses (e.g., Burns 1990; Dyer 1992; Bennett, Barth, and Rutherford 2003; Schwartz-Shea 2003). It may be the case that faculty at research universities are concerned with what they teach their graduate students, not with what those students' backgrounds may have been at their undergraduate institutions, nor what their own undergraduates are learning about methodology.

We were also curious about the timing of the decision to adopt a required undergraduate methods course among departments that have such a requirement. Table 5 reports the results from question 3 of the survey. The majority of departments that have a required course adopted it at least a decade ago. For those departments that do not currently have a required methods course, we asked in question 2 of the survey whether they had any plans to adopt one. Table 6 indicates that the vast majority of those who do not have a requirement have no plans to adopt one in the near future. About one-quarter of respondents reported some general discussion about a future methods course, with only a little over 5% saying that specific plans were underway to require an undergraduate methods course.

Conclusion

The results of this survey are quite provocative given the current state of methodological debate in political science. We discuss at length in our journals, at our meetings, and among colleagues the state of graduate methods training in the discipline. Such debates revolve around what types of methodological training are most appropriate—not whether it is important. By contrast, little discussion is found in our journals about undergraduate methods training. What discussion one finds suggests that little if any such training exists. The results of this survey demonstrate that such a conclusion is unwarranted.

Undergraduates at Ph.D. institutions are the least likely to be required to take a course in methodology. Perhaps this accounts for Hill's skepticism about the state of science education in political science. The institutions that research faculty inhabit, and are most familiar with, are the least likely to educate their undergraduates in social science methodology. Yet, almost half of Ph.D.- granting institutions in our sample have some undergraduate methodology requirement. The numbers improve as one moves to B.A. programs in political science, yet a student is far more likely to receive required training in methodology at an M.A.-granting institution in political science, or in a Combined or Social Science B.A. program. Future study will be needed to determine why these differences exist. One possible explanation for the relative deficiency of methods training at research institutions is that the faculty already incorporates methodological issues into all of their political science classes, making a separate course unnecessary. Whatever the cause, the fact remains that methodological training is occurring, just not as frequently in the places that we have looked for it.

As a result, we believe that there is some room for optimism that political science undergraduates are receiving useful training in social science methodology. What is clear is that our knowledge generating departments must also begin to consider the importance of teaching undergraduates the methods of producing knowledge at their own institutions. This may require a reallocation of teaching resources in some cases, away from substantive coverage in all possible areas to giving students the ability to learn about the world on their own. If our colleagues at B.A. and M.A. institutions with limited faculty resources think it is important enough to require methodology in such high percentages, then perhaps it is time for Ph.D. institutions to reconsider their role in undergraduate education too.

References

Adams Greg. 2001. “Teaching Undergraduate Methods.” Political Methodologist 10 (fall): 2–4.Google Scholar
Alvarez R. Michael, ed. 1992. “Methods Training: Student Perspectives.” Political Methodologist 5 (spring): 2–9.Google Scholar
American Political Science Association. 2000. APSA Survey of Political Science Departments: A Report for the 1999–2000 Academic Year. Washington, D.C.: American Political Science Association.Google Scholar
American Political Science Association. 2002. Directory of Political Science Faculty: 2002–2004, 1st Edition. Washington, D.C.: American Political Science Association.Google Scholar
Bennett Andrew, Aharon Barth, and Kenneth R. Rutherford. 2003. “Do We Preach What We Practice? A Survey of Methods in Political Science Journals and Curricula.” PS: Political Science and Politics 36 (July): 373–378.Google Scholar
Bennett D. Scott. 2002. “Teaching the Scientific Study of International Relations to Undergraduates.” PS: Political Science and Politics 35 (March): 127–131.Google Scholar
Braumoeller Bear F. 2003. “Perspectives on Pluralism.” PS: Political Science and Politics 36 (July): 387–389.Google Scholar
Burns Nancy Elizabeth. 1990. “Methodology in Graduate Political Science Programs.” Political Methodologist 3 (winter): 9–10.Google Scholar
Clawson Rosalee, Aaron Hoffman, and James A. McCann. 2001. “If We Only Knew Then What We Know Now: A Few Reflections on Teaching Undergraduate Quantitative Methods Courses.” Political Methodologist 10 (fall): 4–5.Google Scholar
Couper M. P. 2000. “Web Surveys: A Review of Issues and Approaches.” Public Opinion Quarterly 64: 464–494.CrossRefGoogle Scholar
Dyer James A. 1992. “A Survey of Political Science Research Methods Courses.” Political Methodologist 5 (spring): 17–19.Google Scholar
Garand James C., and Micheal W. Giles. 2003. “Journals in the Discipline: A Report on a New Survey of American Political Scientists.” PS: Political Science and Politics 36 (April): 293–308.Google Scholar
Hewitt J. Joseph. 2001. “Engaging International Data in the Classroom: Using the ICB Interactive Data Library to Teach Conflict and Crisis Analysis.” International Studies Perspectives 2 (November): 371–383.Google Scholar
Hill Kim Quaile. 2002. “The Lamentable State of Science Education in Political Science.” PS: Political Science and Politics 35 (March): 113–116.Google Scholar
Hojnacki Marie. 2001. “Teaching Undergraduates About the Quantitative Study of Politics.” Political Methodologist 10 (fall): 5–6.Google Scholar
Janda Kenneth. 2001. “Teaching Research Methods: The Best Job in the Department.” Political Methodologist 10 (fall): 6–7.Google Scholar
Kasza Gregory. 2001. “Perestroika: For an Ecumenical Science of Politics.” PS: Political Science and Politics 34 (September): 597–599.Google Scholar
Lewis-Beck Michael S. 2001. “Teaching Undergraduate Methods: Overcoming ‘Stat’ Anxiety.” Political Methodologist 10 (fall): 7–9.Google Scholar
McBride Allan. 1994. “Teaching Research Methods Using Appropriate Technology.” PS: Political Science and Politics 27 (September): 553–557.Google Scholar
McBride Allan. 1996. “Creating a Critical Thinking Learning Environment: Teaching Statistics to Social Science Undergraduates.” PS: Political Science and Politics 29 (September): 517–521.Google Scholar
Morrow James D. 2003. “Diversity through Specialization.” PS: Political Science and Politics 36 (July): 391–393.Google Scholar
Odell John S. 2001. “Case Study Methods and International Political Economy.” International Studies Perspectives 2 (May): 161–176.Google Scholar
Poggione Sarah. 2001. “Teaching Undergraduate Methods for the First Time.” Political Methodologist 10 (fall): 10–11.Google Scholar
Rodgers Pamela, and Cecilia Manrique. 1992. “The Dilemma of Teaching Political Science Research Methods: How Much Computers? How Much Statistics? How Much Methods?” PS: Political Science and Politics 25 (June): 234–237.Google Scholar
Schaefer David R., and Don A. Dillman. 1998. “Development of a Standard E-mail Methodology: Results of an Experiment.” Public Opinion Quarterly 62 (autumn): 378–397.Google Scholar
Schwartz-Shea Peregrine. 2003. “Is This the Curriculum We Want? Doctoral Requirements and Offerings in Methods and Methodology.” PS: Political Science and Politics 36 (July): 379–386.Google Scholar
Segal Jeffrey. 2002. “Teaching the Scientific Study of Law and Courts to Undergraduates.” PS: Political Science and Politics 35 (March): 117–119.Google Scholar
Selingo Jeffrey. 2003. “What Americans Think About Higher Education.” Chronicle of Higher Education (May 2): A10–A16.Google Scholar
Sills Stephen J., and Chunyan Song. 2002. “Innovations in Survey Research: An Application of Web-Based Surveys.” Social Science Computer Review 20 (spring): 22–29.Google Scholar
Smee Alan, and Mike Brennan. 2000. “Electronic Surveys: A Comparison of E-mail, Web and Mail.” ANZMAC 2000: Visionary Marketing for the 21st Century: Facing the Challenge, 1201–1204.Google Scholar
Smith Rogers M. 2003. “Progress and Poverty in Political Science.” PS: Political Science and Politics 36 (July): 395–396.Google Scholar
Stanton J. M. 1998. “An Empirical Assessment of Data Collection Using the Internet.” Personnel Psychology 51: 709–726.Google Scholar
Stone Peter. 2001. “Making the World Safe for Methods.” Political Methodologist 10 (fall): 9–10.Google Scholar
Thies Cameron G. 2002. “A Pragmatic Guide to Qualitative Historical Analysis in the Study of International Relations.” International Studies Perspectives 3 (November): 351–372.Google Scholar
Wahlke John C. 1991. “Liberal Learning and the Political Science Major: A Report to the Profession.” PS: Political Science and Politics 24 (March): 48–60.Google Scholar
Figure 0

Response Rates by Degree Program

Figure 1

Representativeness of the Sample Compared to the Universe by Degree Program

Figure 2

Percentage of Departments Having an Undergraduate Methods Requirement

Figure 3

Percentage of Departments Having an Undergraduate Research Design or Methods Course Requirement by Faculty Size and Students Per Faculty Member

Figure 4

Among Department with a Research Design or Methods Course Requirement: How Many Years Ago Was It Adopted?

Figure 5

Department Plans to Adopt a Required Methods Course if None in Place at this Time

Figure 6

Appendix