CURRICULUM REVIEW IN THEORY AND PRACTICE

A faculty that wants to assess the curriculum can draw useful guidance from various sources. However, departments may wonder whether guidance from professional associations, academic literature, and existing data sources is comprehensively relevant to their particular circumstances. The structures and goals of political science programs vary considerably (Wahlke Reference Wahlke1991), and a curriculum review must consider these characteristics. Consequently, we join others (e.g., Robinson Reference Robinson2013; Schomburg and Teichler Reference Schomburg and Teichler2005) in arguing that alumni experiences and reflections also constitute a valuable source of information for a program. Alumni are relatively proximal to the program, which is important given program diversity. Alumni also may provide more reliable answers than current students because of their distance from the day-to-day academic workload. Furthermore, only alumni know how they are using the knowledge and skills acquired during their postsecondary studies, and this information should feed back into the system. Consequently, alumni are key to connecting the past, present, and future of a program.

The goals of a political science program—crucial for this type of evaluation—typically appear in the form of general mission statements and more operationalized outcomes or objectives. These goals often include a combination of learning “how to think” via development of general intellectual abilities in the liberal arts tradition (Wahlke Reference Wahlke1991), skill training, learning of specific content, and/or preparing to be an informed citizen and democratic participant. Increasingly, programs also must respond to pressures to increase retention and graduation rates. Tracking relatively closely with these goals, we distilled the literature and our own preliminary data gathering into the following categories for a curriculum review: (1) program structure, (2) content or knowledge, (3) skill development and competencies, (4) outcomes or student achievements, and (5) learning environment.

The literature provides ample recommendations about program structure in political science. The establishment of a relatively large set of core courses in the curriculum and the strong sequencing of courses are important starting points (Bennett Reference Bennett1991; Ishiyama Reference Ishiyama2005; Wahlke Reference Wahlke1991), with warnings against proliferation of courses (Ishiyama, Breuning, and Lopez Reference Ishiyama, Breuning and Lopez2006). Greater structure in the curriculum is helpful for abstract reasoning and aptitude test scores (Ishiyama Reference Ishiyama2005; Ishiyama and Hartlaub Reference Ishiyama and Hartlaub2003). Exposure to the different modes of inquiry in political science (Wahlke Reference Wahlke1991) and requiring an early research-methods course (Ishiyama Reference Ishiyama2005) are other recommendations. A requirement for off-campus learning (Wahlke Reference Wahlke1991) and an intensive capstone experience (Huerta Reference Huerta2015; Ishiyama Reference Ishiyama2005) constitute additional recommendations. Our questionnaire operationalized program structure by asking about major concentrations, core courses, course sequencing, and flexibility and frequency of course offerings. Moreover, the questionnaire asked about the benefits of curricular “supplements” (not required) such as internships and study abroad.

Student knowledge is another traditional emphasis in the discipline. Although knowledge is the sum of different processes, its most direct relationship is with curriculum content. From the perspective of alumni, knowledge is the more salient concept. Beyond consideration of the traditional subfields of political science (Wahlke Reference Wahlke1991), investigators highlight the need for greater integration of research methods (Parker Reference Parker2010); the scientific nature of the discipline (Hill Reference Hill2002); citizenship (Smith and Graham Reference Smith and Graham2014); gender (Cassese, Bos, and Duncan Reference Cassese, Bos and Duncan2012); ethnic and cultural diversity (Bennett Reference Bennett1991); and related fields including history, economics, and geography (Wahlke Reference Wahlke1991). The questionnaire addressed knowledge via a series of items about how well studies in political science had equipped alumni with basic knowledge in 11 different areas.

Skills have assumed greater importance in the discipline over time (Ishiyama, Breuning, and Lopez Reference Ishiyama, Breuning and Lopez2006). Recommendations about skill development emphasize analytic capacities, research-design skills, statistical skills, and writing and oral-presentation skills (Bennett Reference Bennett1991; Breuning, Parker, and Ishiyama Reference Breuning, Parker and Ishiyama2001; Wahlke Reference Wahlke1991). The questionnaire asked alumni about their preparation to use various communication, research and analysis, and professional skills, as well as the importance of each skill in the respondents’ lives.

The evaluation of outcomes for the program and alumni is typically a straightforward means of evaluating success as measured against program goals. Assessment of learning protocols may evaluate student knowledge and skills throughout the course of studies but often do not examine the knowledge of former students. Previously mentioned questionnaire items captured certain programmatic and self-reported learning outcomes. However, fitting with the recent emphasis on employment, the questionnaire addressed additional alumni outcomes through items about the nature of employment, satisfaction with employment characteristics, and actual and anticipated further studies.

Finally, we asked alumni to evaluate the learning environment because of its strong relationship with the curriculum as well as its emphasis in pre-survey interviews. The questionnaire asked about the quality of teaching and teaching methods, experiences in classes, faculty accessibility, and advising, which are mostly standard items gleaned from student evaluations of their instructors.

QUESTIONNAIRE DESIGN AND DATA COLLECTION

We collected views from alumni of the political science undergraduate program at Montana State University–Bozeman, a research-intensive, public land-grant university in the northwestern United States. The political science program had about 150 majors enrolled at the time of the survey and annually graduated approximately 35 to 40 students in recent years. The political science major also had allowed students to concentrate in one of four areas: international relations, policy and analysis, political institutions, or political theory. The first area was available beginning in 2006 and the other three in 2009. The department also offers a Master of Public Administration (MPA) program with a typical population of about 25 students.

Conducting this curriculum review served a number of pedagogical goals. By assisting in this review, MPA students taking a research-methods course learned about the collection, management, and analysis of both qualitative and quantitative data. They conducted a literature review under the professor’s supervision. They also developed an interview protocol and conducted interviews of purposively selected departmental faculty members and alumni. The interviews identified important concepts not found in the literature that might be important for our specific program. The students coded the interview text into nodes and subnodes and conducted analyses using qualitative analysis software (i.e., NVivo). They then worked with their professors to design the questionnaire. This iterative process involved multiple rounds of meetings as well as reviews of the text and questionnaire structure using web-based survey software (i.e., Qualtrics). The MPA students’ contributions were important in conceptually shaping the questionnaire and in refining the wording and formatting. The hands-on application of ideas and analytical techniques allowed them to experience the course material more tangibly, which resulted in high levels of interest and engagement. During the pilot phase of the project, a few dozen undergraduate students also participated by providing feedback about the construction and wording of the questionnaire. The undergraduates were given an opportunity to participate based on their enrollment in certain courses.

Given resource constraints and the relatively young age of the target population, we decided that a web-based questionnaire was most appropriate. The researchers worked with the university’s administration to obtain information about the population of interest, which included all students awarded a political science undergraduate degree by the institution from the fall of 1999 to the spring of 2014. Students with multiple majors were included only if they identified political science as their primary major. The population included 470 students; however—after supplementing university records with faculty records—valid e-mail addresses were available for only 308 of them. To avoid spam filters, the department head provided a hyperlink to the questionnaire in batches of 20 e-mail addresses. Alumni received a reminder from the department head several days later, and the survey closed after 15 days. The survey yielded a response rate of 34.4% (i.e., 106 of the 308) from valid e-mail addresses, although more recent graduates constituted a larger percentage of respondents. The respondents were 58.5% male and 39.8% female; 1.6% preferred not to answer. ^{Footnote 1} The distribution of respondents in terms of gender and major concentration mirrored the population figures well.

SELECTED FINDINGS

This section discusses selected findings that might be generally interesting or useful. All percentages discussed are the percentage of responses after removing the “not applicable” option.

ACTIONS TAKEN AND FURTHER STEPS

The survey results fed into a collaborative faculty process for curriculum revision, which was a part of the department’s broader, ongoing assessment efforts. The department decided to make significant changes to the curriculum based on the alumni survey results, recommendations from the literature and the discipline, and faculty input. Table 1 outlines the process from goals and objectives to information gathering to outcomes. The first column indicates the university and department goals or objectives under which the changes were categorized. The middle column lists potential areas for improvement based on survey results and review of the literature. The third column shows the consequential actions taken by the department.

Table 1

Actions Based on Alumni Survey Results and Literature Review

Note: Other goal categories in the 2012 strategic plan at the university level are engagement, integration, access, and stewardship. This table addresses the most relevant goals that resulted in departmental action.

As a result of the curriculum review, the department submitted formal paperwork for a curriculum revision. It would expand the core to be more inclusive of subfields, provide for stronger course-sequencing requirements, and better integrate content and skill development across courses and levels. Additionally, the revision would enhance the research-methods sequence, with the primary change being a new mandatory 300-level research-experience course. Furthermore, although alumni viewed the concentrations favorably overall, the department decided that maintaining the concentrations conflicted with other key observations. In particular, the new curriculum would ensure that courses are available and that students have curricular flexibility—both key observations from the survey. The expanded core would improve foundational breadth in terms of knowledge and skills, and students could decide whether to pursue depth or breadth in the upper-division courses (thereby improving flexibility). Finally, the department took seriously the feedback about advising and providing information about internships and other opportunities by adding central advising services and a department newsletter. In summary, our low-cost survey generated findings that have resulted in rapid implementation of programmatic and curricular changes.

The purpose of sharing our curriculum-review process, instrument, and decisional outcomes is to provide an example of a process that engaged students and faculty in meaningful and productive ways. The results fed into our larger assessment activities and provided data about learning and employment outcomes that are of interest to prospective students. We also learned from the data-collection process. For example, a more comprehensive study might locate students who either changed majors or did not graduate to solicit their views. Additionally, a program might consider better incorporating departmental or institutional “values” (e.g., citizenship and freedom; see Smoller Reference Smoller2004) into the questionnaire. Finally, departments may benefit from integrating this type of questionnaire into formal assessment of learning procedures as well as using the results to leverage additional resources. Regardless of the specific approach, we believe alumni constitute a reserve of invaluable information to help departments evaluate their present state and plan for their future.

SUPPLEMENTARY MATERIAL

To view supplementary material for this article, please visit https://doi.org/10.1017/S1049096517000695.