Summary of Sessions

The presentations to the working group examined assessment on three levels: evaluating specific classroom techniques, appraising skills and experiences, and assessing departments and the discipline. Assessment at the micro level—specific classroom techniques—is in some ways the easiest. However, because we often adopt traditional methods in teaching, we may be uneasy about—or may not even consider—asking whether those traditional methods are the best way to help students learn. Only after finding himself soundly bored with his own PowerPoint slides did one presenter decide to switch from lecture to discussion in large classes. Both student satisfaction and performance increased. Using a traditional textbook is habit for many professors; one of our presenters broke out of the habit and tested whether other types of readings, such as original sources, improved student comprehension. The results were inconclusive, but the presenter plans to continue the experiment to track outcomes over time. A presentation on the use of debates highlighted the utility of rubrics in assessing student progress. Showing students a detailed grading rubric, with clearly defined expectations for each component of the assignment, can be a powerful tool in spurring student performance. Assessment not only helps us improve our classes, but can increase student learning in the process.

Our next set of presenters asked a broader question: How can we assess student success in skill development and experiential learning? Critical thinking heads the list of desired skills and one presenter posited that a core set of critical thinking skills could be nurtured even in students in introductory political science classes. Research skills are also important, but how does one foster interest and expertise in this area? Another presenter detailed her university's program to mentor student researchers, concluding that both the program and its assessment need additional investigation. Looking beyond the discipline, one presenter asked to what extent could we measure the success of freshman cohorts. Because his university's program was composed of self-selected students, any assessment needed to account for likely skewing of results. But his experience raised a broader point for the group: What type of methodological precision do we expect in assessing our techniques and programs? Certainly it is better to perform some assessment, even if it is very basic, rather than put off assessment altogether because of inadequate data.

Our final set of presenters addressed the question of departmental and discipline-wide assessment. Without a doubt, departmental assessment generated the most concern, in part because it is frequently imposed by university or external administrators. The view from our participants, however, was that departmental assessment could be tremendously positive. For instance, one department adopted an assessment model from another school. They found that the exercise not only allowed them to define what would be assessed in their program, but it also strengthened the departmental focus and created strong bonds among political science faculty. Another presenter, coming from a department with a long history of successful assessment, outlined how a particular pedagogical tool—capstone courses—improved student learning while achieving programmatic goals.

Core Findings

From these presentations emerged hours of discussion and debate. By the last session, we were able to recognize some key points of agreement in the group. Our first conclusion was the realization that identifying objectives is a critical aspect of assessment, and an underappreciated one. The process of assessment involves recognizing the goals for the program first, and then identifying how to evaluate whether the goals are being accomplished. Assessment is not a value-neutral process. Setting goals requires prioritization; assessment involves choosing what to evaluate. At its essence, the process of assessment is political. Participants in the workshop agreed that faculty goals for education are often at odds with student goals. Critical thinking is valued by most academics, while the perception of the “millennial student” (and their parents) is that they are focused on skills development and occupational outcomes. Participants also recognized that the goals of faculty are likely to vary from a standardizing body that might impose, externally, curricular goals and benchmarks.

With this in mind, we found it important that departments take ownership of the assessment process. In order to do this, we need to establish community: identify our shared interests and collaborate with colleagues and students to determine our goals for curriculum, critical thinking, practical skills development, and ethical awareness. Successful examples presented at the workshop involve embedding assessment into a department plan that its constituents agree to. This includes synthesizing seemingly disconnected objectives between students and faculty. It also means overcoming divides between faculty that employ different teaching (or research) techniques. New active learning techniques may require new assessment strategies; at a minimum, faculty must agree that assessment should address both critical thinking skills and curriculum goals.

Assessment is not necessarily easy, but there are rewards for accomplishing it. Repeatedly, we heard of programs that gathered and used data as leverage for new faculty hires, for improving the quality of student learning, for increasing the quality and quantity of majors, and for developing more cohesive and cooperative programs. At the same time, incentives were found to increase the quality and extent of assessment. It is important to note that good assessment includes a feedback loop: once data is gathered, it must be processed, which leads to the revision of goals and the implementation of new strategies for accomplishing and evaluating those goals. The fear of many faculty is that assessment is a “new” ongoing task that does not lead to a reprioritization of responsibilities but rather to an increased workload. If the departmental or institutional culture is not cooperative, assessment may be used to advance agendas or to undermine alternative views and approaches. In addition, the emergence of multiple accrediting and governing bodies may result in competing plans, reports, and expectations. In such a scenario, faculty may be required to engage in “busy work” that goes nowhere and is read by no one.

We are optimistic that if assessment is done well, taking advantage of the quantitative and qualitative analytical tools that political science faculty possess, the quality of teaching will become increasingly measurable. This, in turn, has the potential to transform the field of political science so that teaching becomes a more important criterion for assessing faculty performance. It also serves as a warning that we should be careful to set goals that are obtainable and demonstrable.

Core Questions

By the end of the weekend, the working group articulated three interrelated questions to present to the membership of APSA and to the discipline.

• What are the positive and negative effects of assessment on our profession and discipline? We heard reports from faculty members who found that assessment increased their control of their curriculum, leading to increased departmental resources. Others reported that assessment at their institutions produced more work for faculty with little or no payoff in increased institutional support or improved teaching.

• How do we as teachers of politics conceive of classroom and program assessment, and how are the two processes connected? Departments that achieved positive results through assessment, as reported to our group, had clear understanding of the purposes of their classroom assessment devices, and could then use classroom assessment data as part of their program assessment.

• How can we use the current assessment debate to reconsider the priorities of the discipline and the role of the APSA? Many participants noted that the absence of disciplinary standards, best practices, or model curriculum made it difficult for departments to leverage university resources, especially with competition from departments with accreditation standards. Our national organization can both support collaboration among political scientists on assessment issues and provide a forum for possible development of a disciplinary conception of the goals and purposes of our teaching. This question also suggests an update of the Wahlke Report on the recommended elements of a strong undergraduate political science curriculum.

Core Suggestions

It seems that overall, both the public and the discipline have expressed renewed interest in and concern for education. Internationally, university teaching and learning centers are established, future college teachers obtain graduate certificates on teaching, and nationally normed surveys investigate to what extent undergraduate students are engaged in their learning experience. These initiatives are important in refocusing on education, but without assessment, there is no way of investigating to what extent we actually improve our students' education. What can political scientists do, then, to facilitate and take control of assessment of teaching techniques, student learning, skills, and programs? Concurring with the conclusion of the 2004 TLC, we emphasize that it is advantageous for the discipline itself rather than external actors to manage its assessment standards and procedures. We thus suggest a series of concrete steps to help political science professors and departments in their assessment efforts.

First, we propose to share information about assessment and learn from the experience of others. To that end, we are offering a pre-conference short course on developing a departmental assessment plan at the 2005 APSA Annual Meeting in Washington, D.C. This will be a hands-on workshop for those beginning the assessment process for their programs; faculty members who have developed successful assessment plans in their departments will be there to guide the participants through the initial steps. This workshop also feeds into the larger plan of developing a network of faculty experienced in assessment, which can function as consultants for departments still in the nascent stages of developing assessment plans. Second, information sharing will also occur through an assessment page housed on the APSA web site. Faculty and departments will be encouraged to post their existing mission statements or assessment practices and measurements for others to view as examples, thus providing a toolbox of resources to assess knowledge, skills, and values.

In order to conduct effective assessment, faculty and departments first need to establish the goals of teaching in the discipline. In other words, what are the learning outcomes that make the discipline distinctive and unique? Defining the goals and objectives of political science education goes beyond defining the essentials of necessary discipline-specific knowledge. It also includes skills—like critical thinking—that are either taught specifically in political science or have a unique interpretation that differs from the way students might acquire the same skill in a different discipline. In order to make a convincing statement for the value of political science education, the particular contribution of our field to undergraduate education has to be made explicit. Assessment can then be conducted against the objectives the discipline has articulated and the department has embraced.

If critical thinking is a generally accepted objective of political science education, then we must develop useful tools to measure it. While smaller programs sometimes use senior projects or capstone courses to assess the level of critical thinking skills that students have acquired, departments with a large number of undergraduates often do not have this option. We thus suggest that one goal for next year's conference is to look more closely at ways in which critical thinking skills can be usefully assessed in a variety of academic contexts. We also realize that as little work as we have completed in undergraduate education assessment, less research and assessment has been done on graduate education in political science.

However, as an organization, APSA can do more than help define goals and objectives in teaching. One of the ways in which APSA—based on faculty input and collaboration—can help facilitate assessment is through the development of a nationally normed comprehensive exam that departments could use for a small fee. This would help departments track student progress rather than just measuring the end result, which would facilitate assessment of programs and curricula. Even greater work is needed in the area of graduate education, where even less work on assessment has been completed in the discipline.

Lastly, there is a need for a future conference on assessment that will include all types of different institutions and facilitate and encourage discussion on assessment across the discipline. This could lead to the development of a discipline-based agenda or plan to guide assessment in political science education in all institutions at the graduate and undergraduate levels.