Hostname: page-component-7b7567c8c-wxvwp Total loading time: 0 Render date: 2026-11-15T08:04:37.147Z Has data issue: false hasContentIssue false
  • English
  • Français

Assessment is Not a Four-Letter Word

Published online by Cambridge University Press:  01 October 2004

Fred Smoller
Fred Smoller
Affiliation:
Chapman University
Rights & Permissions [Opens in a new window]

Extract

The past two decades has seen a rising tide of criticism of American universities. This criticism is expressed in national reports, voter initiatives, legislative reports, and by employers and accrediting agencies who have concluded that there is “a crisis of educational quality in our nation's colleges and universities” (Gardiner 1998, 71–88) and that “a disturbing and dangerous mismatch exists between what American society needs from higher education and what it is receiving” (Wingspread Group on Higher Education 1993). From such criticisms, the assessment movement was born (see, for example, Diamond 1998; Palomba and Banta 1999; Allen 2004).

Information

Type
The Profession
Information
PS: Political Science & Politics , Volume 37 , Issue 4 , October 2004, pp. 871 - 874
Copyright
© 2004 by the American Political Science Association

Introduction

The past two decades has seen a rising tide of criticism of American universities.1

During the 2002–2003 academic year, I was chair of the departmsent of political science, president of the faculty senate, head of the General Education task force, faculty co-director of the institutional self-study for the Western Association of Schools and Colleges, and director of the Ludie and David C. Henley Social Sciences Research Center. Special thanks to Dr. David Fite, associate provost for Institutional Planning and Assessment at Chapman University and my colleagues in the department, especially Arthur Blaser (chair), Donald Will, Gordon Babst, and Ronald Steiner.

This criticism is expressed in national reports, voter initiatives, legislative reports, and by employers and accrediting agencies who have concluded that there is “a crisis of educational quality in our nation's colleges and universities” (Gardiner 1998, 71–88) and that “a disturbing and dangerous mismatch exists between what American society needs from higher education and what it is receiving” (Wingspread Group on Higher Education 1993). From such criticisms, the assessment movement was born (see, for example, Diamond 1998; Palomba and Banta 1999; Allen 2004).2

Assessment is an “ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance. When it is rooted effectively within larger institutional systems, assessment can help us focus our collective attention, examine our assumptions, and create a shared academic culture dedicated to assuring and improving the quality of higher education” (Angelo 1995, 7).

Unfortunately, the need to develop comprehensive student learning outcomes assessment plans has not been well received by our profession.3

For example, a special APSA panel on assessment for department chairs concluded that political science “has not been in the forefront in interpreting assessment objectives and designing procedures … In most departments, assessment is regarded as a lot of work unlikely to be worth doing… The majority of participants' remarks challenged assessment mandates, their costs and their usefulness in informing program development and reform.” See “Summary from the 2001 Annual Meeting Conference for Department Chairs on Program Assessment, Boston” at http://www.apsanet.org/about/chairs/assessment/index.cfm. In addition, a report to the profession from the Task Force on the Undergraduate Major concludes that: “[P]olitical science today collectively presents a picture of disparate and unstructured practices … The loose and unstructured approach taken by many departments permits few political science students to experience much ‘sequential learning’ or to complete their work with any sense of having mastered some ‘common core’ of knowledge that they share with other majors…. Evaluation of students' overall performance … often amounts to little more than a summation of the discrete performances in the courses taken. Ideally, students' learning and performances should be measured … against norms of bench-marks based on expectations of where they ought to be at different stages in their undergraduate career. Unfortunately, we know of no such current practice, and strongly suspect that the faculty time and energy needed to devise and implement such a plan would tax the resources of many departments beyond their capacity.” See John C. Wahlke, “Liberal Learning and the Political Science Major: A Report to the Profession” PS: Political Science and Politics 24 (March, 1991): 48–60.

Most political science departments have made only modest progress in the development of comprehensive learning outcomes assessment plans. This is unfortunate because a well-done assessment can lead to genuine improvement in student learning. This article is a case study of the initial efforts made by Chapman University's political science department to develop such a plan.4

Chapman University is a comprehensive based university located in Orange, California and has approximately 3,500 undergraduates. The political science department has six faculty members, programs in legal studies and peace studies, and a 16-station telephone survey center.

It concludes with some practical recommendations for those embarking on a similar endeavor.

At their core, assessment plans address the following question clusters:

  1. Say what you do. What do we expect students to know and be able to do upon completion of their programs? What skills, values, and knowledge should a graduate of our programs possess?
  2. Do what you say. To what degree do students reflect and demonstrate our educational goals and expected learning outcomes upon completion of our program?
  3. Prove it. To what extent can student learning and development be attributed to their experiences at the university and participation in various programs and activities?
  4. Improve it. How can we use information from assessment of student progress and achievement to improve our program? What changes will bring student learning outcomes closer to the department's objectives?
  5. Document it. How can we document what we did?

Benefits

Important benefits can come from a good assessment plan. First, the assessment process forces faculty to engage in a conversation about something they care deeply about: student learning. Assessment focuses the department's collective attention on the integration of learning objectives within the curriculum (among the subfields in the discipline) and between the curriculum and cognate fields (for example, history, sociology, economics, geography). Assessment also forces the department to consider how the major connects with larger institutional learning objectives, as reflected in the general education or “core” curriculum (e.g., writing, critical thinking, and oral communications skills), and other institutional values, most often reflected in a college's mission statement.

Assessment causes reflection on the relationship of the major to the “co-curriculum” (student involvement in clubs and organizations, internships, service learning activities, and unstructured activities such as speeches, films, panels, and other activities relevant to the study of politics). Research shows that students who are actively engaged in all aspects of the college experience (not just classroom learning) excel on a wide variety of indicators that are consistent with the desire to educate the “whole” person (Astin 1984, 297). Also, by asking how well the curriculum is aligned with the needs of the job market and the requirements of postgraduate education, assessment causes departments to consider what happens to our students after they graduate.

Assessment can also improve department advising, for the simple reason that faculty who “buy into” department goals and learning objectives will have a better sense of why their program is structured the way it is and be better able to quickly and effectively communicate this to students. By understanding how each course fits into the curriculum, students, in turn, will have a better sense of where they are intellectually, not just administratively, within the program, and can therefore be encouraged to take greater responsibility for their progress.

Assessment can also help fundraising efforts because a department that knows “who it is” and where it is going (and why) is better able to package and market itself to individual donors, granting agencies, campus administrators, legislators, and trustees.

Finally, with this feedback, a good assessment program allows departments to adapt to changing conditions (such as student interests, new technologies, trends within the discipline, problems with the curriculum or course scheduling) with incremental changes to its program. This avoids the wholesale changes that sometimes become necessary after a period of prolonged neglect, and preempts criticism of such before it appears in the campus newspaper or shows up in an angry letter to the academic provost.5

Both of which happened in my department when a group of students complained about our failure to offer enough upper division coursework due to not having sufficient faculty.

Steps in the Assessment Process

There is more than one way to develop an assessment plan. Our department implemented the following fairly simple and straightforward process:

  1. Develop department mission statement, learning goals, and learning objectives.
  2. Check for alignment between the curriculum and the identified goals and objectives.
  3. Develop a meaningful, manageable, sustainable assessment plan that systematically examines department goals and objectives.
  4. Collect the assessment data.
  5. Close the Loop. Good assessment leads to collective reflection and action.
  6. Routinely examine your assessment process and correct, as needed.

Develop Department Mission Statement, Goals, and Learning Objectives

The first step in the development of a student learning outcomes assessment plan is to write a department mission statement. In crafting our department mission statement, we considered the mission statement of our university, the “Best Practices” guidelines from professional organizations, discipline-based statements, goals and objectives from similar programs at other institutions, the different needs of the students we serve (majors vs. non-majors), and, finally, departmental and university constraints (available faculty, budget). The mission statement is a brief description of the department's values and philosophy.6

The department's mission statement is: Students in the Political Science Department learn about the discipline of Political Science, the great and enduring ideas of political philosophy, concepts and theories central to an understanding of international relations, and the theory and practice of American politics and government. To achieve these ends the Political Science student will become (1) well-grounded in the social sciences, familiar with the philosophy and methodology of inquiry in the discipline, and acquainted with well-known criticisms of approaches to inquiry in Political Science, and (2) familiar with the breadth and diversity of approachesboth theoretical and practical-and intellectual traditions within the student's chosen sub-field concentration.

Next, we developed student learning goals. These are far-reaching statements regarding the knowledge, skills, and values that the department expects its political science majors to acquire. They define what the department is all about. After some initial prodding, faculty suggested dozens of department goals—from the mundane to the sublime. However, if assessment was going to be meaningful, manageable, and sustainable, these had to be reduced to a select few. Ten seemed like a reasonable and manageable number, one consistent with what other programs are doing. Since assessment is an ongoing process, additional goals can be added over time. The key was beginning the assessment process, even if this meant that the list of outcome goals might be incomplete or flawed.

We divided our goals into three areasknowledge, skills, and values. Knowledge consists of what we want our students to know (for example, terminology, facts, concepts, and principles). Skills are what we want students to be able to do (for example, to be able to speak and write clearly and effectively). Values are the attitudes and qualities we want our students to possess (for example, openness, broadmindedness, empathy, sensitivity, informed judgment, wisdom, integrity, honesty, responsibility, self-knowledge, citizenship, diligence, precision, care, sense of complexity, ambiguity, love of learning, multicultural awareness, aesthetic appreciation, and disciplined creativity).

Under knowledge, we selected assessable goals for each of our fields— Political Philosophy, American Politics and Government, and International Relations and Comparative Politics. We also borrowed freely from other departments' assessment plans.7

Thanks to Vicki L. Golich, director, Faculty Center, chair and professor, political science at California State University San Marcos, www.csusm.edu/golich, for emailing us a list of about a dozen assessment plans which she has assembled. Also see the APSA's web page on assessment.

As we began to discuss our department goals, an interesting question was raised: should the assessment plan reflect current practice or future aspirations? Modest goals didn't inspire us (or our students); overly ambitious goals weren't attainable (for now). They also violated “truth in advertising” dictums. We compromised and went forward with goals that represented who we are, but which provided some room for growth.

Learning Outcomes

The next step was to operationalize the goals in the form of learning outcomes. These are measurable indicators that let the department know if the goals are being reached. They take the form of specific things students should know, be able to do, or the attitudes or values they should possess. To keep things simple this first go-round, we chose one learning objective for each goal.

Knowledge

Goal 1. Students will know the values and beliefs that constitute the Western political tradition, and alternative ideologies and belief systems.

Objective 1. Students will be able to explain the political thought of Plato and Aristotle.

Goal 2. Students will know the institutions and processes of the American political system and its strengths and weaknesses commensurate with citizenship duties and an effective civil society.

Objective 2. Students will be able to discuss the major theories, criticisms, and proposals for reform associated with two of the following institutions: Congress, the Presidency, the Courts, the Bureaucracy, the Media, and California government.

Goal 3. Students will learn the facts and concepts of international relations and be able to relate these to contemporary global issues, including the causes and consequences of economic globalization.

Objective 3. Students will be able to distinguish competing explanations of international conflict, including the causes and consequences of economic globalization.

Skills

Goal 4. Students will be able to communicate effectively in both written and oral form.

Objective 4. Students will be able to research and write a 25-page research paper that adheres to standards of academic integrity, conforms to acceptable standards of academic form and style, and effectively communicates political science concepts. Students will give a 10-minute oral presentation on the same (See Schmidt, 2000 for criteria for evaluating a research paper).

Goal 5. Students will be able to evaluate conflicting arguments, assemble and present empirical evidence, and make reasoned conclusions from the evidence available.

Objective 5. Students will write one empirically based paper demonstrating knowledge of basic survey research methods and elementary statistics.

Goal 6. Graduating seniors will have the capacity to analyze and interpret the significance and dynamics of current political events.

Objective 6. Students will be able to apply in written and oral form a political science theory to a political problem or event.

Goal 7. Students will be able to develop a career goal and a plan for achieving it.

Objective 7. Students will prepare a resume, letter of introduction, and a plan for their post-graduate career.

Values

Goal 8. Students will understand and appreciate the importance of being an engaged and informed citizen who understands the principles embodied in the primary documents of the United States.

Objective 8. Students will write an essay on this topic and/or a reflective essay based on an internship or participation in a service learning project (or both).

Goal 9. Students will appreciate the sanctity and freedoms of individual persons and the constraints upon those freedoms, which emanate from membership in a social and political community.

Objective 9. Students will be able to engage in an informed and literate conversation regarding a contemporary political issue involving human rights.

Goal 10. Students will appreciate the principles embodied in the United Nations Charter and the International Declaration of Human Rights.

Objective 10. Students will write a reflective essay which applies these principles to a contemporary international political problem, and will be strongly encouraged to participate in the Model UN program.

The Politics of Assessment

Being reasonably intelligent and savvy individuals, faculty in our department were not enthusiastic about assessment. Many were not convinced it would lead to tangible improvement in the program, since weaknesses uncovered by a conscientious review would unlikely lead to additional resources, especially faculty lines. Others were convinced that their “institutional building” efforts would come at the expense of teaching and publication-the stuff upon which tenure and promotion decisions are based at most universities. One must concede the truth of such concerns and then soldier on regardless. Here are some suggestions that may help.

Point Person

While departments are given the responsibility for the development of a learning outcomes assessment plan, one person will inevitably shoulder the responsibility for its development. The person who takes on this task must have the requisite release time and other forms of institutional support to do a good job. Be certain to establish a paper trail with the relevant political actors in the tenure and review process so that assessment efforts are duly credited.

Institutional Intent

Some institutions engage in assessment because they have to, not because they really believe in it. If your institution is doing assessment for compliance purposes, direct your energies elsewhere.

Don't reinvent the wheel

My earlier criticisms notwithstanding, many departments have compiled impressive assessment documents. Many of these are available on the web, or through APSA.

Keep it simple

The more complicated your assessment plan, the greater the probability it will not be implemented. Start with something simple and straightforward (and, yes, imperfect) that your department can actually finish, as opposed to a more elaborate plan that will die a slow and tortured death. Much of what assessment is all about is documenting and formalizing what a department is already doing. The assessment instrument and process can always be enhanced. The first time out, just get it done.

Find yourself a guru

There are people in academy who love assessment and have read (or written) extensively about the topic. Major universities, such as California's State University System (CSU), have entire departments set up to help faculty and have produced a number of very helpful unpublished monographs.

Be flexible

Be as flexible as possible in crafting program goals and learning outcomes objectives. This is not the time to get into a major fight about unresolved disciplinary controversies.

One step at a time

Assessment requires a good deal of time and energy. It is easy to be overwhelmed and to overwhelm one's colleagues. Take it one step at a time (Mission Statement, Department Goals, Learning Objectives, etc.).

Building consensus

The development of a successful plan is an exercise in participatory democracy, requiring leadership, tact, and diplomacy. It must be an inclusive process in which all department members' views are solicited. Most important of all, it must be a bottom up, not a top down process. There must be time for meaningful conversations among the faculty involving compromise and consensus. This can be done at department meetings or at a retreat. See if you can figure out some way to make this fun, by, perhaps, “doing lunch” as we say here in California.

Student involvement

In the final analysis, it is the students who have the greatest stake in the quality of a department's curriculum. Seek their input through focus groups, interviews, or, as we have proposed, a student satisfaction survey that they can help design, implement, and analyze.

Recognition and reward

Faculty will not put their time and effort into assessment if such efforts come at the expense of tenure and promotion (see recommendation #2). If there isn't agreement among the relevant decision makers in the personnel review food chain (department, division, personnel committee, dean, provost, and president) that such efforts are important contributions to the institution, leave town!

Conclusion

Our department has developed instruments to assess what students have learned in our courses as well as their opinions concerning our overall program.8

Thanks to Assistant Professor Gordon Babst for the development of these instruments, with input from the department.

These are administered in our introductory courses and in the department's senior seminar. A refinement of these instruments, as well as a discussion of resulting data, will be part of a continuing conversation regarding assessment.

Most faculty react to the prospect of having to develop a learning outcomes assessment plan in a manner which brings to mind the work of Swiss psychiatrist Dr. Elisabeth Kubler-Ross, who, in her seminal work on death and dying, describes the coping strategies of patients who know they have a terminal illness. The first stage is Denial (this isn't happening to me!); the second, Anger (why is this happening to me?); next is Bargaining (I promise I'll be a better person if …); which is followed by Depression (I don't care anymore); and finally, Acceptance ( I'm ready for whatever comes). If done with intention, we feel there can be a final stage to the assessment process: Hope.

References

Allen Mary J. 2004. Assessing Academic Programs in Higher Education. Bolton, MA: Anker.Google Scholar
Angelo Thomas A. 1995. AAHE Bulletin, November.Google Scholar
Astin A.W. 1984. “Student Involvement: A developmental theory for higher education.” Journal of College Student Personnel 25: 297– 308.Google Scholar
Diamond Robert. 1998. Designing and Assessing Courses and Curricula: A Practical Guide. San Francisco: Jossey-Bass.Google Scholar
Gardiner Lion. 1998. “Why We Must Change: The Research Evidence.” Thought and Action (spring): 71– 88.Google Scholar
Palomba Catherine, and Trudy Banta. 1999. Assessment Essentials: Planning, Implementing, and Improving Assessment in Higher Education. San Francisco: Jossey-Bass.Google Scholar
Schmidt Diane E. 2000. Writing in Political Science: A Practical Guide, 2nd ed. New York: Longman.Google Scholar
Wahlke John C. 1991. “Liberal Learning and the Political Science Major: A Report to the Profession.” PS: Political Science and Politics 24 (March): 48– 60.Google Scholar
Wingspread Group on Higher Education. 1993. An American Imperative: Higher Expectations for Higher Education. Milwaukee: Johnson Foundation.Google Scholar