Over the last 10 years in the discipline of political science, there has been a flourishing discussion about the role of departmental assessment. Clearly, one fear has been that these external mandates are infringements on academic freedom at worst, and make extra work for faculty at best. In an early article published in PS: Political Science and Politics, a survey of political science departments discovered that frequently the response to assessment was one of ignorance and often animosity.
A few respondents expressed concern for how the data will be used; some questioned the usefulness of what was being collected; and some stated that the whole issue, in the words of one respondent, is “being forced down our throats, whether we like it or not.” Other comments demonstrated that the writers had little knowledge of developments in the field of outcomes assessment…. Others questioned making such a fuss over the measurement of outcomes, obviously unaware of the nationwide mandate to collect such information. (Julian, Chamberlain, and Seay 1991)
But it is quite clear that assessment is a growing component of accreditation expectations (Julian, Chamberlain, and Seay 1991) and a fundamental requirement for receiving most internal and external grants. Over time and with additional research, political scientists have become more accepting of assessment and have been willing to explore the ways in which it may enhance current objectives and projects. Some of the research has examined how to clearly and carefully take an articulated departmental goal, such as “excellent teaching,” and evaluate that specific expectation (Fox and Keeter 1996). Other articles focus on specific assessment tools, like portfolios and standardized testing, and how best to integrate these elements into their departmental norms and culture (Thompson 1991). Assessment can critically assist departments and faculty that are trying to do it all well—teach, research, and serve various communities. The assessment movement has now placed an additional level of expectations upon faculty; simultaneously, our larger responsibilities have not diminished. In what ways can assessment provide tools to enable us to better practice our profession?
The common thread in the recent research on assessment in political science has been the acceptance of the way it can positively change an academic department and the educational environment that department provides its students. In a discussion of their gradual development of an assessment plan, one department noted “as time wore on, we began to see the value of setting specific program goals. The process forced us to think about the nature of our classes and the education which we are providing to our students” (Lasher and Kitts 1998).
We find assessment can provide the impetus to create a holistic departmental model and then provide an ongoing mechanism to generate feedback and renewal of the model. While we originally served in the same department at a small, private, undergraduate institution, one of us is now working at a larger research-intensive state university. This article discusses our shared experiences in a smaller department. While the product a department develops may vary, this assessment model can work in many academic settings.
Over the past 10 years, our department carefully and deliberately thought and acted on these issues as we completed and implemented our assessment plan. While many of the specific elements we suggest may exist throughout undergraduate programs nationwide and may not be as applicable to larger institutions, we believe the deliberateness with which we have addressed these questions is key to successful implementation of an assessment plan. Our answer to frustrations surrounding assessment has been to rethink the meaning of the department, the purpose of assessment, and the significance of political science education. What has emerged from these discussions has been a claiming of assessment as an inherent part of an academic life, an integration of teaching, research, and service, and a belief that faculty and students can combine into a vital intellectual community to the benefit of all. We suggest that it is precisely the addition of formative assessment to our departmental and faculty goals that allows us to best serve our communities.
An Intentional Process
Our department made assessment a basic part of a deliberately constructed culture; for us it was one based on excellence and the belief in a meaningful community of scholars. Assessment becomes part of the scholars' habit of reflection and refinement. We plan, we implement, we assess, and we plan again.
Creating a Mission
We choose to fulfill our mission and goals through the creation of a community of scholars within the department; a home where all are supported and encouraged to develop their potential, but also one where rigorous assessment holds all accountable through frequent contacts. We should note that the creation of this community presupposes that students and faculty are all able partners, with somewhat different roles at different times. Students serve as our research assistants, co-authors, teaching aides, and help in most of the assessment processes—this is part of their education.
What forms this community? As a department of political scientists and scholars, our ideals inform the goals we have for our students and our expectations of each other. We desire to produce generations of new students who have a deep and abiding intellectual curiosity and the habit of perpetually asking “Why?” and “So what?” We want to instill in students the personal values necessary for participating in a pluralistic democracy; moreover, we insist students begin determining and articulating their own values, specifically as they affect their views of the political arena. To accomplish these goals, the department values different teaching methods in order 1) to adjust to students with different learning styles and 2) to teach different skills. As a department, we have carefully learned and practiced diverse teaching methods, observed faculty from disparate disciplines in the classroom, and joined faculty colleagues from across the entire campus in teaching workshops, often requesting them to comment on our teaching. Students hold faculty accountable through departmentally designed teaching evaluation forms (administered each semester and geared to focus on specific departmental goals) and through such events as the senior exit interview. Students are taught to hold each other accountable through various peer-reviewed public presentations (sophomore majors' conference, senior seminar presentations, specific classroom activities, etc.). We want a community that is able to hold one another accountable, push each to higher expectations, and challenge all with new ways of thinking. This community can only work when professors and students engage in this process both collectively and independently.
The Role of a Departmental Mission
Teaching, scholarship, service, planning, and assessment are all evaluated and pursued in light of how each furthers our goal of creating a community of scholars. Our written mission and our articulated goal of community become the center of our department—not just a cliché on paper. Our mission clearly states our goals and is carefully consistent with the larger university mission.
This deliberateness (creating a clear vision, providing a written mission, articulating related objectives, defining subsequent values) has many benefits. First, we stay focused only on activities that will further what we have previously agreed upon. Second, our program has a coherence that appeals to students and furthers the sense of departmental identity; this may have the additional benefit of heightening retention rates. Third, assessment becomes fairly easy when what is being assessed and the indicators of its success have been identified prior to planning, performing, or evaluating. Finally, in an academic culture of scarce resources, a cohesive department with clearly stated and measurable goals will generally receive greater administrative support that the typical department governed by the idiosyncrasies of individual faculty.
Following from this larger mission, the department then created a series of specific goals that follow from both the departmental mission and university goals. These objectives focus on the knowledge, skills, and values students should obtain prior to graduation. We have also identified precisely where in our curriculum, culture, and assessment these objectives are pursued and evaluated.
Communicating the mission and its justifications early and frequently is vitally important if we are to hold our students to these standards and objectives. In view of this, we have created two documents that we share with all prospective or incoming majors and faculty. The first is a student handbook containing materials regarding the planning of coursework, an overview of learning support and research materials available on campus, helpful advice from successful students and alumni, as well as information about our community of scholars. This resource provides information on how to take responsibility to be an active learner and specific advice on preparing and taking essay examinations. A second means of communication has been our web site. This is a quick way to keep connected with our alumni, letting them know what we have been doing in the department and what our majors have been achieving. It is also a wonderful recruiting device; increasing numbers of prospective students make this their initial contact with the department. Our web site includes links to our popular simulation programs, faculty sites with vitae and course syllabi, and information about our goals and expectations for the department. With new software, updating and maintaining web sites have grown increasingly easy.
Through such assessment devices as our senior exit interview, we have evidence confirming that both the distribution of our student handbook and the utilization of a departmental web site are effective means of communication with current and potential students. By keeping the mission close at hand and commonly understood we are able to return to it as the yardstick against which all is measured. For example, when our students develop their required internship syllabus, they connect their internship expectations with the stated departmental objectives. This centeredness makes it much easier for faculty to parcel out precious time and resources to most effectively meet our goals, and serves as the touchstone by which we evaluate our process. While many departments have these elements (handbook and web site), explicitly connecting them to the mission incorporates them into the assessment process.
Perpetuating the Mission
One necessity common to all communities is that in order to last and thrive, they must recruit new members. Unfortunately, many departments seem to feel that recruiting majors is not their responsibility. We believe we have been well-served by making it a faculty priority to work closely with the Office of Admissions. By using our recruitment initiatives to bring in and retain strong students, we are able to create a better community of scholars—not because they are all alike, but because they share our vision. When a department has a consistent source of strong students it improves the community and the quality of faculty life.
Equally important to the recruitment of additional majors is the corresponding need to recruit faculty who will appreciate and build the community of scholars. In designing a faculty search process, we consider both departmental goals (e.g., specifying the nature of the subfields and experience desired, “growing” the department by hiring dynamic professors, increasing national visibility in professional forums) and pedagogical objectives (e.g., hiring faculty who desire to be excellent in the classroom, finding faculty who can blend their research and teaching goals, seeking innovators in the classroom who value departmental objectives). Toward these ends, we engage students in the hiring process by giving them a real and continuing presence in the search—which reinforces our mission in ways we can document.
Majors in the required methods course and the senior capstone design evaluative forms for interviewing candidates. They discuss and determine the attributes of an excellent faculty member and develop appropriate indicators. Outside of legitimating the search process for students, this process has two important values. First, pedagogically, this is a good way for students to learn to understand the concepts of “measurement” and “evaluation.” Second, students wrestle with such basic educational concepts as effective and ineffective teaching. Our exit interview data shows that this makes them more active partners in their own education. Students observe the candidates' teaching sessions and then use the student-created form to assess each candidate relative to other candidates. We will not hire a candidate who does not have strong student support. In fact, we once postponed a search for this very reason. Feedback from students demonstrates their commitment to our community. One student demonstrated this in her response to a candidate's visit: “One of the best things about Millikin and our department is the collegial atmosphere and mutual respect among students and faculty. I don't see [this candidate] offering either of these attributes.”
Assessing the Mission
Assessment then becomes the key to complete implementation of the departmental mission. Through assessment we are able to test ourselves, our students, and our program. We then find ways to improve, develop, and become more consistent with our goals and values. It is a continual and organic process. Because the faculty has determined the mission (in our case, a mission as simple as a “community of scholars”), we are able to retain control over the assessment process. We determine what will be measured, how it will be measured, how the evaluation will be interpreted, and how the interpretation will be applied. Most accrediting bodies and universities merely want to see a clear process that has made a difference in the life of the department.
A Community of Faculty
In our case, our mission encourages us to work very consciously to develop a strong community of scholars among departmental faculty. Because we value a reputation for rigor, we hold each other accountable for the various components of a faculty member's career—research, teaching, and service. We have a formal, but simple, departmental audit each year where we assess the contributions of each faculty member to the department, university, and larger profession. Recognizing the differing rhythms of an academic career, contributions are compared to what has been proposed by each faculty member in their personal growth plan. While we tend to rely on colleagues in our subfields for accountability in the quality of our research, the departmental community plays a strong role in assessing our teaching. We have been careful to make sure that our departmental activities fit consistently with university requirements for personal three-to five-year plans, faculty portfolios, etc. The goal is not to make more work, but to make assessment work for the department.
Another benefit of a mission-driven assessment plan is that it broadens faculty perspectives on teaching by focusing on departmental missions, goals, and values, rather than independent classroom experiences. All of our courses fit into this broad shared vision that dramatically changes the nature of the assessment of our teaching. Because we are less concerned about individual stylistic approaches and more interested in the ways our diverse approaches fulfill the goals of the larger community, classroom autonomy is greatly respected. Clearly, this communal emphasis on teaching is very different from the way most faculty were socialized into the academy.
This mission emphasis also dramatically alters how we evaluate faculty effectiveness in achieving departmental goals and objectives. We use classroom visits for formative, rather than summative evaluation. We find team visits to be more reliable (with group conversations) than individual visits; perhaps the most valuable part of the classroom visit is the discussion of the shared experience. These discussions allow faculty (and by extension students who are aware of these visits) to see how individual classes connect to the larger departmental community. Fox and Keeter advocate this technique in quoting a department that finds that “through these visits we're exposed to a wide range of teaching styles and receive formal critiques of our in-class performance” (1996, 178). Not only does our department benefit, but these discussions have helped us become better teachers.
Involving Students in the Mission
Like many departments, we advise student organizations, have departmental social functions, involve students in our research, travel with students to conferences, give departmental awards, and devote many hours to meetings with individual students. What we may do differently than other departments is deliberately frame these interactions as part of our departmental mission—in our case, the community of scholars model. In addition, we carefully involve students in our curricular development projects and have sought their feedback as to how well we meet our stated mission.
We use a senior exit interview to assess our success in meeting departmentally determined goals and objectives. Our exit interview is an annual event where graduating students and faculty meet at a local pub. We encourage students to share their insights about the department, a faculty member takes notes, and we make sure the conversation hits the same topics every year (the value of the Methods course, strength of the departmental community, integration of senior seminar). In addition, we designed specific departmental teaching evaluation forms that ask questions relating the course to the departmental goals. Finally, we survey our students regarding their evaluation of the department as a whole. This survey asks questions about satisfaction of majors with curriculum, specific required and elective courses, advising, and the departmental culture in general. We selected specific questions that correspond with our university-wide as well as national student satisfaction surveys. By doing this we gain comparative data and receive specific suggestions from our current students. In addition, this has the added benefit of helping our students maintain ownership in the department, reinforcing our particular mission.
Involving Alumni in the Mission
The final component in implementing and assessing a departmental mission is evaluating the long-term impact of our work. Our alumni provide important feedback regarding the degree of our success and a unique perspective on the work of the department, so it is useful to see how they perceive the department and university five, 10, 15, and 20 years after graduation. The challenge has been how to define their role and involve them in this community. Because the Alumni and Development Offices have such a high stake in alumni involvement, they tend to be very supportive of such ventures. They have helped us create and maintain our departmental database as well as assisted with departmental fundraising.
Our assessment plan provides the incentive for us to survey the department's alumni. In the fall of 1995, a survey was first sent to all located alumni who graduated from 1980–1995 in political science and related interdisciplinary programs. To boost the response rate, a cover letter by the department chair prompted their response and a second mailing included a brief personal note from the faculty member who knew them best. An extremely high response rate (74% of our targeted alumni) enables us to rather safely generalize from our survey. We repeated this process in 2002, surveying majors who graduated between 1996–2001. Programs, faculty, the curriculum, and student outcomes were all addressed. Data generated from these surveys enable us to analyze the perceptions of our alumni regarding a number of our objectives and the performance of the department in fulfilling its mission. It also continues our mission of truly becoming a community of scholars—alumni help faculty by holding us accountable to our own departmental vision. The surveys enable us to answer questions related to the goals we had derived from our mission.
This type of assessment program could easily be adapted to other types of institutions. Larger departments could use this process in a methods course; one class could possibly design the instrument and in another semester a class could interpret the returned data. Again, student involvement in the assessment process will strengthen the department and support at least the teaching aspects of the department's mission.
Closing the Feedback Loop
It is easy for departments to amass a great deal of assessment data each year, yet never change a thing. The data are only valuable when a department uses them to reevaluate and reconsider current departmental practices. To keep this process simple, yet effective, we meet annually to discuss our assess ment findings, interpret the results, and determine changes for the next academic year. We have designed a simple chart with the major criteria we evaluate (e.g., improved success in law school placement, increased student engagement in research methods); we report findings (e.g., LSAT scores went up, fewer students elected to take second semester of statistics than desired); and we note what we will try differently (e.g., maintain LSAT study group, incorporate issues of methodology more clearly in courses across the subfields).
Benefits of Assessment
Our department's willingness to engage in and document assessment activities and demonstrate how we have adapted our teaching and other activities as a result of what we have learned, has provided tremendous benefits. For many departments, reaccredidation or required program reviews are the only times where this sort of reflection and programmatic change occurs. We firmly believe that our model of assessment allows faculty to make more frequent, subtle improvements, and to more closely achieve our mission. Specific benefits include: better political science education, a growing department, increased individual student performance, and increased institutional autonomy and support for the department.
Our administration knows we hold ourselves and our students accountable. Consequently, we have autonomy to do our jobs and funding is generally provided when needed. The tenure and promotion process has become easier because the department is seen as one that is internally rigorous. Our students are well-respected both inside and outside of the institution. Because we respect our students and have carefully taught them to hold themselves accountable, they have become more useful student workers and collaborators. This has made us more effective and efficient in both our classroom and in our research.
Assessment cannot work miracles. But if seen as an opportunity for faculty to deliberately define what they do and why, this intentionality makes the process of defending and advocating for your department a simple one and provides the evidence to support your contentions.
Conclusion
While this approach seems ideal for the smaller undergraduate college, the process is clearly applicable to all types and sizes of schools. The missions themselves will vary, depending on the nature of the university and the type of expectations placed on faculty and students. But all institutions are required to assess their work. Regardless of the source of the accountability—accrediting organizations, state legislatures, Boards of Trustees, or funding sources—we all must demonstrate our excellence and the means of improvement. A mission-driven assessment approach helps keep the power in the hands of the faculty.
If we define our mission and how we will evaluate it, we will be allowed to do our work. However, we must move beyond the “I do what I like and my grades illustrate the outcome” model if we are going to satisfy our outside auditors and improve our own work. Assessment can make us better teachers; by using the same rigorous methodology we use in our research on our teaching and departmental programs, we can become better scholars as well.
The authors presented a version of this article, with the late Robert C. McIntire, at the 2000 APSA conference, where it was awarded best paper by the Undergraduate Education section.
%20-%20https://sketch.com%20--%3e%3ctitle%3e3ECBBF50-FD08-412A-943D-2CFC5E025A16%3c/title%3e%3cdesc%3eCreated%20with%20sketchtool.%3c/desc%3e%3cg%20id='Stylesheet'%20stroke='none'%20stroke-width='1'%20fill='none'%20fill-rule='evenodd'%3e%3cg%20transform='translate(-926.000000,%20-668.000000)'%20id='BUTTONS'%3e%3cg%20transform='translate(100.000000,%20597.000000)'%3e%3cg%20id='BTN-/-RIGHTS'%20transform='translate(816.000000,%2065.000000)'%3e%3cg%20id='ICON-/-Rights'%20transform='translate(10.000000,%206.000000)'%3e%3cpath%20d='M13.5752841,0%20L1.34943182,0%20C0.59375,0%200,0.586666667%200,1.33333333%20L0,22.6666667%20C0,23.4133333%200.620738636,24%201.34943182,24%20L17.6505682,24%20C18.40625,24%2019,23.3866667%2019,22.6666667%20L19,5.36%20L13.5752841,0%20Z%20M13.5752841,1.89333333%20L17.0838068,5.36%20L13.5752841,5.36%20L13.5752841,1.89333333%20Z%20M1.34943182,22.6666667%20L1.34943182,1.33333333%20L12.1988636,1.33333333%20L12.1988636,5.36%20C12.1988636,6.10666667%2012.7926136,6.69333333%2013.5482955,6.69333333%20L17.6235795,6.69333333%20L17.6505682,22.6666667%20L1.34943182,22.6666667%20Z'%20id='Shape'%20fill='%23595959'%20fill-rule='nonzero'%3e%3c/path%3e%3ccircle%20id='Oval'%20stroke='%23595959'%20stroke-width='1.5'%20fill='%23595959'%20cx='9.5'%20cy='13.5355339'%20r='5.5'%3e%3c/circle%3e%3cpolyline%20id='Path-2'%20stroke='%23FFFFFF'%20stroke-width='1.5'%20transform='translate(9.500000,%2013.035534)%20rotate(45.000000)%20translate(-9.500000,%20-13.035534)%20'%20points='8%2016.0355339%2011%2016.0355339%2011%2010.0355339'%3e%3c/polyline%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e)
