DISCIPLINARY CONTEXT OF ASSESSMENT

The first section of the symposium is composed of two articles that address the larger disciplinary context of assessment. The first contribution, E. Fletcher McClellan’s “What a Long Strange Trip It’s Been: Three Decades of Outcomes Assessment in Higher Education,” provides the larger context for the role of assessment. This introduction to the literature on assessment in higher education discusses how the pressure for disciplinary assessment emerged and the impact on both program quality and student learning by examining the continued resistance to assessment as well as the arguments for greater disciplinary engagement. McClellan demonstrates how the discussion has changed from one of mere compliance with accreditors to one in which “institutional commitment to improve and faculty interest in improving student learning were important drivers of assessment activity.”

Using a policy-implementation case-study model, McClellan examines how it has taken more than 30 years for outcomes assessment to become imbedded in institutional practice. He specifically addresses the inherent conflicts between governmental policy goals for higher education and the missions and objectives of those institutions. The consequence of this collision was a top-down implementation process, resulting in a decided lack of enthusiasm for outcomes assessment on the part of institutions compelled to implement these policies. The policy conflicts had a particularly negative impact on the humanities and social sciences. McClellan asserts that the nature of the debate is changing once again; he cites political scientist Peter Ewell in noting that “the question has become more about what kinds of assessment to engage in and under whose control than about whether to engage in it at all.” For McClellan, the remaining question is how the discipline will use assessment to promote the importance of political science in higher education.

Candace C. Young contributes the second article in this section, “Survey of Assessment Practices in Political Science,” in which she empirically evaluates the engagement of political science departments in outcomes assessment. Through the Executive Council and Standing Committee on Teaching and Learning, APSA commissioned a survey on assessment practices in 2010. Young was named the working-group chair and led the group in developing a survey; she summarizes the survey’s findings in her article. The survey was designed to (1) allow APSA to observe how the discipline is performing compared to efforts by other disciplines in curricular and program assessment; and (2) perceive the impact of assessment on our own departments.

Whereas national data reveal that social science disciplines are least likely to engage in the assessment of student learning or to apply assessment results, the APSA survey’s finding that most of the responding departments were engaging in assessment activities may indicate a sea change in the discipline. Young recognizes that departments that engage in assessment or those that explicitly reject the premises of assessment may be the most likely to respond to the survey. Significantly, more than 70% of these departments indicated that they used assessment findings to alter their curriculum—although they had been motivated to engage in their activities because of outside pressures (e.g., accreditation and program-review expectations). However, 64% of departments noted that faculty and staff interest in educational improvement had influenced the assessment process. Young’s comparison of political science with all other disciplines reveals that we now are engaging at a rate more in line with other fields. Although other social science disciplines may be practicing program assessment more consistently than political science, we report using a wider diversity of assessment instruments. Young concludes her report on the data by querying how the APSA and other disciplinary organizations could be of greater utility to political science departments. She recommends a more direct engagement in assessment, such as providing potential student-learning outcomes, compiling national data for departmental comparisons, and serving as a clearinghouse for assessment tools. She notes that although we have national data across social science departments and all disciplines, we are less knowledgeable about the assessment findings within the discipline.

Because of the methodological complexity and broad range of subfields within political science, it has been difficult for the discipline to agree on shared student-learning outcomes and curricular goals. This is not the case for the other social sciences and for the more applied aspects of political science, such as public administration and public policy. A consequence of our discomfort with this discussion has been that many departments continue to rely on the discipline’s most recent statement on undergraduate political science curriculum—the 1991 Wahlke report (Wahlke Reference Wahlke1991)—which was an APSA presidential task force’s determination of the “best practice” in structuring the undergraduate political science major. It continues to be the APSA’s “current” statement on the structure and content of the major. Although this report provided a shared vision of the ideal components of an undergraduate political science curriculum, I have not found similar guidelines developed by political scientists for graduate programs. Whereas fields including public administration agree on specific standards to base accreditation guidelines, political science has gone in the opposite direction and has been unwilling to create even the most general shared consensus regarding what education in our discipline includes.

PROGRAM REVIEW

The second section of the symposium explores the role of program review in both accredited programs (e.g., public administration) and non-accredited programs (e.g., political science). These two contributions focus more on practical application, providing specific information designed to help departments negotiate the program-review process and leverage it for the best outcomes. Program reviews have become standard practice at many institutions in an attempt to compile assessment data across a period of time and to ensure continuous quality improvement within curricular programs. This process can be vastly different for programs that are externally certified by a national professional body—such as the Network of Schools of Public Policy, Affairs, and Administration (NASPAA)—than for those that are subjected to an internal program-review process. These two articles provide specific recommendations and best practices for departments that are confronting these processes, illuminating the key differences between these forms of review. We intended to include an article that focuses on the accredited programs’ site visit from the perspective of a member of several NASPAA external review teams, recommending the elements that make a successful site visit and noting key difficulties in self-study reports and how to avoid them. However, due to compelling circumstances, the commissioned article was not submitted.

David Edwards’s “Assessment for Accreditation: A View from the Trenches” describes the process of preparing for and successfully negotiating the site visit and self-study report for NASPAA reaccreditation. In his capacity as a successful Master of Public Administration program director and former member of a NASPAA site-visit team, he itemizes the accreditation review process, indicating what the faculty should have in place in advance, and discusses lessons he has learned. A primary contention is the importance of being able to demonstrate to the external reviewers—who represent different educational contexts—how his program is “effectively providing a valuable education experience for our students[.]” Edwards also discusses the importance of engaging in “extensive self-examination” to allow for a program to compare its practices, curriculum, and policy with a set of shared national standards that evolve over time.

This author’s article, “Assessing over Time: The Political Science Program Review,” discusses the review process, including a site visit from an external reviewer or review team, from the perspective of both the department and the reviewer(s). Many of the issues raised by the externally accredited review process are relevant in this context; however, in most institutional program reviews, there are no specific disciplinary standards that must be addressed. Instead, a department has more freedom to demonstrate its strengths and to sidestep its liabilities. However, the faculty must craft the narrative such that those who are unfamiliar with the institutional context can comprehend the relevant issues. Drawing on the experiences of several chairs and reviewers from different types of institutions and programs, the article focuses on ways in which the program review can utilize the annual assessment process to improve the departmental curriculum and outcomes. The article also addresses the difficulties of unaccredited programs (i.e., all political science programs) to compete for institutional resources and the way in which the program review might be leveraged on campus and in systems to improve the department. When addressed strategically, program review can use the annual assessment report to better situate a department institutionally and increase the quality of its outcomes.

BROADER ASSESSMENT ISSUES

The third section of the symposium considers larger assessment issues than those addressed in the initial 2009 work. Online education has been transformed from a supplement to an essential part of many institutions’ political science curriculum and general-education offerings at the undergraduate level. However, minimal scholarship in the discipline evaluates what classroom assessment of online teaching has revealed regarding its effectiveness or how online contributions to the curriculum can be assessed as part of all student-learning outcomes. The team of Kerstin Hamann, Bruce Wilson, and Philip Pollock has been a primary contributor to this aspect of the disciplinary Scholarship of Teaching and Learning (SoTL) literature. Joined by Gary E. Smith, their contribution, “Online Teaching and Assessment,” summarizes the findings of prior research and asks how this unique pedagogy will impact program assessment.

In their evaluation of the SoTL literature on online learning in political science, Hamann et al. determined that the research has found consistently that online pedagogy has not impaired student learning; in fact, student-learning outcomes in all modalities (i.e., online, face-to-face, and hybrid/blended) are generally consistent. Their article then discusses how online courses can be integrated into an overall program review. They identify typical limitations in assessing online courses but focus on how to integrate this assessment into the program as a whole to prevent the online elements from being perceived as secondary to or separate from the work of the department. The digital nature of online courses can be beneficial in that students’ work is compiled and archived inherently without further collection from the department. Data regarding student performance can be generated rapidly without additional investment of time. However, the authors recognize the difficulties in assessing online curriculum within a typical face-to-face program; students enrolled in this coursework may be demographically different than those in the face-to-face courses. Regardless, their emphasis on how and why online courses need to be clearly integrated into departmental assessment is beneficial in furthering our engagement in assessment.

The final article in the symposium discusses how to assess the extracurricular learning of political science and public administration students. In “Extending Assessment beyond our own Programs and Campuses: The National Survey of Student Leaders and the Consortium for Inter-Campus SoTL Research,” J. Cherie Strachan and Elizabeth A. Bennion examine the skills and outcomes that experiences with diversity, civic engagement, and leadership may imbue in our political science majors. Specifically for political science, public administration, and public policy programs, the authors explore how departments might integrate these experiences into programmatic assessment. As they note, “Even more so than in other disciplines, what political science students learn in college is intended to be applied by active citizens and civic/political leaders in the public sphere, both on campus and beyond.”

By connecting this work to the SoTL scholarship on civic engagement, we can see how these types of activities allow political science departments to demonstrate how their students meet the civic-engagement missions of many institutions. The article delves further: Much of our SoTL scholarship and assessment work has focused too narrowly on individual campuses, blinding us to our own idiosyncrasies and making our findings ungeneralizable. Strachan and Bennion formed a new Consortium for Inter-Campus SoTL Research that enables political scientists and others interested in these issues to work together to engage in the cross-campus research that is necessary for our scholarship on teaching and learning and for the assessment of our own work to advance. Those who contribute to this research receive reports from the Consortium not only about the students on their own campus but also cross-institutional data that compare analysis and assessment.

As political science confronts a nation that has grown increasingly disinterested in our discipline (Lupia and Aldrich Reference Lupia and Aldrich2015) and as higher education is challenged to justify and defend the value and legitimacy of the social sciences (Shaw Reference Shaw2015), assessment is a tool that we can use to demonstrate our contributions to the classroom, the general-education curriculum, and the lives of our students. We may not agree with the assumptions of those who impose assessment measures or about how these policies have been implemented on our campuses. However, we have the tools to evaluate our own work and the capacity to demonstrate what our degree programs provide, as well as what our students—graduates and undergraduates—achieve and accomplish. The program-review process can equip us with the tools to improve our work and to make our case to crucial internal and external audiences that may not understand our discipline.