%20-%20https://sketch.com%20--%3e%3ctitle%3e3ECBBF50-FD08-412A-943D-2CFC5E025A16%3c/title%3e%3cdesc%3eCreated%20with%20sketchtool.%3c/desc%3e%3cg%20id='Stylesheet'%20stroke='none'%20stroke-width='1'%20fill='none'%20fill-rule='evenodd'%3e%3cg%20transform='translate(-926.000000,%20-668.000000)'%20id='BUTTONS'%3e%3cg%20transform='translate(100.000000,%20597.000000)'%3e%3cg%20id='BTN-/-RIGHTS'%20transform='translate(816.000000,%2065.000000)'%3e%3cg%20id='ICON-/-Rights'%20transform='translate(10.000000,%206.000000)'%3e%3cpath%20d='M13.5752841,0%20L1.34943182,0%20C0.59375,0%200,0.586666667%200,1.33333333%20L0,22.6666667%20C0,23.4133333%200.620738636,24%201.34943182,24%20L17.6505682,24%20C18.40625,24%2019,23.3866667%2019,22.6666667%20L19,5.36%20L13.5752841,0%20Z%20M13.5752841,1.89333333%20L17.0838068,5.36%20L13.5752841,5.36%20L13.5752841,1.89333333%20Z%20M1.34943182,22.6666667%20L1.34943182,1.33333333%20L12.1988636,1.33333333%20L12.1988636,5.36%20C12.1988636,6.10666667%2012.7926136,6.69333333%2013.5482955,6.69333333%20L17.6235795,6.69333333%20L17.6505682,22.6666667%20L1.34943182,22.6666667%20Z'%20id='Shape'%20fill='%23595959'%20fill-rule='nonzero'%3e%3c/path%3e%3ccircle%20id='Oval'%20stroke='%23595959'%20stroke-width='1.5'%20fill='%23595959'%20cx='9.5'%20cy='13.5355339'%20r='5.5'%3e%3c/circle%3e%3cpolyline%20id='Path-2'%20stroke='%23FFFFFF'%20stroke-width='1.5'%20transform='translate(9.500000,%2013.035534)%20rotate(45.000000)%20translate(-9.500000,%20-13.035534)%20'%20points='8%2016.0355339%2011%2016.0355339%2011%2010.0355339'%3e%3c/polyline%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e)
This paper represents a broad survey study of assessment models used in political science programs at 50 universities and colleges across the country. We examine the most frequently used assessment activities and then investigate whether differences between institutions is a function of institutional characteristics. What are the techniques currently employed by departments across the country?
To address this question, we examine data derived from an Internet search of various assessment plans posted by political science departments. Since many students narrow down their choice of colleges by examining their web sites as a prelude (or sometimes alternative) to a campus visit, web sites are becoming an important tool for universities to advertise and explain their majors. While deriving data from university web site may have its limitations, it also provides important insights into each university's offerings as well as an efficient way to gather comparative data.
To begin, it is necessary to provide some general sense of what we mean by “assessment.” Basically, we accept the definition as provided by Angelo (1995) as “an ongoing process of understanding and improving student learning.” This involves, according to Golich (1998):
- making expectations explicit and public;
- setting appropriate criteria and high standards for learning;
- systematically gathering and interpreting evidence;
- measuring outcomes by expectation; and
- using results to explain and improve performance.
Further, as the American Association for Higher Education (AAHE) report on assessment contends, assessment works best when the programs it seeks to improve have clear, explicitly-stated purposes. Clear, shared, implementable goals are the cornerstone for assessment that is focused and useful (Angelo 1995).
Although there are a number of different assessment techniques, as Golich (1998) notes, there is usually an inverse correlation between the quality of measurement methods and their expediency (in other words the best methods usually take longer and cost more). Generally, as Wright (2005) notes, one can distinguish between assessment techniques that require considerable additional effort to implement and those that require less institutional or departmental resources. For example, techniques such as reviewing existing coursework for signs of student learning; examinations of student grades and performance; examinations of existing course syllabi; or student course evaluations require relatively little additional changes to existing departmental offerings and little in the way of adopting techniques beyond what faculty do as part of their day to day activities. On the other hand, other techniques, outside of the normal classroom operations, such as a comprehensive examination, exit interviews, use of pre and post tests, portfolio analysis, a capstone experience, or graduating student surveys, go well beyond the regular operations of faculty members. Recently there have been a number of articles that have described assessment efforts at a number of individual political science programs (see Hill 2005; Deardorff 2005; Deardorff and Folger 2005). In addition the APSA has developed a web site that focuses on sharing examples of several individual assessment programs in political science (see www.apsanet.org/section_562.cfm). However, despite the recent attention in the APSA to assessment, little or no work has been done that has surveyed the kinds of assessment techniques being used by political departments, nor have any studies examined the institutional and departmental variables that are associated with variations in assessment plans.
The Data
The following section provides the results of a survey of reported assessment plans at 50 political science programs and examines whether different plans are related to such institutional and departmental characteristics as whether or not the institution is rated highly, the student/faculty ratio, and the highest degree offered in political science by the institution.
Table 1 reports the schools included in this study. The programs were coded not only for the assessment techniques but also for three institutional departmental characteristics: the extent to which schools were selective (based upon data from U.S. News and World Report), the student/faculty ratio, and the highest degree offered by the program. Of the 50 schools, one was least selective, seven were less selective, 29 were selective, and 13 were more selective. Twenty-five schools had a student/faculty ratio at 17 and below (with a low of 12), and 25 had a student/faculty ratio above 17 (with a high of 28). Ten political science departments offered a Ph.D., 18 offered a masters' level degree (including MA and MPA degrees), and 22 offered only bachelor's degrees.
List of Schools Included in the Study
Table 2 reports the learning outcomes that were most frequently mentioned in the assessment plans of the 50 institutions. The most commonly mentioned learning outcome was knowledge of theories in political science (34); closely followed by knowledge of political institutions and processes (33); knowledge of the subfields in political science (31); and the skills goals of critical thinking (31), methodology (30), written communication (30), and oral communication skills (26). Goals such as the cultivation of citizenship skills (12), career (11), diversity (8), and ethics/values (4) were far less frequently mentioned as goals. By and large, then, most political science programs focused on knowledge of political science as a field and the promotion of skills. Interestingly, relatively few programs promoted citizenship explicitly, nor emphasized career (such as placement in graduate school or employment) as goals or outcomes of a political science degree.
Most Frequently Mentioned Learning Outcomes
Table 3 reports the most frequently mentioned assessment techniques employed by the programs in our sample. As indicated in the table there were a wide variety of different techniques employed, with the most frequent the use of a graduating student survey (24), and the analysis of student grades (22). Several programs also used a comprehensive exam, either standardized or department designed (17), offered the senior seminar/capstone experience as an assessment technique (17), or mentioned the systematic analysis of the senior thesis as an assessment technique (16). Far fewer programs used student portfolios, senior exit interviews, or alumni surveys/interviews as assessment techniques.
Most Frequently Mentioned Assessment Techniques
In the literature on assessment (see Wright 2005) there is a distinction made between assessment based on activities that faculty already engage in “internal” to the classroom (such as analysis of grades, reviewing existing coursework for signs of student learning, examinations of existing course syllabi, or student course evaluations) and assessment activities that are external normal classroom operations, such as a comprehensive examination, exit interviews, use of pre and post tests, portfolio analysis, a capstone experience, or graduating student surveys. Generally the former can be accomplished without much in the way of additional resources or faculty effort, whereas the latter techniques represent, to some degree, a commitment to assessment that goes well beyond the regular operations of faculty members. Thus, by implication, the use of external techniques represents a more rigorous indicator of the level of commitment to assessment by a political science program.
The Results
Figure 1 illustrates the distribution of external assessment techniques used, by frequency of schools. As indicated, eight programs employed none of the external assessment techniques, 12 employed at least one, 16 at least two, and nine at least three. Only six programs used four or more of the external techniques (with four political science programs using all five).
External Assessment Technique
Figure 2 compares schools that were more selective to all other schools to examine whether the level of selectivity is related to whether the political science program used external assessment techniques. To ascertain this, we compiled a simple additive index combining the number of external assessment techniques used. The resulting score created values ranging from 0–5.
External Assessment Techniques Score by Selectivity of Institution
As indicated in Figure 2, more selective institutions tend to use fewer external assessment techniques on average than less selective institutions. However a simple difference of means T-test indicated that this difference was not statistically significant (t = 1.07, p = .30). Thus, nothing definitive can be claimed as to whether less selective schools are more likely to employ external assessment techniques than are more selective schools.
Figure 3 compares schools in terms of the reported student faculty ratio at the institution. Although an imperfect measure of the size of the political science undergraduate program (a more direct measure would be the political science student/political science faculty ratio) the measure nonetheless represents an adequate surrogate—it is reasonable to assume that the general student/faculty ratio reflects the student/faculty ratio in political science.
External Assessment Techniques Score by Student Faculty Ratio
This measure is bifurcated into two categories—schools with student/faculty ratios of more than 17 and student faculty ratios less than or equal to 17. As mentioned above, this bifurcated the sample into two equal parts (of 25 schools for each category). As indicated, schools with a smaller student/faculty ratio were more likely to use external assessment techniques than schools that had a larger student/faculty ratio. However, this difference was not statistically significant (t = −.60, p = .55)—therefore, nothing conclusive can be said about the relationship between the institutional student/faculty ratio and the propensity of a political science department to employ external assessment techniques.
Finally, Figures 4 and 5 compares the average External Assessment Techniques Scores for political science programs that offer a graduate level degree versus programs that only offer a bachelor's level degree in political science, and by the size of the institution in terms of student population. One might expect that since pressure to adopt assessment programs by state legislatures and accrediting agencies has been directed primarily a undergraduate programs, political science departments that offer only undergraduate degrees would be more likely to adopt external assessment techniques. Further, since bachelor's-only departments are likely to be located at smaller schools, and since external assessment techniques would require less effort with smaller numbers of students, it is more likely that such techniques would be adopted at bachelor's-only departments. Indeed, as Figure 4 indicates, bachelor's-only political science departments are, on average, significantly more likely to employ techniques such as exit interviews, comprehensive examinations, and alumni surveys than departments offering graduate degrees (t = −2.05, p = .05). Figure 5 further reinforces the notion that smaller institutions are more likely to adopt external assessment techniques than larger institutions (t = −2.57, p = .01).
External Assessment Techniques Score by Highest Degree Offered by Department
External Assessment Techniques Score by Size of Student Population
Conclusion
The paper provides a brief survey of the assessment practices of 50 political science departments across the country. There are a wide variety of learning outcomes expressed and assessment techniques employed. Most of the learning outcomes deal directly with the content of the political science discipline and the promotion of critical thinking, writing, and oral communication skills as opposed to the promotion of citizenship and ethics and values. Although many assessment techniques are employed, internal techniques (such as analysis of grades, reviewing existing coursework for signs of student learning, examinations of existing course syllabi, or student course evaluations) are more likely to be used than are external assessment activities (such as a comprehensive examination, exit interviews, use of pre and post tests, portfolio analysis, or capstone experiences).
In terms of the relationship between institutional and departmental characteristics and assessment techniques, although the use of external assessment activities does not vary by the level of institutional selectivity or the institution's student/faculty ratio, it does appear to vary according to whether a department offers a graduate degree and the size of the institution. Primarily undergraduate departments and smaller institutions are more likely to employ external assessment techniques than larger graduate degree-granting departments. This might reflect a few things. First, it may indicate that the external techniques are more easily implemented at smaller institutions that have smaller class sizes and fewer majors than the larger state institutions. However, it might also reflect the greater emphasis smaller, primarily undergraduate institutions place on teaching and learning, and these departments seek to go beyond merely employing the easiest assessment techniques to meet the “letter of the law.”
This project represents only a beginning. Indeed, one limitation on this study is our current inability to determine whether a difference exists between public and private institutions. This is because the number of private institutions in this sample was very limited, which might be indicative of how little assessment practices have penetrated private colleges and universities (something we suspect). It may also indicate the greater degree to which public institutions have been compelled to address issues regarding assessment when compared to their private counterparts. However, as a greater number of political science departments adopt assessment plans, there will be more data available from a variety of different institutions. Perhaps, documenting the kinds of assessment models that are currently available might facilitate greater discussion and more careful consideration of assessment as a useful tool in promoting student learning.
Author Bios
John Ishiyama is professor of political science and director of the Ronald E. McNair Program at Truman State University. He can be reached at: jishiyam@truman.edu.
Marijke Breuning is associate professor of political science at Truman State University and can be reached at mbreunin@truman.edu.
