Quiz Activity

One of our first suggestions for fostering critical thinking skills is that instructors should engage students in distinguishing various statements from among the four types outlined previously: normative, interpretive, causal, and factual. We administer a quiz that presents a list of statements and ask students to accurately categorize them. Many items contain “tricks” that challenge students to think carefully. For example, we might use the word “cause” in normative statements that nevertheless divulge the author's values, such as “Land inequality is a major problem in transitional democracies; it causes people not to get what they deserve: a decent life.” We want students to discern that because this sentence incorporates the writer's view about what people deserve and what a decent life is, the statement is normative rather than causal.

The quiz challenges the students to distinguish causal, interpretive, and normative statements from factual statements based on how verifiable they are. For example, after showing a graph displaying presidential approval ratings over the last 50 years, we offer the statement: “We can interpret that the highest approval rating in the last 10 years was in 2001, just after September 11.” This is a factual statement that can be verified with the graph, even though it uses the word “interpret.” Another example we use is: “James Madison said that ‘if men were angels, no government would be necessary.’” This statement is also a fact, because it is a direct quotation. However, the statement “Though Madison seems to believe that man is corrupt, his suggestions rest on his belief in the goodness of mankind” would be an interpretation, because this statement could be wrong and evidence in his writing might support a different interpretation. Again, it is essential that students be able to distinguish arguments from factual statements that are used to provide evidence for arguments.

Other examples of quiz statements include:

• • Young people tend to become more interested in politics when their parents have discussed politics in the household. (Causal)

• • The Second Amendment prevents government from restricting any kind of firearm. (Interpretive; should be given in the context of a class that has included discussion of the text of the Second Amendment)

• • Homework is stupid. (Normative: for comic relief)

• • Weingast says “stable democracy does not simply arise because some countries happen to have the relevant shared set of values” and concludes that “the relationship between citizen values and democratic stability is not a causal one.” (Factual; from Weingast Reference Weingast1997).

The last item is designed to be especially challenging, both because it includes the word “causal” and because a student could use the quotation to interpret Weingast. We advocate using slightly ambiguous and tricky sentences.

After administering the quiz, we allow students a class period to go over it, which invariably leads to a debate about why the statements could fall into several different categories. As an added incentive, we usually offer students extra credit when they argue effectively. This level of nuance has served us well in our teaching, but others adopting this activity might wish to use only the most straightforward items. Other faculty may choose to simplify the exercise if they are concerned that students will lose confidence and become frustrated if the quiz is too tricky. However, another part of the quiz asks students to come up with their own examples of the various kinds of statements, which might strengthen students' confidence by giving them more content control.

In our experience of administering this quiz in senior seminars, the average grade is an 80. The quiz is not easy, but we have found it to be extremely useful in helping students develop their instincts about different kinds of statements. Another result of this assignment has been to bring the lower half of the class up to speed with the more analytically advanced students. In other words, this quiz usually does not help excellent students become more excellent (although the teaching that surrounds it, we would argue, does). Instead, the greatest value of this particular test is that it helps the less advanced students develop their analytical skills.

Student Criticism of Their Own Papers

A slightly more involved adaptation of the statement quiz is an activity in which students evaluate and categorize every statement in a paper that they have written. For this exercise, we refine our four-part categorization of statements to draw a distinction between “relevant” and “irrelevant” factual points. In this assignment, students bring a double-spaced copy of their paper to class the day it is due. Without advance warning (although students can draw on the skills previously developed in the quiz), students are asked to categorize every sentence in their paper. Therefore, if the paper assignment is to make and support a causal claim, student submissions should propose causal arguments and use relevant facts and logic to provide supporting evidence. In our experience, students often mistakenly posit interpretive or normative arguments or supply irrelevant facts. If the assignment is to develop and support an interpretive argument of a particular text, students often err by marshaling as evidence normative assertions or factual points external to the text.

When grading such an assignment, we normally deduct points when students incorrectly combine different types of theses and evidence. However, for this activity, if students identify their own mistakes, then they lose far fewer points overall. This “points back” incentive typically helps motivate students to do well. We find that when students' interpretive arguments are backed up by evidence from outside the relevant text, then they are especially likely to catch their own mistakes. The advantage of this assignment is that students receive the benefits associated with rewriting and being critical of their own arguments. In our experience, students perform much better on papers after this assignment.

Interpretive Arguments

Many political science assignments require interpretation of a particular text, such as classical texts in political theory courses or court opinion interpretations in judicial politics courses. We have found that students have great difficulty with such assignments. We often receive papers that either do not interpret but instead offer a critique based on the students' own normative values, or else offer an obvious summary with no argument. Students will often even make causal arguments about what caused the authors of the text to say what they do. For instance, students have written that “Rousseau was just following the norms of public opinion at the time,” or “the justices were mostly nominated by Republican presidents, which is why they decided what they did.” Some students seem unaware that they are making causal arguments—and what is more problematic, they seem unaware that such causal arguments require evidence.

One assignment that we have found prepares students for more sophisticated interpretive activities is a “baby step” in the formulation of interpretive argument. After completing a reading, students write two logically distinct but plausible interpretations of a particular quotation that they select from the text. During the class period, they spend time in small groups discussing their interpretations. This exercise allows students to provide valuable feedback to each other. The groups then present what they determine to be the best pair of competing interpretations to the class, which allows the students to learn from each other's mistakes. This assignment helps students move in the direction of making an argument but stops short of asking them to make a decision about which of the two interpretations is most plausible. The exercise prepares them to see the possibility of various interpretations of the same text, all of which require textual support, and encourages them to seek out plausible alternatives for the purpose of making counterarguments.Footnote ⁴

Causal Arguments

It is also important to equip students to make and support causal arguments effectively. To do this, it is first necessary to encourage students to think about plausible alternative causal theories. This way of thinking is particularly confusing for students who have not taken a research design course to train them in creating concepts and variables and thinking clearly about units of analysis and variation. To help those students, it is necessary to provide a structure in which they can choose among alternative explanations. In our experience, it is effective to draw on existing theoretical debates within a particular subfield. In comparative politics, for instance, the framework of cultural, rational, and institutional explanation types (Lichbach and Zuckerman Reference Lichbach and Zuckerman1997) can be effectively leveraged. In the area of political economy, a set of approaches identified as ideas, interests, and institutions (Hall Reference Hall, Lichbach and Zuckerman1997) would be equally useful. And the paradigmatic terrain of international relations, which builds on realism, liberalism, and different versions of radicalism (Walt Reference Walt1998), could also serve this purpose.

Here, we offer a specific example from a comparative politics course. Throughout the semester, students are introduced to various and competing explanations of different political outcomes (e.g., war, economic growth, democratic transition, electoral outcomes), with an emphasis on cultural, rational, and institutional theories of politics. Students are routinely encouraged to consider the most likely explanations for those outcomes, given the existing theoretical framework. Thus, they have a limited selection of explanations, which facilitates the organization of causal claims in their minds. For example, discussion might focus on the high propensity of French citizens to engage in public demonstration compared to citizens of other industrialized democracies. Students would be faced with different causal propositions derived from the three major theoretical approaches. A rational explanation could be that the French protest because this approach has proved to be an effective means toward desired political ends. An institutional explanation might argue that the centralized structure of the French state limits people's avenues for effective interest articulation. From a cultural perspective, it may be that demonstrating is simply viewed as normal, habitual behavior among French citizens (on all these points, see Mény Reference Mény and Colomer2002, 103–04). Although this list of explanatory propositions is not exhaustive, its theoretical structure helps students identify competing causal accounts and think about how to assess them using empirical evidence.

To encourage this critical thinking further and practice using these assessment skills, we assign a paper that tasks students with generating a specific “why” question that interests them. Ideally, this question is motivated by a current event so that students can develop the habit of questioning the roots of events as they unfold. Allowing students to identify their own question encourages them to study a subject that they find especially interesting. Examples might be: Why did Swiss voters in 2009 approve a ban on the construction of minarets? Why did Iceland's Best Party (which campaigned partly on the promise of free towels at public pools) fare so well in a recent local election? Why has the South African murder rate fallen so steeply in the past year? Why is prostitution legal in some countries and illegal in others? After identifying their question of interest, students then use the cultural-rational-institutional framework to propose alternative explanations. Finally, they engage in comparative analysis and seek out empirical evidence to support or challenge these various explanations. Ultimately, students draw conclusions about the most likely account(s) of their chosen outcome.

Normative Arguments

One of the greatest challenges an educator faces is getting students to critically evaluate their own normative values. While students are unlikely to take a particular interpretation of a Supreme Court opinion or piece of European Union legislation personally, their ability to dispassionately consider their own worldviews is limited. It can be threatening to have one's values criticized, whether we are asking students to engage in critical self-reflection or raising questions about their beliefs. Nevertheless, since two goals of teaching critical thinking are to help students become better citizens and to challenge their own and others' political viewpoints and decisions, the effort is worthwhile. Given the potential of this kind of exercise to evoke emotions that inhibit learning (Csíkszentmihályi Reference Csíkszentmihályi1990), we advocate an incremental approach.

To deal with the sensitive nature of this form of critical thinking instruction, we use assignments dealing with normative statements that steer away from political areas that are too emotionally involved. Another possible approach is to focus on the mechanics of evaluating assumptions and foundational principles of those normative arguments. Accordingly, we present two assignments that promote critical thinking along normative lines but do not tend to alienate or offend students.

The first exercise involves working through the first principles of fairness when evaluating electoral systems. Since students are not likely to hold ingrained views about which electoral system is best, the debate is not personally threatening to them. Moreover, outlining the first principles of fairness helps students focus on the mechanics of developing an argument rather than the argument itself. To set up this assignment, students are taught that there are various normative considerations that the designers of democratic electoral systems often have in mind. These considerations include the desire to create a system that is highly representative, yielding a legislature that closely mirrors the general population; effective, executing the powers of government in an efficient manner; highly participatory, encouraging a high rate of citizen participation; or high in representative accountability to constituents. Students are also taught that certain aspects of an electoral system will promote some of these desired outcomes. For instance, proportional representation generates a more representative system on averageFootnote ⁵; majoritarian systems can make parliaments more efficient; compulsory voting laws can boost participation; and in systems that select candidates rather than parties, individual representatives are likely more accountable to voters.

To engage with these dimensions of electoral system design and wrestle with the normative considerations involved, students are tasked with designing their own electoral system from scratch. This assignment can be adapted as a paper or an in-class exercise. In the classroom version, students work in small groups to select two normative priorities (e.g., representativeness and efficiency) that they want to pursue. They then create the framework for an electoral system that maximizes their desired normative goals. The assignment grade is not based on which goals they select, but rather whether the system actually maximizes those objectives. In this way, students can be evaluated not on their normative views, but on the basis of whether their suggestions are logically connected to the first principles that they have chosen.

Another strategy for familiarizing students with the logic of normative statements without directly confronting their own values is to structure the lesson around a contemporary controversy that is not likely to be directly salient in their minds. We then provide students with various principles that should serve as reasonable guideposts for decision making. Their grade depends on how well they are able to use those principles in their argument. For example, we might ask students to debate and ultimately make a recommendation about the entry of Turkey to the European Union. The main academic purposes of the exercise are to help students learn about the process of European enlargement and get them to think about the costs and benefits of a new EU member. Although students can and should consider what is best for aspiring member countries, the emphasis in this particular exercise is on making the best decision from the existing member states' points of view.

The issue of Turkey joining the European Union is hotly debated across Europe and is likely shaped to some extent by anti-Muslim biases, as Turkey is a predominantly Muslim society. Therefore, to approach the topic from a relatively neutral point of view, we first introduce students to the debate that occurred in the late 1990s over the entrance of several (Christian) Eastern European countries to the European Union. Students are then encouraged to consider Turkish membership within this framework, thereby minimizing the effects of Turkey's Muslim composition as a key consideration in the debate.

Students are prepared for this class by learning about the criteria formally established by the European Union in 1999 for judging the suitability of countries that apply for membership. These principles focus on economic, liberal democratic, and human rights qualifications, and we encourage students to engage in some basic interpretation of these rules.Footnote ⁶ We also assign a book chapter published before the 2004 eastern enlargement of the European Union that analyzes the costs and benefits of allowing these countries to join (Mayhew Reference Mayhew1998). This reading assignment provides an analytical framework that identifies the economic, political, and security costs and benefits of admitting countries that are (a) not as wealthy as the existing member states, (b) have national minority groups that claim discrimination, and (c) have potentially threatening geopolitical locations. As a result, students are prepared to approach the issue of Turkish membership from a logical-legal point of view.

To wrestle with the membership issue themselves, students are divided into working groups of four or five and given condensed copies of the CIA Factbook report on Turkey (U.S. Central Intelligence Agency 2011). This report is written in bullet-point form and contains information on Turkey's economy, human rights, democracy, and security issues, in addition to other relevant facts for assessing its suitability for membership. The student groups are charged with formulating a normative argument with respect to Turkish membership—that is, the European Union should or should not accept Turkey. To make this judgment, they must use their knowledge of EU membership policies, the framework for assessing the costs and benefits of enlargement for existing EU countries, and the available facts about Turkey. When the groups have made their decisions, the class reassembles and each team presents both their normative recommendation and the factual points about Turkey that are most relevant for this recommendation.

This exercise has proved very successful in the classroom. It never fails that some groups make a pro-membership argument and others an anti-membership argument. This disagreement offers a learning opportunity in itself by demonstrating that even with the same criteria for judgment and the same facts, people can produce opposing normative recommendations. Furthermore, by pushing students to back up their argument with hard facts about Turkey, this exercise encourages them to use empirics and normative judgments to engage with new material. The assignment also allows them to argue the various points and weigh the costs and benefits explicitly in discussion groups.

A more general recommendation for encouraging students to think through normative arguments is to get them engaged in producing a normative research design. We have students read the first chapter of Tom Regan's (Reference Regan and Regan1980) Matters of Life and Death. In this introduction, Regan provides a list of practices to follow (and not to follow) when making normative arguments. He also provides a list of potential first principles that can be used in normative arguments, such as utilitarianism, consequentionalism, the morality of rights, and the value of life. We task the students with generating a normative argument and then identifying the first principle that best provides its foundation. In other words, rather than simply asking students to make an argument—for example, that human beings should not eat meat—they must identify the first principle that serves as the foundation for that argument. In this case, they might invoke a principle that relates to the immorality inherent in allowing the intentional and unnecessary suffering of an animal. They would then argue that the first principle is defensible and logically connected to their specific normative contention.

Having students defend the connection of their argument to first principles is technically a research proposal for normative arguments. When making normative arguments, research proposals must clearly establish the logic that connects a broad first principle to a specific normative argument. For example, students might choose the specific normative argument “War is defensible when attacked.” Then, rather than making an argument based on the conditions under which war is defensible, they would identify a first principle, such as the right of self-defense, and make the case that the right of self-defense is logically linked to the argument that war is defensible when attacked. Their argument would center on the logical relationship between the first principle and the contention. In this way, we ask students to make an argument that is not as threatening to their own sense of right and wrong. Whether war is defensible is likely to be a sensitive topic to both doves and hawks, but the connection of the defensibility of war to the right of self-defense is likely to be less sensitive. Therefore, this assignment is especially likely to engage students in critical thinking and perhaps, on some level, challenge their own thinking. Moreover, because the grades on this assignment are based on the logical evidence that students provide for their argument, participants are unlikely to believe that their grades hinge on the instructor's own normative beliefs.

Each of these activities encourages students to engage with normative assessments and normative claims in various ways, yielding many educational benefits. However, these exercises also rely on intellectual material that encourages students to think about normative views in general without treading on ground that might be especially sensitive. Still, we expect that learning about normative claims in a relatively neutral way will ultimately urge students to think about the subjects on which they themselves hold normative views.