%20-%20https://sketch.com%20--%3e%3ctitle%3e3ECBBF50-FD08-412A-943D-2CFC5E025A16%3c/title%3e%3cdesc%3eCreated%20with%20sketchtool.%3c/desc%3e%3cg%20id='Stylesheet'%20stroke='none'%20stroke-width='1'%20fill='none'%20fill-rule='evenodd'%3e%3cg%20transform='translate(-926.000000,%20-668.000000)'%20id='BUTTONS'%3e%3cg%20transform='translate(100.000000,%20597.000000)'%3e%3cg%20id='BTN-/-RIGHTS'%20transform='translate(816.000000,%2065.000000)'%3e%3cg%20id='ICON-/-Rights'%20transform='translate(10.000000,%206.000000)'%3e%3cpath%20d='M13.5752841,0%20L1.34943182,0%20C0.59375,0%200,0.586666667%200,1.33333333%20L0,22.6666667%20C0,23.4133333%200.620738636,24%201.34943182,24%20L17.6505682,24%20C18.40625,24%2019,23.3866667%2019,22.6666667%20L19,5.36%20L13.5752841,0%20Z%20M13.5752841,1.89333333%20L17.0838068,5.36%20L13.5752841,5.36%20L13.5752841,1.89333333%20Z%20M1.34943182,22.6666667%20L1.34943182,1.33333333%20L12.1988636,1.33333333%20L12.1988636,5.36%20C12.1988636,6.10666667%2012.7926136,6.69333333%2013.5482955,6.69333333%20L17.6235795,6.69333333%20L17.6505682,22.6666667%20L1.34943182,22.6666667%20Z'%20id='Shape'%20fill='%23595959'%20fill-rule='nonzero'%3e%3c/path%3e%3ccircle%20id='Oval'%20stroke='%23595959'%20stroke-width='1.5'%20fill='%23595959'%20cx='9.5'%20cy='13.5355339'%20r='5.5'%3e%3c/circle%3e%3cpolyline%20id='Path-2'%20stroke='%23FFFFFF'%20stroke-width='1.5'%20transform='translate(9.500000,%2013.035534)%20rotate(45.000000)%20translate(-9.500000,%20-13.035534)%20'%20points='8%2016.0355339%2011%2016.0355339%2011%2010.0355339'%3e%3c/polyline%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e)
Lecture is, arguably, the oldest known instructional technique used in the university setting. Since it was first employed in Plato's Academy, lecture has become an indispensable part of teaching favored across the college and university curriculum. Recently, this time-honored method of instruction has come under attack for its presumed inability to foster higher order cognitive and attitudinal goals (Cashin Reference Cashin1985; Day Reference Day and McKeachie1980; Frederick Reference Frederick, Pescosolido and Aminzade1999; Renner Reference Renner1993). Critics of traditional lecture-based formats call for their replacement with active learning approaches that provide students with an opportunity to meaningfully talk, interact, write, read, and reflect on the content, ideas, and issues of an academic subject (Meyers and Jones Reference Meyers and Jones1993, 6).
Shall we cut back on lecturing in favor of novel “active” methods of instruction? Are lectures less effective than active learning techniques in promoting students' learning? This study compares student learning in traditional lecture and debate formats. Educational debate is an active learning strategy designed to engage students in the practice of important cognitive skills, such as critical thinking and deliberation, among others (Bonwell and Eison Reference Bonwell and Eison1991; Scannapieco Reference Scannapieco1997). Developing these skills and shaping learners' attitudes toward divisive topics is widely assumed to be the greatest educational value of debate (Bauer and Wachowiak Reference Bauer and Wachowiak1977; Brembeck Reference Brembeck1949; Budesheim and Lundquist Reference Budesheim and Lundquist1999; Combs and Bourne Reference Combs and Bourne1994). There is, however, a lack of compelling evidence linking debates to improved student learning (Hill Reference Hill1993; Nandi et al. Reference Nandi, Chan, Chan, Chan and Chan2000).
Much of the research on the educational value of debate assesses the performance of students enrolled in argumentation courses, or those involved in formal debate tournaments. Since these analyses extend over a long period of time, during which debaters experience a medley of influences on their thinking, isolating the impact of debates on student learning has proven problematic. The attraction of students with high cognitive abilities to formal debates hindered the establishment of an irrefutable causal link between participation in debates and improved critical thinking (McGlone Reference McGlone1974, 140). The dearth of scholarship comparing debates with other instructional techniques, and problems with the measurement of learning outcomes, have undermined the quality of empirical findings (Antepohl and Herzig Reference Antepohl and Herzig1999, 107). Furthermore, since the majority of past studies have analyzed debating students but drawn conclusions about the impact of debates generalized to all students, to date no research has convincingly demonstrated whether and how educational debates contribute to the learning of both debaters and non-debaters.
Our research analyzes the immediate effects of debating on the well-defined levels of student learning and compares the impact of debates on the students' higher order cognitive skills to that of traditional lectures. To do so, we conduct a classroom experiment which exposes students to both debates and lectures over the course of the study. We also develop improved measurements of students' cognitive skills and learning. These skills—knowledge of factual information, comprehension of complex concepts, the higher level cognitive skills of application and evaluation, and affective learning—encompass the majority of levels of intellectual behavior important in learning (Bloom Reference Bloom1956; Krathwohl, Bloom, and Bertram Reference Krathwohl, Bloom and Bertram1973).
The Effect of Lectures vs. Debates on Student Learning
The persistent popularity of the lecture-based teaching model among teachers has been attributed to its efficiency as a method for instruction. It is inexpensive, since one instructor can teach a large group of students, and familiar to students and teachers alike. Lectures are effective in covering large chunks of the material quickly, and can be easily adapted to fit the needs and interests of a particular audience (McKeachie Reference McKeachie1969). It is for its assumed ineffectiveness as a method for student learning that lecture fell into disrepute among educators. Lecture's strength, “speed teaching and learning,” is also one of its major weaknesses as this type of presentation leaves little room for fostering important intellectual virtues (Paul Reference Paul, Pescosolido and Aminzade1999, 128). Green and Dorn (Reference Green, Dorn, Pescosolido and Aminzade1999, 60) find that when students take an alienated and superficial approach to learning, as they often do in lecture, the result is short-term memorization of lecture content.
Thus, the memorization of information, or the transfer of basic knowledge, is the only learning outcome that has been attributed to lecture (Antepohl and Herzig Reference Antepohl and Herzig1999; Nandi et al. Reference Nandi, Chan, Chan, Chan and Chan2000). The lecture format is conducive to direct reproduction of information laid out by the instructor. Since the lecturer has maximum control over the flow of information, and is able to provide students with precise, up-to-date, and not otherwise available information more effectively than debaters, who may deliberately or inadvertently misrepresent or omit data, we expect that lecture will be more effective than educational debate in transmitting factual knowledge to students.
Many educators contend that lectures are ill-suited for fostering higher order cognitive skills. Comprehending complex material, for example, requires the ability to connect several components of a phenomenon in a logical and meaningful way. It requires greater student engagement with the material and more “mental energy” than students typically expend in a traditional lecture mode, unless the lecture is accompanied by discussion, short papers, etc. (Renner Reference Renner1993; Ruyle Reference Ruyle, Kelly and McGraw-Hill1995). Subsequently, we expect that lectures will be less effective than debates in fostering students' understanding of complex concepts.
Lectures also do not excel in fostering the skills of application, analysis, and evaluation. In a lecture class, students are passive learners, engaged in extensive note-taking. This format of instruction enhances students' ability to memorize and reproduce the presented information directly without examining and analyzing it. Educational debate, on the other hand, can help students learn how to formulate clear, precise, and logical arguments. The latter is one of the most complex cognitive tasks as it requires research, analysis, synthesis, organization, and evaluation of information (Puchot Reference Puchot2002). In debates, students often invoke real-life examples to corroborate their theoretical arguments. Motivated to persuade the audience to their side, debaters can appeal to familiar practices to make their arguments relevant to students' lives. Therefore, we expect that debate will surpass lecture in promoting students' skills of application. Participants of debates are encouraged to weigh facts, compare arguments from various perspectives, generalize, and reveal fallacies in their own positions and in arguments of their opponents. For this reason, we expect that, compared to lectures, debates will better facilitate students' evaluation skills.
Effective learning is impossible without engaging students' emotional side—their attitudes, feelings, preferences, and values. Learners who have no desire to acquire and use new knowledge cannot obtain an in-depth understanding of the complex issues and transferable skills of well-reasoned thinking. For the purpose of this study, we tested the impact of lectures and debates on three affective components of learning: students' interests, concerns, and attitudes toward the subject of a lecture or debate.
Some scholars argue that lecture is an effective way of communicating the intrinsic interest of the subject matter (Cashin Reference Cashin1985, 54; Frederick Reference Frederick, Pescosolido and Aminzade1999, 63). Others contend that lecture experiences are “boring, irrelevant, and not useful” (Renner Reference Renner1993; Nandi et al. Reference Nandi, Chan, Chan, Chan and Chan2000). Debates, on the other hand, often generate a highly-engaged learning environment. Many students rate debates highly both for their ability to enhance learning and for the personal enjoyment they derive from participating in them, and some report important attitude changes attributed to the debate format (Bauer and Wachowiak Reference Bauer and Wachowiak1977; Budesheim and Lundquist Reference Budesheim and Lundquist1999; Green and Klug Reference Green and Klug1990). Consequently, we expect that debates will generate higher interest and greater concern with the debated topics than will lecture, and that students will be more likely to change their points of view following a debate than after a lecture.
To recapitulate, we believe that lecture is an effective method to disseminate factual knowledge, but inappropriate for developing high-level cognitive skills. We expect that students who are presented with the material in the debate format will have a better understanding of complex issues, and will demonstrate better application and evaluation skills than those students learning from lectures. Further, we hypothesize that students who participate in the debate format will be more engaged with the subject matter than will students who participate in the lecture format.
Research Design
Overview
To examine the comparative advantages of lectures and debates in fostering student learning, we conducted an experiment. The participants were 60 undergraduates (27 females and 33 males) enrolled in introductory political science courses at a large public research university. The data were collected during fall 2005 and spring 2006 semesters in two classes.
Our research was conducted in the classroom setting. We selected six controversial topics from the classroom curriculum taught by one of the researchers. The topics were reformulated into questions with two opposite and well-documented positions. In the first class, topics 1, 3, and 5 were presented using debate format; topics 2, 4, and 6 were presented in lecture format. In the second class, we switched the formats and presented the first trio of topics using lecture, and the second trio using debate. Both lectures and debates were structured similarly. At the beginning of a class meeting, an instructor (one of the researchers) posited a question and summarized the opposite perspectives on the raised controversy. Then, students were offered a brief questionnaire (a pre-test questionnaire) asking about their degree of interest, level of concern, and attitudes toward the topic.
Following this introduction, in the debate sections, two teams—a pro and a con team—had to defend opposite sides of the topics; in the lecture sections, the instructor presented, defended, and challenged the topics' contrary positions. In the debate sections, the teams were required to conduct a thorough analysis of the debate topics and prepare arguments in defense of their own positions and in refutation of those of their opponents. In the lecture sections, the entire class received reading assignments, but only the instructor carried out additional research of the pertinent literature.
To ensure every student's direct involvement with the debates, we took a pre-debate poll of students' opinions on the debated issue and assigned students the role of debate judges. At the end of each debate session, students were asked to vote for the team that was able to better defend its position on the issue. Another strategy for involving the non-debating students was through the Q&A session. Questions from the audience, during which the non-debating students could request the debaters to clarify their arguments, present additional evidence, or consider alternative approaches to the issue at hand, was the lengthiest segment of debate.
At the end of the class meeting, the students were administered the same questionnaire (a post-test questionnaire) tapping into their interests, concerns, and attitudes toward the topic. In addition, students had to answer six questions that we used to measure learning outcomes.
Measurements of Learning Outcomes
We conceptualize learning outcomes as students' knowledge of factual information, comprehension of complex concepts, and higher order cognitive skills of application and evaluation. Knowledge, the simplest cognitive skill, involves recall and recognition of dates, events, places, or major ideas (Bloom Reference Bloom1956). To measure students' knowledge of the topic we offered them three multiple-choice questions. Students had to choose a correct place, date, or name and recognize a perspective on the debated issue. For each correct answer a student earned 1 point. We added the points to form a 3-point scale.
Comprehension of concepts is characterized by an ability to grasp and explain their meanings in one's own words (Bloom Reference Bloom1956). For every topic, we picked one key concept and asked students to explain what it means. We identified the core elements of each concept and evaluated students' responses against this definitional template on a 5-point scale.
Application is a higher order cognitive skill that enables students to see how the acquired knowledge can be relevant to situations not discussed or considered in class. To measure students' ability to apply the learned material, we developed hypothetical scenarios describing real-life problems within the scope of the debated topics and asked students to devise informed solutions to those problems. For each scenario, we identified five elements of an “ideal” (educated) solution and used this template to gauge students' responses on a 5-point scale.
Critical evaluation, the most complex cognitive skill, consists of making judgments based on reasoned argument. It encompasses an ability to derive conclusions, justify and verify decisions, and think without bias and prejudice. To measure students' skills of evaluation, we formulated an argument representative of one position on the discussed issue and asked students to respond to the argument from the same or a different perspective. These evaluation tasks tested students' ability to trace an argument to one of the opposite viewpoints, deduce conclusions from an approach to the problem, and justify the answer. The evaluation exercises also required students to resist individual biases and preferences. We identified standards for successful completion of each evaluation task and graded students' responses on a 5-point scale. Appendix I contains sample templates that we used to evaluate students' skills of comprehension, application, and evaluation in one of the lectures.
We also examined the impact of lectures and debates on students' level of interest, concern, and attitudes on the topics under consideration. To measure students' interests and concerns, we used 5-point Likert scales with 1 indicating no interest or concern, 3 indicating a neutral point, and 5 indicating the presence of interest or concern. To measure students' attitudes toward an examined question, we asked which of the two positions they supported, or whether they were undecided on which side to take. For the final analysis, we subtracted the post-lecture/debate ratings of interests, concerns, and attitudes from the pre-lecture/debate ratings. The positive scores indicate an increase in the students' interests and concerns, or change in the attitudes following a lecture or debate.
Results
All students' responses were combined into a large sample consisting of 180 cases with students' scores recorded after six debates, and 180 cases with students' scores recorded following six lectures. We used the runs (Wald-Walfowitz) test procedure to examine whether our data violated the assumption of randomness. Since the runs test demonstrated that the order of values of the dependent variables was irrelevant, and the data largely satisfied the assumption of normality, we used the Independent Samples t-test to examine the impact of lectures and debates on the described learning outcomes. Table 1 contains the results of the t-tests.
Independent Samples t-tests of Cognitive Skills
Our expectation that lecture students would score higher on factual knowledge was somewhat supported by the data. The average knowledge scores from lectures (M = 2.42) were higher than the average knowledge scores from debates (M = 2.29), and the difference between the two groups of scores is statistically significant at the .10 level. Substantively, however, the difference is rather small, 0.13 or 4.3% on a scale of 0–3.
Debate students received higher comprehension scores than lecture students (M = 2.09 for lectures, M = 2.67 for debates, t = −4.53, p = 0.000). The average difference is 0.58 on a 5-point scale, equivalent to 11.6%, or a whole letter grade. In line with our expectations, the average scores on students' application and critical evaluation skills were also higher for those students who learned from debates instead of lectures, and these results are statistically significant at the .05 and .01 levels. It is important to note, however, that substantively, both the debate and lecture application and evaluation scores were disappointingly low. The means of the application scores were only 1.91 for lectures and 2.19 for debates, and the means of the evaluation scores were 1.47 for lectures and 1.91 for debates, all measured on a 5-point scale. These data imply that neither lectures nor debates seem to excel in developing higher order cognitive skills.
To examine the impact of instructional methods on students' affective learning, we used the two-sample Wilcoxon rank-sum test to examine whether there are differences in the degree of change in students' interests, concerns, and attitudes caused by their exposure to different modes of instruction. We opted for a nonparametric test because the distribution of differences in scores failed to satisfy the assumption of normality. The Wilcoxon test is performed on the absolute values of differences between the two test variables. These differences are referred to as ranks. Negative ranks indicate differences below 0, positive ranks indicate differences above zero, and ties indicate when differences equal zero. In our case, negative ranks mean all those instances in which lectures generated higher interest and greater concern in students than did debates, whereas positive ranks indicate the cases in which students' interests and concerns increased more following the debates than the lectures. Table 2 reports the results of the Wilcoxon tests.
Two-sample Wilcoxon Rank-sum Test of Affective Skills
a Based on negative ranks
Since the number of negative ranks is greater than the number of positive ranks on the variable of interests, lectures in our study stirred more student interest in the topics presented by the instructor than did debates. However, this difference is not statistically significant. The analysis reveals significantly higher numbers of positive ranks on the measure of students' concerns. This finding shows that students' concern for the issue increased more following the debates than the lectures. Consistent with our expectations, debates impact students' attitudes more than lectures. A greater number of students changed their position on the issues discussed in the class following debates than following lectures, a finding significant at a .05 level.
Discussion of Findings
The empirical analysis largely supported our expectations of the impact of lectures and debates on student learning conceptualized as the knowledge of facts, comprehension of complex concepts, production of higher order cognitive skills of application and evaluation, and engagement with the subject matter. The tests demonstrated that in our classes students acquired better comprehension, application, and critical evaluation skills when a controversial topic was taught in the debate format. With regard to basic knowledge, lectures better facilitated students' memorization, recall, and recognition of information, however, this finding was only significant at the .10 level.
Although students' application and evaluation scores were higher after debates than after lectures, neither lectures nor debates excelled in promoting students' application and critical evaluation skills in our classes, as demonstrated by low ratings of student responses to the application and critical evaluation tasks. To facilitate students' application skills, instructors must focus part of their teaching on practicing the transfer of the acquired knowledge to novel situations. Furthermore, if we want our students to practice higher order cognitive skills, we must ensure that they are capable and motivated to carry out these complex intellectual tasks. The former can be accomplished by developing and discussing criteria for making well-reasoned judgments, communicating the instructor's expectations to students, and explaining how those expectations can be met. Students are typically more motivated to invest their intellectual energy in solving complex problems when they know that their work will be graded, and when they have sufficient time to complete intellectual tasks. Since we did not grade students' responses, they may not have been motivated to think in a highly reasoned way. Instead, students may have been motivated to arrive at quick solutions in order to finish by the end of the class.
With regard to emotional involvement with the subject matter, to our surprise, lectures triggered greater interest in students, but this result was statistically insignificant. It is well known that enthusiasm breeds interest. The increase in student interest in the lecture class could be a result of the instructor's enthusiasm for the topic. Debates, on the other hand, encouraged students to reevaluate their positions and change their attitudes toward the discussed issues. It is important to note that our students were repeatedly reminded that winning the debate was not its purpose. The instructor admonished debaters against propagandizing for their position. The students were aware that their presentations were to be evaluated on the quality of reasoning and empirical and theoretical support for the arguments. Therefore, we believe that the change in the students' opinion occurred largely due to the deliberation accompanying the debates.
The findings of this study confirm that the traditional and active learning methods of instruction can produce different learning outcomes. Lectures can be used for promoting basic knowledge of the subject material. Debates appear to be more effective in developing students' comprehension of complex concepts and application and critical evaluation skills. A combination of both the conventional and active-learning curricula may provide the most effective training for undergraduate students. We wish to emphasize, however, that debates are not the only way to get students to think more critically and analytically, and they, perhaps, are not the most effective active learning strategy for improving students' critical thinking. The most effective instructional curriculum has yet to be found.
From a methodological perspective, ours was not a true experiment in that the subjects were not assigned at random to the lecture and debate treatments. We have tried to rule out the effects of several potential sources of bias. For example, the distributions of students by year in college, gender, and major field were similar between the classes. We found no significant impact of the order in which we administered lectures and debates over the course of the study on student learning. We also calculated the power of the study to ensure that we did not miss a possibly relevant influence, other than the teaching format, on the groups.
Appendix 1. Sample Questions and Evaluation Templates
Issue: Can workfare help poor women with children escape from poverty?
Question 1 (Comprehension): Define workfare under Personal Responsibility and Work Opportunity Act
Evaluation guidelines: a satisfactory answer must contain the following five points:
-
ʼn It is a policy requirement – 1 point
-
ʼn It is a basis for a welfare check or other welfare provisions – 1 point
-
ʼn Recipients have to work – 1 point
-
ʼn Any legal job can be held – 1 point
-
ʼn 30 hours per week for a period of time of not more than three years – 1 point
Question 2 (Application): Imagine the following situation. You are the head of the Indiana Office on Welfare and Social Affairs. The PRWORA is a federally recommended program. In this situation, which policy direction will you choose: (1) Establish welfare on the basis of need for unemployed people without a term; or (2) Follow the PRWORA recommendations and establish workfare requirements for those recipients who spend 5 years on welfare. Explain your position.
Evaluation guidelines: five points are awarded to the answer that specifies the following:
Clearly states a policy position – 1
-
ʼn Specifies the reason behind this decision: (1) For the first position, the possible reasons can be to encourage people to get off welfare to work; to take low-income jobs; to create work attachment and work ethics; or to establish training and educational programs. (2) For the second position, the possible reasons can be to understand different situations of people; to understand time constraints that are not enough for people to get out of poverty; to understand the importance of other provisions for women with children, such as childcare, healthcare, food stamps, housing – 3 points maximum.
-
ʼn Refute/criticize the opposing position; recognize/mention some limitations – 1 point
Question 3 (Critical Evaluation): How would an opponent of workfare respond to the following statement: “The research demonstrates that from 1993 to 1998 there was a significant decline in caseloads – by 44%. This decline demonstrates that the PRWORA is a success”?
Evaluation guidelines: five points are awarded for the following considerations:
-
ʼn The PRWORA was designed to decrease the caseload, so people try to get off welfare when they find a job in order to retain the welfare option if they become unemployed again – 2 points
-
ʼn The decrease in caseload does not mean that people live better. They simply may not have an option to use welfare – 2 point
-
ʼn Economic growth and the low unemployment rate in that period contributed to the decline in caseloads. If economic crises hit and unemployment goes up, the caseload will increase – 1 point
