%20-%20https://sketch.com%20--%3e%3ctitle%3e3ECBBF50-FD08-412A-943D-2CFC5E025A16%3c/title%3e%3cdesc%3eCreated%20with%20sketchtool.%3c/desc%3e%3cg%20id='Stylesheet'%20stroke='none'%20stroke-width='1'%20fill='none'%20fill-rule='evenodd'%3e%3cg%20transform='translate(-926.000000,%20-668.000000)'%20id='BUTTONS'%3e%3cg%20transform='translate(100.000000,%20597.000000)'%3e%3cg%20id='BTN-/-RIGHTS'%20transform='translate(816.000000,%2065.000000)'%3e%3cg%20id='ICON-/-Rights'%20transform='translate(10.000000,%206.000000)'%3e%3cpath%20d='M13.5752841,0%20L1.34943182,0%20C0.59375,0%200,0.586666667%200,1.33333333%20L0,22.6666667%20C0,23.4133333%200.620738636,24%201.34943182,24%20L17.6505682,24%20C18.40625,24%2019,23.3866667%2019,22.6666667%20L19,5.36%20L13.5752841,0%20Z%20M13.5752841,1.89333333%20L17.0838068,5.36%20L13.5752841,5.36%20L13.5752841,1.89333333%20Z%20M1.34943182,22.6666667%20L1.34943182,1.33333333%20L12.1988636,1.33333333%20L12.1988636,5.36%20C12.1988636,6.10666667%2012.7926136,6.69333333%2013.5482955,6.69333333%20L17.6235795,6.69333333%20L17.6505682,22.6666667%20L1.34943182,22.6666667%20Z'%20id='Shape'%20fill='%23595959'%20fill-rule='nonzero'%3e%3c/path%3e%3ccircle%20id='Oval'%20stroke='%23595959'%20stroke-width='1.5'%20fill='%23595959'%20cx='9.5'%20cy='13.5355339'%20r='5.5'%3e%3c/circle%3e%3cpolyline%20id='Path-2'%20stroke='%23FFFFFF'%20stroke-width='1.5'%20transform='translate(9.500000,%2013.035534)%20rotate(45.000000)%20translate(-9.500000,%20-13.035534)%20'%20points='8%2016.0355339%2011%2016.0355339%2011%2010.0355339'%3e%3c/polyline%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e)
Although large classes have become common in higher education, many instructors are not fully prepared to address the challenges they present. These challenges include low student interest, irregular class attendance, low grades, and limited student–instructor interaction. Furthermore, research shows that increased class size has “a negative and statistically significant impact on the amount learned, instructor rating, and course rating” (Monks and Schmidt Reference Monks and Schmidt2010, 15). Facing some of these challenges, we adopted instant-response technology, or “clickers,” in each of our sections of Introduction to Comparative Government and Politics, which normally enroll 110–190 students. Clicker use refers to “an electronic application where a receiver located in the instructor's computer captures information from student keypads in response to questions posed by the instructor” (Premkumar and Coupal Reference Premkumar and Coupal2008, 146).
We share the results of a course redesign to introduce clickers. We analyzed the restructured lectures and how we incorporated clickers, organized a learning experiment, and gathered survey results. The work is based on two courses taught in the fall semester of 2010. Section 1 had 108 students enrolled and met three times a week in 50-minute lectures. Section 2 enrolled 151 students and met twice a week in 75-minute lectures. The teaching experiment was performed in section 2, and both sections produced data for the survey. Here we argue that while the use of clickers is a time-consuming endeavor that presents the instructor with a steep learning curve, using clickers makes teaching large classes more effective and promises to increase student learning.
HOW TO INCORPORATE CLICKERS INTO THE CLASSROOM
Incorporating clicker questions into lectures requires a major restructuring of the course slides or outlines. We find that at most, five or six questions can be included in a 50- to 75-minute lecture. Incorporating clicker questions is time-consuming work, and instructors need to plan the pace of the lecture accordingly. We also find that student participation rises considerably when responses receive a grade or when students know that some clicker questions will also appear in the exams.Footnote 1 However, instructors should specify which questions are graded and which are not.
Our analysis of the way we incorporated clicker questions into our lectures revealed that the technology enhanced the following learning and teaching strategies: (1) student attentiveness, (2) problem-solving group work, (3) real-time assessment of student comprehension, (4) critical thinking skills, and (5) simulation exercises. Although this is not an exclusive list, it does address some of the main pedagogical challenges confronting instructors of large classes.
Student Attentiveness
Maintaining student attentiveness is a major challenge in large classrooms (Bunce, Flens, and Neiles Reference Bunce, Flens and Neiles2010; Hoekstra Reference Hoekstra2008) for several reasons. First, many university students hold jobs and often work full time (Perna Reference Perna2010, xiii) and may have trouble finding a proper balance between academic and work responsibilities. Second, students have ready access to disruptive electronic devices that they often use during class. Third, a large classroom fosters disruptions coming from physical movements and noise by other students. Thus, one of our main concerns was to keep students engaged during class time.
In a traditional lecture without clickers, instructors use a number of teaching methods to maintain attention by raising questions, using slides, or showing photographs and videos (table 1). Although these methods are useful, they keep students in a passive role.
Example PowerPoint Presentation for Traditional Lectures
Alternatively, in lectures that use clickers (or clicker lectures), the instructor organizes the lectures around questions that require instant student feedback (table 2). In other words, clicker lectures include all of the elements used in traditional lectures, but, in addition, they pose questions every 15 to 20 minutes that require students to use their clickers to answer (table 2). In addition, providing a question for students to answer when they arrive in the classroom motivates them to get ready for the lecture and set up their clickers. As explained in detail, because the goal is to keep students' attention, they are not expected to know the correct answer to every question.
Sample PowerPoint Presentation for Clicker-Integrated Lectures
In general, clicker questions aimed at keeping students engaged may pursue various objectives. Questions based on the reading materials encourage students to come to the class prepared by reading in advance. Opinion or polling questions give students the opportunity to anonymously express their views on any subject. Clickers can keep the answers anonymous while still showing that a particular student voted. Answers to these questions may coincide with society-wide attitudes and help the instructor make an abstract topic seem more relevant. Questions to stimulate discussions enhance participation and help students learn from each other or after deliberation. Review questions inform instructors whether students have generally understood a topic allowing instructors to focus on areas that require attention.
Problem-Solving Group Work
Problem-solving group work is especially helpful when the correct answer ratio of a question is low. Students are asked to turn to their neighbors and explain the reasons behind their responses. A two- to three-minute discussion is usually sufficient. As expected, the percentage of correct answers generally improves after discussions. Having students discuss a question in small groups breaks up the routine, gives students a chance to rethink their responses, and provides an opportunity to interact with their classmates. More correct answers following this group-work exercise lends support to findings that students effectively learn from each other (Slavin Reference Slavin1995).
Real-Time Assessment of Student Comprehension
Posing a set of questions at the end of the lecture gives students a chance to review the lecture and test their knowledge, while the instructor gets feedback on whether the lecture material needs further clarification. Assigning a grade to these questions greatly increases students' motivation to be more attentive to the lecture content.
Critical Thinking Skills
Clickers have the potential to enhance critical thinking skills. Five distinct critical thinking skills are description, application, recognition of assumptions, inference, and analysis (Greenlaw and DeLoach Reference Greenlaw and DeLoach2003). While writing assignments increase critical thinking (Çavdar and Doe Reference Çavdar and Doe2012), these skills could be also developed through multiple-choice questions (table 3). We inform our students that our clicker questions will slowly move from description to analysis during the semester. More challenging questions require students to move beyond memorizing what the lecture or reading material stated and analyze the information. Asking challenging questions that require critical thinking keeps the questions interesting throughout the semester.
Sample Questions for Building Critical Thinking Skills
Simulation Exercises
Clickers can be used to construct simulations as shown by this example on political ideologies and electoral systems. After a brief lecture on political ideologies, students form a political party with four to five other students and choose their party's ideology from among liberalism, socialism, and social democracy. Groups are expected to work in class to prepare a campaign speech in accordance with their ideology. They are asked specifically to address taxes and health care and are encouraged to be creative. After 10 to 15 minutes, volunteers from each ideology come to the podium and present their campaign speeches. The classroom is divided into districts, and a general election allows all students to vote for competing parties using clickers. The results are tabulated according to different electoral systems. After a brief lecture, the instructor poses several clicker questions on the advantages and disadvantages of different systems of representation, and students witness first-hand how the three parties fare under each system.
DO CLICKERS INCREASE LEARNING?
Instant-response technology can enhance instructional strategies, but does it support learning? Research in various disciplines examining the impact of clickers on learning report small but positive effects (Hoekstra Reference Hoekstra2008; Morling et al. Reference Morling, McAuliffe, Cohen and DiLorenzo2008; Sevian and Robinson Reference Sevian and Robinson2011; Shapiro and Gordon Reference Shapiro and Gordon2012). Although based on a small sample, our own research confirms these claims. We also considered the impact of the timing of the lectures during the semester and found that the clicker's positive effect on learning wears off toward the end of the semester.
The experiment compared learning results from traditional lectures (the control group) with those from clicker lectures (the treatment group) and found that clickers have some positive effects on student learning. In traditional lectures (see table 1), the instructor would talk uninterrupted for 20–30 minutes, often integrating PowerPoint presentations, photographs, or short videos into the lectures. The clicker lectures, lasting about 20–30 minutes, incorporated the same supporting materials, but to increase student engagement additionally included five multiple-choice questions in the actual presentation, as illustrated in table 2. These questions, asked during the lectures, were designed to focus student attention, test prior knowledge, or check immediate understanding of new concepts.
Each lecture type was followed by five multiple-choice questions to test how effectively students had learned the material covered. In the clicker lectures the five test questions were different from the clicker questions included throughout the lecture. During the semester, six traditional and six clicker lectures were conducted (table 4). Colorado State University's Institutional Review Board evaluated the research protocol and required student consent for participation in the research. All students received a recruitment e-mail and listened to a recruitment speech by our graduate assistants. In all, 125 students signed consent forms; of these 59 participated in the teaching experiment, and 122 answered the survey. Of the 59 students who participated in the learning experiment, 20 were freshmen, 24 sophomores, 9 juniors, and 6 seniors; there were 21 female students and 38 male students; 48 students identified as white and 11 as nonwhite or no reply; and 22 were political science or history majors, and the rest were nonmajors or undecided. The topics of the 12 lectures were randomly assigned to the lecture types.
Lecture Topics
The lecture data was analyzed using a randomized blocked design (RBD) blocking on students. First, the model treats each student as a block allowing us to focus on the variable of primary concern, in this case performance in clicker versus traditional lectures, while controlling for the effects of variability between students (i.e., gender, major, year in college, or any other individual attribute that might influence performance). Except for a few students who participated in either a clicker lecture or a traditional class but not in both, every student has performance scores for clicker and traditional lectures. The estimates were adjusted for student effects that were assumed to be random.
Second, the RBD allows us to determine if the clicker effects depend on time sets, which were treated as six pairs. The lectures were thus paired with respect to students, who were generally exposed to both lecture types. The RBD produced mean estimates based on the model and adjusted for students and missing data. So, if one high performing student missed the class, the model assumed that the estimate would be higher, and vice versa. In addition, when computing overall treatment and control means, the model averaged the dates as if students had equal attendance for each date/lecture-type combination. In sum, the 59 students constituted 59 blocks exposed to 12 lectures that were paired into six time sets during the semester.
As table 5 shows, our research found that clicker lectures, which were designed to increase student engagement, produced higher mean scores on multiple-choice questions than traditional lectures. Table 5 shows statistically significant results and highlights that overall, clicker lectures obtained a higher proportion of correct answers (0.75) compared to traditional lectures (.69). In other words, student scores increased by 6% in clicker lectures, suggesting that stronger engagement with the instructor and the material with the help of clickers increases their understanding.
Analysis of Mean Scores by Type of Lecture using Randomized Block Design (POLS 241-02 Fall 2010)
a In a RBD the error term for comparing methods would be the student by time-set by method interaction. With no missing observations it would have (59 − 1) * (6 − 1) * (2 − 1) = 290 degrees of freedom. However, with 33 missing observations, as was the case in this experiment, the DF equal 257.
Table 6 presents the results of the 12 lectures and when they were delivered. The results show that in general, clicker lectures obtained a higher percentage of correct answers than the traditional lectures (ranging from 1 to 10 points higher). The model also takes into account the effect of time on student performance by pairing lectures into six time sets that included one clicker and one traditional lecture taught within proximate periods of time. This approach allows a time-sensitive comparison of student performance and demonstrates that as the semester advanced, the difference between the proportions of correct answers obtained in clicker versus traditional lectures began to narrow. The last column in table 6 presents the P-values of a comparison of means within each of the six sets of treatment versus control lectures. The analysis produces only two statistically significant results during times 2 and 3, a set of lectures delivered in the middle of the semester. Lower student performance as the semester advances may be accounted for by the pressures coming from a semester's worth of accumulated work—including midterms, papers, and finals. Lower performance may also result because of increasing complexity in the lecture materials as lectures integrate more concepts, or because students must use critical thinking skills to make intricate connections between theories and cases.
Analysis of Mean Scores by Lecture and Time using Randomized Block Design (POLS 241-02 Fall 2010)
Although some results are not statistically significant and would require more data, the overall analysis of the data gathered on the lectures suggests a positive connection between increased student engagement aided by clicker technology and learning in large classes. But do students perceive the same connection? To test this question we conducted a survey to see whether students felt that the technology helped them with learning.
STUDENT PERSPECTIVES OF CLICKER USE
The literature reports that student reactions toward using clickers are mostly positive (Berry Reference Berry2009; Elicker and McConnell Reference Elicker and McConnell2011; Evans Reference Evans2012; Hoekstra Reference Hoekstra2008; Shaffer and Collura Reference Shaffer and Collura2009). To evaluate student perceptions on the impact of clickers, in the last week of classes we administered an opinion survey in both sections. Again, following IRB recommendations, 125 consented to take the survey, and 122 completed it. Students who consented to take the survey were 56 freshmen, 44 sophomores, 16 juniors, and 9 seniors; 57 female students and 68 male students; 100 students identified as white and 25 as nonwhite or no reply; and 43 were political science or history majors, 82 were nonmajors or undecided.Footnote 2
Survey questions appraised student learning, motivation to learn, engagement with the course, incentives to attend class, and overall opinions of the technology (see table 7). We conducted a series of least square means analyses and found no statistically significant results suggesting that student perceptions on clicker technology differed with respect to gender, major, year in college, or ethnicity/race.
Student Opinion Survey on the use of Clicker Technology (POLS 241 Sections 1 and 2 Fall 2010, N=122)
* 1 = Strongly Agree, 2 = Agree, 3 = Neutral, 4 = Disagree, 5 = Strongly Disagree
The survey presents consistently high results in answers to questions on understanding of the subject matter and engagement. Most students agreed that clickers helped them understand lectures (79%), concepts (74%), and course material (70%), although fewer supported the statement that clickers helped them prepare for the exams (57%). Students overwhelmingly agreed that clickers offered ways to test their knowledge (89%) and kept them engaged with the subject (75%) and with instructor questions (71%). They also found that clicker questions were interesting (52%). A minority of students agreed that clickers offered opportunities to engage with their classmates (25%), however, we offered few opportunities for collaboration.
Far less agreement occurred with respect to whether clickers helped students learn more in this course (46%), increase their motivation (29%), or make exploring different concepts more enjoyable (39%). We speculate that students considered the fullness of learning opportunities that go beyond the use of interactive technology, and include other activities delivered by the instructor, and their own personal tasks reading, preparing notes, and completing assignments. Forty three percent of students agreed that clickers made attending class more interesting, and between 54% and 58% agreed that they preferred classes that use clickers. We did not use clickers to take attendance, and many students wrote comments in the survey rejecting the use of clickers to take attendance. Overall, students were satisfied with the use of clickers in the course (58%), but only 39% preferred that clickers be used in all other courses. Thus, students still value a good lecture whether or not it uses clickers.
IN CONCLUSION, DO WE RECOMMEND USING CLICKERS?
Thanks to clickers, we now have more students attending our lectures, and we enjoy channeling their energy from one activity to another. We also appreciate the opportunities to actively involve them with the class material, we appreciate the dynamism that using clickers helps create in the classroom, and we very much like that our students listen carefully to our lectures. Our research largely agreed with the consensus in the literature that clickers do contribute to student learning. However, we do have a few recommendations.
First, instructors must allow a significant amount of time to learn about the technology and ways of incorporating clicker questions into their lectures. Technical difficulties emerge and becoming familiar with the technology requires significant preparation. Universities that employ technical staff often organize clicker training sessions, and online support is also available. However, we find that long-distance support of the software is not as efficient as on-campus support. Thus, we recommend being familiar with the clicker literature that describes potential and actual problems associated with this instrument (Cole and Kosc Reference Cole and Kosc2010; Dallaire Reference Dallaire2011; Duncan Reference Duncan2011; Evans Reference Evans2012; Milner-Bolotin, Antimirova, and Petrov Reference Milner-Bolotin, Antimirova and Petrov2010; Premkumar and Coupal Reference Premkumar and Coupal2008; Sevian and Robinson Reference Sevian and Robinson2011).
Second, we strongly recommend clickers as an optional tool for students. We used the technology that requires students to use a remote, not to download an application for their smartphones. In our case, we found that about one-third of our students did not have the clicker and were not willing to purchase it unless it was required. We anticipate a similar situation with smartphones since not all students might own one or be willing to use the technology. We typically assign some extra credit to students who use the clickers consistently and who have a high percentage of correct responses.
Lastly, we caution against using clickers in administering exams or taking attendance until the software is significantly improved. Common problems include batteries running out or clickers failing to capture student data even if the student has chosen an answer. Allowing students to miss 20% to 25% of the clicker questions gives some room for such failures.
Instant response devices increase instructor effectiveness and student learning by allowing students and instructors to engage in real-time assessments of knowledge gained during lectures. Our lecture redesign had a positive effect on teaching by making classes more engaging, and our experiment found a positive correlation between this technology and learning. However, the results need to be corroborated by similar experiments in other courses. Finally, the survey confirmed that students find clickers useful.
Therefore, we recommend use of this technology with some reservations. Since it involves investing considerable time in redesigning lecture materials, instructors should plan accordingly. Following these recommendations will likely maximize the teaching and learning benefits of this new technology.
ACKNOWLEDGMENTS
The research was funded by a grant from Colorado State University's Institute for Teaching and Learning in 2010–2011 and was the recipient of the American Political Science Association's 2011 Annual Award for Learning Innovation Using Information Technology. We would like to thank James zum Brunnen for his help with the statistical analysis, and Amy Lewis, Jamie Barringer, Theresa Jedd, and Caitlin Wall for collecting classroom data. Special thanks go to Sue Doe for her critical feedback.
