RESEARCH METHODS, BLENDED AND FLIPPED

The great challenge of a blended flipped classroom in research methods is to provide sufficient “structured flexibility” to allow students to work at their own pace without leaving them completely to their own devices (O’Flaherty et al. Reference O’Flaherty, Phillips, Karanicolas, Snelling and Winning2015). This course design creates structure in two ways. First, online modules are divided into three distinct categories: data collection, data analysis, and academic skills. Whereas the academic-skills modules are mandatory for everyone, students are free to choose which module(s) from the data-collection and data-analysis sections they take. Second, clear connections are made between the online and offline components of the course: what students learn in the online modules is immediately applied in the offline workshops, which then is integrated into the concluding assignment for each module. Each edition of the course has four distinct phases (figure 1).

Figure 1

Course Design of the Blended Flipped Classroom

Phase 1 is a face-to-face meeting in which instructors and students meet one another for the first time. With much of the teaching taking place online, it is important to establish from the onset that this is not an only-online course such as a massive online open course or a small personal online course. After the introductory meeting, students have six weeks to select and follow several online modules, attend workshops, and write their assignments (i.e., phase 2). The online-course environment within the online platform currently consists of 11 modules organized around the most common methods for data collection and analysis used in the political and policy sciences (see figure 1). Students must take a minimum of seven modules and can volunteer to do more. A module typically consists of three to four knowledge clips (a maximum of 7 minutes), one or two readings, a quiz, and an assignment.

In phase 3 of the course, students participate in offline workshops related to the online modules. They attend workshops only for the mandatory and voluntary modules that they followed online: typically, three or four because not all modules (e.g., data management) currently have an associated workshop. Workshops are two- or three-hour meetings that occur multiple times throughout the seven-week period of the course. To gain entry to a workshop, students must submit their draft assignment as instructed in the online part of the course. During the workshop, they use their assignment during in-class exercises, and strategies for successful completion of the assignment are discussed. Examples include mock interviews in a “fishbowl” format, an online scavenger hunt to locate quality primary sources, and—taking inspiration from Collier (Reference Collier2011)—a criminal investigation to apply process-tracing (see supplementary material). These face-to-face components of the course are important: during the workshops, students engage in group-based learning activities and receive direct feedback from an instructor. To give students flexibility and to maintain small class sizes, all workshops are offered at least twice and in different weeks.

Phase 4 of the course is assessment: six assignments (30% of the total grade, 5% each) and a written exam (70%). The graded assignments are revised versions of those that students submitted to gain entry to the workshops—when students take the workshop, many realize that they made mistakes. They then are given the opportunity to revise and resubmit their earlier work, before the assignments are graded. After a final face-to-face meeting with all groups of students and instructors, they take a written exam. The exam mirrors the course structure with three different sections: academic skills, data collection, and data analysis. Students must answer one question for each of the mandatory academic-skills modules and choose one question from the data-collection and data-analysis sections. Those who took more than one data-collection or data-analysis module are allowed to choose which question from each section they want to answer. For instance, students are asked to edit a short text that contains obvious flaws in writing, assess the quality of questions in an interview transcript, or interpret a statistical output.

WHICH RESOURCES ARE NEEDED?

The blended flipped classroom approach to teaching research methods relies heavily on digital technologies to deliver course content. Although it is argued that such technologies are particularly beneficial to tech-savvy millennials (Roehl, Reddy, and Shannon Reference Roehl, Reddy and Shannon2013), they do require a substantial investment in time and resources. Therefore, we decided to engage as much as possible with technologies already available to us. First, we used the existing online-course environment used by our university (i.e., “Blackboard”) as the platform for the online modules. Second, we restricted our time in the film studio by recording only the introductory videos for each module. For the substance of each module, we produced our own knowledge clips, using Prezi—a high-quality microphone for voiceovers and screen-capture software. This approach made us less dependent on outside expertise and material and also made it possible to produce content with minimal material requirements. Other scholars have produced and published excellent knowledge clips online; instead of making a similar clip, we linked our modules to those videos. Third, we used the interactive features of Blackboard to set up platforms where students could sign up for workshops, ask questions about the course material, and report problems with the technology.

Nevertheless, the project would not have been possible without substantial institutional support provided by the university. The blended flipped classroom was developed with funds received from the Leiden University’s ICT & Education program, the Faculty of Governance and Global Affairs, and the Netherlands Ministry of Education. Financial support included a grant that allowed instructors to hire teaching assistants, reduce their overall teaching load, and purchase recording material and software. Material support included the availability of an on-campus studio to film the introductory videos, supervised by a professional videographer. Instructional designers, employed by the university’s Online Learning Lab, helped with course design.

Neither the online modules nor the face-to-face workshops required more material resources from students than a traditional classroom setting. They already had access to the existing course-management system and they could complete the online modules on any university computer if they did not have their own computer. Although readings were mandatory for all modules, the course did not require an expensive textbook; the modules were linked to the university library’s catalogue, where a digital copy of publications could be accessed. However, the course required a substantial time investment—admittedly more than the non-flipped version of the course that was taught previously. For that reason, each module component provided an estimate of how much time was needed to complete it. When we received complaints that some estimates did not reflect the actual time a student spent on a component, we made adjustments accordingly.

UNEXPECTED OUTCOMES AND LESSONS LEARNED

Although flipped classrooms have become a veritable fad in recent years, from K–12 to higher education, evidence of positive effects on learning outcomes is still inconclusive. Many studies show a positive effect on student performance, but these tend to be restricted to (predominantly STEM) courses with high levels of foundational-knowledge transmission. Studies also often suffer from a lack of comparative data on student performance in flipped and traditional classrooms. Instead, they rely on self-reporting by students, which is shaped by considerations other than the pure “learning value” of courses. Research also indicates that the observed benefits of the flipped classroom actually may be caused by the use of active-learning techniques rather than the inverted classroom. For instance, the difference in performance between flipped and traditional classrooms disappears when both use active-learning techniques (DeLozier and Rhodes Reference DeLozier and Rhodes2017; O’Flaherty et al. Reference O’Flaherty, Phillips, Karanicolas, Snelling and Winning2015).

Similar caveats apply here: we cannot provide a systematic comparative assessment of the added value of the flipped classroom in comparison with a “traditional” methods course. The previous non-flipped edition of the course did not use such active-learning techniques. Moreover, we could cover a broader range of topics in the flipped-classroom version of the course, which makes the “flipped” and “non-flipped” versions not directly comparable. However, a short qualitative assessment of students’ written feedback provides insight into what they perceived as the main benefits and problems of the flipped classroom. For instance, the first iteration of the flipped classroom made some students feel “like a lab rat being experimented on.” However, feedback addressing the format was generally positive, highlighting the advantages of its flexibility—for instance, the possibility to “work at my own pace and in my own time” and “pick the specific research instruments you needed.” This flexible approach, however, also raised “problems with self-discipline”; because there was no weekly class, “I tended to push back doing the modules.” Critical comments focused on the course workload, which indeed was expanded with the introduction of the flipped classroom.

We also observed—in line with the literature—that to be successful, flexible pedagogies require “flexible students” (Wanner and Palmer Reference Wanner and Palmer2015). Hence, we encountered resistance to the format of the blended flipped classroom. Particularly in the first edition of the course, some students found the instructors to be too absent from their daily learning experiences. One student commented in the course evaluation as follows: “The teachers are only using this format to reduce their own workload. Why can’t they just give a traditional lecture instead of letting us do all the work?” The introductory meeting for each new edition of the course has been changed to begin with an extensive explanation of the rationale for the blended flipped classroom. It offers students flexibility and specialization but also requires active involvement in their own learning experience. It is a message that is repeated continuously, from the homepage of the online-course environment to the various offline workshops that are offered.

Mindful of the limitations of comparison, the average scores of student evaluations for the first iteration of the flipped classroom were not surprisingly lower than the last iteration of the non-flipped version. Course evaluations are administered after the final exam and consist of 15 statements covering a range of topics from perceptions of workload to perceptions of the instructor’s performance. Students must indicate their agreement with the statements on a five-point Likert scale. For “The quality of the course was [1=poor, 2=insufficient, 3=sufficient, 4=good, 5=excellent],” the non-flipped course scored 3.8 (N=26) and the first edition of the flipped classroom scored 2.7 (N=74). However, these scores came closer to old levels in subsequent course editions, as issues related to the format and integration with other courses were resolved. This highlights the idea that “flipping” the classroom is an investment over time, which requires fine-tuning and improvement throughout a period of trial and error.

Moreover, students showed a greater mastery of research-methods skills in the assignments and on the final exam than in the previous course format. By the time students take the final exam, they already have practice with the course material, which often pays off: the final exam has a high passing rate, despite that students find it more demanding. So far, it has fluctuated between 84% and 94% each course edition (pass rates for the course tend to be lower because students often need to revise and resubmit module assignments). Furthermore, colleagues involved in thesis supervision report an across-the-board improvement in research skills that students display in their individual projects. This has benefited not only the students but also the thesis supervisors who spend less time correcting students’ work. Unfortunately, we do not have data available to demonstrate the effect of the curriculum on courses other than our own.

A final unexpected outcome of the flipped-classroom project was the new connection it forged between the research-methods course and the education provided by faculty in the department. To some extent, this connection already was embedded in the course design: the flipped classroom includes a podcast channel on which our colleagues discuss their use of particular methods in their research. They also are helping to design new modules that teach the methods in which they specialize. With all knowledge clips created under a Creative Commons Share Alike license, moreover, colleagues can use the curriculum in their own courses or in thesis supervision.

Outside of the course, the curriculum has served two purposes. First, colleagues teaching other courses have assigned parts of our curriculum if they believed students needed a quick refresher on a particular topic. This has reduced their workload, particularly in the context of thesis supervision. Second, the course has set explicit standards in academic writing, research ethics, and data management throughout the graduate program, where these were previously implicit. In this respect, the blended flipped classroom on research methods has acted as a common good in the teaching profile of our institution.

SUPPLEMENTARY MATERIAL

To view supplementary material for this article, please visit https://doi.org/10.1017/S1049096519000581