Finding the Right Simulation

There are three preliminary steps to choosing the right simulation for your course. The first step is finding or developing an exercise that connects to and supports your course content. Second, you must select the most appropriate format for the simulation from the many options available. Finally, it is necessary to find an exercise that fits into the schedule and time available within the course. There are many possible combinations of these choices, so at each step, it is important to consider the course objectives to ensure that the end result will serve your specific needs. You should identify these objectives before evaluating possible exercises, as a means of both limiting your search and forming clear evaluation criteria as you consider options. For example, if you are teaching a course on economic development in Africa, it makes little sense to look at U.S. Congress simulations.

This section presents the most common options for creating simulation exercises using four distinct format options, which are also summarized in figure 1. Although these options are presented as a simple dyadic choice, it is important to note that there are shades of variation within each option, and it can often be effective to combine these choices if time and resources permit. The combination of options can be extremely worthwhile in more complex exercises for drawing out different communication, research, and interaction styles for students or providing opportunities to reconsider information from different perspectives. For example, you may find it appropriate to conduct an exercise with a mixture of online and face-to-face delivery methods to give students with different communications styles the chance to lead their group.

Figure 1

Decision Stages for the Use of Classroom Simulations

Delivery Method

Although there are variations within each type, simulations are delivered through methods that can be best categorized as online or face-to-face formats. While both can be effective and each has its pros and cons, the choice of delivery method has an effect on the overall atmosphere of the exercise. Simulations can be run entirely in one of these formats or may use a combination of delivery methods if time, space, and resources permit.

The use of technology as an intermediary in online simulations can remove interpersonal aspects of negotiation (e.g., body language, personal charisma) from the mix, forcing a focus on clear written communication. The software used (whether a dedicated software tool or simple e-mail) will generate an archive of the negotiation that can be used in debriefing or student reflection papers. In addition, because students generally have access to their own computers or computer labs on campus, it is possible to schedule the majority of the simulation outside of regular class times so that instruction can continue during the exercise.

However, online simulations also have drawbacks that should be balanced against their positive aspects. Students who are uncomfortable with technology or have limited access to the Internet may not be able to participate at the same level as their peers. This limitation will vary based on lab availability or student characteristics at your institution (e.g., commuter or working students may have limited access to campus computer labs). Instructors must have a relatively high level of comfort with technology, because even the most seemingly intuitive online system will require some introduction, training, and support for students by the instructor, and the instructor must be prepared to troubleshoot and respond to student questions. Finally, there is an inherent bias in using computers in this fashion that should be considered. Some students are going to be able to maximize the technology—for example, by using html coding to make their messages more visually appealing or leveraging their experience with advanced online research tools—and this difference in skill level will result in a hierarchy among teams. (One useful approach to this problem is to identify these students in advance and assign them to different teams.)

Face-to-face exercises are the quickest activities to create and require the least investment for instructors in terms of resources (e.g., fees for software development or customization). Some of the positive aspects of this approach are that students can develop their public speaking skills, hone their use of visual aids (e.g., charts, presentation software), and experience the procedural aspects of a formal conference environment. This format also creates much stricter time constraints for negotiations, which forces students to be much more focused and careful in their use of time during the session. Although this restriction could also be seen as negative, it presents strong incentives for teams to prepare in advance, since they will not be able to conduct additional research or internal discussion during the session. Finally, the instructor has complete autonomy in scheduling the exercise, because there are not other parties involved, so exercises can be concluded or extended as progress dictates.

On the other hand, face-to-face exercises also bring some potential pitfalls to the table. Because all students must be present for the simulation and the logistics of bringing together students from different locations can be complex and expensive, these activities work best in a single classroom. This constraint can limit the variety of perspectives brought to the table, particularly since students with minority opinions may feel less willing to express them in a face-to-face setting. In-person simulations generate no archive of interactions; details will be lost unless arrangements are made to record the proceedings or someone is designated to take notes, so debriefing must be done immediately after the exercise to ensure maximum retention of these points. Finally, instructors must either allocate class time to the exercise or undertake the difficult task of aligning schedules and reserving space outside of class for the exercise. Scheduling is not a major factor with a one-session exercise, but for longer and more complex exercises, it can be a significant burden for the instructor.

Participant Universe

The participant universe for a simulation is defined by who is taking part in the exercise—either a single-classroom or a distributed format (i.e., multiple classes or locations). Although this article specifically focuses on individual courses, it can often be desirable to format an exercise to include multiple classes or locations. For example, a combined exercise involving a political science and an economics class would allow students to gain new knowledge and perspectives by linking the two disciplines. Similarly, it would also be possible to combine these two approaches by separating the negotiation levels (e.g., breaking down a two-level game) or adding a single-classroom pre- or post-simulation exercise to the distributed simulation.

Distributed exercises offer the most potential for the expression of multiple opinions and values, because student teams can be drawn from a variety of locations, backgrounds, and experiences. Many more teams can take part in a distributed exercise—because the potential pool of participants includes classes from around the world, it is theoretically possible to create a simulation involving dozens, even hundreds, of parties (e.g., a Model United Nations exercise). This broadened scope can often produce more complex and creative outcomes as students seek to reconcile often divergent opinions. Finally, because they are engaging with peers from other institutions, students tend to take the exercise more seriously and exhibit competitive traits to prove that their school is more capable. It can be effective, for example, for students to face academic, geographic, or athletic rivals in order to create a powerful motivation for good performance.

However, distributed exercises also pose some challenges in terms of time, space, and expense. It can be very difficult to schedule a distributed exercise, given the variation in holiday and break scheduling. For example, scheduling an exercise with other instructors in the spring is often nearly impossible as a result of varying spring break times between late February and late April, which leaves windows only at the beginning of the semester before students are prepared or the end, when they may be already checked out. It can also be very expensive to arrange travel and physical space for a distributed face-to-face exercise. Online simulations can alleviate this problem, although a group debriefing of all participants remains an obstacle. Third-party distributed exercises may also have different goals or agendas than the instructor has and may even conflict with intended course outcomes. Finally, variations in preparedness and course goals among the locations can produce an uneven experience in terms of quality and learning outcomes.

Single-classroom exercises are by far the easiest simulations to arrange and supervise, because the entirety of the exercise can take place under the supervision of the instructor. Since everyone is from the same class, they all have had the same preparation and expectations prior to participating. This uniformity helps keep the exercise focused and build momentum, because students can refer to information they have all previously learned and can expect everyone to understand. At the end of the exercise, the teacher can easily debrief all participants in a single location, either immediately after the conclusion of the simulation, or after a period of individual reflection.

Some of the drawbacks of the single-classroom simulation are the same as its attributes. Although common preparation and expectations can enhance speed and focus, divergent views are far less likely to be represented. In part, this homogeneity may be due to a lack of awareness of different perspectives, but more often, it stems from students' tendency to defer to the authority of the instructor in terms of perspective and approach. The roleplay aspect of the exercise can be more difficult to maintain within a single class, because students already know each other. They may struggle to stay in their roles or maintain the formality required for an effective exercise. (In other words, outbreaks of silliness have been known to occur.) Finally, the perceived stakes are often lower for students in a single-classroom exercise, since the sense of competing with others is less acute.

Interaction Style

Interaction style refers to the mode of communication, generally distinguished between synchronous (simultaneous) and asynchronous (serial) forms. While most simulations use some combination of these two formats, it is possible to use them independently either throughout or in separate phases of an exercise. In this section, I consider them separately to draw out their distinct characteristics.

Synchronous interactions occur in real time, with all parties sending and receiving communications (verbal or written) either simultaneously or in turns. When using this style, all parties have the ability to observe and take part in all communications, and the instructor can observe these interactions as they occur. This approach forces students to focus on quick assimilation and analysis of information to respond to their peers, and therefore encourages the development of efficient team processes. This format is particularly effective for crisis or other time-sensitive simulations in which the accuracy and timeliness of response is important. Although synchronous interactions are ideally suited for face-to-face simulations, approaches such as scheduled conferences or limited negotiation times can also be incorporated into online exercises.

There are many topics or situations in which synchronous communication can detract from the experience, however. In particular, when used alone, this style tends to produce less complex outcomes, because communications tend to occur in a more linear fashion. This simplifying effect is a product of both the difficulty of holding more than one conversation at once and the natural tendency for discussions to focus more narrowly on one topic at a time. The result is generally an outcome with less opportunity for issue linkage or detailed implementation features. Less time remains for reflection and internal team discussion, as time taken away from the group generally means missing out on group negotiations.

Despite the appeal of synchronous communications, in many cases, an asynchronous model can be more effective. Because a reply to a statement does not need to occur immediately in this model, groups have time for reflection, team discussion, or private consultation with allies about content and strategies. Communications tend to be more complex, since there is more time to craft messages and opportunities that include greater specificity and detail. If the instructor has access to these messages, he or she also has more opportunity to provide feedback and teachable moments during the exercise. This approach is well-suited for longer or more detailed simulations, or simulations that deal with particularly complex issues.

However, while asynchronous communication allows for more reflection and detail in responses, students may not always use the time constructively. Students have more opportunities to check out of simulations when they are not constantly engaged, or if they feel they are not receiving an adequate amount of communication. Instructors may struggle to monitor exchanges in the absence of a designated messaging system or framework, leading to lost teaching opportunities and less thorough debriefing. The teams must accept the responsibility for coordinating their response mechanism for messages, including conducting internal discussions and crafting their reply, and teams that are poorly organized can fall behind quite quickly.

Role Assignment

Students can undertake roles in a simulation exercise either as teams or individuals. In my experience, teams are far more effective, because they add the additional layer of internal negotiation into the exercise and build valuable collaboration skills, but there are situations (such as experimental or training simulations) in which individual roles make the most sense. Although exercises available from vendors generally specify how roles should be distributed, teachers should still consider the relative merits of each approach.

Team role assignments, in which students are assigned in groups to portray a single role, offer a number of advantages. As indicated previously, these exercises allow for negotiations within the team, emphasizing collaborative aspects of international relations. This structure offers students the option of specializing in tasks or issues, allowing them to emphasize their strengths or develop new ones. This format is useful for large classes; for example, it can be effective to divide students into teams by discussion section. Small classes can be divided into teams or can form a single team (divided into issue specialties as appropriate) in a distributed simulation as well.

Placing students in teams can provide a temptation to free-ride within the group, which is often exacerbated by the reluctance of other students to confront the offenders or complain to the instructor. Peer pressure can alleviate this issue to some degree, and confidential peer grades that factor into student grades for the exercise are often quite effective as well. Teams can occasionally produce internal conflict, requiring instructor intervention to resolve and to ensure that students on the losing side of the disagreement maintain engagement with the larger group. It can also be challenging to assign individual grades for team work, although individual assignments within the exercise, peer grading, and observation by the instructor can alleviate this difficulty.

Individual roles are most effective in small groups, both because it is easier to observe each student and because each role is likely to have more relevance within the exercise. With individual roles, each student is explicitly accountable and consequently has a greater investment in the outcome. Grades are much easier to assign to individuals than to groups, and students will generally learn the most in terms of breadth (although not always depth) about the subject matter. This approach can also produce greater diversity of opinion, since each individual must be actively involved.

Simulations with individual roles do, however, tend to be dominated by a few personalities. It is difficult to ensure that all parties have an opportunity to be heard in this case, because there are no team members to balance the dominant personalities. A single student absence or underperformance can disrupt the entire exercise, particularly if that student represents a critical role. Preparation for a simulation of this type is also more complex, because each role must have roughly equivalent supporting material and relevance within the exercise. Although it is usually simple to identify five or six roughly equal roles on a topic, it is significantly more difficult to find twenty or thirty that are equivalent and distinct.

Final Thoughts on Simulation Selection

Selecting the most appropriate format from these options requires you to think through both your expectations for learning outcomes and the level of engagement that you expect from your students. If you want a one-day simulation experience and do not have ready access to a computer lab, it makes sense to use a simulation that is face-to-face, single classroom, and synchronous. If you want an exercise that addresses semester-long content, it may be better to choose an online, asynchronous approach. A smaller class size is often well-served by distributed exercises that increase the number of parties and perspectives represented. It is important to keep in mind that even an exercise that matches your content objectives will not be particularly effective if the format of the exercise does not fit your available time, technology, and logistical resources, so you may need to revise these materials for your particular situation.

A final, although certainly not inconsequential, question for the instructor looking for an off-the-shelf exercise is whether you should pay for a simulation exercise from a vendor or seek free options. A variety of providers can be found online by searching for specific topics: lists of simulations can be found in a number of places such as APSAnet (http://www.apsanet.org/content_15404.cfm), and many journals now include simulations as part of their content. As a general rule, free simulations require more instructor preparation in terms of printing materials and setting up the exercise, and they generally offer minimal, if any, support. These exercises are good options for experienced instructors or teachers with limited access to departmental resources, and they are a cost-efficient way of experimenting with simulations or learning about simulation design in preparation for creating your own. Paid simulations can be relatively costly, but they offer staff support and extensive instructor guides, and provide all necessary materials for the classroom. Online paid simulations (e.g., exercises offered by the ICONS Project at the University of Maryland, http://www.icons.umd.edu) offer a specialized online simulation interface in addition to materials. Instructors are even using online “virtual worlds” (e.g., Second Life) for classroom applications such as structuring environments and scenarios for distance learning and specialized classroom exercises (Foster Reference Foster2007; Childress and Braswell Reference Childress and Braswell2006).

Pre-Simulation Student Preparation

Once a simulation exercise has been selected, you should allocate ample time for student preparation, particularly introducing the content, format, and expectations and allowing time for research and team meetings. Whenever possible, students should be encouraged to connect course content with the simulation in the lead-up to the exercise. This approach encourages active learning in the pre-simulation phase by allowing students to apply knowledge to a concrete task and also increases the likelihood that they will be well-prepared for the exercise itself. I have had success in the past in using simulations as capstone experiences near the end of the semester, allowing students to accumulate learning artifacts through assignments that they can actively apply to the exercise at the end of the course.

There are many ways to link the exercise to content, and this decision should be guided by considerations of how your content fits within the framework of the exercise, and vice versa. One effective approach is to link the issues in the simulation exercise with course content in discrete units—that is, the issue areas in the simulation can become an organizational tool for course content. Having students give classroom presentations on this content, particularly from the perspective of their assigned role, can further increase their learning. Another approach is to integrate simulation preparation into written class assignments and assessments. Assignments and tests that include simulation-specific material serve to emphasize the importance of the exercise as well as increase preparation opportunities. These two options are not mutually exclusive and, in fact, complement each other very well.

Depending on your needs, it may not be possible or desirable to create a course that revolves around a simulation exercise, particularly if you are an instructor new to simulation. However, some level of integration between content and simulation is important for student success in a simulation exercise. As a general rule of thumb, based on personal experience, the best ratio of preparation to simulation is approximately 1:1—a one-day exercise can generally be adequately prepped in one day, while a more complex, several-week-long simulation may require several weeks of content and team preparation. However, much of this preparation can be assigned outside of class time in the form of research papers, readings, or team meetings, and because the simulation will be matched to course goals, this preparation phase will serve these ends as well.

Simulation teams should spend time together in meetings to work through their strategies and tactics. This approach helps overall preparation (content and role information) and allows students to develop a specific team approach for the exercise. Preparation meetings also give individuals a chance to hone their group-work skills by requiring them to reach consensus as a team prior to the start of the exercise. To be effective in a simulation, a team must speak with one voice and maintain a consistent approach to other teams, regardless of who is involved in an interaction at a given time. This internal team negotiation is a key learning outcome for team-based simulations and gives students necessary experience for future academic or professional endeavors.

Facilitator

In nearly all cases, the instructor will undertake a facilitation role during the exercise. This role could be limited to simply serving as a timekeeper or ensuring that sessions start on time and exercise procedures are followed. However, depending on the construct of the exercise, the facilitator role could include moderation, mediation, or even arbitration roles. Ideally, these tasks should be assumed by students, but if this is not feasible, the instructor should take measures to ensure that this allocation does not interfere with the simulation. For example, you might create a structure that compartmentalizes your role by limiting your availability or requiring students to produce a substantive agreement before submitting it to you for arbitration.

Regardless of your function, you should create a clear set of guidelines and expectations for this role and share it with your class. These guidelines are also for your own use as the exercise unfolds and should be referred to often. During a simulation, it is always tempting to expand your role by working with students or offering advice, but you should avoid interfering whenever possible, because students can often perceive an instructor's offhand opinion as an authoritative declaration. If the exercise becomes too instructor-focused, much of what makes a simulation unique can be lost. Less is more when it comes to direct instructor involvement in a simulation exercise.

Control Team

There are occasions on which instructors find that they must adopt an active role as a control team, as a result of class size or the specific characteristics of a given role. This involvement may be necessary when a particular role is more powerful than others and could upset the balance of the exercise, requires specialized knowledge beyond the scope of the course, or has such a limited part in the exercise that it would require little student preparation or execution. Ideally, students would not know that anyone is serving as a control team in the exercise, although in most cases, this concealment is impossible. (Hiding the role is easier with an online exercise, although it is difficult to sustain the deception.) If a control team is necessary, using a teaching assistant or a student not currently enrolled in the course (e.g., part of an independent study, paid student work, or extra credit in another class) is preferable to having the instructor take the role. If the use of an outside individual is not possible, then caution is recommended to preserve the integrity of the exercise. In this situation, the same recommendations from the facilitator role are applicable. If your participation is known, it is important that you and your students have a set of guidelines regarding the capabilities and limitations of your role, and that all parties adhere to these rules during the exercise. You should be aware that while all roles may be created roughly equal, not all participants are. As the instructor, your opinions and preferences will carry far more weight than those of other teams, so you should make an effort to ensure that your presence does not unduly influence other roles.

Observer

The most important functions of the instructor during the exercise are to monitor the simulation to identify issues and problems as they arise, intercede if necessary, and collect material for the post-simulation debriefing. Regardless of any other roles that the instructor may assume during the simulation, the observer is the one role that the instructor must always take. By following the exercise as it unfolds, the instructor is able to provide useful input for students and advice when asked. More important, however, is that the instructor can take advantage of teachable moments as they arise. For example, in the event that a team's strategy has proven unsuccessful, you can encourage that team to reflect on events and persuade them to adjust their approach during subsequent classes. A quick response in these cases can prevent a team from disrupting the exercise or completely checking out.

Some other objectives for instructors can be to monitor students' communication styles and application of course content. The incorporation of concrete simulation examples of why teams were or were not successful will enrich the debriefing process. In addition, occasional interventions may be appropriate to reinforce course goals or provide encouragement to student teams. Instructor involvement during the simulation will emphasize the importance of the exercise as well, keeping students focused and minimizing temptations for free-riding within the group.

Finally, for longer exercises, it is useful to provide students with time to meet in their teams to discuss their progress and address obstacles that may have arisen in the simulation. These meetings can occur in or outside of class, or even through virtual means such as an online class forum or other chat venue. Some ways of managing this process are by requiring a written summary or online transcript of the meeting, or setting aside time during office hours to meet with the group. In this way, students can mutually reinforce course content for each other, and the instructor can provide additional input and encouragement as appropriate.

Group Debriefing Session

The first part of the debriefing process (and generally the stage most anticipated by students) is often the group debriefing session. This should be conducted as quickly as possible after the conclusion of the exercise, and ideally, immediately after it ends, so that recollections are fresh and students remain in their assigned roles. This approach is often referred to as a “hot” or “flash” debrief, because participants have just stepped out of the simulation. Although these sessions can occasionally become tense, the instructor can redirect this passion to increase student understanding of the complexities of the subject matter. (For example, they could draw lessons or linkages from their frustrations to real-life examples of conflict resolution.) If the debrief is not immediate, it is helpful to take a few minutes to recap the exercise and ask students to assume their roles in their group once again.

The first aspect of the group debrief should be an opportunity for students to vent or express their general feelings about the exercise. Frustration and excitement about the results are natural outcomes, and students should have an opportunity to express these reactions at the beginning of the session. Encouraging this feedback first allows students to decompress a bit and also reveals particular aspects for discussion during the remainder of the session. While the instructor must maintain order in the classroom, some back-and-forth between teams is normal in this phase of the group debrief and can be directed into positive learning opportunities. For example, confusion about the behavior of another team can be used to illustrate differences in preferences expressed by rational actors in a negotiation context. This kind of dialogue is often most effective if the instructor steers the discussion to roughly follow the narrative of the simulation, focusing on clear decision points and actions taken by the parties.

A direct outcome of the venting phase can be insight into the motivations, positions, and interests of other parties. Teams can explain or defend their actions in the context of their assigned roles and their outcome preferences for the simulation. Once the simulation has ended, students should be encouraged to share confidential information about their roles, such as data to which they alone had access or the results of their own research and planning. This stage allows students to talk about their team preparation and strategies for the exercise and affords an opportunity for the instructor to relate these elements back to course learning objectives. In particular, discussions about the difficulty that parties face in cooperating or reaching formal agreements will fit into this discussion well, since the motivations of the various parties are transparent and available for questioning by others.

At the end of this session, it is useful to engage in a discussion about group processes, including the difficulties encountered by individual students in working with a group and the strategies that they used for internal organization and interactions with other teams. As students relate experiences within their groups, they can be connected to specific content areas and group dynamics issues, including topics such as aggregation of interests and general group-work skills applicable to the workplace. This discussion serves to emphasize useful skills and approaches to working within groups, as well as to move the focus back to the level of individual students. A discussion of the process of the simulation can be included here, with a focus on student learning at each step of the process, including preparation and various simulation phases. It can be helpful to ask students to relate these aspects of the exercise to their program of study or future academic and professional goals (e.g., how their internal team discussions might relate to their courses in business or help prepare them for law school). In other words, this discussion seeks to emphasize the takeaways from the experience, in addition to content and process knowledge.

The final element of the group debrief should be an instructor-led discussion that specifically relates the exercise back to course content. This discussion serves as a way to fit the exercise within the framework of the overall course and to allow the instructor to resume his or her role as classroom leader. Ideally, this wrap-up discussion should move from simulation-specific examples to increasingly more general applications to encourage students to leave their roles behind and return their focus to classroom outcomes and content. Key concepts from the simulation can be emphasized by taking events from the exercise and attaching names to them (e.g., log-rolling in a government simulation or north-south power differentials in an international relations simulation) or making comparisons with cases covered in class. This approach can serve as a bridge to reconnect the exercise to the rest of the course and return to a more traditional classroom format.

Individual Assignment

After working in a group for a period of days or even weeks, most students are eager for an opportunity to offer their own reactions to the exercise. An individual assignment, generally in the form of a short reflection paper, is a good forum for these opinions. Instructors should link this paper to course objectives and teachable elements of the exercise itself, such as those concepts related to specific outcomes or events that took place. The length of the paper should be correlated to the length of the exercise, but it should ideally be long enough to focus on both process and content issues. This paper could be a formal research paper, although a simple reaction paper drawing from student experience rather than cited works is generally more effective. The instructor should provide a list of questions related to process and content, including elements such as team preparation, internal team organization, negotiation strategies and tactics, perceived relations to course content, and personal reactions to provide guidance to students. It is recommended, although not necessary, that the instructor include an element of analysis in this assignment, either by asking students what they may have done differently, given what they now know, or how they might apply their experiences to future situations. Finally, this assignment is a good opportunity to ask students to explain what they did individually within their groups during each phase of the exercise. While the instructor should have a good idea of what each student contributed, this can serve to either reinforce or fill gaps in this observation.

Simulation as Case Study

Provided that the simulation takes place early enough in the course, it is possible to use this exercise as a form of case study for future class material. Unlike traditional case studies, the simulation exercise is a shared experience that all students take part in and of which they have an understanding that is both detailed and complex. One particularly effective pedagogical approach is the use of specific examples from this experience to illustrate concepts or compare to other cases. I have found it very effective, for example, to relate the simulation exercise to concepts introduced later in the course, such as components of game theory, which can seem somewhat abstract to students. The ability to place names (e.g., “iterated games” or “brinksmanship”) on behaviors that students have already experienced and strategies that they have already used is a powerful pedagogical tool for an instructor.

It can also be useful to use the actual episode or setting on which the exercise was based as a case study separate from the simulation itself. The research that students conducted in preparation for the exercise can serve as a starting point for a more sophisticated examination of the actual case, essentially building on what they have already done with additional information drawn from new course content. Although it is possible to overuse a case in this manner, a great deal of value lies in the occasional use of this approach to provide an increasingly complex understanding of a case study.

Assessing the Impact of Simulations

Assessing the impact of simulation exercises on student learning is difficult, given that a proper research design for doing so does not merely assess learning within a course, but also compares courses with simulations to identical ones without simulations. Because most instructors rarely teach a course more than once per semester, most of the evidence up to now has been anecdotal or, at best, has consisted of comparisons of different iterations of the same course before and after simulations were adopted. Both of these research designs are admittedly problematic, given the limitations of the data and potential intervening variables such as differences in student cohorts. These limitations are often cited in criticisms of simulations, as it is difficult to demonstrate a return on the investment when most of the evidence is based on anecdotal or proxy measures such as teaching evaluations, registration for these courses, or student feedback.

Efforts to quantify the effect of simulation often generate seemingly contradictory results, showing high student satisfaction but negligible testing outcomes (Raymond Reference Raymond2010). However, a growing body of empirical support is demonstrating the positive effects of simulations in the classroom, beyond their effect on grades. Recent work demonstrates that simulations can improve student understanding and retention of content (Yoder and Hochevar Reference Yoder and Hochevar2005), increase student interest in the subject matter more generally (Shellman and Turan Reference Shellman and Turan2006), and even enhance student ability to gain admittance to and succeed in graduate school (Kember and Leung Reference Kember and Leung2005). Pre- and post-unit tests can be useful tools in establishing the effectiveness of simulations versus other pedagogies in individual classes by comparing gains in units with and without simulations. Although this approach remains a difficult-to-quantify aspect of teaching political science, evidence is accumulating in support of simulations.