Design of the Simulation

Smith and Boyer pointed out that designing a simulation involves clarifying its goals, creating a compelling scenario, identifying the major actors, and establishing the basic rules (1996, 691–3). I based the simulation's design on the combination of two real-world multilateral negotiations on biological weapons controls, the first of which took place between 1995 and 2001 and failed to produce a tangible result in the context of the so-called Ad Hoc Group (AHG) (Littlewood 2005). The second set of discussions among member states of the BWC were undertaken from 2003 to 2005 and had been conceptualized as a much more limited replacement of the first set of negotiations (Tucker 2004).

These negotiations guided the simulation both in terms of content and country delegations. In terms of the latter, I chose six states for the simulation, which took opposing positions on a number of issues in the AHG negotiations. The states were: Brazil, China, Iran, United Kingdom, United States of America, and South Africa. The issues to be addressed during the negotiations were (1) verification measures, including declarations and inspections; (2) export controls and international cooperation measures; and (3) national implementation measures and the creation of an international organization for biological weapons control. I established a negotiating committee for each of these three clusters, mirroring the activities of the “Friends of the Chair” in the AHG negotiations. Table 1 gives an overview of the relative negotiating positions of the six countries with respect to the issues. These positions largely reflected the positions these countries had taken in the AHG negotiations and were only modified to accommodate the goals of the simulation.

As you can see from the columns in Table 1, country positions on most of the substantive issues ranged from strong support to strong rejection. These initial country positions were part of confidential instructions each delegation received in week three as part of the introduction to the simulation. This introduction also addressed the scenario in which the negotiations are embedded, pointed towards additional sources available for the conducting the simulation, and established the ground rules. I directed students towards resources they should consult, including two web sites: www.opbw.org and www.brad.ac.uk/acad/sbtwc/. While the first of these hosts has practically all official documents submitted to and produced during the two sets of biological weapons arms control negotiations mentioned above, the latter one contains a large number of briefing papers analyzing various aspects and overall progress of the negotiations to strengthen the BWC.

The ground rules included the following:

• take role assignments seriously;
• treat country instructions as confidential;
• assign issue area responsibilities within delegations;
• reference all materials used in any written contribution to the simulation, e.g., position papers; and
• none of the materials found on the two web sites mentioned above can be used to sabotage the simulation.

This last rule was a precaution to avoid a replay of the failure of the AHG negotiations in 2001 as an easy way out. In order to allow for caucusing, I set up three regional groups: the Southern Group (Brazil and South Africa), the Eastern Group (China and Iran), and the Western Group (UK and the U.S.). For each of the substantive negotiating sessions there was a series of student activities: clarify (and develop) country positions, discuss in regional groups, discuss in subject committees, participate in informal consultations, and meet in plenary. Students agreed to set up three committees on verification measures, international cooperation and export controls, and national implementation measures and international organization. Each committee was chaired by a representative of a different regional group. Lastly, I announced that I would serve in the role of the “secretary to the negotiations” in order to aid the chairperson of the negotiations and the three committee chairs conduct their business. This role, as well as the option for the professor to act as the various “capitals” of delegations participating in the negotiation provided the necessary tools to prevent an early end of the exercise either through early resolution of all differences or through a collapse of negotiations. While the secretary to the negotiations role allowed for some influence on the procedural level, the capability to send additional instructions from “capitals” let me steer the simulation—only if the need arose—in relation to substantive negotiating positions.

Conduct of the Simulation

The introductory session formed the first of five sequential sections of the simulation. The remaining four sections are: negotiating round one (weeks four through six), negotiating round two (weeks seven through nine), a drafting exercise (week 10), and the debriefing (week 12). The first two of these sections, plus the corresponding report writing assignments and the latter two sections taken together, can each be conceptualized as a learning cycle according to Kolb's model—adding up to three learning cycles over the duration of the simulation. The modular approach of the simulations should also allow for its utilization in differently structured courses. With its spread over practically a whole semester, it is certainly at the extreme end of the typical range of simulations' duration. However, it would easily be possible to have only one negotiating round or to reduce the number of negotiating committees through a reduction of topics to be addressed. One might even consider placing the simulation at the end of a semester and having only a short negotiating round plus the drafting exercise in one afternoon. This, obviously would require much more preparatory work, which could be organized either in the form of lectures or group work by students themselves. Any of these changes, however, would also have to be reflected in the assessment structure with a likely reduction in the proportion of the final grade that the simulation determines.

Table 2 below shows that in the extended version of the simulation each of the three committees held one meeting during both negotiating rounds. As country delegations in the simulation were composed of two or three students at most, each student had an opportunity in both negotiating rounds to present his/her country's position on particular issues and discuss the topic he/she had taken responsibility for. This concrete experience served as a platform for the reflective observations that found their expression in the negotiating journal students had to keep. The short reports at the end of each negotiating round, which form part of the summative assessment of the course, then led students to Kolb's abstract conceptualization stage. This was achieved by linking the coursework not only to the simulation, but also to the required reading for the sessions on international regimes and international negotiations.¹

This, in turn, informed the active experimentation stage at the end of the first negotiating cycle, thereby paving the way for round two of the negotiations. As Brock and Cameron have noted, such experimentation “actions, of course, are a new concrete experience. The learning cycle begins anew” (1999, 251).

For this second cycle the students were tasked to have their position papers for each of the committee meetings prepared in advance. While the presentations of country positions during the first negotiating round occurred only orally and were largely based on the provided confidential country instructions, I expected contributions to the second round of negotiations to be more substantial, detailed, and based on additional research. These national position papers in turn served as the basis for the drafting exercise in week 10 of the course. For this I provided an “empty shell” of a treaty structure to students, which they then filled with text from their position papers prepared for the three committees.

Discussions about which text was consensual and which was controversial, and had thus to be bracketed, provided the third concrete experience for the students participating in the simulation. The reflective observation and abstract conceptualization stages of this last learning cycle were scheduled to take place during the debriefing of the simulation in week 12.

Minute-Paper Returns

Because these are open-ended questions, answers are not necessarily related to the above mentioned aims and learning outcomes of the course. Still, students' responses provide interesting snapshots over the course of the semester on how prominently these aims and ILOs featured on students' minds. Table 3 summarizes their responses immediately after their first exposure to the simulation (week three), after conducting two negotiating sessions (week five), after a further three negotiating sessions halfway through the second learning cycle (week eight), and after the attempted conversion of national position papers into a draft arms control treaty (week 10).

A general feature of minute-paper returns is that a high percentage of student responses addressed the process aspects of the simulation, not its content. With the exception of the first minute-paper survey in week three, skills got even fewer mentions than the subject matter of the negotiations. It would seem that the outlier response in week three has its origins in the fact that there was no standard lecture that would have covered new substantive content in an introductory presentation, and that the whole session revolved around the set-up and organization of the negotiations. In this context students had to come to terms with what was expected of them individually and as a member of a smaller group, i.e., their “country delegation.” Not surprisingly then, as students were grappling with their own roles in this largely unfamiliar situation of a simulation—only two students had prior simulation experience—many were unclear about the simulation process and how it would develop over the course of the semester, resulting in the only instance in which the “unclear” remarks on the process dimension outweighed the “useful” ones. Noteworthy in this context is that at the end of the introductory session, no student expressed uncertainty about the skills required to partake in the simulation.

In addition, the steady increase of comments on positive aspects of the process dimension in the minute-paper returns is worth mentioning. Especially if combined with the “outstanding” elements that students identified, these positive comments at least balance the “unclear” ones (week three) and outrank them by a factor of more than three to one at the end of the simulation (week 10). Another trend is related to the distribution of positive and negative remarks in relation to the process dimension: while at the beginning of the simulation practically all positive comments refer to the complexity of the real world negotiations and the large majority of negative remarks are concerned with the conduct of the simulation and how this complexity will be tackled in the simulation, this pattern changed over the course of the semester with more and more positive responses relating to the simulation process. However, the simulation and its complexity was in a sense a moving target for students throughout the semester. This is revealed by a closer examination of the process related responses in the “unclear” category in week eight. Here, responses fall into two categories: on one hand comments focused on the increased complexity of the second negotiation round with more detailed position papers being presented by country delegations—as compared to round one's overview presentations. On the other hand some comments were already looking ahead to the next stage of the simulation when the national positions had to be translated into treaty language, posing yet another procedural hurdle for students to overtake. Somewhat surprisingly, this heightened awareness of the complexity of the negotiations did not result in a corresponding increase in relation to the substance matter of the negotiations. From this it would seem that students did not regard the objective complexity of the subject matter as as much of a concern as they did the procedural aspects of the negotiations.

It is reassuring that between 12 and 15 students found something useful to remark upon that was either related to one of the ILOs or the skills. “Teamwork” was the most often mentioned term in the latter category. In general, the high number of responses that relate to the ILOs can be taken as an indicator that teaching methods and learning goals, at least in the four sessions with minute-papers, were aligned to a substantial degree. As a matter of fact, all five of the responses in week five that relate to the “substantive” learning outcome actually address the specific learning outcomes of the session. According to Biggs (2003) such an alignment is the necessary precondition for deep learning on part of the students. It is also encouraging to see that between eight and 14 of the responses under the heading “outstanding” address either ILOs or skills that the course seeks to help develop. The responses also show a reduction of uncertainty related to the aims and outcomes of the course and the simulation over the course of the semester. A noteworthy piece of information in this context is that one of the two students who in week five remarked on being unclear about a substantive issue covered in the session also noted that he needed to do more research into this particular issue and did not expect the instructor to clarify matters for him.

Focus Group Results

As mentioned above, the focus group discussion took place in week 10 of the semester. The group of six (of the 16) students was self-selected, as they responded positively to my query a couple of weeks before the focus group meeting. The discussion was based on the following four questions that were taken from Kille (2002, 280):

• What lessons did you draw from the simulation experience?
• What surprised you about the simulation? I.e., what occurred that you did not expect?
• Overall, what do you think was best about the simulation?
• Overall, what would you suggest changing about the simulation?

In response to the first question about the lessons learned, participants in the focus group were quick to point out that the simulation clearly provided an opportunity to improve their negotiation and presentation skills. In the words of one student, the simulation allowed for “acting out what you've learned during the lectures.” One recurring theme in this context was the tension between students' beliefs about biological weapons disarmament and the country instructions they were expected to follow. While in some instances this tension led to more research on the background of the country positions and subsequent identification with a particular country's approach, in other cases this led to the bending or outright disregard of country instructions with the aim of making progress in the negotiations. In the case of one student on the U.S. delegation this latter approach was fueled by the perceived lack of leverage as a student participating in the simulation (in contrast to the assumed leverage of real world negotiators). Still, other participants in the focus group discussion maintained that this was the closest they had ever experienced “the real thing.” In any case, the few deviations from country instructions did not endanger the overall aims and ILOs of the role playing exercise or the course as a whole. Therefore, it did not become necessary to interfere in the simulation via additional country instructions.

One side effect of some delegations' flexible approach to their country instructions led to the apparent non-confrontational negotiating mode and compromise-orientation of many delegates. This came as a surprise to most of the focus group participants, especially those with previous simulation experience. However, as one student cautioned, the agreement reached in some of the negotiating committees might well have been superficial, and could well have collapsed, had the simulation gone through one or more additional negotiating rounds.

With respect to the best aspects of the simulation, students felt that this teaching tool also provided an opportunity for less extroverted students to participate in the course and provided a continuous learning experience—which was additionally strengthened through the continuous assessment of the two short reports—and thus made it more difficult to cover up a lack of preparation. Students generally appreciated the “hands on” mode of operation and most of them reported an increased level of activities within and among country delegations in between the negotiating sessions in class as compared to “regular” courses with standard seminar sessions. Lastly, another positive aspect students identified relates to the “mental frame of reference for approaching the course reading,” which, in the opinion of one student was less of a one-way causal relationship than a cyclical one where the simulation and course readings were influencing each other, thus leading to a better understanding of the issues.

Students in the focus group generally believed that even more time should be made available for the simulation. Alternatively, it was suggested that the mandate might be more focused, so as to allow more time for more in-depth negotiations on a specific issue. This, as well as the above mentioned activities in between negotiating sessions, clearly shows that students got engaged in the simulation and can also be taken as a reflection of the frustration over the fact that students were unable to actually draft a treaty text. That this was far from being a failure on their part was an important message to convey during the debriefing session. Other suggestions for future improvements focused on the inclusion of more detailed guidance on non-negotiable positions in the country instructions and giving feedback on the negotiations in the lectures where possible.

The Debriefing of the Simulation

The debriefing session was subdivided into two parts, and I asked students in the first to address the above four questions of the focus group discussion. Given the large overlap of students participating in both of these sessions, the first part of the debriefing by and large followed the pattern of the focus group discussion. The tensions between personally held beliefs and country instructions—which in some cases reflected a different political and cultural background—allowed us to revisit how cultural influences affect multilateral negotiations, which we addressed at the beginning of the semester. Having gone through the experience of the role playing exercise, several students expressed that they now could more easily relate to the notion that cultural differences actually influence negotiating positions and styles.

As Lederman has pointed out with respect to debriefings, “in the educational setting, the debriefer knows what the experience was designed to be like, but wants to find out how it was for the participants and to help them sort their interpretations … in experience-based learning, debriefing is an acknowledgement that ambiguity is expected” (1992, 149). In order to help students “sort their interpretations” and explore the degree of ambiguity in perceptions of the simulation, the second part of the debriefing session introduced students to Kolb's experiential learning cycle. I presented the four stages of the cycle—concrete experience, reflective observation, abstract conceptualization, and active experimentation—without revealing their sequence in the cycle. When I asked the students which stages related to the simulation, practically all students identified “concrete experience” and “active experimentation,” and most thought they also had engaged in “reflective observation.” While students could easily relate the first two activities to what they had done during, and preparing for, the role play, the element of “reflective observation” was usually associated with keeping a negotiating journal. Somewhat surprisingly, those few students who thought “abstract conceptualization” was another part of their experience also related this to journal writing. None of the students brought up the two essays in this context. These essays, from the design point of view, were a central component in the learning cycle and were intended to cover the “abstract conceptualization” element of the cycle. Students not making this connection clearly attests to the ambiguity inherent in perceiving and interpreting a role playing exercise. Remarkable in this context is also that despite being unable to identify this stage of the learning cycle as part of their experience, most students had actually successfully mastered it, as evidenced by the high quality of many of the essays. When trying to identify the reasons for this disconnect between design intentions and student perceptions, many students agreed with the opinion voiced by one of their peers that he regarded essays (or any other form of assessment) as an activity to demonstrate what had already been learned, not as part of the learning process. It came as a pleasant surprise when at the end of the debriefing session a student suggested that this session might as well be understood as another instance of abstract conceptualization.