Numerous studies have examined the relationship between education and civic outcomes such as political participation and political knowledge (Lindgren, Oskarsson, and Persson, Reference Lindgren, Oskarsson and Persson2019; Nie, Junn, and Stehlik-Barry, Reference Nie, Junn and Stehlik-Barry1996; Persson, Reference Persson2015b). Scholars have emphasized the roles of the length and types of education in shaping such outcomes, yet far less research has focused on what teaching practices are the most effective. There is a consensus in the literature that in order for democracies to sustain, citizens need at least some basic level of civic competence, such as a baseline level of political knowledge, adherence to democratic norms and that many enough are willing to participate in political activities (Carpini and Keeter, Reference Carpini and Keeter1996; Dahl, Reference Dahl1989). Hence, a central task for social science classes is not only to increase students’ knowledge per se but also to encourage them to reflect on and develop their own views on political and democratic issues and principles.
This article examines whether deliberative education increases students’ civic competences. In contrast to a more traditional teacher-centered education style, deliberative education emphasizes the importance of student-to-student communication for the learning process. In an ideal deliberative teaching situation, students bring different perspectives to an open discussion and they present their views and respectfully listen to, and reflect upon, the reasoned arguments put forward by others. In addition, they strive to find agreement, or at least they agree on what there is disagreement about. The central idea is that if the teaching is focusing on reasoned deliberation between students, they will form a better understanding of the content and develop more well thought through views of the world. Thus, the deliberation method is assumed to translate into not only higher levels of knowledge of the subject being studied but also into stronger commitment to democratic values and principles (Englund, Reference Englund2000; Gutmann, Reference Gutmann1999). Hence, we test the hypothesis that deliberative education (compared to teacher-centered education) increases students’ political interest and political knowledge and makes them discuss politics more often and embrace democratic values to a higher degree. Throughout the text, we use the term “civic competence” as a collective term for key prerequisites for democratic participation: political discussion, political interest, political knowledge, and democratic attitudes (Ekman and Amnå, Reference Ekman and Amnå2012).
Initial experimental analyses have supported the arguments put forward by deliberative education theorists (Bogaards and Deutsch, Reference Bogaards and Deutsch2015; Deslauriers, Schelew, and Wieman, Reference Deslauriers, Schelew and Wieman2011; Latimer and Hempson, Reference Latimer and Hempson2012). However, a limitation of most of these analyses is that they draw on either short-term interventions (a one-day project or less) or small sample sizes. We address both of these issues in this study.
More specifically, we propose a way to put the deliberative teaching ideal into a continuing pedagogical teaching practice. Then, we test the model empirically by presenting results from a field experiment designed to analyze both the short- and long-term effects of deliberative education to teacher-centered education on a set of different civic competences: democratic values, political discussions, political interest, and political knowledge. The experiment was conducted among more than 1,200 students in 59 Swedish upper secondary classrooms. In contrast to previous research, we find little evidence that deliberative education significantly increases civic competence.
Empirical research on deliberative education
While large-scale empirical studies on deliberative education are rare, there are related studies looking at the impact of classroom discussions or specific curricula on civic outcomes. These studies have used both observational and experimental data. Within the former group of approaches, most have focused on the impact of an open classroom climate. In these studies, an open classroom climate has been measured by questions, for example about, whether students feel free to express different opinions than those of their teachers, whether they are encouraged to form their own preferences, and whether their opinions are respected. The studies using observational data have identified a positive relationship between an open classroom climate and civic outcomes such as political knowledge (Campbell, Reference Campbell2008; Hess and McAvoy, Reference Hess and McAvoy2014; Persson, Reference Persson2015a; Torney-Purta, Reference Torney-Purta2002a,b), interest in politics (Hess and McAvoy, Reference Hess and McAvoy2014; Kahne, Crow, and Lee, Reference Kahne, Crow and Lee2013), and political engagement (Kahne, Crow, and Lee, Reference Kahne, Crow and Lee2013; Neundorf, Niemi, and Smets, Reference Neundorf, Niemi and Smets2016). However, given the endogeneity problems related to these studies, we should not interpret the conclusions as evidence of causality.
There are a few experimental analyses on specific curricula that include deliberation in their design. However, most of these studies rely on short-term interventions (a one-day project or less) or small sample sizes or lack proper random assignment. What the experimental studies suggest, however, is that deliberation is largely beneficial for students’ development of political knowledge and civic virtues and engagement. For instance, experimental analyses involving one-day projects (or less), such as deliberation polls in school, show that university students who engage in these deliberative practices and discussions are likely to increase their civic engagement and political knowledge (Bogaards and Deutsch, Reference Bogaards and Deutsch2015; Deslauriers, Schelew, and Wieman, Reference Deslauriers, Schelew and Wieman2011; Latimer and Hempson, Reference Latimer and Hempson2012). In addition, a more long-term interventional but small-sample study (only six high-school classes) showed that deliberative teaching practices during a four-week civics course were beneficial for the political knowledge and “democratic virtues” in one group of students: those in male-dominated education tracks (Andersson, Reference Andersson2015). Finally, a long-term intervention (covering 21 lessons) by Green et al. (Reference Green, Aronow, Bergan, Greene, Paris and Weinberger2011), conducted on a relatively large scale (59 high-school classrooms), also emphasizes the impact of teaching practices on civic outcomes. However, the treatment in their study differs from ours: their treatment constituted (exposure to) a curriculum that enhanced the attention to constitutional principles. The authors found that treated students increased their knowledge about, but not their support for, civil liberties.Footnote 1
The conclusion to draw from previous studies is, on the one hand, that they largely support the hypothesis that deliberative education increases civic competence. On the other hand, they provide little reliable evidence on the impact of the long-term effects of deliberative teaching as a continuing practice. Most previous studies have problems related to weak treatments or low power, while the well-designed exception by Green et al. (Reference Green, Aronow, Bergan, Greene, Paris and Weinberger2011) does not consider deliberation as a teaching practice.
As a consequence, it is an open question as to whether there are any causal effects of deliberative education – and how far reaching any such effects are. It is not obvious that deliberation will affect all, if any, outcomes. For instance, it is reasonable to assume that, to the extent that deliberation has a positive effect, it starts with increasing knowledge and political interest. If that triggers cognitive processes, the effects might spill over to changed values and attitudes and ultimately to changes in political behavior. However, we cannot rule out another possibility, namely that more teacher-centered education (compared to deliberative education) increases (some types of) civic competence. As a suggestion, teacher-centered education might be more oriented toward knowledge content than the discussions in the deliberative treatment. Or there might be a crowding out phenomenon in relation to political discussions: students in the deliberation treatment might feel fatigue of political discussion, while the control group might feel a stronger desire to discuss the issues they have worked with. Thus, when testing the hypothesis about deliberative education and civic competence, we also take these alternative suggestions into account.
Deliberative versus teacher-centered teaching
The experimental program included two different teaching practices – deliberative and teacher-centered – each involving a series of seven lessons covering the first third of the mandatory social science course in Swedish upper secondary education (in Sweden students start upper secondary school the year they turn 16). The two teaching practices included the same subject content, with the first three lessons covering issues concerning human rights and the remaining four covering democratic decision-making. The participating teachers used the lessons sequentially as one continuous unit. We provided treatment teachers with teaching instructions/manuals and classroom assignments students were to complete through group discussions. The teachers in the control group received an equally carefully developed set of lessons based on a teacher-centered approach. To ensure that the same texts were used in all classrooms, we developed two textbook-style chapters: one on human rights and another on democratic decision-making. The chapter on human rights, in total six pages, covered issues such as the history of the Universal Declaration of Human Rights and its ratification in Swedish law, and its implications for freedom of speech and potential restrictions on citizens’ freedoms and rights. The students also read a nine-page chapter titled “What is democracy?” outlining the basic idea of democratic decision-making which also included issues like the concept of, and criteria for, democracy. In addition, the chapter discussed non-democratic governance and examples of totalitarian regimes throughout history.
The deliberative teaching instructions were operationalized from the theoretical ideal of deliberation, that is, using conversation as a transformative force, highlighting different opinions and perspectives, and shaping and reshaping preferences. The teaching material was created to provide repeated problem-solving situations in small-group deliberation. Students were to discuss issues of human rights and democratic decision-making in an orderly manner. The general instruction to students was to focus on discussing and listening, helping each other to develop arguments, respecting each other’s opinions, and not offending one another. The core part of the teaching material consisted of the group exercises termed “deliberation scenario dilemmas”. Each lesson included two or three deliberation scenario dilemmas.
In the teacher-centered teaching practice, we focused on developing a model that would minimize student interaction. Consequently, we developed instructions and exercises for individual student reflection and teacher-student-driven dialog. We used the same scenario dilemmas as in the deliberative education teaching practice, although the discussions between students were kept to a minimum. Instead the students were asked to reflect individually and write down their thoughts. After that, the teacher started an I-R-E sequence – initiating questions concerning the dilemma, letting the individual student respond, and opening up for the class (and the teacher) to evaluate. After finishing the I-R-E sequence, the teacher summarized the main outcome, commenting on the individual ideas raised.
This setup had the advantage that we were in control of what happened in both groups, which would not have been possible with an “untreated” control group and their standard teaching practice (cf. Bogaards and Deutsch, Reference Bogaards and Deutsch2015). Using this strategy, we were able to compare teaching with a maximum of student interaction with teaching with a minimum of student interaction. The complete teaching material is provided in the supplementary material, Appendix Section A13.
Study design
The experimental program was carried out in 25 upper secondary schools and included 59 classes and 36 teachers. In total, 1,283 students participated in the study. The selection strategy was to target, and try to recruit, all upper secondary schools (“gymnasieskolor”) in western Sweden, more exactly those with post codes beginning with 2, 3, and 4 including large rural areas, six smaller towns, and the two bigger cities of Gothenburg and Malmö. During the spring of 2015, we contacted a total of 93 upper secondary schools in southern and western Sweden. Civics teachers at each school received an information letter outlining the study. The strategy for recruiting teachers was to offer a complete package of lesson plans based on the two different teaching practices being evaluated. This included teaching material for a total of seven lessons, teaching instructions, and exercises for students. We also offered a follow-up report and dialog on the results.Footnote 2
It is important to note that in the Swedish system, there is a common social science course curriculum with clearly defined “goals”. However, teachers have a lot of autonomy when it comes to how to achieve these goals, that is, regarding which books or other material to use, how to structure the education, etc. This also implies that both of our two sets of study material are fully legitimate to use in the educational setting at hand.
We did not receive any response from 20 of the schools. The most common reason as to why we were unable to recruit a school was that the teachers had not yet received information from the school principal about which classes they would be teaching during the subsequent autumn. Only a handful of principals and teachers responded that our experiment did not fit into their planning or that they were not interested.
The program was executed during the academic year of 2015/2016. The teaching material packages (each with a classroom set for 30 students), including surveys, were distributed by a courier company to the teachers in early August of 2015, giving them roughly 2 weeks to study the material before beginning. During the program, the researchers and research assistants were constantly available to discuss any questions or issues raised by the teachers. The research group had continuous contact with the participating teachers in order to keep track of how the field experiment was proceeding.
The students completed a survey before the study and another after the seven lessons of the field experiment. These surveys contained questions about political knowledge, democratic values, interest in politics, etc. In order to be able to evaluate whether the experiment had any long-term effects, we also administered a follow-up survey at the end of the school year (in late May and early June 2016, i.e., approximately 7 months after the series of lessons ended).
The randomization process was carried out at the class level through cluster random assignment. The participating teachers could have up to five classes to participate in the experiment. Twenty-two teachers participated with one class, seven with two classes, six teachers with two, six with three classes, and one teacher participated with five classes.
Since we did not know until the semester started how many teachers would participate with how many classes, we prepared a randomization scheme. For each possible number of classes (one to five), we conducted a large number of completely random assignments in random order in which classes were assigned to treatment or control. When the teachers contacted us to inform us about how many classes they were going to teach, we went to the random sequence for the next teacher with that specific number of classes. We made sure that teachers with odd numbers of classes came out with a predominance of control or treatment classes in half of the cases. This procedure ensured that the assignment of the teaching practice was blocked by and balanced across teachers (cf. Gerber and Green, Reference Gerber and Green2012, pp. 71–85).
In the Supplementary material Appendix Section A2, we present randomization checks showing that there were no significant differences in the proportions of boys and girls, immigrant origin or students from specific socio-economic backgrounds between the treatment and the control classes. A further issue relates to the composition of the schools that agreed to participate in the study versus the schools that did not participate. In the Supplementary material Appendix Section A3, we show that the participating schools were not atypical compared with average Swedish schools.
As manipulation checks, we included 11 questions on an open classroom climate that are identical to those used in the country comparative Civic Education (CivEd) study administered by the International Association for the Evaluation of Educational Achievement (IEA). The results presented in the Supplementary material Appendix Section A4 show that those in the treatment group perceived their classroom climate as being significantly more open than did their peers in the control groups. Hence, the experiment appears to have had the desired effect of creating a more open and deliberative discussion climate in the classrooms.
To further test whether the experiment was carried out satisfactorily, we performed two surveys, before and after the experiment, with the participating teachers. The results presented in the Supplementary material Appendix Section A5 show that more than 90% of the teachers felt motivated and were able to implement the treatment and follow the instructions well.
Measurements
As outcome variables, we focus on civic competence defined broadly (cf. Ekman and Amnå, Reference Ekman and Amnå2012). First, the most basic concept that we look at is political interest. This is an important factor, since it captures citizens’ basic concern for politics. Many of the effects that deliberative education is supposed to have are expected to emanate from an increase in political interest. We use a standard single item measure of political interest.
Second, we study factual political knowledge. We use a seven-question index inspired by “the ideal political knowledge index” measures proposed by Carpini and Keeter (Reference Carpini and Keeter1996). These are multiple choice questions with four response options. The questions are not directly related to the educational content, but rather represent a standard battery of political science research questions about generic political knowledge. The idea here is to see whether deliberative education increases learning in a way that makes it spill over into general political knowledge.
Third, we measure students’ democratic values. In order to achieve comparability with previous studies, we rely on measures similar to those used in the comparative CivEd Study. The questions measure students attitudes to the democratic system.
Fourth, we test whether deliberative education increases how much the students discuss politics with people in their surroundings.
The full wordings of the questions can be found in the Supplementary material Appendix Section A1.
Results
Now, let us turn to the main results. Since the randomization was carried out at the class level, we use ordinary least square models with heteroscedastic-consistent standard errors clustered at the class level (cf. Green and Vavreck, Reference Green and Vavreck2007).Footnote 3 Table 1 shows the treatment effects, illustrating the differences between deliberative and teacher-centered groups after the experiment. All dependent variables are recoded to vary between 0 and 1, except for political knowledge, which indicates the number of correct answers given (0 to 7). To increase precision, we have included individual level covariates (gender, mother’s and father’s country of origin, respectively, and number of books at home) as well as the individual baseline levels of the dependent variables as stated by the students in the first survey that was conducted before the experiment started.Footnote 4 The results show only insignificant differences between the treatment and control groups, and the sizes of the insignificant effects are small.Footnote 5
Effects after the experiment
Notes: Clustered standard errors (class level) in parentheses, *p < 0.10, **p < 0.05, ***p < 0.01.
Thus far, we have looked at the results immediately after the experiment, but does deliberative education have any long-term effects? Table 2 shows the effects by the end of the school year. Unfortunately, not all classes were able to participate in the second step of the study, although the majority of the classes did (50 classes).Footnote 6 The results show no or fairly small effects at the end of the school year. The levels of political interest, discussion, and democratic values are close to zero and not statistically significant. However, the difference in political knowledge did increase between the groups, to on average 0.3 more correct answers in the deliberative group. Although this effect is not statistically significant, it is – in contrast to our other outcome variables – of some substantial interest.Footnote 7
Effects at the end of the school year
Notes: Clustered standard errors (class level) in parentheses, *p < 0.10, **p < 0.05, ***p < 0.01
In the Supplementary material Appendix Section A12, we also present analyses of whether the treatment affected student’s grades in the social science course. We find that grades are somewhat higher in the deliberation classrooms, but the differences are not statistically significant.
Conclusion
In this article, we have tested the hypothesis that deliberative education (compared to teacher-centered education) increases students’ civic competence. The theoretical literature put a lot of promise in the potential of deliberative education. However, well-designed studies to test these claims have been rare. We have presented the hitherto, to our knowledge, largest experimental investigation of the effects of deliberation in an educational setting. In contrast to initial empirical investigations of the effects of deliberative teaching practices in the classroom, we find in general little evidence that deliberative education positively impacts civic competence. In one respect, this main conclusion should be slightly nuanced: there is some equivocal and sporadic evidence that political knowledge is positively affected by deliberative education. Although the long-term effect on political knowledge is not statistically significant, it is of some substantial interest.
Despite the lack of, or small, effects after our 7-week intervention, our study does not rule out the possibility that effects would be more substantial with more extensive interventions, or in specific subgroups. Our study focused on the average treatment effect, but further studies would benefit from carefully testing how the effects of deliberative education across different groups of students (cf. Sumaktoyo, Nickerson, and Keane, Reference Sumaktoyo, Nickerson and Keane2016).Footnote 8 However, such tests carried out over a long period of time require a large number of participating classes in order to meet the statistical power requirements. Most likely, it will also require close collaboration with school organizations that are willing to participate. When performing such tests, it is important to pay specific attention to political knowledge. This civic outcome has been important in previous research on deliberative education, and it is the only type of civic competence that was of some substantial interest in this largest field experiment of deliberative education to date.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/XPS.2019.29
