1. Introduction
Deliberative democracy is spreading both globally and locally and the current global trend – the digitalized and more fragmented society – also suggests that such a society may benefit from well-organized citizen deliberations (Bächtiger et al., Reference Bächtiger, Dryzek, Mansbridge and Warren2018). The Oxford Handbook of Deliberative Democracy defines deliberation as ‘mutual communication that involves weighing and reflecting on preferences, values, and interests regarding matters of common concern’ (Bächtiger et al., Reference Bächtiger, Dryzek, Mansbridge and Warren2018: 2) and mini-publics where ‘a diverse body of citizens is selected randomly to reason together about an issue of public concern’ (Smith and Setälä, Reference Smith, Setälä, Bächtiger, Dryzek, Mansbridge and Warren2018: 1) have demonstrated their capability to produce high-quality deliberations.
At the same time, inequality and inequity among discussants even in such exemplary mini-publics have been increasingly acknowledged both in theory and practice (Abdullah et al., Reference Abdullah, Karpowitz and Raphael2016). Although the long-term solution to this problem may be promoting societal changes for equality in a broader sense, the ongoing reality of inequality among discussants calls for more immediate and direct measures to be applied to the practices of citizen deliberation (Fung, Reference Fung2005).
Gender inequality, one of the long-term structural inequalities, exists in every country to different extents (Evans, Reference Evans2016) and feminists have warned that deliberation could serve as ‘a mask for domination’ (Mansbridge, Reference Mansbridge and Murchland1991: 123). Despite the importance and seriousness of this problem, not many have studied remedies for the gender gap in deliberation (Lupia et al., Reference Lupia, Krupnikov and Levine2013; Karpowitz and Mendelberg, Reference Karpowitz and Mendelberg2014). Our experimental study, following the aforementioned definition of deliberation and aiming to contribute especially to mini-publics, narrowed our focus to gender equality and examined whether introducing an evidence-driven style of deliberation can mitigate gender inequality in policy deliberation. Before explaining the method and results of our study and discussing the results, we will review prior studies on the gender gap in deliberation and proposed institutional remedies for the gender gap in deliberation.
2. Gender gap in deliberation
Although the importance of citizen deliberation in democratic decision-making is widely recognized today, concerns have also been raised about gender inequality in deliberation. Young (Reference Young2000: 52–57) distinguishes between external exclusion and internal exclusion to illustrate her points. In the case of gender inequality in deliberation, external exclusion means that women are excluded from gaining access to political discussions. The lower female legislative representation in many countries suggests that the external exclusion of women from political discussions remains a problem to be solved for representative democracy.
Against this problem of external exclusion in deliberation, mini-publics offer a solution by randomly recruiting citizens of different backgrounds. If randomly recruited men and women attend a deliberative meeting at an equal rate, women and men should be equally represented at the discussion table. However, even if women are represented equally at the discussion table, their statements may be ignored or dismissed for some reasons. Such a situation is called internal exclusion. Indeed, Young (Reference Young and Benhabib1996, Reference Young2000) argued that certain interpretations of deliberative norms praise dispassionate and rational discussions, which tend to favor the speech culture of white, middle-class men.
Mouffe's criticism of deliberative theorists of ‘first generation’ (Bächtiger et al., Reference Bächtiger, Dryzek, Mansbridge and Warren2018: 3–9) resonates with Young's argument (Mouffe, Reference Mouffe2000: Chapter 4). Mouffe criticized Rawls and Habermas for their emphasis on ‘rational consensus,’ which would belittle the role of passions and confrontation in democracy. Instead, Mouffe endorsed ‘agonistic pluralism,’ which acknowledges the unavoidability of exclusion even in democratic politics and encourages the mobilization of passions for democratic objectives. Similarly acknowledging the problem of masking inequality in the name of democracy, Sanders (Reference Sanders1997) pointed out that if we do not see inequality in deliberation and consider deliberation as a signal of democratic practice, those who are already underrepresented in formal political institutions are further disadvantaged.
Studies in psychology and sociology suggest that gender gaps in deliberation are caused by the differing communication styles of men and women. Previous studies have confirmed the traditional gender stereotypes: women's language tends to be more communal whereas men's language tends to be more agentic (Carli, Reference Carli, Ryan and Branscombe2013; Kroska, Reference Kroska, McLeod, Lawler and Schwalbe2014). Although some of the results were mixed, the overall tendency in those studies was that men used more assertive language, interrupted others more, and exerted greater influence over others' opinion. Conversely, women's contributions were ignored or devalued more often and women were penalized when they communicated in an agentic or competent manner. The only exception was when the topic was feminine. When discussing feminine topics, women were slightly advantaged (Dovidio et al., Reference Dovidio, Brown, Heltman, Ellyson and Keating1988; Ridgeway, Reference Ridgeway2001; Kroska, Reference Kroska, McLeod, Lawler and Schwalbe2014).
Two well-known explanations for such gender differences are social role theory (Eagly and Wood, Reference Eagly, Wood, Van Lange, Kruglanski and Higgins2012) and expectation states theory (Ridgeway and Nakagawa, Reference Ridgeway, Nakagawa, McLeod, Lawler and Schwalbe2014). Both theories assume that the stereotypical beliefs on gender differences function as social norms. Such norms about gender differences expect men to have higher status and to be competitive and expect women to be supportive and communal. Accordingly, men acquire agentic language as they grow up and tend to penalize women who violate gender role norms. In contrast, women grow up learning that they have lower status than men and are expected to be supportive and obedient.
In their book on gender inequality in deliberation, Karpowitz and Mendelberg (Reference Karpowitz and Mendelberg2014) articulated three pathways by which women's participation in group discussion may be restrained. First, women tend to think they are less competent than others in the deliberation, and people who are not confident about their competence speak less. Second, women tend to dislike competition or conflict and thus prefer to avoid discussion, which can be conflictual. Third, women tend to prioritize relationships and social ties and thus are hesitant to speak out when disagreements arise (Chapter 3). The women's tendencies behind these three pathways can be explained by the expectations women receive to have lower status and to be supportive and communal.
When we turn to empirical studies, mixed results have been reported regarding the gender gap in group discussions. In her review of earlier studies on jury deliberations and group interactions, Mendelberg (Reference Mendelberg, Delli Carpini, Huddy and Shapiro2002) found that women participated less and tended to agree more in jury deliberations, at least until the 1980s, and that women tended to have less influence in group interactions (p. 165). However, a more recent survey of actual jurors in Massachusetts found no statistically significant gender effect (York and Cornwell, Reference York and Cornwell2006) and another study of juror surveys from four different sites in the USA found a statistically significant gender gap only among Asian American jurors in Los Angeles (Cornwell and Hans, Reference Cornwell and Hans2011).
Regarding face-to-face policy deliberations, a content analysis of deliberative forums in Philadelphia did not find statistically significant gender gaps, but the signs of coefficients were in favor of men (Dutwin, Reference Dutwin2003). A study examining the Danish Deliberative Poll reported that ‘there is a gender gap only among the most passive, but not among the most active participants’ (Hansen, Reference Hansen, Wolf, Morales and Ikeda2010: 31). A Finnish deliberative experiment in 2006 found that ‘men and those with a higher education have dominated the discussion’ (Himmelroos, Reference Himmelroos2017: 14). Another Finnish study in 2012 found men talking more than women whereas women and men did not differ in their evaluation of how equal their discussions were (Himmelroos et al., Reference Himmelroos, Rapeli and Grönlund2017).
Meanwhile, experimental studies in the USA have found gender gaps under certain conditions. An experimental study reported by Karpowitz and Mendelberg (Reference Karpowitz and Mendelberg2014) found an interaction effect of decision rule and the number of women in a group. Under majority rule, significant gender gaps were found mainly when women were a minority. Under unanimity rule, significant gender gaps were found when women were a majority. Their mediation analysis also found that 59% of the interaction effect on the gender gap in influence was mediated by the gender gap in speech participation. Another experimental study found that the gender gap between white men and women disappeared in racially diverse groups (Toosi et al., Reference Toosi, Sommers and Ambady2012). In this experiment, white women spoke less and were rated as being less persuasive than white men in all-white groups. However, when they deliberated in racially diverse groups, white women spoke as much as white men and were rated as being equally persuasive. They also found that white women's confidence increased over time in diverse groups.
Comparing face-to-face and online deliberations, Showers et al. (Reference Showers, Tindall and Davies2015) found a gender gap in one of the three formats of face-to-face deliberations but no gender gap in online deliberations. Two other studies on online deliberations in the USA found no gender gap and one study found women contributing more words to online discussion (Price, Reference Price, Davies and Gangadharan2009; Polletta and Chen, Reference Polletta and Chen2013). However, a content analysis of an online discussion forum on the 2014 Scottish independence referendum suggests that a gender gap in online deliberations could still occur under certain conditions. They found that 91% of the self-identified contributors (representing 38% of all contributors) were men (Quinlan et al., Reference Quinlan, Shephard and Paterson2015). This result may be partly explained by Clark et al. (Reference Clark, Bordwell and Avery2015), which compared anonymous and named online deliberations among ninth-grade students in the American Midwest. They found that the gender gap in named deliberation disappeared in anonymous settings and that female students preferred discussing under anonymous conditions.
One of the reasons for the mixed results may be found in their different approaches to measuring the gender gap in deliberation. Previous studies on face-to-face deliberation have used a range of variables to measure gender gaps, including time spent speaking and perceived influence (Karpowitz and Mendelberg, Reference Karpowitz and Mendelberg2014); the amount of speaking (lines of text, number of utterances, and number of thought statements), the number of topics addressed, and the number of significant argumentative contributions (Dutwin, Reference Dutwin2003); and types of deliberative behavior (e.g., most active, most passive, and expert) (Hansen, Reference Hansen, Wolf, Morales and Ikeda2010). Studying online deliberation, Polletta and Chen (Reference Polletta and Chen2013) measured message length and the number of messages. In their studies of jury deliberation, York and Cornwell (Reference York and Cornwell2006) measured perceived influence and Cornwell and Hans (Reference Cornwell and Hans2011) used self-reports on participation.
Also, most of the studies reviewed here were conducted in the USA and Europe and the recent mixed results combined with the reports of gender gaps prior to the 1990s may be interpreted as a gradual disappearance of the gender gap in those countries. Even if that is the case, the experimental studies strongly suggest that female participation in deliberation depends on the settings that women find themselves in and a question remains as to whether the gender gap in deliberation exists outside of those countries.
3. Institutional remedies for gender gaps in deliberation
In response to inequalities in deliberation, previous studies have suggested improving the designs of deliberative institutions. A well-known measure to address inequality in group discussions is the use of facilitators or moderators, who would encourage quiet discussants to speak more (Pierce et al., Reference Pierce, Neeley and Budziak2008; Trénel, Reference Trénel, Davies and Gangadharan2009). Yet, we should be cautious about what those facilitators are capable of. An experimental study has reported that moderators influenced discussants simply by using body language such as nodding their heads when they agree and shaking their heads when they do not agree (Spada and Vreeland, Reference Spada and Vreeland2013).
As mentioned in the previous section, Karpowitz and Mendelberg (Reference Karpowitz and Mendelberg2014) found that when women are a minority they speak more under a unanimity rule and thus recommended the use of a unanimity rule when women are a minority in discussion groups. Lupia et al. (Reference Lupia, Krupnikov and Levine2013) reviewed psychological research on gender inequality in communication and recommended institutional adjustments such as adding team-building exercises before deliberation and introducing rules against interruption.
Another suggestion for institutional remedies is the use of a certain deliberation style, which is the focus of this paper. In their study of mock jury deliberations, Hastie et al. (Reference Hastie, Penrod and Pennington1983) distinguished two styles of deliberation. The ‘verdict-driven’ style starts deliberation with a public ballot. After the ballot, each juror advocates one position at a time with supporting evidence. In contrast, ‘evidence-driven’ deliberation starts with a review of evidence and public balloting takes place later in deliberation. Hastie and his colleagues found that jurors in evidence-driven juries rated their persuasiveness, open-mindedness, and seriousness higher than those in verdict-driven juries. Their other findings include that verdict-driven juries reached their verdicts more quickly. Referring to this study, Marder (Reference Marder1987), in her discussion of gender inequalities in jury deliberations, explained the benefits of evidence-driven discussion by contrasting the two deliberation styles:
Verdict-driven discussion creates adversarial factions preoccupied with winning the point and silencing the dissenters. Verdict-driven deliberations also involve fewer participants; each side, rather than every individual, articulates its view…. Evidence-driven discussion relies upon open communication so that all members feel that they have had a fair chance to influence the decision. Those who have a different recollection of the facts or who hold a minority opinion in the group speak because others are willing to listen and to consider what they are saying (p. 602).
Sanders (Reference Sanders1997) also discussed how deliberations can be structured to tackle dominance and hierarchy in group deliberations and acknowledged the potential of evidence-driven deliberations. She noted that an evidence-driven jury, which defers voting while discussing the evidence, could hold off the force of conformity that allows some individuals to dominate deliberations (pp. 366–367).
4. Hypotheses
Building on the arguments of Marder (Reference Marder1987) and Sanders (Reference Sanders1997), we hypothesized that introducing an evidence-driven style of deliberation reduces the gender gap in policy deliberation. Regarding the mechanism of how an evidence-driven style reduces the gender gap, we acknowledge the importance of understanding deliberation as an act of communication. As reviewed above, there are gender differences in communication and previous studies have reported that men are more assertive and women are more likely to express agreement (Ridgeway and Smith-Lovin, Reference Ridgeway, Smith-Lovin and Chafetz1999). Marder (Reference Marder1987) acknowledges such a gender difference and points out that communication styles of women and men match evidence-driven style and verdict-driven style, respectively:
As studies have shown, women in general express more concern than men about accommodating different points of view among members of a group and translate that concern into an evidence-driven mode of discussion. In studies of all-female groups, the more active speakers tried to draw out the more silent members, which is a key characteristic of evidence-driven juries. In contrast, in all-male groups, the more active members eventually ignored the less active members, which is a key characteristic of verdict-driven juries. The men displayed competitiveness with other men, whereas the women expressed cooperation with other women. In mixed groups, however, the women consistently became more silent (p. 603).
The combination of the gender differences in communication and the characteristics of deliberation styles suggests that by creating ‘adversarial factions preoccupied with winning the point and silencing the dissenters’ (Marder, p. 602), verdict-driven deliberation may encourage men to be more assertive, dominate the discussions, and pressure women to agree with men's verdicts. In contrast, if discussants feel more open-minded with an evidence-driven style, women may speak more with greater confidence and not feel as pressured to agree with men. As a result, women's deliberative contribution may increase.
As for how to measure the gender gap in deliberation, the biggest concern regarding the gender gap in deliberation is the possibility that deliberation becomes dominated by men and favors men's opinion. Therefore, measurement of the influence of male and female discussants is important in this respect. In addition, Young (Reference Young2000) argues the importance of understanding others' positions through public discussion (Chapter 3) and Niemeyer and Dryzek (Reference Niemeyer and Dryzek2007) argue that deliberation should yield meta-consensus and inter-subjective consistency as ideal outcomes. In an attempt to also address such epistemic goals of deliberation (Beauvais and Baechtiger, Reference Beauvais and Andre Baechtiger2016), this study measured each discussant's ‘contribution’ in deliberative participation instead of measuring only ‘influence.’ This deliberative contribution is intended not only to capture influential speeches but also to capture speech that enhances discussants' understanding of important facts, beliefs, and values regarding the topics of discussion.
Accordingly, our hypotheses are as follows:
H1: The gender gap in deliberative contribution is smaller with an evidence-driven style than with a verdict-driven style.
H2: The gender gap reduction from using an evidence-driven style is mediated by the amount of speech.
H3: The gender gap reduction from using an evidence-driven style is mediated by the level of agreement.
H2 and H3 hypothesize the mechanism of our main hypothesis (H1). Figure 1 illustrates the hypothesized direct and mediated relationships between deliberation style and the gender gap in deliberative contribution. H2 hypothesizes that an evidence-driven style reduces the gender gap in the amount of speech. According to this hypothesis, men speak more than women in verdict-driven deliberation. In contrast, as men and women speak more equally in evidence-driven deliberation, they enjoy more equal deliberative contributions. H3 hypothesizes that an evidence-driven style reduces the gender gap in the level of agreement. According to this hypothesis, verdict-driven deliberation allows men to receive more agreement from other discussants and thus men are regarded as more convincing. In contrast, as men and women receive agreement more equally in evidence-driven deliberation, they enjoy more equal deliberative contributions.
Hypotheses of this study.
The type of deliberation both Marder and Sanders had in their minds was jury deliberation and there are differences between jury deliberation and policy deliberation, most notably regarding the subject and the goal of deliberation. Although jurors discuss only matters related to the assigned court case, matters of common concern are discussed in policy deliberation. Also, jury deliberations aim at doing justice to the accused whereas policy deliberations aim at choosing the best policies.
However, there are also similarities between these two types of deliberations. Both jury and policy deliberations recruit citizens as discussants and both types include the processes of fact-finding and values-based decision-making. Furthermore, the differences in the subject and the goal of deliberation would not affect the gendered communication as long as the topic of deliberation is not feminine. When the topic of discussion is female-oriented such as child-rearing, women may speak more and be more assertive. Therefore, with the possible exception of female-oriented topics, our hypotheses about deliberation style should apply to both jury deliberation and policy deliberation.
It is true that many mini-publics do offer information sessions or briefing material before the discussions (Gastil and Levine, Reference Gastil and Levine2005) and it may encourage the discussants to focus on factual information. However, evidence is also presented before jury deliberations and Hastie et al. (Reference Hastie, Penrod and Pennington1983) demonstrate that receiving factual information beforehand does not guarantee evidence-driven deliberations.
Although many studies have examined the effect of decision rules (such as majority vs unanimity decision-making) on deliberation, fewer studies have investigated the effects of deliberation styles or rules that regulate how discussions should proceed (Kameda, Reference Kameda1991; Morrell, Reference Morrell1999; Baccaro et al., Reference Baccaro, Bächtiger and Deville2016). In addition, the abovementioned study by Hastie et al. (Reference Hastie, Penrod and Pennington1983) was observational and the two deliberation styles were not randomly assigned to the jurors, making causal inference difficult. In this study, we tested whether an evidence-driven style reduces the gender gap in policy deliberation, using a randomized experiment.
5. Method
Forty-eight students from two universities around Tokyo in Japan participated in our experiment in June and July of 2012. The participants were recruited through temporary job ads posted on campus and online and received 3,700 yen (about US$35) after completing their participation in the experiment. Some of the discussion groups included only men or only women and thus the participants who did not discuss in mixed-gender groups were excluded from the analyses for this paper.Footnote 1 As a result, data from 38 participants (23 men and 15 women; 19–39 years old) were analyzed.
5.1 Experimental design and measurements
The participants' main task was to discuss divisive policies in groups of four participants. Each participant had two sessions of deliberation. The treatment variable was deliberation style (evidence-driven style or verdict-driven style). A repeated measures (within-subjects) design (Kraska, Reference Kraska and Salkind2010; Verma, Reference Verma2015) was used for this experiment. By measuring a dependent variable (DV) on the same subjects under different conditions or over time periods, repeated measures experiments increase statistical power by using the subjects as their own control. When the same subject receives a control treatment and an experimental treatment, much of the difference in the DV is attributable to the experimental treatment.
One notable disadvantage with the repeated measures method is possible carry over effects. The subjects may become tired or bored when receiving the second treatment which is similar to the first treatment. Also, if the effect of the first treatment stays with the subjects, the effect of the second treatment may be a combined effect of the first and the second treatments. To minimize such possible carry over effects, we randomly assigned the discussion groups to Procedure A and Procedure B so the order of experimental conditions was counterbalancedFootnote 2 (Figure 2).
Experimental design and procedure.
The participants of Procedure A discussed the first topic with an evidence-driven style and then discussed the second topic with a verdict-driven style. The participants of Procedure B discussed the first topic with a verdict-driven style and then discussed the second topic with an evidence-driven style. After the first session, those who discussed with one deliberation style were randomly allocated to the groups of the other deliberation style in the second session to avoid possible effects of knowing other discussants' ideological positions from the previous discussion.
We consider that our main DV, each discussant's deliberative contribution, is best measured through subjective evaluations by discussion group members. Because ‘very much of the work of deliberation, even in external-collective settings, must inevitably be done within each individual's head’ (Goodin, Reference Goodin, Fishkin and Laslett2003: 54), it is very difficult to find out who was influential or helpful in understanding the issues during a discussion without asking each member of the discussion group. Accordingly, we created an index for ‘perceived contribution,’ using post-deliberation surveys. After each deliberative session, participants were asked to rate how much each of the four group members (including themselves) contributed to the deliberation as a percentage of the total discussion so that the sum of the four ratings equaled 100% (see Appendix A for the survey questions used in this paper). In this way, each participant received ratings from four different people (including oneself) and the index of perceived contribution was created by calculating the mean value of the four ratings given to each participant.
The variable required for the second hypothesis was each discussant's amount of speech. As mentioned in Section 2, there are various ways of measuring the amount of speech such as time spent speaking and lines of text. Considering that people may take different amounts of time to utter the same statement, we counted transcribed letters. Recordings of all the discussions were transcribed and the number of letters in each discussant's statements was counted.
To test the third hypothesis, we needed a variable that describes how much agreement each discussant received from group colleagues. In the post-deliberation surveys, participants were asked whether they agreed with the opinion of each group colleague using a seven-point scale.Footnote 3 Based on this question, we created an index for ‘level of agreement’ by calculating the mean value of the ratings each participant received from the three other discussants in the group.
5.2 Procedure
As shown in Figure 2, the experiment consisted of two group deliberation sessions, each of which was accompanied by a pre-survey, introduction to the discussion topic, the deliberation itself, voting, and a post-survey. At the start of the experiment, participants were instructed not to interact with each other outside of the deliberative sessions. After submitting the first pre-deliberation survey, participants were divided into groups of four. Fact sheets about the first topic of discussion (foreigner voting rights)Footnote 4 were handed out to the participants, who read the fact sheets in silence for 15 minutes. The fact sheets covered information about the topic such as the legal bases of foreigner voting rights, statistical data about foreigners in Japan, and excerpts from parliamentary reports.
Each group was randomly assigned to either Procedure A or Procedure B. Each deliberative session lasted for 50 minutes. Before starting the first deliberation, the discussants using Procedure A were instructed not to say their individual verdicts (whether they support or oppose permanent residents' right to vote in local elections) for the first 15 minutes and to talk about facts and data. They were also told that for the remaining 35 minutes, they could state their verdicts. Meanwhile, the discussants using Procedure B were instructed to state their individual verdicts first and then continue the discussion freely (see Appendix B for the instructions given to participants). The two styles of deliberation were referred to as ‘rules’ during the experiment to make sure the treatments were fully implemented.Footnote 5 The participants kept the information sheets for reference during the discussion.
After the 50-minute discussion, each group was instructed to cast a secret vote in favor or against granting permanent residents the right to vote in local municipal elections in Japan. After the voting, experimenters announced the result for each group and the participants completed a post-deliberation survey. The second deliberative session repeated the same process except that the groups using Procedure A had the verdict-driven style and the groups using Procedure B had the evidence-driven style for this round of deliberation. The topic for the second deliberation was whether to abolish capital punishment.Footnote 6
6. Results
To test the first hypothesis, a two-way repeated measures analysis of variance (ANOVA) was conducted with ‘perceived contribution’ as a DV and ‘gender,’ ‘deliberation style,’ and their interaction as independent variables (IVs) (Table 1).Footnote 7 The interaction effect of gender and deliberation style, MSE = 75.63, F(1, 32) = 4.92, p = 0.034, partial η 2 = 0.13, was statistically significant at the 0.05 level. This effect size of η 2 = 0.13 (Cohen's f = 0.39) is close to Cohen's operational definition of a large effect size (f = 0.40)Footnote 8 (Cohen, Reference Cohen, Cohen, Adad, Cohen, Cohen, Mansfield and Cohen1988).
Interaction effect of gender and deliberation style
Note: N = 68. *p < 0.05.
Figure 3 shows the margin plot based on the ANOVA result. With the verdict-driven style, the average female contribution was 21.9%, 95% confidence interval (CI) [19.9, 24.0], and the average male contribution was 27.2%, 95% CI [25.4, 29.1]. With the evidence-driven style, the average female contribution was 24.5%, 95% CI [22.4, 26.6] and the average male contribution was 25.5%, 95% CI [23.7, 27.4]. Figure 3 shows that a statistically significant gender gap in deliberative contribution was observed with a verdict-driven style but not with an evidence-driven style, and thus this result supports the first hypothesis.Footnote 9
Interaction effect of gender and deliberation style on perceived contribution. Note: Error bars indicate 95% CI.
6.1 Mechanism of reducing gender gap
The second and third hypotheses address what factors mediate the deliberation style's effect on the gender gap in perceived contribution. Testing the mediated relationships as shown in Figure 1 requires causal mediation analysis at the group level, the result of which will be reported in the next section.
In this section, we test the relationships between deliberation style and the two possible mediators, gender gap in the amount of speech (H2) and gender gap in the level of agreement (H3), at the group level. The results of these analyses prepare us for the causal mediation analysis, which is reported in the next section. The gender gap in speech was calculated by subtracting the average amount of speech by women from the average amount of speech by men (within each group). The gender gap in agreement was calculated by subtracting the average agreement received by women from the average agreement received by men (within each group).
The second hypothesis is based on the assumption that women may find it easier to speak with the evidence-driven style and thus gender gap in speech is smaller in evidence-driven style. However, a one-tailed t-test found no statistically significant difference between the two deliberation styles (p = 0.40). Because there was no relationship between deliberation style and the gender gap in speech, this result rejects the second hypothesis.Footnote 10
The third hypothesis concerns the level of agreement received by discussants and assumes that gender gap in agreement is smaller in evidence-driven deliberations. The result of a one-tailed t-test showed that the difference between the two deliberation styles was statistically significant at the 0.10 level (p = 0.08).
6.2 Causal mediation analysis
So far, regarding the mechanism of deliberation style's effect on the gender gap in deliberation, H2 was rejected but H3 was not conclusive. In this section, we report the result of a causal mediation analysis, which was conducted using R package ‘mediation’ (Imai et al., Reference Imai, Keele and Yamamoto2010; Tingley et al., Reference Tingley, Yamamoto, Hirose, Keele and Imai2014). The outcome variable of the causal mediation analysis was gender gap in deliberative contribution, defined as men's average perceived contribution minus women's average perceived contribution (as a percentage within each group). The treatment variable was deliberation style of the group: evidence-driven style (1) or verdict-driven style (0). The mediator for H3 was gender gap in agreement, as defined in the previous section.
Control variables were gender gap in the number of opposing discussants,Footnote 11 gender gap in the frequency of talking about politics, gender gap in speech, gender gap in age, and experimental site. The indices of gender gap for the control variables were created by subtracting women's mean value from men's mean value within each group for each relevant variable. Gender gap in speech was used as a control variable because this variable had no statistically significant relationship with deliberation style (as reported in the previous section) but was highly correlated with gender gap in perceived contribution (r = 0.70). Table 2 shows the fitted models and Table 3 shows the result of the mediation analysis.
Mediator model and outcome model for mediation analysis
Note: Standard errors are in parentheses. *p < 0.05, **p < 0.01.
Mediation effect of gender gap in agreement on gender gap in deliberative contribution
Note: N = 18. *p < 0.05.
Table 3 shows that when the evidence-driven style was used, the gender gap in deliberative contribution was reduced by 5.83 percentage points compared with verdict-driven deliberation. Because the indices of gender gap were created by subtracting women's average score from men's average score in each group, each index can be regarded as a gap between one man and one woman, which is equivalent to half of the gender gap in a four-member group. Therefore, the estimated effect of 5.83 percentage points is equivalent to a gender gap reduction of 11.66 percentage points if the group consists of two men and two women. Out of this total effect of 5.83 percentage points, a gap reduction of 3.52 percentage points (60% of the total effect) was mediated through a gap reduction in the level of agreement. This result supports our third hypothesis.
6.3 Gender differences in the level of agreement
The result of our causal mediation analysis at the group level supported the third hypothesis, showing that the effect of deliberation style on the gender gap in deliberative contribution was mediated by the gender gap in the level of agreement. Yet, the group-level analysis does not tell us whether deliberation style had an effect on only women, only men, or both. Because the act of agreeing requires two sides of a person who agrees and the other person who receives the agreement, there are six possibilities in total. First, deliberation style could have an effect on the level of agreement received by men, women, or both. Second, deliberation style could have an effect on the level of agreement expressed from men, women, or both. This section returns to the individual-level analysis to examine these possibilities.
As mentioned in Section 4, earlier studies suggest that women are more likely to express agreement than men. To test this possibility, we distinguished women's agreement from men's agreement. The variable ‘women's agreement’ indicates how much agreement a discussant received from female group colleagues and is the mean value of the rating(s) each participant received from female group colleague(s), as rated on a seven-point scale. For comparison, the variable ‘men's agreement’ was also created by calculating the mean value of the rating(s) each participant received from male colleague(s). The ‘level of agreement,’ which was defined earlier and combines agreement from both men and women, is called ‘agreement in total’ in this section.
Table 4 shows the results of two-way repeated measures ANOVAs with ‘agreement in total,’ ‘women's agreement,’ and ‘men's agreement’ as DVs and ‘gender,’ ‘deliberation style,’ and their interaction as IVs.Footnote 12 For ‘agreement in total’ and ‘women's agreement,’ the interaction effect of gender and deliberation style, MSE = 3.87, F(1, 32) = 7.11, p = 0.012, partial η 2 = 0.18 and MSE = 6.78, F(1, 27) = 5.44, p = 0.027, partial η 2 = 0.17, respectively, was statistically significant. Both of these effect sizes of η 2 = 0.18 (Cohen's f = 0.47) and η 2 = 0.17 (Cohen's f = 0.45) can be considered as large according to Cohen's operational definition of effect sizes (Cohen, Reference Cohen, Cohen, Adad, Cohen, Cohen, Mansfield and Cohen1988).
Interaction effect of gender and deliberation style on agreement
Note: *p < 0.05.
Figure 4 is a margin plot from the ANOVA for women's agreement. The main effects of the ANOVA were not statistically significant and accordingly, the error bars overlap in this figure. However, this figure shows the directions of the statistically significant interaction effect of gender and deliberation style, which are not shown in Table 4. With the verdict-driven style, the average agreement received by women was 4.60, 95% CI [3.88, 5.32], and the average agreement received by men was 5.60, 95% CI [5.08, 6.13]. With the evidence-driven style, the average agreement received by women was 5.10, 95% CI [4.38, 5.82] and the average agreement received by men was 4.67, 95% CI [4.14, 5.19]. This result suggests that women's tendencies to agree more with men and less with women in verdict-driven deliberations switched to the opposite directions in evidence-driven deliberations. Combined with the non-significant interaction effect for ‘men's agreement,’ these results confirm that women tended to follow gender role norms in verdict-driven deliberations.
Interaction effect of gender and deliberation style on women's agreement. Note: Error bars indicate 95% CI.
7. Discussion
This study compared two distinct deliberation styles and found a smaller gender gap in evidence-driven deliberation. We acknowledge that the statistically non-significant gender gap with the evidence-driven style may become statistically significant if the number of participants in the experiment is increased. Nevertheless, the effect of the evidence-driven style (when compared with the verdict-driven style) in reducing the gender gap of 5.83 percentage points, or the equivalent of 11.66 percentage points for a group of two men and two women, is not negligible. This result suggests that an evidence-driven style can be useful in addressing gender gap in policy deliberations.
Regarding how evidence-driven deliberation reduces the gender gap, causal mediation analysis found that 60% of the effect of the evidence-driven style was mediated through the level of agreement. Furthermore, the individual level analysis regarding the effect of deliberation style on women's agreement suggests that the evidence-driven style discouraged women's agreement with men's opinion, which then reduced the gender gap in deliberative contribution.
These results in turn suggest the possibility that an evidence-driven style counteracts gender differences in communication (i.e., women's tendency to agree with men, who tend to be more assertive). An evidence-driven style is likely to encourage women to establish their own opinion by giving them time to focus on factual information and also by deterring men from expressing their opinion earlier in the discussion. Also, a focus on evidence (not verdict) may create a less confrontational environment and women may not feel the need to express agreement. Through such processes, women may become more confident with their opinion, rather than just agreeing with men's opinion.
At the same time, deliberation style did not make a statistically significant difference in discussants' amount of speech. This is an interesting finding considering that the amount of speech was highly correlated with deliberative contribution at the individual level (r = 0.67) and the group level (r = 0.70). This relationship between the gender gap in the amount of speech and the gender gap in deliberative contribution is similar to the finding of Karpowitz and Mendelberg (Reference Karpowitz and Mendelberg2014). In their study, the interaction effect of decision rule and the number of women in a group on the gender gap in influence was mediated by the gender gap in speech participation. What is new in our study is that the gender gap reduction using an evidence-driven style was not mediated through the amount of speech. In other words, our results showed that the evidence-driven style had an effect through a mediator other than the amount of speech, which has been the main explanatory variable for deliberative participation in previous studies.
This finding in turn has an implication for deliberative practices. To enhance equality in participation, group discussions tend to rely on facilitators or moderators who manage discussants' speech time (Justice and Jamieson, Reference Justice and Jamieson2012; Schuman, Reference Schuman2012). Our finding suggests that, even when facilitators are present at group discussions, the use of an evidence-driven style is beneficial because it reduces the gender gap through a pathway other than the amount of speech. The effect of evidence-driven style, which is mediated by the level of agreement, is a psychological factor that is harder for facilitators to control and thus an evidence-driven style can be effective even with the presence of facilitators.
One limitation of this study is the relatively small number of participants in the experiment. This study did not recruit many participants for the experiment and instead used a within-subjects (repeated measures) design that increased statistical power at the individual level. It is still possible that, with many more participants, the gender gap observed in evidence-driven deliberation could have been statistically significant. However, the statistical power was large enough for the individual-level repeated measures ANOVA to detect that the gender gap observed in verdict-driven deliberation was larger than the gap in evidence-driven deliberation.
Conversely, a repeated measures design was not used at the group level. As mentioned in Section 5.1, instead of keeping the same members in each group for the second session, those who discussed with one deliberation style in the first session were randomly allocated to the groups of the other style in the second session. Therefore, the groups of the first session and those of the second session did not match. As a result, t-tests of the two deliberation styles at the group level had lower statistical power. However, the causal mediation analysis with control variables was able to detect the mediation effect significant at the 0.05 level.
Another limitation of this study is its generalizability or external validity. As is often the case with laboratory experiments, our study is not free from this problem. In particular, because most of the participants in this study were Japanese university students,Footnote 13 the generalizability of our findings to other populations in the world remains to be tested. A review of communication studies that compared American and Japanese participants found that culture was more important than gender in explaining behaviors (Mare and Waldron, Reference Mare, Waldron, Dindia and Canary2006). This suggests that our findings about the effects of deliberation style detected with Asian people may not be found elsewhere, particularly in Western cultures. However, as repeatedly mentioned above, gender differences in communication and the benefits of evidence-driven styles of deliberation have mainly been discussed in the context of the USA, and thus we expect similar findings if our study designs were applied to Americans as well. In such cases, differences may be found in the effect sizes.
Although our quantitative content analysis found that the amount of speech did not mediate the effect of the evidence-driven style on deliberative contribution, conducting additional content analyses is desirable. Previous studies suggest that the frequency and the types of interruptions a speaker receives have an effect on the amount of speech and the perceived influence of the speaker (Karpowitz and Mendelberg, Reference Karpowitz and Mendelberg2014: Chapter 8). The next step would be to conduct qualitative content analyses to further investigate the mechanisms that cause and reduce the gender gap in deliberative participation.
Although the average gender gap was reduced by using an evidence-driven style, this style was not effective for all the female discussants. Although the results are not shown in this paper, some women had lower perceived contribution and some men had higher perceived contribution when the evidence-driven style was used. One possible explanation of these results, at least partially, is that the evidence-driven style actually empowers people with less power, and not one particular gender. Obviously, people with less power are not always women and less powerful men might benefit from evidence-driven style as well. Previous studies point out similarities in the communication styles of men and powerful people, as well as in the communication styles of women and less powerful people (Henley, Reference Henley1977, Reference Henley, Kalbfleisch and Cody1995; Kalbfleisch and Herold, Reference Kalbfleisch, Herold, Dindia and Canary2006). If an evidence-driven style changes the perceptions and tendencies of less powerful people and encourages their deliberative contribution as it did with many women, this style may be useful when inequalities exist among discussants due to other social or economic statuses as well. Because our study was conducted with university students whose social and economic statuses were more or less the same, other experimental settings will be required to investigate this hypothesis.
Although discussants in our study mainly used fact sheets during the first 15 minutes of deliberation, ‘evidence’ is not necessarily restricted to impersonal data. Personal stories, also referred to as narratives and testimonies, are likely to have similar effects as long as they are not strongly opinionated or judgmental. In his review of research on the use of storytelling in deliberation, Steiner (Reference Steiner2012) found that although personal stories can be used in a manipulative way and stories sometimes go off-topic, they ‘are more often told at the beginning of a decision process’ (p. 66) and help ‘the socially disadvantaged to get a better voice’ (p. 85). Furthermore, men are said to be more interested in facts and analysis, whereas women are more interested in feelings and personal experiences (Polletta and Chen, Reference Polletta and Chen2013: 305). Previous studies also report that women are more likely to use narratives than men (Polletta and Lee, Reference Polletta and Lee2006: 710; Steiner, Reference Steiner2012: 82). Therefore, the gender gap in deliberation may be further reduced if an evidence-driven style encourages discussants to tell their personal stories during the deliberation.Footnote 14
When deliberation occurs among equal participants, there may be no need to introduce additional rules or styles. However, current practice suggests unequal deliberative participation based on gender and other factors. This study found that the introduction of an evidence-driven style that encourages participants to start their discussion by focusing on factual information and avoiding outright expression of opinion reduces the gender gap in deliberation. We also found that evidence-driven deliberation reduces the gender gap by lowering women's tendency to agree with men rather than by increasing women's speech. This psychological effect is a factor that is harder for facilitators to control and thus an evidence-driven style can be effective even with the presence of facilitators. As long as there is substantial inequality among discussants, the introduction of effective remedies should be beneficial and an evidence-driven style can be considered as a potential remedy.
Acknowledgements
This study was supported by JSPS KAKENHI Grant Number 19046001 and the research plan of this study was approved by the institutional review board of the University of Tokyo (Approval No. 11-45). We thank Tatsuya Kameda and Yoichi Hizen for their support during the preparation of the experiment and Ken'ichi Ikeda, participants at the Japanese Studies Association of Australia Conference and the Australian Political Studies Association Conference in 2015 and anonymous reviewers for their valuable comments on earlier versions of this paper.
Hiroko Ide Levy is an honorary research fellow in the School of Humanities and Social Sciences at La Trobe University and a tutor in the Asia Institute at the University of Melbourne. Her research focuses on deliberative democracy and psychological analysis of political behaviour.
Shiro Sakaiya is Associate Professor of Law and Politics at the University of Tokyo. His research interests include Japanese politics, public opinion, and political participation.
Appendix A
Survey Questions Used in This Paper
The following survey questions were given to participants (original language was Japanese).
1. Perceived contribution
Please rate each member's contribution to the previous deliberation including yourself so that the sum of the ratings is 100%.
2. Agreement with other
Please rate each member on a scale of seven.
3. Preference on policy
Please circle one number from 1 (strongly disagree) – 4 (neutral) – 7 (strongly agree).
4. Frequency of talking about politics
Do you talk about public policies or politics in your daily conversation? (Circle one.)
(1) Almost never (2) A few times in a month (3) A few times in a week (4) Almost every day
5. Gender and age
Please tell us about yourself.
Appendix B
Instructions before the discussion
The following instructions were provided to discussants (original language was Japanese). ‘XXX’ for the first deliberation was ‘granting permanent residents the right to vote in local municipal elections’ and ‘XXX’ for the second deliberation was ‘abolishing capital punishment.’
Evidence-driven rule
First, please state the facts and data you know or learned from the information sheets about the topic. Start from the person with the smallest number and take turns clock-wise. Once everyone has spoken, anyone can speak freely. However, please refrain from expressing your opinion about whether you support or oppose XXX for the first 15 minutes. Please spend the first 15 minutes discussing facts and data about the topic using the information sheets. During the remaining 35 minutes, you can state your opinion.
Verdict-driven rule
First, please state whether you support or oppose XXX and why, starting from the person with the smallest number and take turns clock-wise. Once everyone has spoken, anyone can speak freely. Please start discussing the topic.
