Do elites exhibit gender bias when responding to political aspirants? Relative to their share of the voting population, women are significantly underrepresented in politics. The political recruitment process—the process of going from an eligible citizen to an elected representative—occurs in three distinct stages (Krook and Norris Reference Krook and Norris2014; Norris and Lovenduski Reference Norris and Lovenduski1993). First, eligible citizens must select themselves as potential candidates. Second, those who aspire to political office must be selected as candidates by political parties. Finally, candidates must be selected as representatives by the voters. Gender distortions can occur at each of these stages, and studies have generally shown that women face higher barriers to representation than similarly situated men (Anzia and Berry Reference Anzia and Berry2011; Fox and Lawless Reference Fox and Lawless2004, Reference Fox and Lawless2010; Fulton Reference Fulton2012; Fulton et al. Reference Fulton, Maestas, Sandy Maisel and Stone2006; Norris and Lovenduski Reference Norris and Lovenduski1993). Most research has focused on the last stage of the recruitment process, where voters select their representatives. Here, though, I focus on a much earlier stage by examining whether elites discriminate against female political aspirants. Specifically, I examine whether elites are equally responsive to female and male political aspirants who signal an interest in a political career.
Most studies of the political recruitment process rely on observational data, which makes it difficult to isolate the effect of gender discrimination because of the potential for omitted variable bias, selection bias, and post-treatment bias. Even when we observe differences between men and women, it is hard to know whether these differences are really due to the gender of the candidate or to some other unobserved or difficult to measure factor such as quality (Fulton Reference Fulton2012; Fulton and Dhima Reference Fulton and Dhima2020), experience (McDermott Reference McDermott2005), or attractiveness (Ahler et al. Reference Ahler, Citrin, Dougal and Lenz2017; Lenz and Lawson Reference Lenz and Lawson2011). The issue of post-treatment bias, which has received relatively little attention in the literature, is particularly problematic (King and Zeng Reference King and Zeng2006; Montgomery, Nyhan, and Torres Reference Montgomery, Nyhan and Torres2018). An individual's gender is often determined early in life and affects most things they do or experience. Many of the things that scholars like to control for to isolate the effect of gender in their analyses, such as education and experience, are likely influenced by an individual's gender. This makes it extremely difficult to control for all of the potential confounders in an observational study of gender discrimination without inducing post-treatment bias (Crabtree Reference Crabtree2019). Worryingly, post-treatment bias can be in any direction (Montgomery, Nyhan, and Torres Reference Montgomery, Nyhan and Torres2018). The methodological difficulties faced by observational studies in accurately identifying gender discrimination can have enormous costs. For example, some observational studies may erroneously indicate the presence of gender discrimination, leading policy makers and other actors to waste significant time and money attempting to remedy a problem that does not in fact exist. Similarly, other observational studies may incorrectly indicate that gender discrimination is absent, inducing policy makers and other actors to overlook the very real negative consequences of discrimination felt by particular gender groups.
To avoid the potential methodological problems associated with observational studies, I employ an experimental research design, specifically an audit (or correspondence) experiment, to examine elite gender discrimination. Audit experiments are especially useful for investigating sensitive topics, such as gender discrimination, because they allow researchers to avoid both selection bias concerns that arise when the people who are likely to discriminate opt out of studies and social desirability concerns that arise when people have incentives to downplay their discriminatory behavior to avoid perceived social and legal sanctions.Footnote 1 With an audit experiment, the researcher varies some characteristic of individuals, keeping everything else the same, and then sends these individuals, or messages from these individuals, into the field to see whether the randomized characteristic affects some outcome of interest (Bertrand and Duflo Reference Bertrand, Duflo, Duflo and Banerjee2017). Unlike survey experiments, which can only get at reported attitudes toward discrimination in a hypothetical scenario, audit experiments provide a real-world behavioral measure of discrimination. In my audit experiment, I send an email message from a political aspirant inquiring about a career in politics to elected representatives at different levels of government in Canada.Footnote 2 By varying the sex of the political aspirant, I can examine whether political elites respond to men and women at the same rates. Replying (versus not replying) to an email message like this is recognized as important because responses can be considered “a type of ‘micro’-mentorship where even a small act of encouragement can teach an aspirant about the profession and provide cues about whether he or she will be welcome” (Kalla, Rosenbluth, and Teele Reference Kalla, Rosenbluth and Teele2018, 338).
My study is one of the few gender and politics audit experiments to be conducted outside the United States and the first to address one of the stages of the political recruitment process.Footnote 3 To my knowledge, there has been only one audit experiment that looks at elite gender discrimination in the political recruitment process. In contrast to previous observational studies of political elite behavior, the audit study conducted by Kalla, Rosenbluth, and Teele (Reference Kalla, Rosenbluth and Teele2018) in the United States finds no evidence of any gender discrimination by legislators against female political aspirants. A common concern with experiments is that they are often not replicated, and as a result, it is hard to know whether their findings are generalizable to other contexts (Clark and Golder Reference Clark and Golder2015). Indeed, much of the discussion regarding the drawbacks of experiments is framed around concerns with external validity.Footnote 4 One way to address concerns with external validity and evaluate the robustness of results is to replicate an experiment in a different context (Krupnikov and Levine Reference Krupnikov and Levine2014). Replication is especially pertinent when the findings of an experiment, such as the one conducted by Kalla, Rosenbluth, and Teele (Reference Kalla, Rosenbluth and Teele2018) run counter to expectations in the literature. My audit experiment builds on and extends Kalla, Rosenbluth, and Teele's analysis to the Canadian context.
There are at least three reasons why Canada is a good context for evaluating elite gender discrimination against female political aspirants. First, Canada is useful from a practical perspective when it comes to extending Kalla, Rosenbluth, and Teele's (Reference Kalla, Rosenbluth and Teele2018) study. While Canada is a parliamentary democracy and the United States is a presidential one, both countries employ a single-member district plurality electoral system. And, with the exception of Quebec, which has a large French-speaking population, both countries are predominantly English speaking. This means that I can use the same experimental treatment, thereby addressing potential concerns about minimal replication in research and the generalizability of experimental studies. Moreover, Canada has a qualitatively similar level of women's legislative representation to that found in the United States. Second, prior research on Canadian politics suggests that the persistent underrepresentation of women has less to do with voter reluctance to elect female candidates (Bashevkin Reference Bashevkin2011; Black and Erickson Reference Black and Erickson2003; Goodyear-Grant Reference Goodyear-Grant, Anderson and Stephenson2010; Sevi, Arel-Bundock, and Blais Reference Sevi, Arel-Bundock and Blais2019; Young Reference Young2006) and more to do with discrimination in earlier stages of the political recruitment process (Erickson Reference Erickson1991; Thomas and Bodet Reference Thomas and Bodet2013). This suggests that a focus on elite gender discrimination is particularly pertinent in the Canadian case. Finally, the inclusion of women in politics has become a salient issue over the last few years in Canada, and there has been growing pressure to increase diversity and make political representation more inclusive. During the 2015 federal elections, for example, the Liberal Party made a commitment to gender-equal cabinets. Upon coming to power, the Liberal Party fulfilled some of its promises by forming Canada's first gender-balanced government at the federal level, further propelling issues of gender equality onto the national agenda.
Based on responses from 1,774 legislators in Canada, I find no evidence of elite gender discrimination against female political aspirants. Indeed, I find consistent evidence that Canadian legislators are significantly more responsive to female political aspirants than male ones and more likely to provide them with helpful advice. This pro-women bias in a very early and informal phase of the political recruitment process exists at all levels of government in Canada and tends to be stronger among female legislators and those associated with left-leaning parties. These results obviously do not imply that female political aspirants have not historically faced elite discrimination in this phase of the political recruitment process or that they do not face higher hurdles than men in later and more formal stages of the recruitment process. They do suggest, however, that contemporary political elites in Canada may be open to increasing female political representation. This should be treated as positive news. When combined with the absence of elite gender discrimination at the same stage of the political recruitment process in the United States (Kalla, Rosenbluth, and Teele Reference Kalla, Rosenbluth and Teele2018), the results from my experiment should serve as welcome encouragement for women to put themselves forward as potential candidates and pursue their political ambitions.
SHOULD GENDER AFFECT ELITE RESPONSIVENESS?
On the whole, research suggests that gender distortions are greater earlier in the political recruitment process than later. While studies of voter discrimination find that stereotypes are still pervasive in voters’ evaluations of candidates (Bauer Reference Bauer2015), there is mixed evidence as to whether and when voters are unwilling to support female candidates (Schwarz, Hunt, and Coppock Reference Schwarz, Hunt and Coppock2018). In fact, some analyses find that voters are more willing to vote for female candidates than male ones (Black and Erickson Reference Black and Erickson2003; Golder et al. Reference Golder, Stephenson, Van der Straeten, Blais, Bol, Harfst and Laslier2017; Lawless and Pearson Reference Lawless and Pearson2008).Footnote 5
Research suggests that parties are less enthusiastic than voters when it comes to selecting female candidates. For example, evidence indicates that party leaders have historically been more likely to recruit men than women to run for political office (Fox and Lawless Reference Fox and Lawless2010; Niven Reference Niven1998, Reference Niven2006; Sanbonmatsu Reference Sanbonmatsu2006; Schwindt-Bayer Reference Schwindt-Bayer2011) and that parties, when they do select female candidates, often nominate them in districts where they are less likely to win (Erickson Reference Erickson1991; Sanbonmatsu Reference Sanbonmatsu2002; Thomas and Bodet Reference Thomas and Bodet2013) or place them lower on party lists (Luhiste Reference Luhiste2015). Gender distortions are arguably even stronger at the self-selection stage, with numerous studies showing that women are much less likely to put themselves forward as potential candidates than similarly situated men (Fulton et al. Reference Fulton, Maestas, Sandy Maisel and Stone2006; Lawless and Fox Reference Lawless and Fox2010; Pruysers and Blais Reference Pruysers and Blais2019) because of gender role socialization (Clark, Hadley, and Darcy Reference Clark, Hadley and Darcy1989), family obligations (Fulton et al. Reference Fulton, Maestas, Sandy Maisel and Stone2006), perceptions of qualifications (Fox and Lawless Reference Fox and Lawless2011), lack of party support and recruitment (Fox and Lawless Reference Fox and Lawless2004, Reference Fox and Lawless2010, Reference Fox and Lawless2011; Fulton et al. Reference Fulton, Maestas, Sandy Maisel and Stone2006; Niven Reference Niven1998; Sanbonmatsu Reference Sanbonmatsu2006), and election aversion (Kanthak and Woon Reference Kanthak and Woon2015).
As the early stages of the recruitment process appear to be more distortionary, women's underrepresentation is often considered a supply-side issue rather than a demand-side one (Fox and Lawless Reference Fox and Lawless2004; Htun Reference Htun2016).Footnote 6 More specifically, it is frequently assumed that the low level of female representation is primarily caused by the lack of women running for political office rather than low voter demand (Htun Reference Htun2016, 90–91). But it is important to remember that demand also comes from political elites. Elite demand matters not only because elites have the power to change formal institutions, such as electoral rules, and thus make the political opportunity structure more permissive to female political inclusion. It also matters because their informal messages and behavior can have a significant impact on encouraging female political aspirants to put themselves forward as potential candidates. Research suggests, for example, that political aspirants are twice as likely to think about running if they are encouraged by political elites (Fox and Lawless Reference Fox and Lawless2004) and that women are more likely to run and get elected if elites promote messages of female inclusion (Karpowitz, Monson, and Preece Reference Karpowitz, Quin Monson and Preece2017). Encouragement from political elites is especially important for female candidates as “women are simply unlikely to run in the face of elite discouragement” (Niven Reference Niven2006, 485).
Can we expect political elites to be equally responsive to female aspirants when they seek advice on how to start a political career? By looking at political aspirants who contact legislators for advice, I focus on an informal phase of the recruitment process to examine possible elite gender bias. To date, relatively little is known about how informal institutions shape candidate emergence. Most studies of gender bias tend to focus on the impact of formal institutions such as quotas (Frechette, Maniquet, and Morelli Reference Frechette, Maniquet and Morelli2008; Jones Reference Jones1998; Krook Reference Krook2009), electoral systems (Kittilson and Schwindt-Bayer Reference Kittilson and Schwindt-Bayer2010), district magnitude (Schmidt Reference Schmidt2009; Shugart Reference Shugart, Rule and Zimmerman1994; Taagepera Reference Taagepera, Rule and Zimmerman1994), and ballot structures (Jones and Navia Reference Jones and Navia1999; Luhiste Reference Luhiste2015; Schmidt Reference Schmidt2009; Thames and Williams Reference Thames and Williams2010). The scholarship that exists on informal institutions tends to address the challenges that female representatives face once they are in the legislature (Escobar-Lemmon and Taylor-Robinson Reference Escobar-Lemmon and Taylor-Robinson2009; Hawkensworth Reference Hawkesworth2003; Heath, Schwindt-Bayer, and Taylor-Robinson Reference Heath, Schwindt-Bayer and Taylor-Robinson2005; Kathlene Reference Kathlene1994). With a few exceptions (Bjarnegård Reference Bjarnegård2013; Bjarnegård and Kenny Reference Bjarnegård and Kenny2015), this scholarship on informal institutions is not mirrored to the same extent in the literature that addresses political recruitment.
There are several reasons why we might expect political elites to exhibit gender bias when responding to individuals who are thinking about a career in politics. As noted earlier, existing research suggests that there is gender discrimination in the political recruitment process, and numerous studies find that elites have historically been less likely to recruit female candidates than male ones (Fox and Lawless Reference Fox and Lawless2010; Fox and Oxley Reference Fox and Oxley2003; Sanbonmatsu Reference Sanbonmatsu2006). As an example, Niven (Reference Niven1998) finds that the majority of women holding local office across four U.S. states report having been discouraged from running for office by party leaders. Even when female and male candidates report receiving similar levels of encouragement from political elites, there seems to be some bias when it comes to the districts in which they are selected to run. For example, female candidates for state house and senate races in Florida in 2000 and 2002 report having been discouraged from running in favorable districts and instead encouraged to run in unfavorable districts, while men received the opposite messages (Niven Reference Niven2006). Similarly, there is evidence that female candidates for federal office in Canada in 2008 and 2011 were more likely than men to be nominated in noncompetitive districts (Erickson Reference Erickson1991; Thomas and Bodet Reference Thomas and Bodet2013).Footnote 7 One potential reason why political elites discriminate against female candidates has to do with how implicit and explicit gender stereotypes influence whom they deem appropriate for political office. When envisioning a strong legislative candidate, studies have found that party leaders tend to describe someone with stereotypically masculine traits (Niven Reference Niven1998). If political elites believe that female candidates are not suited to holding political office, then they will be less likely to encourage them to run for office. This reasoning leads to the gender bias hypothesis.
Gender bias hypothesis: Political elites will be less responsive to female political aspirants than to male political aspirants.
There are reasons to believe that the level of gender bias may vary depending on the gender of the political elite because of in-group and out-group bias. According to social identity theory (Tajfel and Turner Reference Tajfel, Turner, Austin and Worchel1979), individuals have a natural tendency to categorize people into groups based on shared identity traits such as gender, race, and religion. These shared group identities create a sense of connection and belonging, which can lead to a more favorable evaluation and treatment of in-group as opposed to out-group members, even in the absence of any conscious pro-in-group bias (McPherson, Smith-Lovin and Cook Reference McPherson, Smith-Lovin and Cook2001). According to social identity theory, therefore, female political aspirants should experience more discrimination from political elites who do not share their gender (men) than those who do (women). Consistent with the claim that people favor members of their in-group, Niven (Reference Niven1998) finds that male party chairs in the United States prefer candidates who resemble themselves on a range of traits, including gender, occupation, and personality characteristics. Since women are not free of these biases, female political elites should favor female political aspirants. While Tremblay and Pelletier (Reference Tremblay and Pelletier2001) find that female party presidents in Canada do not prefer candidates with stereotypically feminine traits, Cheng and Tavits (Reference Cheng and Tavits2011), as well as Medeiros, Forest, and Erl (Reference Medeiros, Forest and Erl2019), find that they are more likely to nominate women candidates in their constituency. This reasoning suggests a gender affinity story in which male elites will be more responsive to men and female elites will be more responsive to women. Note that this gender affinity story is consistent with the predictions from the gender bias hypothesis. The fact that contemporary political elites are primarily made up of men means that we should observe a gender bias on average against female political aspirants.
Gender affinity hypothesis: Political elites will be more responsive to political aspirants who share their gender. In other words, male political elites will be less responsive to female political aspirants, and female political elites will be more responsive to female political aspirants.
The gender affinity story suggests that we will see less discrimination against female political aspirants when the numeric or descriptive representation of female elites is high. This is because female political aspirants will enjoy more positive in-group bias and less negative out-group bias when women make up a larger percentage of the political elite. A common claim in the literature is that fewer women hold political office as we move up the levels of government (Baxter and Wright Reference Baxter and Wright2000; Blais and Gidengil Reference Blais and Gidengil1991; Palmer and Simon Reference Palmer and Simon2001, Reference Palmer and Simon2010), leading some scholars to talk of a “glass ceiling” when it comes to women's representation (Baxter and Wright Reference Baxter and Wright2000; Cotter et al. Reference Cotter, Hermsen, Ovadia and Vanneman2001; Ferree and Purkayastha Reference Ferree and Purkayastha2000; Folke and Rickne Reference Folke and Rickne2016). Political hierarchies, such as those that often exist between different levels of government, are commonly associated with increased discrimination against marginalized groups such as women, with the most powerful offices typically restricted to men and other high status elites. Putnam (Reference Putnam1976, 33) refers to this as the “law of increasing disproportion.” Bashevkin (Reference Bashevkin1993, 92) finds evidence of this hierarchical impact on women, which she summarizes as “the higher, the fewer” within Canadian parties. She later suggests that a similar result holds across the different levels of government in Canada (Bashevkin Reference Bashevkin and Bashevkin2009, 4). The reasoning is captured in the levels of government hypothesis.
Levels of government hypothesis: Political elites will be less responsive to female political aspirants than to male political aspirants at all levels of the government. However, this negative effect will grow as we move from the local to the national level.
The extent to which this general theoretical hypothesis applies in the Canadian case is somewhat open to question, though. One reason is that there is debate about the degree to which the different levels of government—municipal, provincial, and federal—represent a clear political hierarchy.Footnote 8 Constitutionally, the federal and provincial levels enjoy equal status, with municipalities occupying a subordinate status. However, some have argued that provinces, despite their formal and constitutional equality, are also subordinate in practice to the federal level, or at least perceived to be so, partly because of their limited monetary resources (Dyck Reference Dyck1998, 225). There is also debate as to whether the descriptive representation of women declines as we move from the municipal to the federal level. Tolley (Reference Tolley2011, 585) finds that women experienced a “municipal advantage” in about 60% of jurisdictions in 2009 and that the level of women legislators at the federal level was lower than at both the provincial and municipal levels from 2004 to 2009. However, these differences were substantively small and not necessarily reflective of earlier time periods. It remains an open empirical question, therefore, whether the degree to which political elites respond differently to female and male political aspirants will vary across the different levels of government in Canada.
How much gender bias political elites exhibit against female political aspirants should also depend on their partisan affiliation. Political elites are nested within political parties, and there is compelling evidence that parties differ in their ideological and behavioral commitment to gender egalitarianism. Past studies have shown, for example, that parties on the left of the ideological spectrum are more responsive to gender-related demands than parties on the right (Caul Reference Caul1999; Kittilson Reference Kittilson2006; O'Brien Reference O'Brien2018; Salmond Reference Salmond2006). The commitment of left-wing parties to issues of gender equality is discernible in their gender-egalitarian policies (Beckwith Reference Beckwith2000; Young Reference Young2000), their greater incorporation of women within leadership structures (Caul Reference Caul2001), and their initiatives to increase the presence of women in politics by recruiting more female candidates. Since legislators select into parties and egalitarian attitudes on gender issues are associated with left-wing party membership (Trembley and Pelletier Reference Tremblay and Pelletier2000), it is likely that political elites from left-leaning parties will be more responsive to female political aspirants than political elites from right-leaning parties. This reasoning is captured in the left-wing partisan hypothesis.
Left-wing partisan hypothesis: Political elites from left-leaning parties will be more responsive to female political aspirants than political elites from right-leaning parties.
Although most research assumes that political elites will be biased against women who are thinking about a political career, there are also reasons to expect that they will be equally responsive to male and female political aspirants. First, some scholars argue that when it comes to their reelection and political careers (Fiorina Reference Fiorina1989; Grose Reference Grose2011), interactions with constituents, like personal communications with them, are as important, if not more important, for political elites than legislative behavior (Fenno Reference Fenno1978). Since elites are likely to be vote-maximizing agents and voters can sanction them based on their interactions, they have an incentive to be responsive to all of their constituents irrespective of any potential biases they might otherwise have. Evidence for this comes from a recent audit experiment by Loewen and MacKenzie (Reference Loewen and MacKenzie2019) showing that 202 randomly selected Canadian legislators at the federal and provincial levels were equally responsive to requests for assistance from men and women. Second, if political elites believe that voter demand for female candidates is similar to voter demand for male candidates, as some existing research indicates (Black and Erickson Reference Black and Erickson2003; Golder et al. Reference Golder, Stephenson, Van der Straeten, Blais, Bol, Harfst and Laslier2017; Lawless and Pearson Reference Lawless and Pearson2008; Schwartz, Hunt, and Coppock Reference Schwarz, Hunt and Coppock2018), then strategic incentives will again encourage political elites to be equally responsive to female and male political aspirants. This reasoning is captured in the equal response hypothesis.
Equal response hypothesis: Political elites will be equally responsive to female and male political aspirants.
EXPERIMENTAL RESEARCH DESIGN
Identifying gender bias in the responsiveness of political elites is difficult with observational data because of potential problems with omitted variable bias, selection bias, and post-treatment bias. I avoid these methodological problems by conducting the first gender and politics audit experiment on an aspect of the political recruitment process outside the United States. In the audit experiment, I sent an email message from a political aspirant inquiring about a career in politics to legislators at the municipal, provincial, and federal levels in Canada. By randomizing the sex of the political aspirant, it is possible to determine whether political elites respond at equal rates to women and men. Whether political elites are willing to reply to an email from a political aspirant seeking advice on how to start a career in politics is important as responses serve as a visible signal of inclusion, indicating whether the female and male aspirants are welcome in the political profession. This type of “micro-mentorship” is often considered especially important for female aspirants as women are significantly more likely to put themselves forward as candidates if they are encouraged and actively recruited to run for office (Carroll and Sanbonmatsu Reference Carroll and Sanbonmatsu2013; Fox and Lawless Reference Fox and Lawless2004, Reference Fox and Lawless2014). Audit experiments, like the one conducted here, are well suited to investigating sensitive topics, such as elite gender discrimination, as they allow researchers to directly evaluate actual behavior, as opposed to attitudes or reported behavior, while mitigating selection bias and social desirability concerns.Footnote 9
The email message I sent to legislators in Canada is shown in Figure 1.Footnote 10 The email contained a request to learn about how the legislator entered politics and to receive advice on how to start a career in politics. The email was sent from a hypothetical university student, and each legislator received just one email. Having a university student, as opposed to a high school or middle school student, ask for advice about how to start a political career allows for a more credible inquiry since university students are more likely to have thought seriously about their career choices and taken steps in pursuit of their career objectives.Footnote 11 The only difference in the email sent to each legislator was whether the email was sent from (and signed by) an email account with a female or male first name. In effect, the randomized experimental treatment is the gendered name of the putative student. In line with the broader literature on audit studies (Bertrand and Mullainathan Reference Bertrand and Mullainathan2004), I used multiple female and male names to avoid the possibility that differences in elite responsiveness might be driven by a particular name effect as opposed to the gender of the political aspirant.Footnote 12
Email sent to legislators.
The names I used were the same as those used by Kalla, Rosenbluth, and Teele (Reference Kalla, Rosenbluth and Teele2018) in their study of political elite responsiveness in the United States. The 13 first names for women were Amanda, Ashley, Brittany, Emily, Hannah, Jessica, Kayla, Lauren, Megan, Rachel, Samantha, Sarah, Stephanie. The 13 first names for men were Andrew, Brandon, Christopher, Daniel, David, James, John, Joshua, Matthew, Michael, Nicholas, Ryan, and Tyler. The 26 surnames were Allen, Anderson, Brown, Clark, Davis, Hall, Harris, Jackson, Johnson, Jones, King, Lee, Lewis, Martin, Miller, Moore, Robinson, Smith, Taylor, Thomas, Thompson, Walker, White, Williams, Wilson, and Young.Footnote 13 I considered adding last names that would signal the race/ethnicity of the student, for example, a French- or South Asian–sounding last name. However, I ultimately refrained from doing this as the relatively small number of Canadian legislators limits my statistical power and makes factorial experimental designs that jointly manipulate the gender and race of the student less practical. What this means, though, is that my audit experiment is limited to testing whether political elites discriminate against Anglo-Canadian college-educated women. I randomly combined the first and last names to create 26 unique names. Finally, I generated Gmail accounts for each hypothetical student that took the following form: firstname.lastnameXXXX@gmail.com, where XXXX represents four random digits.
Given my interest in political elite bias, the population of interest is the universe of legislators in Canada. The names and contact information for Canadian legislators come from the Represent Civic Information API. The original sample consisted of 1,936 legislators.Footnote 14 However, I dropped several legislators for two reasons. First, I was forced to drop those legislators for whom an email address was not provided. Second, I dropped duplicate legislators. These were legislators who held multiple elected positions, such as city counselor and regional counselor. This left me with 1,779 unique legislators across all of the levels of government in Canada. Specifically, there are 854 municipal legislators (28.6% women), 591 provincial legislators (31.6% women), and 334 federal legislators (26.7% women). Of the 1,779 legislators in the final sample, five could not be reached because of an invalid email address. In line with common practice, I exclude these observations from the upcoming analyses (Butler and Broockman Reference Butler and Broockman2011). This means that the results reported in the article refer to the 1,774 legislators who actually received an email.Footnote 15
The emails were sent on January 20 and January 21, 2018, with legislators randomly assigned to receive their message on one of these days. To better test whether male and female legislators respond at different rates, I block-randomized the email messages on the gender of the legislator (Moore and Schnakenberg Reference Moore and Schnakenberg2012). This means that I first divided the legislators into two groups—male and female—and then I randomly assigned the treatment within these two groups.Footnote 16 The benefit of block randomization is that we can ensure that roughly equal numbers of male and female legislators are assigned to each experimental treatment (Gerber and Green Reference Gerber and Green2012). The information contained in Table 1 confirms that the randomization procedure was successful and that the two experimental treatment groups (male or female sender) are balanced demographically. Because I am interested in the responsiveness of political elites, my outcome measure, email response, is coded 1 if a response came from an email account associated with the legislator within two weeks, and 0 otherwise; I do not count auto-responses as replies.Footnote 17
Demographic balance across treatment groups
Note: Table 1 indicates the means for different demographic variables across the two treatment groups; standard deviations are shown in parentheses. The column p-value of difference refers to the p-value from a difference-in-means test across the two treatment groups.
RESULTS AND DISCUSSION
Before evaluating the specific hypotheses, I discuss the response rate in general. Almost half of the legislators in Canada responded to the email they received. Specifically, 864 (49%) of the 1,774 emails that were successfully sent received a response. This response rate was significantly higher than the 26% response rate in the United States (Kalla, Rosenbluth, and Teele Reference Kalla, Rosenbluth and Teele2018). The response rates decrease as we move from the local to the national level—the response rate was 57% for legislators at the municipal level, 46% for legislators at the provincial level, and 31% for legislators at the federal level. The upcoming reported response rates are based on responses that were received within two weeks of the original email message being sent. In Figure 2, I show the cumulative response rates across the various levels of government over time. As Figure 2 indicates, the temporal pattern of responses was very similar across the three levels of government, and almost all of the legislators who responded did so within two weeks of receiving the email from the hypothetical student. This is consistent with previous audit studies dealing with political elites, in which almost all responses were received within an initial two-week window (Costa Reference Costa2017).Footnote 18 Interestingly, there is no substantive difference in the overall response rates for female (48%) and male (49%) legislators.
Cumulative response rates by level of government. This figure shows the cumulative response rate across the different levels of government. The horizontal axis indicates the number of days since the original email message was sent; the last response was received 76 days after the original email was sent. The vertical axis indicates the cumulative proportion of responses received. The vertical dashed line at 14 days indicates the two-week cutoff for my upcoming analyses.
Table 2 provides information about response rates by treatment name and legislator gender. The first row shows how the overall response rate of the legislators varies depending on whether the hypothetical student sending the email message is female or male. The second and third rows show the response rates broken down by legislator gender. The 95% confidence intervals are shown in square brackets. Recall that the gender bias hypothesis predicts that legislators will be more responsive to male political aspirants than female ones and that the equal response hypothesis predicts that the gender of the political aspirants will have no effect on the response rates. The results from the audit experiment falsify both hypotheses. Canadian legislators respond to female students (52%) at higher rates than male students (45%). This pro-women bias of 7 percentage points is statistically significant (p = .01) and can be attributed solely to the gendered name manipulation in the experiment.Footnote 19 When Kalla, Rosenbluth, and Teele (Reference Kalla, Rosenbluth and Teele2018) conducted the same experiment on legislators in the United States, they found no evidence of gender discrimination by political elites against female aspirants. This result, which ran counter to expectations in the literature, raised concerns about external validity and made one wonder whether similar results would be found in other contexts. The results from my audit experiment in Canada also reveal no evidence of gender discrimination against female political aspirants. Indeed, unlike the study in the United States, I find evidence of a pro-women bias.
Response rates by treatment name and legislator gender
Note: The first two columns show the response rates to the email messages sent from male and female students for different sets of legislators. The third column indicates the percentage-point-difference in response rates, with positive differences indicating a pro-women bias. The 95% confidence intervals are shown in square brackets. The fourth column indicates whether the differences are statistically significant. The p-values come from a linear probability model where email response is the dependent variable and the model includes strata fixed effects (gender of legislator) and robust standard errors clustered at the email account level.
But does the gender of the legislator matter? According to the gender affinity hypothesis, male legislators are expected to be more responsive to male political aspirants and female legislators are expected to be more responsive to female ones. The fact that we already know that women are more likely on average to receive a reply than men is, on its face, an argument against the gender affinity story. Since there are more men in elected positions in Canada, the gender affinity story predicts that we should have observed a higher average response rate for male political aspirants. Nonetheless, we can examine the gender affinity hypothesis more directly by looking at the response rates of female and male legislators separately. The core finding is that both female and male legislators exhibit a substantively large and statistically significant pro-women bias in their response rates. The pro-women bias exhibited by female legislators (11 percentage points) is, after accounting for rounding, twice as large as that exhibited by male legislators (6 percentage points).Footnote 20
There is evidence that this pro-women bias in response rates also translates into a pro-women bias in terms of meaningful responses and micro-mentorship more generally. To evaluate this requires looking at the actual content of the email responses. There are several ways to do this.Footnote 21 One approach is to examine whether the responses contained substantive advice (Kalla, Rosenbluth, and Teele Reference Kalla, Rosenbluth and Teele2018).Footnote 22 For example, email responses in which the legislator suggested that the student should (1) run for student government, (2) learn about the issues, (3) attend local party or political meetings, (4) learn to be extroverted, or (5) always put their values first were coded as providing substantive advice. About 26% of the responses were coded as giving substantive advice. Of the responses that did not specifically offer substantive advice, many encouraged the student to call or set up a meeting. Overall, there is a statistically significant 3 percentage points pro-women bias among all legislators when it comes to receiving substantive advice.
A second approach employs a new measure of elite responsiveness for audit studies developed by Costa (Reference Costa2020). This measure distinguishes between responses that are meaningful and those that only satisfy some minimum required effort on the part of the legislator. Using this “quality of response” measure, I again find a statistically significant pro-women bias (6 percentage points) among Canadian legislators. Finally, we might suspect that longer responses are more substantively meaningful than shorter ones. On this metric, I find that the responses to female political aspirants are significantly longer and contain more characters than those to male political aspirants.
While there is an overall pro-women bias in terms of responsiveness among legislators in Canada, it is still possible that there is a glass ceiling effect where female political aspirants do less well as we move from the local to the national level. Table 3 provides information about response rates by treatment name, level of office, and legislator gender. There are three main sections that each relate to legislators at either the municipal, provincial, or federal levels. Within each section, the first row shows how the overall response rate at the specified level of government varies depending on whether the hypothetical student sending the email message is female or male. The second and third rows in each section show the response rates broken down by whether the legislator is female or male. Again, 95% confidence intervals are shown in square brackets. Contrary to the levels of office hypothesis, there is a pro-women bias at all three levels of government. Significantly, the magnitude of this bias is fairly consistent across the different levels. Specifically, the pro-women bias is 7 percentage points at the municipal level, 8 percentage points at the provincial level, and 6 percentage points at the federal level. This overall pro-women bias is only statistically significant at conventional levels at the municipal level. However, this may well be due to the fact that the sample size shrinks markedly as we move from the municipal to the federal level. While a pro-women bias is exhibited by both female and male legislators at each level of government, the magnitude of the pro-women bias is typically larger for female legislators. At the municipal level, the pro-women bias exhibited by female legislators (12 percentage points) is 2.5 times larger than that exhibited by male legislators (5 percentage points). At the provincial level, the pro-women bias exhibited by female legislators (15 percentage points) is three times larger than that exhibited by male legislators (5 percentage points). There is no substantive difference in the magnitude of the pro-women bias across female and male legislators at the federal level. Indeed, it is only at the federal level that the pro-women bias exhibited by female legislators is not statistically significant.
Response rates by the treatment name, level of office, and legislator gender
Note: The first two columns show the response rates to the email messages sent from male and female students for different sets of legislators separated by level of office. The third column indicates the percentage-point-difference in response rates, with positive differences indicating a pro-women bias. The 95% confidence intervals are shown in square brackets. The fourth column indicates whether the differences are statistically significant. The p-values come from a linear probability model where email response is the dependent variable and the model includes strata fixed effects (gender of legislator) and robust standard errors clustered at the email account level.
Are there partisan effects? According to the left-wing partisan hypothesis, legislators from left-leaning parties will be more responsive to female political aspirants than legislators from right-leaning ones. In what follows, I focus on the three largest political parties at the federal level in Canada: the New Democratic Party (NDP), the Liberal Party, and the Conservative Party. On a left-right ideological scale, the NDP is on the left, the Liberal Party is center-left, and the Conservative Party is on the right. Demographically, Canadian parties differ in their gender composition. In line with the literature (Caul Reference Caul1999, Reference Caul2001; Kittilson Reference Kittilson2006; Medeiros, Forest, and Erl Reference Medeiros, Forest and Erl2019; Salmond Reference Salmond2006), the proportion of female legislators in my sample is highest in the left-leaning NDP (101 men and 88 women) and lowest in the right-leaning Conservative Party (215 men and 48 women); the proportion of female legislators in the center-left Liberal Party is in between (200 men and 85 women). Ideologically, the three parties differ in their support for enhancing political access for women. The NDP is a social democratic party that has been at the forefront of enhancing the electoral representation of women (Cross Reference Cross2004; Matland and Studlar Reference Matland and Studlar1996; Pruysers and Cross Reference Pruysers and Cross2016; Young and Cross Reference Young and Cross2003). Furthermore, the NDP and the Liberal Party have both taken more concrete steps than the Conservative Party to increase diversity and in particular increase the representation of women (Cross Reference Cross2004; Pruysers and Cross Reference Pruysers and Cross2016; Young and Cross Reference Young and Cross2003). At the provincial level, there is considerable variation in party systems, both in terms of the identities of the parties competing but also in the extent to which these parties are organizationally integrated with the parties competing at the national level (Thorlakson Reference Thorlakson2009). It is possible, however, to classify many of the provincial parties into broad ideological categories associated with the left, the center, and the right.Footnote 23 Elections at the municipal level tend to be nonpartisan; as a result, my upcoming discussion focuses on the effect of partisanship at only the federal and provincial levels.
Table 4 provides information about response rates by treatment name, party ideological type, and legislator gender. There are three main sections in the table depending on whether the legislator is associated with a left, center, or right party. Within each section, the first row shows how the overall response rate in the specified party varies depending on whether the hypothetical student sending the email message is female or male. The second and third rows in each section show the response rates broken down by whether the legislator is female or male. As before, 95% confidence intervals are shown in square brackets. The legislators from all three party types exhibit an overall pro-women bias. However, in line with the left-wing partisan hypothesis, the legislators from the left-wing parties respond at higher rates (57%) to female political aspirants than legislators from the right-wing parties (48%). Moreover, the pro-women bias exhibited by legislators from the left-wing parties (11 percentage points) is almost two times larger than that exhibited by legislators from the right-wing parties (6 percentage points). On the whole, both female and male legislators from each of the party types respond to female political aspirants at a higher rate than male political aspirants. The only time when this is not the case comes when we look at female legislators from the right-wing parties; these particular legislators demonstrate a pro-men bias (–7 percentage points).
Response rates by the treatment name, party ideology, and legislator gender
Note: The first two columns show the response rates to the email messages sent from male and female students for different sets of legislators separated by party ideology. The third column indicates the percentage-point-difference in response rates, with positive differences indicating a pro-women bias. The 95% confidence intervals are shown in square brackets. The fourth column indicates whether the differences are statistically significant. The p-values come from a linear probability model where email response is the dependent variable and the model includes strata fixed effects (gender of legislator) and robust standard errors clustered at the email account level.
CONCLUSION
Women remain significantly underrepresented in politics in virtually every country in the world. The political recruitment process consists of three stages: self-selection, party selection, and voter selection. While there is evidence that gender discrimination against women exists at all three of these stages, the earlier stages appear to be more problematic for women's representation (Fox and Lawless Reference Fox and Lawless2004, Reference Fox and Lawless2010; Thomas and Bodet Reference Thomas and Bodet2013). In this article, I have examined whether there is elite gender discrimination in an informal phase of the recruitment process where political aspirants are seeking encouragement or mentorship from elites to start a political career. This is a particularly important phase of the recruitment process as studies have repeatedly shown that elite encouragement, especially for women, plays an influential role in getting political aspirants to put themselves forward as potential candidates (Fox and Lawless Reference Fox and Lawless2004; Karpowitz, Monson and Preece Reference Karpowitz, Quin Monson and Preece2017; Niven Reference Niven2006).
To identify if there is gender discrimination against female political aspirants, I employ an email audit experiment. Audit experiments are particularly well suited to investigating sensitive topics such as gender discrimination as they mitigate concerns that researchers might have with social desirability and selection biases and help overcome methodological problems with omitted variable and post-treatment biases that affect studies that rely on observational data. Audit studies also have the advantage that they provide us with a behavioral, as opposed to a reported behavioral or attitudinal, measure of discrimination. For those interested in increasing women's political representation, especially in Canada, the results from my experiment are promising. Overall, I find no evidence that Canadian legislators discriminate against female political aspirants who contact them.Footnote 24 Indeed, legislators in Canada appear to be both significantly more responsive and helpful to female political aspirants than male ones. This pro-women bias, which exists at all levels of Canadian government, is stronger among female legislators and those associated with left-leaning parties.
One of the goals of this research was to examine the generalizability of the results from a similar audit experiment conducted by Kalla, Rosenbluth, and Teele (Reference Kalla, Rosenbluth and Teele2018) in the United States. That study found that there was no significant difference in the response rates of legislators to female and male political aspirants. One of the common concerns raised with experimental research has to do with external validity. To what extent do the results from one experiment generalize to other contexts? This concern is particularly pertinent when the findings of an experiment, like the one conducted by Kalla, Rosenbluth, and Teele (Reference Kalla, Rosenbluth and Teele2018), run counter to expectations in the literature. As my audit experiment indicates, the result that female political aspirants do not experience elite gender discrimination in this informal phase of the political recruitment process does generalize beyond the United States, at least to Canada. I encourage scholars to further evaluate the external validity of these findings by examining whether they generalize to additional cases.
While the responsiveness of legislators to female political aspirants who express an interest in politics does not necessarily imply that parties are actually going to nominate them as candidatesFootnote 25 the results of these audit experiments in the United States and Canada should be encouraging for women who are thinking about a career in politics. Furthermore, these results suggest that to the extent that gender discrimination against women does exist in the early stages of the political representation process, it does not occur in this informal phase—when political aspirants are seeking advice on how to start a political career—but at some other point in the process of going from a citizen to a legislator. On this point, it is worth noting that the email requests for advice in these audit experiments came from “self-starters” who had already self-identified as political aspirants and decided to reach out for help (Kalla, Rosenbluth, and Teele Reference Kalla, Rosenbluth and Teele2018). It is possible that a gender bias exists even earlier in the recruitment process, when men and women are thinking about whether a career in politics is for them. To the extent that women are concerned that they will not receive support if they put themselves forward, the results from these audit experiments should be particularly reassuring.
Ultimately, if women's political representation is to increase, it is important to identify exactly where gender discrimination occurs and why. Audit experiments are an important part of the methodological toolkit for studying gender discrimination as they can help us understand exactly where and how gender distortions are occurring.
SUPPLEMENTARY MATERIAL
To view supplementary material for this article, please visit https://doi.org/10.1017/S1743923X20000227.
