Double Selection Bias

Perhaps the major shortcoming of the existing evidence is that it suffers from potentially severe selection bias (Dancygier and Laitin Reference Dancygier and Laitin2014; Hainmueller, Hangartner, and Pietrantuono Reference Hainmueller, Hangartner and Pietrantuono2015; Kesler and Demireva Reference Kesler and Demireva2011). In order to isolate the causal effect of naturalization, we need to compare two groups of immigrants that differ in their naturalization status, but are otherwise similar on all other characteristics that can independently affect integration. The fundamental problem is that such a comparison is hard to come by empirically with typical observational data, because there is a complex two stage selection bias that determines which immigrants obtain citizenship.

Figure 1 illustrates the two stages in the double selection process. In the first stage, immigrants choose to apply for naturalization and this decision is based on a whole host of reasons that have independent effects on integration. For starters, only immigrants who are sufficiently motivated and have the resources to apply for naturalization will obtain citizenship, while the group of non-naturalized immigrants contains many immigrants who lacked the resources or motivation to apply for citizenship in the first place. Arguably, the motivation and resources to apply are among the most important confounders when trying to estimate the effects of citizenship because the resources and motivation to apply are strong determinants of integration into the host country. In addition, there are many other potential differences that explain why immigrants choose to apply. Plenty of evidence suggests that those who choose to apply typically have resided in the country for a longer period of time (in part simply due to residency requirements), and are better informed, better integrated, and perhaps more educated or more fluent in the local language (see, for example, Chiswick and Miller (Reference Chiswick and Miller2009)). Immigrants who apply might also identify more strongly with the host country and its culture or have differences in other traits, such as their intention to stay or political interest, that lead them to seek citizenship, compared to the group of immigrants who do not (see, for example, Yang (Reference Yang1994)). The unadjusted comparison of non-naturalized and naturalized immigrants is therefore one of apples and oranges.

FIGURE 1.

Double Selection Bias

Note: Illustration of the double selection bias that confounds the comparison of naturalized and non-naturalized immigrants.

In the second stage, decision makers then review the applications and often interview the applicants to decide who gets citizenship and who is denied. The problem here is that decision makers typically have much more information about the applicants than is observed by the researcher, and they would typically use this information to decide on the applicants. For example, applicants who fail to make a “good impression” in the application interview (in terms of appearance, language skills, familiarity with the host country, etc.) might be more likely to be rejected because they are perceived to have a low potential to integrate. As a result of this screening, the comparison between accepted and rejected applicants is again like comparing apples and oranges because the reasons that determine why an applicant is rejected might be correlated with the integration outcomes of interest. For example, those who are judged to have a lower integration potential might, in fact, be less likely to integrate successfully. Overcoming this double selection bias with typical observational data is a fairly hopeless endeavor. We cannot measure the myriad unobserved confounders that determine immigrants’ selection into applying as well as all the unobserved confounders that determine the decision makers’ selection among the applicants. In fact, we typically have little information about whether and why immigrants applied and also much less information about the applicants than the decision makers when they make their screening decisions. But unless we can control for all the confounding characteristics that determine the selection in both stages, we will end up with biased estimates of the effect of citizenship since the unmeasured confounding characteristics are correlated with the outcomes and the application decision.

Note that a similar selection bias applies when trying to estimate the effect of the timing of the naturalization. The timing of when immigrants naturalize is again far from randomly assigned, and there are many potential differences that explain why some immigrants choose to apply early and others choose to apply only later into their residency period. For example, more motivated or better informed immigrants might apply right after they become eligible, while less motivated or informed ones delay their naturalization until they have been in the host country for a long time.

In order to eliminate the double selection bias and isolate the causal effect of citizenship from the effect of differences in background characteristics, the ideal design would involve an experiment where we randomly assign citizenship among a group of eligible immigrants. Random assignment forms the gold standard for causal attribution, because it ensures that the treatment group of immigrants who obtain citizenship is similar to the control group of immigrants who do not obtain citizenship on all measured and unmeasured characteristics in expectation. Our research design exploits a natural experiment in Switzerland that mimics this ideal experiment.

The Swiss Naturalization Regime in Comparative Perspective

Naturalization has been a divisive issue in Switzerland for many decades given its unusually large immigrant population. To compare the Swiss naturalization regime with that of other European and North American Countries, Figure 2 plots the Citizenship Policy Index (CPI) for various countries for the year 2005.Footnote ⁶ The CPI is a standard measure developed by Howard (Reference Howard2005) that uses an additive formula to measure a country's citizenship policy between very liberal (6) and highly restrictive (0). It is based on the three main components of citizenship policy: whether citizenship is granted by place of birth or by citizenship of the parents, the length of the residency requirement for naturalization, and the acceptance of dual citizenship for immigrants (see Howard (Reference Howard2005) and the Online Appendix for details).

FIGURE 2.

Citizenship Policy Index for European and North American Countries

Note: The Citizenship Policy Index (CPI) measures a country's citizenship policy between very liberal (6) and highly restrictive (0) based on citizenship by birth, residency requirements, and acceptance of dual citizenship.

The plot reveals that the score for the Swiss citizenship regime is the modal category on the CPI, on par with other restrictive countries like Germany or Italy that also use the jus sanguinis principle (i.e., citizenship is passed on from the parents’ citizenship). While Switzerland does require a fairly long residency period, its regime is more liberal insofar as it allows dual citizenship in contrast to many of the more restrictive countries.Footnote ⁷

Naturalization Referendums

Naturalization applications in Switzerland are decided at the municipal level. An immigrant who has cleared the eligibility requirements and seeks naturalization is required to apply with the municipality in which he or she resides. The municipal authorities then process and green light the application until it is eventually put to a vote (see Hainmueller and Hangartner (Reference Hainmueller and HangartnerForthcoming) for an overview). We focus on the group of so called ballot box municipalities who until 2003 used secret ballot referendums to decide on the applications.Footnote ^{8 ,} Footnote ⁹ A naturalization referendum typically had two phases. In the first phase, a voting leaflet was mailed to all Swiss voters in the municipality that informed the voters about the pending naturalization requests with a short résumé that described each applicant. The résumés typically included information about the applicant's origin, gender, marital status, number of children, year of arrival, education, occupation, and an assessment of their language skills and integration levels based on the application interview.Footnote ¹⁰ In the second phase, voters then cast a secret ballot where they voted “yes” or “no” on each applicant, and Swiss citizenship was awarded only to applicants who received a majority of positive votes. Note that voting on referendums occurred in regular intervals, and naturalization referendums appeared on the ballot alongside other questions about municipal matters that are all typically decided via referendums in Switzerland, such as decisions about the local budget, infrastructure, urban planning, etc. The use of naturalization referendums ended in 2003 when the Swiss federal court ruled that secret ballot referendums can no longer be used for naturalization decisions (see Hainmueller and Hangartner (Reference Hainmueller and HangartnerForthcoming) for details).

The naturalization referendums allow us to devise two identification strategies that overcome the thorny double selection bias and get at the long-term effects of naturalization. The identification strategies guard against selection bias in two ways. First, we can avoid the selection into applying by limiting the analysis to only those motivated immigrants who applied and cleared the eligibility criteria such that they faced a naturalization referendum. Second, we can remove the second stage selection into who is accepted for naturalization using two strategies that exploit the use of voting leaflets and the occurrence of close referendums, respectively.

Instrumental Variable Strategy

In the first strategy we utilize the fact that we can measure and control for all the applicant characteristics that were reported to voters in the voting leaflets when they decided on the applicants and therefore rule out omitted variable bias. In contrast to the situation where an immigration official decides on the applicants based on information that is unobserved to the researcher, here we do observe all the relevant applicant characteristics that were reported to voters who decided on each request. Once we adjust for the reported characteristics and compare applicants who applied in the same municipality, in the same time period, have the same gender, country of origin, marital status, number of children, education, occupational skill, years of residency, assessed integration level, and language proficiency, such matched applicants are observably equivalent to voters, and therefore voters cannot systematically discriminate between applicants based on their unobserved characteristics. Therefore, among such observably equivalent applicants who are matched on the characteristics that voters see on the leaflets, who wins and who loses is not driven by systematic differences in the integration potential of the individual immigrants, but by idiosyncratic shocks that affect the aggregate vote outcomes such as what else appeared on the ballot or the weather on the day of the referendum. Hainmueller and Hangartner (Reference Hainmueller and Hangartner2013) provide substantial evidence for this selection on observables assumption. For example, they show that the effect of the reported applicant characteristics on the vote outcomes are similar in large and small municipalities which rules out the possibility that private information about the applicants might have a systematic effect on the outcomes of the referendums.

One remaining issue that we have to address with this strategy is the issue of noncompliance, by which we mean the fact that a sizable proportion of applicants who lost their first naturalization referendum reapplied and subsequently obtained citizenship. Fortunately, we can directly address the issue of reapplications by exploiting the exogenous variation in naturalization status that results from winning or losing the first referendum that each applicant faces. For this we apply the instrumental variable (IV) framework with heterogeneous treatment effects as developed in Angrist, Imbens, and Rubin (Reference Angrist, Imbens and Rubin1996) which allows us to treat the outcome of the first referendum like a randomized encouragement design experiment where those applicants who win their first referendum are encouraged to get citizenship, while those who do not win their first referendum are encouraged not to get citizenship.

Employing the framework of Angrist, Imbens, and Rubin (Reference Angrist, Imbens and Rubin1996) the population of applicants is made up of two subgroups. The subgroup of so-called compliers are the applicants who comply with the encouragement. In other words, they get naturalized if they win their first referendum but do not get naturalized if they lose their first referendum. The other subgroup are the so-called always-takers. These are the applicants who always get naturalized; even if they lose their first referendum, they reapply and subsequently get citizenship.Footnote ¹¹

To identify the local average treatment effect of naturalization (LATE) for the subgroup of compliers we compute the intention-to-treat effect (ITT), which is the effect of wining the first referendum on social integration, and divide it by the proportion of compliers in our sample, which is given by the first stage effect of winning the first referendum on the probability of naturalization or equivalently the difference between the proportion of winning applicants who do get Swiss citizenship and the proportion of losing applicants who nonetheless get citizenship through a reapplication.

To estimate the LATE, we code a binary treatment indicator that captures whether the immigrant is naturalized or not and a binary instrument that captures whether the immigrant won or lost his or her first referendum. We then run a two-stage least squares model regressing the integration outcome on the reported applicant characteristics from the leaflets, municipality and time period fixed effects, and the treatment variable which we instrument with the instrumental variable (Angrist, Imbens, and Rubin Reference Angrist, Imbens and Rubin1996). Importantly, this strategy relies on the fact that we have enough compliers in our sample and therefore the first stage effect is sufficiently strong. Below we test this assumption and find that the instrument is indeed sufficiently strong.

Fuzzy Regression Discontinuity Design Strategy

We also apply a second, complementary, empirical strategy based on a fuzzy regression discontinuity (RD) design which similarly removes the second stage selection into who is approved for naturalization. The fuzzy RD design exploits the exogenous variation that is generated among the subset of applicants who barely won or lost their first naturalization referendum by just a few votes. In narrowly decided referendums, the outcome of the referendum is largely decided by random factors, such as the weather on election day or other agenda items that appeared on the ballot, rather than the characteristics of the applicants. In other words, who loses and who wins is as good as randomly assigned and we can therefore isolate the causal effect of citizenship on the downstream integration outcomes just like in a randomized experiment. The required identification assumption in the RD design is that the potential integration outcomes of the immigrants are continuous at the threshold (Hahn, Todd, and Van der Klaauw Reference Hahn, Todd and Klaauw2001). This assumption could fail if immigrants could sort around the threshold such that barely rejected and barely accepted applicants would differ systematically. However, sorting around the threshold would require that individual immigrants have precise control over the aggregate referendum outcome, which is not plausible in the large, secret ballot referendums under study (see Eggers et al. (Reference Eggers, Fowler, Hainmueller, Hall and Snyder2015) for support for the no sorting assumption in a wide variety of elections).

Figure 3 illustrates the logic of the fuzzy RD design. The top left panel plots a balance test where the vertical axis is the applicants’ vote share margin from the first naturalization referendum and the horizontal axis is the applicants’ years of education—as reported on the voting leaflet. The vote margin is computed as the difference between the applicants’ share of yes votes and the threshold of 50% of yes votes that the applicant had to exceed to win the referendum and thereby receive Swiss citizenship. The plot is focused on the sample of “competitive” applicants who are within a ± 15% window around the threshold of winning. The grey and black line summarize the average years of education on both sides of the threshold, respectively. We see that in close referendums, which are decided by just a few votes, who loses and who wins is as good as random and therefore the education level of close winners and close losers are similar on average at the threshold. Given this local random assignment, we expect close winners and close losers to be similar on all other observed and unobserved confounders, just like in a randomized experiment and this covariate balance allows us to remove the selection bias. Figure B.2 in the Online Appendix shows that close winners and close losers are similarly balanced on other background characteristics; the distribution of p values from the balance tests closely approximates the uniform distribution as expected given randomization at the threshold.Footnote ¹²

FIGURE 3.

Fuzzy Regression Discontinuity Design: Identification Checks and the Effect of Naturalization on Long-Term Social Integration

Notes: Upper left panel indicates that years of education (a pretreatment covariate) is balanced at the victory threshold in the naturalization referendums. Upper right panel shows that the density is smooth across the victory threshold, suggesting that there is no evidence of sorting. Lower left and right panels show that long-term social integration and the probability of naturalization sharply increase when comparing applicants who barley won and barley lost their first naturalization referendum. (Loess lines; 95% confidence intervals for binned averages.)

The top right panel shows another important identification check for the fuzzy RD design where we follow McCrary (Reference McCrary2008) and explicitly test for the no sorting assumption by computing the density of the vote margin variable. If applicants had precise control to manipulate their voting results we should see an unusually large (small) number of applicants who barely won (lost). Instead, we see that the density is smooth across the threshold, which implies that there is no evidence for sorting of applicants around the threshold. This is what we expect given that it is implausible for applicants to precisely control the outcome of referendums that involve thousands of voters.

The plot in the bottom left panel previews the main result for the ITT effect. The lines summarize the applicants’ average score on the social integration scale, the summary measure of social integration measured in our recently administered follow-up survey, as a function of the vote share margin. We see that levels of social integration jump considerably at the threshold such that applicants who barely won their first referendum and received Swiss citizenship are today much better integrated on average compared to otherwise similar applicants who barely lost their first referendum. Given the local random assignment at the threshold, we can attribute this effect to winning the referendum as opposed to differences on omitted variables.

Note that this ITT effect, which amounts to about a 0.14 increase on the social integration scale, understates the effect of naturalization for compliers because many applicants who lost their first referendum eventually naturalized by way of re-applications and therefore also received the treatment. To correct for this noncompliance and identify the LATE of naturalization for compliers at the threshold, we need to scale the intention-to-treat effect by the inverse of the compliance ratio at the threshold (Hahn, Todd, and Van der Klaauw Reference Hahn, Todd and Klaauw2001).

The bottom right panel visualizes the first stage effect. The lines show the share of naturalized applicants as a function of the vote margin. The probability of naturalization increases sharply by about 0.28 at the threshold, and therefore the LATE of naturalization for compliers amounts to about 0.14/0.28 = 0.5. Note that the social integration scale has a standard deviation of 0.5 so the LATE estimate implies that naturalization considerably increased the long-term social integration of immigrants by about a full standard deviation unit. In the results section below we formally estimate the fuzzy RDD effect at the threshold by fitting a similar two-stage least squares model which regresses the integration outcome on the treatment indicator and instrument this indicator with a binary instrumental variable that captures whether applicants succeed in their first referendum or not. To this regression we also add the vote margin and the interaction of the vote margin with the instrumental variable such that the LATE of naturalization is identified for compliers only right at the threshold of winning.

Note that the two empirical strategies are complementary to each other in that they identify the same naturalization effect based on slightly different assumptions. However, there is an important difference in the external validity between the two designs since they identify this effect for different subgroups of applicants. The IV design offers higher external validity because it identifies the LATE of naturalization for the subgroup of compliers in general, while the fuzzy RD design is limited in its external validity as it only identifies the LATE of naturalization for the subgroup of compliers who are right at the threshold of winning. Because of this local identification we also lose precision in the fuzzy RD design and have less power to detect potential naturalization effects.

Sample and Covariates

We draw on a variety of original data to implement our empirical strategies. The basis for our sample is the data compiled by Hainmueller and Hangartner (Reference Hainmueller and Hangartner2013) based on the voting leaflets and voting outcomes for all 2,225 applicants who faced naturalization referendums between 1970 and 2003 in all the 46 ballot box municipalities that used secret ballot referendums with voting leaflets (see Online Appendix for details).

Our covariates capture the applicants’ characteristics reported on the leaflets. They include the applicants’ gender, age, number of children, country of origin, marital status, highest educational attainment, occupational skill, years of residency prior to the application (including an indicator for immigrants born in Switzerland), language proficiency, and integration status. The Online Appendix describes the coding of all variables used in our analysis and provides the descriptive statistics (Tables B.2 and B.3).

To measure the social integration outcomes, we administered a survey of all immigrants who faced naturalization referendums. We first extracted the addresses of these immigrants at the time of their naturalization referendum and then tracked down the applicants to the best of our abilities and administered a survey by phone. As expected, several of the addresses were outdated as immigrants had moved, died, or left the country. Nonetheless, we interviewed 768 applicants, which amounts to a cumulative response rate 3 (RR3) as defined by the American Association for Public Opinion Research of 34.5%. Among competitive applicants who were within a ± 15% vote margin of winning, the response rate was even higher, and we interviewed 474 applicants for an RR3 of 45.9%. This is a higher response rate than is typically achieved by phone surveys in Switzerland or the United States, let alone for surveys of immigrants (see Online Appendix for details).

One potential concern might be that the probability of being interviewed is correlated with naturalization and integration. In the Online Appendix we provide evidence that this is not a concern in our study. In particular, we find that the probability of being interviewed as well as the characteristics of those being interviewed are no different for immigrants who were narrowly accepted and narrowly rejected for naturalization (see Figure B.1 and Table B.1). Moreover, in our context we would expect that differential attrition would, if anything, lead to an attenuation bias in our effect estimates if naturalized immigrants are more likely to stay in Switzerland than non-naturalized immigrants and, among the non-naturalized, those with lower levels of integration are more likely to leave than those with higher levels of integration.

Outcome Measures

Immigrant social integration is a latent and multifaceted concept that involves several dimensions such as social inclusion, social engagement, intergroup contact, social capital, and discrimination (Berry Reference Berry1997; Carens Reference Carens2005; Castles et al. Reference Castles, Korac, Vasta and Vertovec2002; OECD 2012). Studies have used different measures to capture these various dimensions of social integration, and there is no single commonly agreed upon measure that is consistently applied in the literature. Given this, our study takes a pragmatic approach to measurement and combines four existing measures in a social integration scale to obtain a summary measure of social integration. The use of a scale ensures that the results are not driven by a single survey question that might only tap into one dimension of social integration. Averaging across multiple measures also addresses the well-known problem of potentially serious measurement error that typically arises in survey research when trying to measure a latent concept with single survey questions (Achen Reference Achen1975; Ansolabehere, Rodden, and Snyder Reference Ansolabehere, Rodden and Snyder2008).Footnote ¹³

Our scale is generated from four survey questions that have been used in prior work and tap into various dimensions of social integration. The first item, Plans to stay in Switzerland, is a question that measures whether immigrants are planning to stay in Switzerland permanently or whether they have plans to leave Switzerland. It is coded with values 1, 0, and − 1, for immigrants who have plans to stay forever, those who are not sure, and those who say they plan to eventually leave Switzerland, respectively (the Online Appendix provides all the question wordings). This item captures whether naturalization has changed the long-term attachment and settlement plans of immigrants and thereby increased their incentive to invest into a future in Switzerland and reduced the uncertainty associated with potential return migration (Dustmann Reference Dustmann1996). This measure is of theoretical importance because it directly taps into the theoretical mechanisms through which naturalization might increase social integration by shifting the time horizons of immigrants towards a long-term future in the host country. It is important to emphasize that the immigrants in our sample who did not naturalize still have fulfilled the long residency requirements that make them eligible for naturalization, and they all hold a permanent residency permit which allows them to stay in Switzerland for as long as they wish. This comparison group therefore provides a rather high bar for finding an effect of naturalization on changing long-term settlement plans. Permanent residents who have been in Switzerland for a long time can be expected to already have a fairly long time horizon and a high likelihood of wanting to stay in Switzerland for good. It is therefore far from obvious whether naturalization would have enough of an effect to shift time horizons even further towards permanently settling in Switzerland.

The second item, Discrimination, is a standard measure of perceived discrimination that is coded as 1 for immigrants who describe themselves as being a member of a group that is discriminated against in Switzerland, and 0 if not (Kesler and Demireva Reference Kesler and Demireva2011). Discrimination is an important barrier to social integration and a potent source of marginalization and strained intergroup relations between immigrants and host country nationals. As Kesler and Demireva (Reference Kesler and Demireva2011; 215) put it, “perceptions of discrimination are therefore likely to be important in generating a sense of social cohesion among immigrants.” This measure also taps into the psychological mechanisms discussed above where naturalization might act as a signal of acceptance and increases a sense of belonging among immigrants and the recognition of naturalized immigrants as equals by natives.Footnote ¹⁴

The third item, Club membership, is a measure of social capital that captures whether immigrants are currently active members of a social club or association in which they participate in regular meetings. The answer options included several potential organizations such as a youth organization, social club, volunteer firefighters, carnival club, political association, a local charter of a charitable organization, or “other,” and we code the measure as 1 for immigrants who actively participate in at least one of these organizations and 0 otherwise.Footnote ¹⁵ These clubs form an essential part of the social life in Swiss communities, and are a standard measure of social integration in official statistics in Switzerland and many other European countries (Kesler and Demireva Reference Kesler and Demireva2011; Avitabile, Clots-Figueras, and Masella Reference Avitabile, Clots-Figueras and Masella2013; Kristensen Reference Kristensen2014). This item therefore taps into whether naturalization increased the social capital and community engagement of immigrants and thereby their exposure to and interaction with natives, which is another relevant dimension of social integration.

The fourth item, Swiss newspaper, is a question that measures whether immigrants read newspapers from Switzerland or foreign newspapers from their home country. The answers are coded on a five-point scale ranging from 5 for immigrants who read exclusively Swiss newspapers to 1 for immigrants who exclusively read newspapers from their home country. This item also taps into the theoretical mechanisms and captures whether naturalization has indeed shifted the orientation of immigrants towards Switzerland and away from their homelands in the sense that immigrants feel the need to acquire information and knowledge about the host country environment as opposed to their country of origin (Avitabile, Clots-Figueras, and Masella Reference Avitabile, Clots-Figueras and Masella2013; Dustmann Reference Dustmann1996).

To construct the social integration scale from these four items, we extract the first principal component from a polychoric principal component analysis (PCA), which has the advantage that it takes into account the binary and categorial distribution of the items (see Online Appendix for details). To aid the interpretability, we rescale the first principal component, which explains about 45% of the total variance, to have a mean zero and standard deviation of 0.5. Note that the results of all models are virtually identical if we use a simple equal weighted average of the four items instead.

It is important to emphasize that in contrast to other studies of naturalization, our outcomes capture the long-term effects of naturalization. Given that the use of naturalization referendums ended in 2003, at the time of our survey, the average naturalized immigrant had possessed Swiss citizenship for about 15 years. Our design therefore enables us to examine whether naturalization had any lasting effects in promoting the long-term social integration of immigrants, rather then resulting in only temporary short-term changes.