Critique of the Empirical Analysis

The first point of critique is Woodberry's (Reference Woodberry2012) choice of the Bollen-Paxton (BP) data as a measure of democracy. The BP democracy score is constructed using four measures of democratic rule (political rights, competitiveness of nomination process, elected chief executive and effectiveness of elected legislature body) and four measures of political liberties (freedom of broadcast media, freedom of print media, civil liberties and freedom of group opposition). The equally weighted mean of the indices of political rights and legislative effectiveness was then averaged with the index of political opposition. In addition, the score accounts for restrictions on the franchise by capping the political rights variable (and its average with legislative effectiveness) at the franchise percent in a given country (Paxton et al. Reference Paxton2003). The resulting democracy indicator assumes values between 0 and 100.Footnote ² Woodberry uses Bollen's (Reference Bollen2009) definition of a liberal democracy: ‘the degree to which political liberties and democratic rule exist in a country’ (369).

While conventional democracy indices may lead to some rater bias, there is no apparent reason to favor the BP data over other, more comprehensive dichotomous or continuous indicators of democracy that also cover recent years. In particular, Woodberry's analysis limits the timeframe over which the dependent variable is averaged to only 44 years between 1950–1994.

Therefore, this letter extends Woodberry's data by first using the Boix-Miller-Rosato (BMR) (Reference Boix, Miller and Rosato2013) and Polity IV (PIV) indices.Footnote ³ The broader historical scope of the new variables is relevant, as Woodberry does not present any theoretical predictions regarding the duration of the effect of Protestantism on democracy. Therefore, Woodberry's choice to limit the analysis to the 1950–94 period is rather arbitrary given that more recent data is available.

The BMR index is dichotomousFootnote ⁴ and defines a country as democratic if it satisfies a minimum set of conditions for democratic contestation and participation. The contestation requirement states that elections need to be free and fair, and the participation requirement requires that the majority of adult men are able to vote. Inspired by Dahl's (Reference Dahl1971) classification, it is the most comprehensive dichotomous measure of democracy available and unlike other indices, it takes a minimal suffrage requirement into account. We also employ the PIV index,Footnote ⁵ which reduces the sample size by 15 per cent (that is, 121 instead of 142 countries) but provides a comprehensive continuous assessment of democracy (ranging from −10 to +10) based on the presence of electoral institutions, constraints on the exercise of power as well as civil liberties. To further ensure robustness, we replicate the analysis using twenty-four other democracy measures, of which we predominantly report regressions using the Varieties of Democracy (V-Dem) and Freedom House data. From the V-Dem index we use the Electoral Democracy,Footnote ⁶ Liberal DemocracyFootnote ⁷ and SuffrageFootnote ⁸ scores since they most closely capture liberal democracy in the sense of the BP measure. From the Freedom House index we employ the Political RightsFootnote ⁹ and Civil LibertiesFootnote ¹⁰ measures, as well as a combined averaged score of both measures. The relationship between the original BP variable and the alternative measures of democracy is reported in Table A17 and illustrated graphically in Appendix Figure A1. As the table shows, seventeen of the twenty-six measures are correlated with a correlation coefficient larger than 0.80, and only three measures are correlated with a coefficient smaller than 0.70 with the BP measure.

Evidence from New Data

Results with Alternative Democracy Measures (1950–94)

We first replicate Woodberry's analysis using the same time period over which the democracy measure is averaged (1950–94) but instead employ the BMR and PIV datasets (all regressions in this letter follow Woodberry's replication code and use robust regression). Woodberry uses three proxies for CPs: (1) Protestant Missionaries per 10,000 Population in 1923, (2) Years Exposure to Protestant Missions and (3) Percent Evangelized by 1900.Footnote ¹¹ Catholic versions of the first two variables are included as well. Descriptive statistics are displayed in Table A1 and the complete list of countries can be found in Appendix Table A2.

Table 1 (using BMR, 1950–94) merges the results presented in Columns 3 and 4 of Woodberry's Table 2 and Columns 3–5 in Woodberry's Table 3. Appendix Table A3 conducts the same exercise but uses the PIV data.Footnote ¹² Table 1 shows that the relationship between CPs and democracy is not robust: the Protestant Missionaries in 1923 variable is only significant in specifications without controls (Columns 2 and 5), which raises concerns about omitted variable bias. Columns 3–5 control for a set of additional variables that relate to the historical influence of colonization processes, such as the date of ‘discovery’, literacy rates and the number of years between ‘discovery’, colonization and missionary contact. Years Exposure to Protestant Missions is insignificant. Percent Evangelized is significantly and robustly associated with democracy, yet, as we explain below and as Woodberry acknowledges, this variable is not a direct measure of Protestantism since it is a ‘retrospective estimate and includes both Europeans and Catholics—not only indigenous people exposed to Protestant missions’ (257, emphasis added).

Table 1.

Predicting democracy in non-Western societies (Columns 1–2) and controlling for the process of colonization (Columns 3–5), 1950–94, Boix-Miller-Rosato Reference Boix, Miller and Rosato2013

Note: standard errors reported under coefficients in parentheses (robust regression). The table replicates Table 2, Columns 3–4 (equivalent to Columns 1–2 in current table) and Table 3, Columns 3–5 (equivalent to Columns 3–5 in current table) from Woodberry (Reference Woodberry2012). *** p < 0.01, ** p < 0.05, * p < 0.1

Table 2.

Predicting democracy in non-Western societies (Columns 1–2) and controlling for the process of colonization (Columns 3–5), 1950–2015, Boix-Miller-Rosato Reference Boix, Miller and Rosato2013

Notes: standard errors reported under coefficients in parentheses (robust regression). See notes to Table 1. *** p < 0.01, ** p < 0.05, * p < 0.1

Table 3.

Instrumental variable regression: mean level of democracy from 1950–2015, Boix-Miller-Rosato Reference Boix, Miller and Rosato2013

Note: robust standard errors reported under coefficients in parentheses. Table 3 replicates Table 6 from Woodberry (Reference Woodberry2012). *** p < 0.01, ** p < 0.05, * p < 0.1

Robustness Checks

An important concern about Woodberry's results is that the inclusion of post-treatment covariates may lead to significant bias. To account for this concern, we ran models that excluded all variables for which at least some of the coverage occurs after the first year of the democracy average (1950) or the year of the Protestant Missionaries variable (1923), such as Percent Muslim in 1970, Major Oil Producer, Years of Protestant Missions until 1960, Years of Catholic Missions until 1960, various religious composition variables measured in 1970, log GDP per capita (1960–94), education (1960–85) and newspaper circulation (1975–95). There are no changes to the previous findings (results available upon request).

To make sure that our results are not driven by the idiosyncrasies of a particular democracy measure, Appendix Tables A10–A15 replicate Table 2 for six additional democracy measures: Freedom House – Average of Political Rights and Civil Liberties, as well as each index separately, and V-Dem – Electoral Democracy, Liberal Democracy and Suffrage. The results are consistent with the specifications with the BMR and Polity measures. While Percent Evangelized is positive and significant across specifications, Protestant Missionaries and Years Exposure to Protestant Missions are only significant in the specifications with no or few controls. In fact, in the V-Dem Suffrage specification, the effect of Protestant Missionaries is negative and significant. Once again, this raises the issue of omitted variable bias in the cross-sectional regressions, and means that Woodberry's results are unlikely to be causal.

In Table A16, we run regressions with a plethora of additional democracy variables, focusing on the period 1950–94 (as in Woodberry's original specifications), as well as from 1950 until the latest year of coverage of the democracy measure. The only measures for which we obtain consistent and significant positive results for the impact of Protestant Missionaries (across both periods) are the Vanhanen Contestation and Democratization scores. While the former measure is based on the level of the opposition's share of votes in parliamentary or presidential elections, the latter also takes into account the percentage of the population that actually voted. However, neither measure accounts for the state of liberal democracy based on civil liberties, political rights, or the existence of effective and fair legislative and executive bodies – a distinction that is critical for Woodberry's theory.

In Table A18, we present the results using Bayesian model averaging in order to account for uncertainty in the choice of explanatory variables. This approach distinguishes between focus regressors (which are always present in the model) and auxiliary regressors (about which the researcher is not certain). We thus specify Percent Evangelized as the focus regressor, and all other variables as the auxiliary regressors. We highlight independent variables with a posterior inclusion probability (pip)>0.5 or t-ratio>1 (in absolute value) to indicate that a regressor is robustly correlated with the dependent variable and thus should be included in the model.

The results show that only in Woodberry's (Reference Woodberry2012) original specification using the BP measure are all three Protestant variables robustly correlated with democracy. Using eleven alternative measures of democracy from the Freedom House, Boix-Miller-Rosato Reference Boix, Miller and Rosato2013, Polity IV and V-Dem datasets, we estimate fifty-five BMA models in total and up to 17 million models per specification. The variable Protestant Missionaries is significant only in specifications without controls in select Freedom House and V-Dem models. Years Exposure to Protestant Missions is correlated more robustly with democracy measures, but overall there is no consistent pattern using the BMA method. On average, control variables, such as dummies for colonial history or the state of being landlocked, are identified as predictors with the highest posterior inclusion probabilities. This is further proof of the fragility of Woodberry's argument that Protestantism, rather than colonial history or exogenous characteristics, contributes to the development of democracy.

In Table A19, instead of considering the entire period 1950–2016, we present the results by decade, for a total of twenty-six democracy measures. Across all these specifications and democracy measures, we find no consistent pattern linking Protestantism and democracy over time (measured as Protestant missionaries per 1,000). Indeed, for some measures signs and significance alternate from decade to decade.

Questioning the Uniqueness of Protestantism

Our extension of Woodberry's results showed that Percent Evangelized is significant and robust across specifications. This suggests that further research and better data are needed on the role of different strands of Christianity in democratic emergence and consolidation. The analysis in this letter questions the uniqueness of Protestantism, and raises the possibility that it is (Western) Christianity overall that matters. For instance, Djankov and Nikolova (Reference Djankov and Nikolova2018) show that Catholics and Protestants (unlike those of Eastern Orthodox faith) have similar attitudes and values, including perceptions of government and democracy. Building on the previous literature,Footnote ¹⁶ Caicedo (Reference Caicedo2019) shows that Protestant as well as Catholic missions had a positive effect on educational attainment and income levels by increasing literacy and teaching various crafts to indigenous populations. This evidence undermines Woodberry's case for the exceptionalism of CP. However, more granular data on different Christian missions is needed to examine this question rigorously.