We wish to begin by thanking Crabtree and GolderFootnote ¹ for the time and effort they have spent replicating the results in Powell and TuckerFootnote ² and providing further evidence in support of the primary substantive conclusion of that article. We also want to thank the British Journal of Political Science for offering us the opportunity to revisit the topic of electoral volatility in post-communist countries.

The primary goal of P&T (2014) was to rigorously conceptualize a new approach to thinking about electoral volatility – by disaggregating electoral volatility into volatility between parties that were present across both elections in a pair of consecutive elections (‘Type B’ volatility) and volatility due to new party entry and party exit (‘Type A’ volatility), an approach that is especially important in the context of post-communist countries – and to provide a comprehensive dataset for two decades of post-communist elections that incorporates these new measures. To be clear, P&T (2014) was not the only piece arguing for the importance of disaggregating measures of electoral volatility,Footnote ³ but the article makes a contribution by systematically laying out a set of rules for exactly how to code these two different types of volatility (itself a complex task), making a case for why volatility should be coded in this particular manner and providing a substantially expanded set of measures relative to previous work.

Concurrently, in the course of introducing these new measures and data, P&T (2014) replicates and extends the existing literature on volatility in post-communist countries by adding additional countries and years of data beyond what had been included in previous analyses. To do so, P&T (2014) relies on a very specific algorithm to determine a priori which elections to include in the analysis.Footnote ⁴

These analyses produced an interesting empirical finding: once these additional countries and years were added to the analysis, almost all of the previous results disappeared,Footnote ⁵ leaving practically no variables that were correlated with either Type A or Type B volatility. Moreover, the data clearly demonstrated that most volatility in post-communist countries was due to Type A volatility. These were the two most important substantive contributions of P&T (2014), and are listed as such in the conclusion of the article on pp. 142–3.

P&T (2014) did, however, note one exception to this general conclusion that there were no variables correlated with either Type A or Type B volatility, which was that GDP change since 1989 (hereafter GDP-89) was negatively correlated with Type A volatility (see Table 4, p. 139 of P&T 2014). C&G (2016), while replicating every other finding reported in P&T (2014), present two new findings that cast some doubt on whether GDP change since 1989 is indeed negatively correlated with Type A volatility. First, by adding data on GDP in Bosnia-Herzegovina from 1989–93 to the analysis that was unavailable at the time P&T (2014) was submitted to the British Journal of Political Science for review, C&G (2016) find that the coefficient for GDP-89 increased in magnitude (from −4.6 to −6.0), but so too did the standard error of this estimate (from 1.3 to 6.6), thus decreasing our statistical confidence in this relationship below conventional levels (see C&G 2016, Table 1).Footnote ⁶ Secondly, deleting Bosnia-Herzegovina from the analysis altogether also results in a similar increase in the size of the coefficient to −6.1, with the standard error also increasing by a roughly similar amount to 7.2 (see C&G 2016, Table 1).

Table 1

Country Dropping Robustness: Type A Volatility

Note: robust standard errors in parentheses. The full regression model is from the original Powell and Tucker (Reference Powell and Tucker2014) article. For space reasons, other coefficients are omitted from the table. Full results are available on authors’ webpages. Standard errors clustered by country. ***p<0.01, **p<0.05, *p<0.1.

While these new results clearly strengthen the primary substantive conclusion of P&T (2014) – that little is known about the correlates of either Type A or Type B volatility in the first two decades of post-communist elections – caution is still in order in claiming that the book is definitely closed on the possible link between growth since the start of the post-communist period and Type A electoral volatility. First, it remains the case that both using the new Bosnia-Herzegovina data and list-wise deleting the Bosnia-Herzegovina cases from the analysis increases the size of the coefficient on GDP-89. Many methodologists more sophisticated than we have long argued that too much attention is paid to standard significance levels at the expense of substantive effects, and this may serve as an example. Certainly, our confidence that there is no relationship between GDP growth since 1989 and electoral volatility would have been enhanced had the coefficient in the GDP-89 variable moved closer to 0 in the new analyses as opposed to heading in the opposite direction. Secondly, using the original data from P&T (2014), the GDP-89 results are robust to dropping every other country (in sequence) from the analysis besides Bosnia-Herzegovina (see Table 1).Footnote ⁷

Of course, this does nothing to change the finding that dropping Bosnia – or using the new version of the Bosnia-Herzegovina data – increases the size of the standard error on the coefficient for GDP-89, but it does suggest that the relationship between GDP-89 and Type A electoral volatility was not so weak that dropping countries at random could produce similar effects. Furthermore, dropping each country in turn – including Bosnia-Herzegovina – results in a fairly consistent set of coefficients for GDP-89, ranging from −4.1 (dropping Hungary) to −6.1 (dropping Bosnia-Herzegovina).Footnote ⁸ Taken together, it appears that there is even more evidence in support of P&T (2014)’s primary conclusion that little is known about the correlates of electoral volatility in post-communist countries through the end of 2009, but that change in GDP since the start of the transition is probably worth continuing to analyze as a potential correlate of Type A volatility as new data become available.

Before closing, we wish to offer a dissenting view of the characterization in C&G (2016) of the primary contribution of P&T (2014). As noted previously, the primary contributions of P&T (2014) were to introduce a new framework for analyzing electoral volatility, to provide a detailed set of coding rules to operationalize this framework and to provide data for these new measures of volatility over two decades of post-communist election results. Substantively, the primary finding of P&T (2014) is that there are almost no correlates of electoral volatility in post-communist countries in the years 1989–2009 among a standard set of explanatory variables – in contrast to limited sub-sections of the post-communist cases included in previous studies (see Tables 3–5) or Western European countries (see Table 6) – and that most of the volatility in post-communist countries is of the Type A, as opposed to Type B, variety. Nevertheless, C&G (2016) present their contribution as a ‘challenge [to P&T 2014’s] central claim that replacement volatility in post-communist Europe is driven by long-term economic performance’ (emphasis added).

As further justification of the importance of making this challenge, C&G (2016) note that P&T (2014) has been ‘widely cited’ with ninety-five Google Scholar citations as of April 2015.Footnote ⁹ To better understand how readers saw the central contribution of P&T, we tracked down as many of these citations as we could find and identified the reason why they cited P&T.Footnote ¹⁰ Of these ninety-five citations, twelve were essentially different versions of other articles already in the collection (which is an interesting finding in itself concerning the use of Google Scholar as a metric for academic influence, but beyond the scope of our discussion here), leaving eighty-three unique papers/articles/books that had cited P&T (2014) as of April 2015. We were able to locate seventy-seven of the eighty-three. Of these, only six (8 per cent) referenced the finding that economic conditions impacted electoral volatility. In contrast, forty-six (60 per cent) cited the use of Type A and Type B volatility, while twenty-eight (36 per cent) referenced the data regarding overall levels of volatility in post-communist countries.

In net, an exhaustive re-examination of the data in P&T (2014) by C&G (2016) upholds the major substantive findings of the former – that we know little about the determinant of electoral volatility in post-communist countries – which should add to the authority of these findings moving forward. We continue to believe, though, that the major contribution of P&T (2014) going forward will be the framework presented for thinking about electoral volatility – especially in new democracies – in terms of Type A and Type B volatility, as well as the finding that so much of the first two decades of post-communist electoral volatility was of the Type A variety. Nevertheless, we are hopeful that the findings from P&T (2014) and C&G (2016) will both spur further work on the important topic of the determinants of electoral volatility, especially in new democracies and transitional societies.