Binary Measures

Researchers studying the politics of financial crises mostly rely on two sources of cross-country information about crises: Reinhart and RogoffFootnote ¹ and Laeven and Valencia.Footnote ² Reinhart and Rogoff classify countries as being in crisis if they experience at least one type of event: (1) one or more bank run, closure, merger or public sector takeover and/or (2) closure, merger, takeover or large-scale public assistance of an important financial institution that marks the start of a string of similar events.Footnote ³ Laeven and Valencia take a similar approach that emphasizes public interventions. A country-year is in crisis when there is both significant distress in the banking system and policy makers respond with significant interventions.Footnote ⁴

These binary crisis indicators come from detailed comparative work that identifies key features of crises. Yet they present a number of problems for studying political behavior. Crises are identified post hoc by researchers who know what happened after the fact. Financial market stress that policy makers successfully address, and therefore prevent a crisis, is not included. Similarly, stress that a government temporarily dampens through unsustainable policy measures, only to flare up later, is not recorded. The measures are dichotomous: they do not indicate crisis severity, or how it changes over time. Having an annual dichotomous measure also means that measurement errors – incorrectly timing the start or end of a crisis – can bias statistical model estimates.

There are large inconsistencies between the timing of crises in the Laeven and Valencia and Reinhart and Rogoff datasets.Footnote ⁵ Moreover, there are significant differences in crisis timing between different versions of the Laeven and Valencia data.Footnote ⁶ While the measures use precise definitions of when a crisis started, the reasons for dating the end of a crisis are either unstated, as in the case of Reinhart and Rogoff, or are ad hoc. Laeven and ValenciaFootnote ⁷ determine that a crisis has concluded when real GDP and real credit growth are positive for two years, or five years have elapsed from the crisis start year.

Despite these shortcomings, the indicators have been widely employed in the literature. Fielding and Rewilak and Herrera, Ordoñez, and Trebesch use Laeven and Valencia’s indicator as a dependent variable to understand how capital flow bonanzas affect the probability of crises occurring.Footnote ⁸ Danielsson, Valenzuela and Zer use Reinhart and Rogoff to understand how stock price volatility predicts crises.Footnote ⁹ Keefer and Rosas use earlier versions of the Laeven and Valencia dataset to identify periods of crisis and to argue that electoral competitiveness affected policy responses.Footnote ¹⁰ Reischmann employs the Laeven and Valencia measure to examine the electoral effects on ‘creative public budget accounting’ during crises.Footnote ¹¹ Ha and Kang use Laeven and Valencia to find that developing country governments respond to crises with fiscal and monetary tightening, though this is moderated by political constraints.Footnote ¹² Gandrud and Kleibl combine the two data sources to understand how financial regulatory structures are changed after crises.Footnote ¹³ Broz integrates the datasets to find a ‘partisan financial cycle’.Footnote ¹⁴ Crespo-Tenorio, Jensen and Rosas, and Chwieroth and Walter, as well as Pepinsky use the datasets in their research on the political effects of crises.Footnote ¹⁵

Continuous Measures

There are less frequently used approaches to measuring financial stress that aim to code extreme stress as well as continuous variation in severity over time. For example, Romer and Romer use a sixteen-point scale of the cost of credit intermediation to provide an indication of stress intensity. They code twenty-four countries using information from the Organisation for Economic Co-operation and Development’s (OECD’s) semi-annual Economic Outlook reports from 1967 to 2007. Relying on contemporaneous reports allows for the construction of a real-time measure of credit market distress and addresses potential problems with ex post coding.Footnote ¹⁶

While their approach is an improvement over previous work, it is limited. First, their data source confines them to a relatively small sample of OECD countries. Secondly, because their measure is laborious and time consuming to create and update, even if there were a more encompassing corpus, applying the method would be costly. Thirdly, human coders can introduce well-known problems of inter-coder reliability and unreproducibility.Footnote ¹⁷

Why the EIU?

The EIU compiles real-time, third-party qualitative assessments of financial market conditions within country-time contexts. Reports are released at regular monthly or, for a small subset of countries, quarterly intervals. These reports contain perceptions of economic conditions and represent an important channel of dissemination to public and private actors. They form a large corpus – more than 20,000 texts – of reports for more than 180 countries over the period 1997–2011. From 2003, they follow a consistent style and format, thus minimizing the extent to which text analysis picks up stylistic features, rather than information on financial market stress. The regularity and frequency of reporting for such a large group of countries in a consistent format distinguishes the EIU from other sources of financial information that we might analyse, chiefly newspaper reports. To take full advantage of format comparability we concentrate on data from 2003.

Summarizing Financial Market Stress in the EIU

To develop an index that classifies financial conditions on a continuous more-stressed/less-stressed spectrum for as many country-months as possible, we need an efficient way to summarize the vast quantity of information in the EIU reports. We first collected and pre-processed the EIU texts – focusing specifically on sections regarding banking and finance (more details are available in the Appendix). We then used KPCA to place the texts onto a financial market stress spectrum.

In political science, texts are frequently summarized using unordered ‘bags-of-words’ approaches that aim to find clusters of topics within texts or clusters of texts around topics.Footnote ²⁰ We would like to preserve word order in our texts. Many financial terms such as ‘credit growth’ and ‘borrowing costs’ have different interpretations depending on the adjectives that modify them; consider, for example, ‘slowing credit growth’ vs. ‘expanding credit growth’ or ‘falling borrowing costs’ vs. ‘increasing borrowing costs’. Likewise, adjectives can have very different implications for describing market conditions depending on the nouns that they modify. For example, ‘increasing’ can indicate worsening conditions – for example, ‘increasing non-performing loans’ – or improving conditions – for example, ‘increasing lending’. A bags-of-words approach treating each word as having meaning as an individual unit, rather than in ordered associations with other words, would not adequately capture commonly used and radically different meanings.

KPCA – developed by Scholkopf, Smola, and Muller and Lodhi et al. – allows us to address these issues. This approach can extract structure from our likely high-dimensional EIU corpus while preserving word order.Footnote ²¹ Note that we also conducted the analysis using bag-of-words PCA. While less computationally intensive to estimate, it resulted in a less valid measure (see the Appendix).

Our unit of analysis is a sub-string kernel: a short sequence of lettersFootnote ²² that can be shared within and across words. This allows us to distinguish between two simple documents with the stemmed strings ‘slow credit’ and ‘expand credit’. They share the five character kernels ‘credi’ and ‘redit’, but differ on ‘low c’ and ‘and c’, among others. We summarize the similarity of these documents with the frequency distribution of five-length strings that they have in common, standardized by document length. We find these pairs for all of the documents in our corpus to create a kernel matrix. We then scale the documents using principal component analysis,Footnote ²³ using the first component as our indicator (see the Appendix), and made two final transformations. First, we rescaled it so that it would be between 0 and 1Footnote ²⁴ in order to ease interpretation. Secondly, we smoothed the results by taking a two-period – usually two-month – moving average.

Qualitative Examination

Table 1 shows a progression of texts in the lead-up to and height of the 2008 US financial crisis. The first text is from the observed United States minimum FinStress month, April 2006. The text discusses strong business investment. A year later, the FinStress score increased significantly to 0.41. Likewise, the language changed. The EIU notes a ‘slump in the housing market’ and that liquidity for lower-quality mortgages is ‘drying up’. The following year, shortly after the failure of Bear Sterns, the FinStress score increased to 0.5. The text notes that the bailout of Bear Sterns has ‘increasingly swung’ market participants to think that the worst is over. FinStress is clearly higher than before, but far from its peak, reflecting the cautiously positive assessment in the EIU report. FinStress rose to 0.62 in September 2008 (a report that was released before Lehman Brothers’ failure). This report notes increasing lending risks as well as threats to the ‘stability of weak financial institutions’. In November 2008 FinStress rose close to the US maximum, with the EIU noting that there is a ‘credit crunch’. FinStress closely tracks EIU reports describing increasingly worrying financial market conditions.

Table 1

Portions of Selected EIU Reports for the United States (2006–2008)

Comparison with other Indices

We now compare FinStress to the dichotomous measures of banking crisis developed by Reinhart and RogoffFootnote ²⁶ and Laeven and Valencia,Footnote ²⁷ as well as Romer and Romer’sFootnote ²⁸ continuous measure.Footnote ²⁹ We expect that our measure is capturing many of the same events, but with more nuance in magnitude and over time.

Figure 1 compares FinStress to three other measures of financial stress. In many cases – conditional on the coverage of each data seriesFootnote ³⁰ – the indices substantively overlap. Comparisons with Romer and RomerFootnote ³¹ are limited by data availability, but clearly there are cases in which FinStress and their index are similar. For example, both indices increase in the United States from early 2007. A notable difference is how Romer and Romer classify Japan as being without stress from mid-2005, while FinStress decreases, but to a high level compared to many other economically developed countries at that time. Both indices classify Iceland as being under stress in the late 2000s, but the timing is different. Romer and Romer classify Iceland as being in stressFootnote ³² in 2006–2007, which is earlier than a marked increase in the FinStress Index, as well as Reinhart and Rogoff and Laeven and Valencia’s timing.

High FinStress levels should appear where there is crisis in the dichotomous codes, but the reverse may not be true: the dichotomous indices may miss a building crisis as perceived at the time. Laeven and Valencia and Reinhart and Rogoff should have more Type II errors – there are signs of stress but they miss them. Laeven and Valencia’s coding is stricter because countries both have to experience distress and have a particular policy response, so the associated FinStress levels should be higher.

We do find that the mean FinStress level during periods that Laeven and Valencia classify as crises – that is, financial market stress reached the point where politicians responded using a pre-specified set of policies – is higher than during non-crisis periods.Footnote ³³ Though developing countries tend to have higher FinStress scores, countries defined by Laeven and Valencia as in crisis in both groups have FinStress scores around 0.59 on average. Those not in Laeven and Valencia crises have lower scores.Footnote ³⁴ The fact that mean FinStress is higher during periods that Laeven and Valencia classify as crises, but not dramatically so, supports the proposition that they miss considerable periods of budding and high stress, and perhaps suffer from more Type II errors.

Laeven and ValenciaFootnote ³⁵ comment that part of the problem with dating financial crises is that each one develops differently: ‘Some crises evolve gradually, gaining speed as the ripple effects from a seemingly small shock propagate forward in time ... other episodes happen more abruptly and are often the result of sudden stops.’

FinStress’ real-time continuous measurement and monthly periodicity allow us to distinguish these types. We can see in Figure 1 that financial market difficulties in the United States built over a long period, with a few spikes during notable banking difficulties, such as Bear Stearns’ and Lehman Brothers’ collapse. Conversely, countries such as Germany, Hungary and Iceland have more sudden periods of perceived financial distress. The Greek case presents an interesting trajectory. There is a notable spike in Greek financial stress in 2008 – when both Reinhart and Rogoff Reference Reinhart and Rogoff2009 and Laeven and Valencia Reference Laeven and Valencia2013 determine a crisis had started. This is then followed by a stable period, and then a sudden uptick in 2010, when a possible government default severely threatened the banking system.

Kazakhstan is notable for another path: rapid stress onset and just as rapid dissipation. In late 2009 there was a prominent spike in perceptions of stress directly related to a credit crunch resulting from the Global Financial Crisis. According to an International Monetary Fund assessment, a large and quick policy response – facilitated by the country’s sizable National Oil Fund – returned stability to the country’s banking system within a few months.Footnote ³⁶ Kazakhstan’s FinStress score returns almost to its previous trend level at that time. Annual binary measures of crises do not capture these changes.

We can use FinStress to identify periods in which financial market conditions were perceived to be worsening, though for whatever reason these perceptions changed before other measures would record a financial crisis. Australia in late-2008/2009 is a notable example. It had a clear spike in perceptions of stress shortly after Lehman Brothers’ collapse. Fairly quickly thereafter, its FinStress score returned to its previous level. Laeven and Valencia and Reinhart and Rogoff do not distinguish these periods.

FinStress’ advantages are also apparent for timing the end of heightened distress. This is a difficult issue for the binary indicators. Crisis onset is well defined by these measures, but they rarely have a clear or non-ad hoc way of determining when a crisis ended. Though we are limited by the time period coverage of the EIU texts, it is clear that some countries, notably Switzerland, the United Kingdom and the United States, were perceived to have had improved financial market conditions from about 2010. Other countries, particularly eurozone countries, plateaued at a high level through the end of 2011, a period that corresponds with the eurozone crisis. Laeven and Valencia’s measure uses an ad hoc definition of crisis termination and so classifies these entire periods as crises with equal intensity.

Exploring the Functional Form of the Relationship Between Crisis and Electoral Volatility

Following the Global Financial Crisis there has been considerable interest in how financial stress may hasten government failure,Footnote ³⁷ how crises increase electoral support for extremist parties,Footnote ³⁸ and how negative economic events and crises increase electoral volatility – that is, the degree to which voters switch support for parties from one election to another.Footnote ³⁹ Crespo-Tenorio, Jensen, and Rosas, and Funke, Schularick, and Trebesch both rely on Laeven and Valencia’s binary measure,Footnote ⁴⁰ while Hernández and Kriesi use an even simpler measurement of crisis: whether or not an election occurred after November 2008.Footnote ⁴¹

Figure 2 presents hypothetical scenarios to illustrate possible measurement error when estimating the effect of financial market stress on elections. The top-left panel illustrates time-order measurement error. Using annual indicators of crisis may ascribe a level of stress to an election that was not perceived at the time. This type of error could lead to systematic underestimation of the effect of financial market stress.

Fig. 2

Hypothetical scenarios of error in the measurement of financial market stress at elections using an annual binary measure vs. FinStress Note: Shaded areas indicate years classified as a crisis by a hypothetical binary crisis measure. Dashed vertical lines indicate an election.

The upper-right panel of Figure 2 illustrates incorrectly measuring the intensity of financial market stress for two elections held during a period of heightened stress. Using a binary crisis measure codes the elections as occurring at equivalent stress levels. This error could lead to underestimating the effect of a crisis on election outcomes. Additionally, a binary measure would not allow us to examine whether there was a non-linear effect of stress; it may be the case, for example, that very high stress has an exponentially stronger effect on outcomes than medium levels.

Finally, the lower-left panel of Figure 2 provides a scenario in which neither a binary crisis measure nor FinStress levels would capture potentially important features of how voters perceived stress at elections x _c and y _c . Even though the binary and FinStress levels are the same, in the former stress is worsening, while in the latter conditions are improving. Voters might be more hostile to incumbents as conditions worsen and more favorable when conditions improve. The binary measure would not allow us to examine this possibility.

These are not just hypothetical examples. Figure 3 compares national election dates in nineteen European countries to FinStress scores and Laeven and Valencia crisis periods. In some elections, notably elevated FinStress levels and Laeven and ValenciaFootnote ⁴² crises overlap: Belgium (June 2010), Germany (October 2009) and Iceland (May 2009). There are also notable examples of the binary crisis indicator coding an election as occurring in a crisis well before voters would likely have perceived one in their country: Austria (October 2008), Portugal (October 2009) and Spain (March 2008). It would be inaccurate to estimate the effect – in these cases on voters’ actions – of a crisis that had not yet occurred. There are also instances in which financial market stress had declined or was notably declining by the time of the election, but the binary crisis measure still coded a full-intensity crisis: Sweden (October 2010) and the Netherlands (June 2010).

Fig. 3

Comparing national election timing, FinStress, and Laeven and Valencia’s (Reference Laeven and Valencia2013) Banking Crisis Measure in Europe Note: Dashed vertical lines indicate national government elections. Lightly shaded areas represent Laeven and Valencia (Reference Laeven and Valencia2013) banking crisis periods.

In light of these measurement issues, and assuming that electoral volatility will be higher under more stressful economic conditions, we anticipate that the binary crisis measure will underestimate the effect of financial market stress on volatility compared to FinStress.

To examine this we use data on electoral volatility in Western European countries from Emanuele.Footnote ⁴³ He codes the total electoral volatility of an election by summing vote switching between existing parties, new parties and failed parties. Because we want to compare the estimated effects of the two financial market stress measures, we separately ran simple normal linear models of volatility including the stress measures on the right-hand side.Footnote ⁴⁴

Figure 4 shows predicted electoral volatility – which ranges from 2.5 to 29.6 in the sample – under various fitted levels and measures of stress. The first row of plots shows results from simple linear specifications. As expected, while both the binary and continuous measures are estimated to have a significant (at the 10 percent level) positive effect on volatility, the predicted effect of FinStress is somewhat stronger. The bottom-left panel of Figure 4 shows predicted electoral volatility from a model estimated by adding a quadratic term. Above FinStress scores of about 0.6 we estimate a rapidly increasing effect of FinStress on volatility. At the highest levels, volatility is predicted to be much higher than anticipated by the binary measure of crisis and linear FinStress. The average predicted volatility at the highest FinStress levels under the quadratic specification is 20.9, while it is 15.3 in the linear FinStress specification and 14.4 for the binary measure.

Fig. 4

Predicted electoral volatility for representative levels of FinStress and Laeven and Valencia (Reference Laeven and Valencia2013) Banking Crisis Note: shaded regions/vertical lines represent 95 per cent confidence intervals.

The middle-left panel of Figure 4 shows the predicted volatility for the range of observed changes to FinStress from the previous twelve-month moving average. Large increases in stress compared to the average level over the previous twelve months are associated – at the 10 per cent significance level – with increased electoral volatility. Stated conversely, improving financial market conditions are predicted to lead to less vote switching. The closest analogue to examining the effect of stress changes on volatility for the binary measure is to create a binary crisis-start year variable.Footnote ⁴⁵ In contrast to the estimates using the FinStress change variable, we did not find a statistically significant effect at any level of Laeven and Valencia crisis onset on electoral volatility (Figure 4, middle-right panel).

Comparing Model Fit

We further compare the usefulness of FinStress to the Laeven and Valencia binary crisis measure by comparing the adjusted R ² from models with the two indices. A higher adjusted R ² would indicate that a variable explains more of the variance in the dependent variable.

We use regressions with electoral volatility as the dependent variable, as well as a substantively very different variable: cumulative revisions to EU member state government debt statistics made by the European Union’s statistical agency, Eurostat. Gandrud and HallerbergFootnote ⁴⁶ used this variable to examine the political and economic conditions associated with Eurostat – an independent statistical agency – revising government debt statistics. They found that governments facing unscheduled elections during a financial crisis were more likely to have their debt statistics revised upwards. The authors argue that this is likely because incumbents under-report debts during stressful times to make themselves more attractive to voters.

We ran identical linear regression models with the dependent variables, changing only whether we included FinStress, the FinStress polynomial discussed earlier or the binary crisis measure. The results are shown in the Appendix. For electoral volatility, the model with polynomial FinStress has an adjusted R ² about 1.8 times higher than that with the binary measure (0.193 vs. 0.109). For debt revisions the difference is more modest: the FinStress model has an adjusted R ² 1.08 times that of the binary model (0.405 vs. 0.375). When using debt revisions as the dependent variable, monthly FinStress was collapsed into its annual average to put it on the same time scale as the annual dependent variable. It is not surprising that the relative model fit is better with monthly FinStress than when it is collapsed into an annual average, and thus is more similar to the annual binary measure. The former operationalization leverages FinStress’ continuous measurement level and high frequency, whereas the latter reduces this information.