Genealogy of the comparative method

The development of QCA followed from several methodological challenges that Charles Ragin confronted in the 1970s and 1980s. He was trained as a quantitative sociologist, but became increasingly frustrated with the limitations of this approach.

Early in his postgraduate studies, he read Barrington Moore's (1966) Social Origins of Dictatorship and Democracy. This book proved to be highly influential in the development of QCA in the sense that it provided an excellent in-depth comparative analysis of a limited number of cases, focusing on the combinations of antecedent conditions linked to specific, large-scale historical transformations. While breathtaking in scope and ambition, the analytical foundation of the work centred on a series of pair-wise case comparisons, without an overarching, systematic treatment of cross-case patterns. The key problem for Ragin, who had been taught to frame problems in terms of the net effect of variables, is that the argumentation of the work resisted the language of variables and net effects. Hence, here was a highly influential social science book, which could not be squared with the existing quantitative discourse.

The challenge Ragin faced was to formalize a technique that would enable researchers to systematically integrate within-case and cross-case analysis. An important concern was to remain true to the nature of the qualitative argumentation with its key focus on the question of ‘how things happen’. Answering the ‘how question’ should remain the starting point of any technique that tries to formalize comparative case analysis. In his dissertation (Ragin, Reference Ragin1975), he used both qualitative and quantitative evidence to assess the origins and social bases of Welsh and Scottish nationalism within the context of political regionalism in Great Britain. The historical analysis provided a contextualization of the quantitative analysis and the quantitative analysis provided further questions for the historical analysis. In a way, it was a mixed methods approach avant la lettre. Still, the frustration was that it proved difficult to integrate the different analyses fully because they addressed different questions. While offering, in the end, a more complete picture, the two analytical modes stood at a distance from each other.

A second work that influenced the drive to combine qualitative and quantitative approaches was a collaborative research project on the Romanian peasant revolt of 1907 with one of Ragin's advisers, Daniel Chirot (Chirot and Ragin, Reference Chirot and Ragin1975). One concern in this project was to investigate the conjunction of conditions linked to the outbreak of peasant revolts, using quantitative evidence. Building on the macro-sociological tradition in research on revolutions, the paper identified two models that were tested using historical data on the Romanian peasant rebellion. One model built on the work by Eric Hobsbawm (Reference Hobsbawm1959), Charles Tilly (Reference Tilly1967), and Barrington Moore (Reference Moore1966) and stressed the conflict between the survival of peasant traditionalism and the intrusion of capitalist market forces. In the paper Ragin, building on the work of John Stuart Mill (1843), developed the idea of chemical causation (see also Ragin, Reference Ragin1987: 25), arguing that causal conditions must often combine in order to generate qualitative change. Ragin operationalized this argument via the statistical analysis of interaction effects. The paper showed that the model does explain the Romanian revolt and that the important variable explaining the intensity of the rebellion is the interactive effect of peasant traditionalism and the penetration of market forces in agriculture.

Identifying interaction effects as a key explanatory strategy started a 5-year journey looking systematically into interaction models (for an extensive discussion of interaction terms in empirical research see Delacroix and Ragin, Reference Delacroix and Ragin1978). This increased attention to interaction effects within the context of statistical modelling also resulted in increased frustration for a variety of reasons. The results of a test for interaction are shaped in part simply by the metrics of the component variables and their degree of correlation. Substantial co-linearity between an interaction term and its components is a recurring issue. When higher-order interaction terms are considered, co-linearity is exacerbated considerably, and researchers often have their choice of which interaction effects to report. Ragin concluded that working with interaction effects, especially three-way and higher-order terms, seemed to be an extremely fragile enterprise. However, this was one of the areas Ragin wanted to explore further in order to assess the conjecture that there could be mixtures of four, five, or six conditions generating a qualitative change. This way of viewing social phenomena was much more akin to the rich historical analyses of macro-sociological phenomena. Ragin worked for years on interaction models in order to make them work for the purpose of analysing high-order interaction effects and to make them more robust. In the end, he concluded that this was not a fruitful path and that alternative techniques had to be developed.

A third challenge was related to the way quantitative scholars dealt with research populations and the assumption that researchers should use ‘given’ populations. Ragin and other researchers (see, for example, also Pzewroski and Teune, Reference Przeworski and Teune1970) challenged this assumption and reflected on how best to construct populations for research purposes [see also the discussions in Ragin and Becker (Reference Ragin and Becker1992) on casing]. Again, a confrontation with how qualitative researchers select their cases revealed a very different approach to population construction and definition. In qualitative work, cases are selected to serve specific theoretical purposes and not assumed to be exogenously given.

These issues and related issues stimulated a search for a new approach. The aim was, in essence, to formalize some of the practices that are common (and often implicit) in qualitative research. In order to formalize how different conditions combine to generate a qualitative change, Ragin turned to books he had read as an adolescent on Boolean algebra, set theory, and switching circuits. At about the same time, he moved from Indiana University to Northwestern University, which at that time was more open to qualitative-oriented studies and methodological innovation. In this environment, it was possible for him to experiment with new analytical approaches, and he presented his first lectures on his methodological ideas at seminars in Northwestern.

The end product of this exploration of alternative analytical foundations culminated in the development of QCA. A first application, jointly with Susan Mayer and Kriss Drass, appeared in the American Sociological Review in 1984 (Ragin et al., Reference Ragin, Mayer and Drass1984). This paper focused on employment discrimination and addressed the appropriateness of the statistical techniques used to assess discrimination, especially in legal disputes. The paper compared and assessed the distinctive strengths of QCA vis-à-vis logistic regression for assessing discrimination, with a special focus on the legal principle that the groups compared in such cases should be ‘similarly situated.’ Ragin argued that this legal principle demanded a form of comparison that attends to cases as configurations of characteristics. In 1987, The Comparative Method was published, which elaborated further the configurational principles first presented in the 1984 paper. The Comparative Method was not intended as an end product of a process of methodological innovation, but more as a first step in a ‘work in progress’, which could be communicated to a wider audience.

1987 – The Comparative Method

One of the key aims of The Comparative Method was to develop a research approach, including analytical tools, which would make it possible for researchers to combine case-oriented approaches with variable-oriented approaches. The goal was to ‘integrate the best features of the case-oriented approach with the best features of the variable-oriented approach’ (Ragin, Reference Ragin1987: 84). The research approach and technique, as presented in the 1987 book, had five main components.

First, it emphasized the case-based nature of comparative research, an emphasis that demands that each case be considered a complex entity (a whole) and that its integrity as a case should be maintained in the course of the analysis. Different parts of each case should be understood in relation to one another and in terms of the whole that they form together. The organizing idea was that the parts of a case constitute a coherent whole and that the effects of variables should be assessed in the context of the case and not detached from it. In order to operationalize this idea, cases are represented as configurations of variables. The essence of the analytical approach was to link configurations of causally relevant conditions to outcomes.

Second, the approach was comparative in the sense that it enabled researchers to explore similarities and differences across comparable cases by comparing configurations and pooling similar cases together. The analytical device that allowed this was the truth table, which displays the data in a matrix of all logically possible configurations of causal conditions. By bringing together cases in a truth table, a researcher could assess which cases display identical configurations of causal conditions and which cases differ on one or more conditions.

Third, an iterative way of developing an explanatory model was proposed so as to facilitate a dialogue between theory and evidence. The key mechanism for developing an explanatory model in QCA is the presence of contradictions (Ragin, Reference Ragin1987: 113–118; Rihoux and De Meur, Reference Rihoux and De Meur2009: 48–50). Contradictions occur in QCA when an identical configuration of conditions is linked to both the presence and absence of an outcome. Contradictions are revealed through the transformation of a data matrix into a truth table. If a truth table reveals contradictions, they should be resolved, primarily by identifying omitted causal conditions (Ragin, Reference Ragin1987: 113; see also Rihoux and De Meur, Reference Rihoux and De Meur2009: 48–49 for complementary strategies). Hence, the development of an explanatory model in QCA goes hand in hand with resolving contradictions. This back and forth process of including and excluding theoretically and empirically relevant conditions in a model until a model has been identified with no or only a few contradictions is the key mechanism for developing an explanatory model for analytical purposes. As Ragin (Reference Ragin2005: 34) argues, ‘the resolution of contradictions […] deepens knowledge and understanding of cases and also may expand and elaborate theory’.Footnote ²

Fourth, as a result, QCA allows for the assessment of multiple conjunctural causation, which implies that: (1) most often, it is a combination of conditions that produces a phenomenon – the outcome; (2) several different combinations of conditions may produce the same outcome; and (3) a given condition may have a different impact on the outcome depending on the context (i.e. depending on the causal ‘conjuncture’; see also Berg-Schlosser et al., Reference Berg-Schlosser, De Meur, Rihoux and Ragin2009: 8–10). This feature of QCA is premised on the idea that different causal paths may lead to the same outcome. Hence, QCA developed a context-specific notion of causality. This allowance for greater causal complexity also implied that a causal condition could have opposite effects depending on the context in which it operated. As a result, by using QCA, the researcher is urged not to ‘specify a single causal model that fits the data best (as one usually does with statistical techniques), but instead to determine the number and character of the different causal models that exist among comparable cases’ (Ragin, Reference Ragin1987: 167). In order to achieve this aim, The Comparative Method introduced the analysis of necessary and sufficient conditions via QCA. The latter allowed researchers to identify necessary (i.e. a condition must be present for a certain outcome to occur) and sufficient (i.e. a condition can by itself produce a certain outcome) conditions (Ragin, Reference Ragin1987: 99–101).

Fifth, QCA gave researchers the possibility to determine the degree to which they wanted to reduce the empirical complexity of the cases they were analysing, to achieve greater parsimony. The analytical procedure that facilitated the reduction of complexity was Boolean logic. Boolean algebra allowed one to identify causal regularities that are parsimonious, that is, which combine the fewest possible conditions within a set of conditions that are considered in an analysis. The key procedure was Boolean minimization – that is, reducing a full case description to the shortest possible expression (the minimal formula) that displays the causal regularities in the data (Ragin, Reference Ragin1987: 93).

In sum, The Comparative Method introduced a new formal logic to compare cases, explore causal diversity, and reduce the abundance of case information into more parsimonious explanations. The first version of QCA was developed for the use of dichotomous (crisp-set) variables and introduced software tools (QCA under DOS, developed by Ragin and Drass) to analyse the data.

First applications in social sciences

After the publication of The Comparative Method, several scholars adopted the approach, especially in specific subfields of political sociology such as industrial democracy, welfare states, revolutions, social movements, and trade unions. Table 1 shows the number of applications by year for the first decade after the publication of The Comparative Method.

Table 1

Early adoption: QCA applications per year (1984–1997)Footnote ^a

^a Based on the COMPASSS database (see footnote 1).

^b Others include working papers and reports.

The table shows that early adoption was piecemeal. Only a relatively small number of applications appeared in the first 10 years in peer-reviewed journals (39 in total). However, many of these papers appeared in leading academic journals such as the American Journal of Sociology (Amenta et al., Reference Amenta, Carruthers and Zylan1992; Brown and Boswell, Reference Brown and Boswell1995), American Sociological Review (Hicks et al., Reference Hicks, Misra and Nah NG1995; Cress and Snow, Reference Cress and Snow1996) Sociological Methods and Research (Amenta and Poulsen, Reference Amenta, Dunleavy and Bernstein1994; Coverdill et al., Reference Coverdill, Finlay and Martin1994; Hicks, Reference Hicks1994), Social Forces (Amenta and Poulsen, Reference Amenta and Poulsen1996; Hollingsworth et al., Reference Hollingsworth, Hanneman, Hage and Ragin1996), Economic and Industrial Democracy (Abell, Reference Abell1990), International Journal of Comparative Sociology (Griffin, et al., Reference Griffin, Botsko, Wahl and Isaac1991; Wickham-Crowley, Reference Wickham-Crowley1991), Third World Quarterly (Berntzen, Reference Berntzen1993; Foran, Reference Foran1997), Law and Policy (Weinberg and Gould, Reference Weinberg and Gould1993; Gregware, Reference Gregware1994) Comparative Politics (Berg-Schlosser, Reference Berg-Schlosser and De Meur1994), International Studies Quarterly (Kiser et al., Reference Kiser, Drass and Brustein1995), Studies in Comparative International Development (Blake, Reference Blake1996), Journal of European Social Policy (Peillon, Reference Peillon1996), Work and Occupations (Brueggemann and Boswell, Reference Brueggemann and Boswell1998), Policy Studies Journal (Kiser and Baker, Reference Kiser and Baker1994), Sociological Quarterly (Biggert, Reference Biggert1997), and Historical Methods (Griffin et al., Reference Griffin, Caplinger, Lively, Malcom, McDaniel and Nelsen1997).

This overview shows that while early adopters were small in number, they were significant in impact and exposure because several of the papers were published in top-ranked sociological journals. Some authors such as Edwin Amenta and Dirk Berg-Schlosser were influential in this respect. As a result, the uptake of QCA-based papers led to the diffusion of the approach and a discussion of its strengths and weaknesses, with several publications focused explicitly on the merits of the approach.

The comparative method in methodological debates

Following the publication of The Comparative Method and the first applications, a lively intellectual debate developed around the possibilities and limitations of QCA (Lieberson, Reference Lieberson1991, Reference Lieberson1994; Bollen et al., Reference Bollen, Entwisle and Alderson1993; Savolainen, Reference Savolainen1994; Goldthorpe, Reference Goldthorpe1997; Scharpf, Reference Scharpf1997; for an extensive overview and discussion of the debates see Rihoux, Reference Rihoux2003; De Meur et al., Reference De Meur, Rihoux and Yamasaki2009). Early debates focused on five issues.

A first debate revolved around case sensitivity. Opponents argued that QCA was too sensitive to individual cases, since the inclusion or exclusion of a single case can modify the results of an analysis (Goldthorpe, Reference Goldthorpe1997). Proponents on the other hand argued that, in QCA, each case did indeed matter. By adding one new case, one might, in fact, discover another explanatory (or causal) path, which was considered to be one of QCA's unique strengths. The additional causal path might not have much explanatory power (or ‘coverage’, in more recent versions – see below) but it might be theoretically significant (Marx et al., forthcoming).

A second debate focused on the use of dichotomous variables, which was considered a crude measurement for many social scientific concepts (Goldthorpe, Reference Goldthorpe1997). Although generally acknowledging that dichotomous variables are limiting, it was also emphasized that the main advantage of working with crisp sets was the elegance of simplicity. In addition, in the debates on concept operationalization, especially in comparative politics and macro-political sociology, there was the view that one should not always pursue gradualism at all cost (Sartori, Reference Sartori1970, Reference Sartori1984); however, depending on the aims of the research, the researcher could work with dichotomous variables and achieve sufficient explanatory insight (Collier and Adcock, Reference Collier and Adcock1999). In addition, many statistically inclined researchers use dichotomous variables in their analysis and several techniques have been developed to pursue the analysis of dichotomous variables.

A third debate focused on the limitations of the number of conditions QCA can take into account, which in turn has a potential impact on the omitted-variable bias, that is, the fact that the applied explanatory model overlooks an important explanatory condition (King et al., Reference King, Keohane and Verba1994: 168ff.). According to some critics, the selection of the relevant conditions is more difficult in QCA than in other types of analyses, since the researcher is limited to only a few conditions. This is a result of the Boolean procedure. If one considers five conditions, there are 32 possible combinations (2⁵) in the truth table. Increasing the number of conditions to eight produces 512 possible configurations and an increase to 12 explanatory conditions would produce 8192 possible configurations. This will lead to a situation for which no analytical reduction is possible and one is confronted with the fact that each case is unique, which in turn would restrict researchers to descriptive accounts (Aarebrot and Bakka, Reference Aarebrot and Bakka1997; Scharpf, Reference Scharpf1997). Although valid, this criticism was not considered to be unique to QCA, since other approaches which analyse models have equal restrictions on the number of variables. Also from a theoretical point of view, it is not straightforward to deal with models containing many interacting terms.

A fourth debate focused on the static nature of the approach and the inability to include a time dimension or sequence of variables in an analysis (Boswell and Brown, Reference Boswell and Brown1999: 181). Criticisms focused on two issues. First, QCA was criticized for the lack of a longitudinal perspective towards analysis. In the criticism, it was assumed that conditions were measured in a snapshot way (one moment in time) as was the case in traditional cross-sectional research. This criticism emerged against the background of a period that saw an increased interest in time series analysis and the opportunities that it seemed to offer at that time for macro-sociological and comparative politics research (for a critical reflection see Kittel, Reference Kittel2006). Many users considered this criticism to be unfair, since it failed to make a distinction between cases and observations and the measurement of conditions in QCA could just as well be based on time series data (i.e. many observations). Hence, conditions could be operationalized in a way that makes them dynamic – that is, the time dimension can be injected into the conditions themselves (Rihoux, Reference Rihoux2001). A second criticism focused on the difficulty of including a sequence of conditions in the analysis. In explaining long-term changes, as is the case of many political sociological accounts, the timing of variables can determine the outcome (see also Pierson, Reference Pierson2003), and it is important to include a procedure that makes it possible to sequence the causal conditions. This issue was addressed by developing techniques that allowed for a sequencing of conditions (see Caren and Panofsky, Reference Caren and Panofsky2005; Ragin and Strand, Reference Ragin and Strand2005; Schneider and Wagemann, Reference Schneider and Wagemann2012: 263ff.).

A final debate revolved around the notion that QCA assumes case independence. The cases are compared under the assumption that they do not influence each other. This assumption is assuredly present in all variable-oriented techniques of analysis and hence not unique to QCA. The relevance of this issue is largely dependent on the research question and the topic under investigation. In diffusion studies, the interconnectedness of cases is relevant, while in other cases this might be far less relevant. If it is theoretically relevant, there are several ways to proceed. First, working within the framework of QCA researchers can include conditions that take the interrelatedness of cases into account. In addition, further in-depth follow-up within case research (i.e. process tracing, see below) can reveal the interconnectedness between cases. Third, other available methodological tools specifically designed to study interrelatedness, such as social network analysis, can complement QCA.

From crisp sets to fuzzy sets

Following these criticisms, several innovations were introduced in the QCA framework. Concerning the measurement of variables in QCA, two major developments occurred. First, quite soon after the publication of The Comparative Method, Charles Ragin started to work on the development of fuzzy sets as a complement to crisp sets. This line of inquiry led to the publication of Fuzzy-Set Social Science. In fact, fuzzy-set social science predated many of the debates focused on the use of crisp sets. The fuzzy-set approach was a natural extension of the crisp-set approach. Fuzzy-Set Social Science (Ragin, Reference Ragin2000) was primarily concerned with introducing fuzzy sets for social science research and developing its potential for analysing necessary and sufficient conditions, which at that time was gaining increasing recognition in social scientific research (Goertz and Starr, Reference Goertz and Starr2003; Goertz, Reference Goertz2006b). This led to the development of new fsQCA software.Footnote ³ Second, in Marburg, the research group around Dirk Berg-Schlosser worked on the issue of including more refined measurement techniques (multi-value sets) in QCA, which led to the development, by Lasse Cronqvist, of the new software named TOSMANA (Cronqvist and Berg-Schlosser, Reference Cronqvist and Berg-Schlosser2009; http://www.tosmana.net/).

A second area of further development in the use of QCA focused on the selection of conditions and the specification of models. Amenta and Poulsen (1994) outlined several strategies for the selection of conditions for a QCA analysis and discussed the strengths and weaknesses of each strategy (see also Yamasaki and Rihoux, Reference Yamasaki and Rihoux2009: 125–130). They also proposed configurational strategies, which were especially suited for QCA. Another approach is to develop macro-conditions, which combine different indicators in higher-level conceptual constructs.

A third area of further development addressed the issue of the sequence of conditions. Here the work of Schneider and Wagemann (Reference Schneider and Wagemann2006) focused on developing a two-step QCA protocol, based on the distinction between ‘proximate’ and ‘remote’ conditions. This approach was applied in research, which sought to explain the consolidation of democracy, and later also in research on other topics (e.g. Maggetti, Reference Maggetti2009; Sager and Andereggen, Reference Sager and Andereggen2011). Caren and Panofsky (Reference Caren and Panofsky2005) also worked on the sequencing of conditions and introduced a technique called temporal QCA (TQCA; see also Ragin and Strand, Reference Ragin and Strand2005 for a further elaboration of this approach).

This line of work, conducted by several scholars at the end of the previous millennium and the start of the new, not only resulted in several innovations but also in a wider acceptance and an increasing number of applications. Figure 1 shows both the cumulative development of all articles, which apply QCA, and also the articles within the broad array of political science.

Figure 1

Evolution of number of QCA applications overall and in political science. Figure based on the COMPASSS database (see footnote 1).

QCA in political science research

The graph shows a strong increase from 2003 onwards in political science. This growth can probably be explained by two dynamics. First, since 2000 there has been increased interest in case-based research and comparative case research in political science, following a range of influential papers. Second, particularly in Europe, a group of researchers applying QCA got together in the COMPASSS network, which resulted in the diffusion of courses, best practices, the organization of panels at conferences, and developing courses for the European Consortium for Political Research (ECPR) summer school. As a result, more researchers were exposed to QCA and started to apply it in their PhD dissertations, which in turn resulted in journal publications.

The appendix provides an overview of all English-written QCA applications (as peer-reviewed journal articles) for the period 2003–2011. Besides the first author, short title, and journal, the table also gives an overview of the type of QCA analysis (crisp set, fuzzy set, and/or multi-value) and the numbers of cases and conditions involved in the application. The table in the appendix reveals a few interesting observations with regard to the diversification of QCA papers.

First, QCA is now applied to a wide range of topics in political science including topics dealing with democracy, party politics, welfare state research, public administration, policy analysis (for an extensive review, see Rihoux et al., 2011), governance, regulation, and political sociology. Second, the number of journals featuring QCA-based articles is also expanding. While in the first decade, there was a high concentration in mainly sociological journals; we now see a move towards a wide variety of journals. In the COMPASSS database, more than 200 peer-reviewed journals have now published QCA-based articles. Third, and strikingly, notwithstanding early criticisms on the use of dichotomous variables and the emergence of fuzzy sets and multi-value QCA, crisp-set QCA (csQCA) continues to be the most frequently used technique. Of the 94 applications listed in table A1, 52 use csQCA, some use several approaches, and only five use multi-value QCA. Hence, csQCA remains a relevant and complementary analytical tool (see also Grofman and Schneider, Reference Grofman and Schneider2009). Fourth, with regard to the number of cases, the appendix shows that QCA remains a tool for small- and intermediate-N analysis. With a few exceptions on the high and low end, most applications (76 in the appendix) use a number of cases, which ranges from 10 to 90. The number of conditions ranges from 2 to 10 with most applications using four or five explanatory conditions, thereby allowing researchers to capture complex configurations.

The appendix does not reveal the qualitative differences between the different applications of QCA. A discussion of this goes beyond the scope of this paper. However, it is noteworthy that a closer inspection will reveal that different authors apply QCA in different ways with different research designs. Depending on the evaluation criteria for assessing an application (use of set-theoretic configurational arguments, model specification criteria taken into account, carrying out robustness tests, return to the cases after a QCA analysis, and possibly other criteria) some applications will score well, others will score less. Emmenegger et al., Reference Emmenegger, Kvist and Skaaning(2013 forthcoming) recently reviewed 19 published QCA applications in the area of comparative welfare state research. They assessed these 19 articles against a set of criteria of ‘good QCA practice’ and found that only half of the studies made complex propositions formulated in set-theoretical terms. Despite the case-oriented nature of QCA and the applications, only few scholars went back to the cases after the formal analysis. Finally, they showed that only a few studies carried out robustness checks of their findings. The latter is not surprising, since most robustness checks are of recent nature. They argue that improvements on these issues are needed to increase the quality of studies using QCA.

Recent developments in QCA

Besides a wider range of applications, there has also been an increased attention to the methodological development of the QCA approach on several fronts – all of which are still in progress.

First of all, Charles Ragin developed several new features to QCA in his 2008 book Redesigning Social Inquiry: Fuzzy Sets and Beyond (Ragin, Reference Ragin2008). Two key innovations involved the development of new measures to aid the interpretation of the results of QCA applications, namely, consistency and coverage [see also Ragin, Reference Ragin2006, for related alternative approaches see Goertz (Reference Goertz2006a: 95–128); Eliason and Stryker (Reference Eliason and Stryker2009); Schneider and Wagemann (Reference Schneider and Wagemann2012: 220–250)]. These new measures were developed primarily for the assessment of sufficient combinations of conditions identified via truth table analysis. In this context, consistency evaluates the degree to which a combination of conditions constitutes a subset of an outcome. Coverage, by contrast, evaluates a consistent combination's empirical importance by assessing the proportion of cases with the outcome that the combination covers (crisp sets) or the proportion of the sum of the membership scores in the outcome it covers (fuzzy sets). These measures enable researchers to assess the degree to which a model explains the outcomes observed in the cases and also the relative weight of each causal combination. Parallel measures exist for the evaluation of necessary conditions. In this context, the consistency of a necessary condition is based on the degree to which the condition is a superset of the outcome; the coverage (or empirical relevance) of a necessary condition is based on the degree to which the outcome covers the necessary condition, assuming the outcome is a consistent subset. If the necessary condition dwarfs the outcome, it is likely to be omnipresent and thus a trivial necessary condition (e.g. air is a necessary condition for social revolution).

Second, Marx (Reference Marx2010, see also Marx and Dusa, Reference Marx and Dusa2011) focused on one assumption on which csQCA is based, namely that contradictions would naturally occur if the explanatory model is flawed. It was hypothesized that contradictions should always occur when csQCA is applied to the analysis of random data. This hypothesis was tested on the basis of a simulation in which random data sets were analysed by csQCA. Marx found that contradictions are not naturally occurring phenomena. In some instances, csQCA generated no contradictions on the basis of random data. The occurrence of contradictions is a function of the design of the explanatory model in terms of number of cases and conditions included in the analysis. This finding has implications for how to specify models in a csQCA analysis in terms of the number of conditions, which can be included in csQCA (see also Berg-Schlosser and De Meur, Reference Berg-Schlosser and De Meur2009).

Third, authors have started to develop measures and procedures to assess the degree to which the results of a QCA correspond to the empirical data. Eliason and Stryker (Reference Eliason and Stryker2009) develop goodness-of-fit tests for fuzzy-set analyses to formally assess the fit between empirical information and various causal hypotheses while accounting for measurement error in membership scores. They also develop descriptive measures to complement these tests. Skaaning (Reference Skaaning2011) focuses on the sensitivity of QCA results in relation to the calibration of raw data into set-membership values (both crisp set and fuzzy set), the frequency of cases linked to the configurations, and the choice of consistency thresholds. In order to assess the sensitivity of QCA results with regard to these three elements, he developed robustness tests using systematic procedures. In their book on Set-Theoretic Methods for the Social Sciences, Schneider and Wagemann (Reference Schneider and Wagemann2012: 284ff.) discuss extensively how to assess robustness and evaluate the effects of changing set-membership calibration, changing consistency levels, and adding or deleting cases.

Fourth, several efforts have been made to better account for time and sequence, on the one hand, and process, on the other, using QCA. With regard to time and sequence, the problem is that the QCA as a technique (the computer-run part of it) was not initially conceived to directly tap sequences. One answer has been developed by Caren and Panofsky (Reference Caren and Panofsky2005), in the form of the direct inclusion of sequences into QCA, with an enrichment of QCA notation into a ‘TQCA’ (see also Ragin and Strand, Reference Ragin and Strand2008). Several other, more indirect strategies have been applied, such as the sequencing of QCA with Event Structure Analysis (Stevenson and Greenberg, Reference Stevenson and Greenberg2000; Duckles et al., Reference Duckles, Hager and Galaskiewicz2005), the combination of QCA with softer forms of sequence analysis (Bleijenbergh and Roggeband, Reference Bleijenbergh and Roggeband2007), the incorporation of time series into QCA (Hino, Reference Hino2009), the sequencing of QCA with Optimal Matching (Watanabe, Reference Watanabe2004; Krook, Reference Krook2006), or with dynamic game theory (Brown and Boswell, Reference Brown and Boswell1995). With regard to process, that is, the focus on causal mechanisms and causal chains, different options have been developed to combine QCA with Process Tracing – in particular, in order to better identify critical junctures (Emmenegger, Reference Emmenegger2010) or to use QCA upstream to identify particular ‘typical’ or ‘deviant’ cases that may then be analysed more in-depth via process tracing (e.g. Schneider and Rohlfing, 2013; Beach and Pedersen, Reference Beach and Pedersen2012). Several other ‘softer’ strategies have been implemented, especially in the dialogue between QCA and thick case studies (Rihoux and Lobe, Reference Rihoux and Lobe2009).

Fifth, and more broadly, and even if a broad majority of QCA applications so far are ‘mono-method’ at least in the format of journal articles (Rihoux et al., Reference Rihoux, Álamos-Concha, Bol, Marx and Rezsöhazy2013), there is a significant amount of work done on how to better combine, sequence, or ‘mix’ QCA with other techniques, both qualitative and quantitative. On the quantitative front, there is now a significant diversity of QCA applications that have performed one or the other form of triangulation with quantitative/statistical analyses (for a review, see Rihoux et al., Reference Rihoux, Ragin, Yamasaki and Bol2009: 170–172).

Sixth, and in concrete terms, in the field of textbooks and introductory texts and software development, several new initiatives were launched and are in development. Rihoux and Ragin (Reference Rihoux and Ragin2009) published an introductory textbook on QCA and related techniques. Recently, Schneider and Wagemann (Reference Schneider and Wagemann2012) published an extensive textbook on set-theoretic approaches and QCA, discussing systematically all components related to QCA as an approach and technique. Several scholars are also involved in introducing the ‘nuts and bolts’ of QCA in journal articles or book chapters. (See, for example, Fiss, Reference Fiss2007, Reference Fiss2011; Grofman and Schneider, Reference Grofman and Schneider2009; Marx et al., Reference Marx, Cambre and Rihouxforthcoming). Concerning software developments, there are currently two main software packages freely available, which offer QCA: fsQCA and TOSMANA. In addition, QCA has been introduced in R [Dusa, Reference Dusa2007, Reference Dusa2010; and a technical textbook on QCA with R (Thiem and Dusa, Reference Thiem and Dusa2012)] and STATA (the program Fuzzy developed by Longest and Vaisey, Reference Longest and Vaisey2008). Combinations with other methodological analytical tools such as social network analysis are also generating new software tools (APES) with which QCA can be combined (Serdült and Hirshi, Reference Serdült and Hirshi2004). A continuous follow-up of recent developments in the area of QCA is available through COMPASSS (see also footnote 1), a global network of researchers interested in the development and application of systematic cross-case comparative methods. COMPASSS is an extensive resource website including full bibliographic references, overview of activities including training courses, summer schools (most notably, the ECPR summer and winter methods courses), and conferences. The COMPASSS website also hosts a dedicated and peer-reviewed working paper series. COMPASSS regularly circulates a newsletter.