The study of democracy is a flourishing theme in political science. Particularly since 1989, an increasing number of journals, articles, books and conferences have paid attention to this topic. To give some examples, two new academic journals were launched with the aim of fully focusing on the study of democracy and democratization: Journal of Democracy first appeared in 1990 and Democratization in 1994. Moreover, within a very short period (from 1990 until 1993), no fewer than 17 academic journals decided to dedicate one or more issues to this theme.Footnote 1 In addition, not only journals and special issues, but also many stand-alone articles focus on this key topic within political science. Based on analyses of about 4,500 articles published in 19 leading political science journals, we can confidently state that the theme of democracy has lost none of its attraction (Doorenspleet and Freyburg Reference Doorenspleet and Freyburg2014).
Scholars have focused not only on trying to explain democracy in comparative perspective (democracy as a dependent variable) and on exploring the effects of democracy (democracy as an independent variable), but also on the conceptualization and measurement of the phenomenon. The term ‘democracy’ first appeared in ancient Greek political and philosophical thought, and scholars have thought about different conceptualizations of democracy for several centuries (see Dunn Reference Dunn1994), while the actual quantitative measurement and ranking of countries according to their level of democracy are more recent endeavours (for example, Alvarez et al. Reference Alvarez, Cheibub, Limongi and Przeworski1996; Arat Reference Arat1991; Beetham Reference Beetham1994; Bollen Reference Bollen1980; Bollen and Grandjean Reference Bollen and Grandjean1981; Coppedge et al. Reference Coppedge, Gerring, Altman, Bernhard, Fish, Hicken, Kroenig, Lindberg, McMann, Paxton, Semetko, Skaaning, Staton and Teorell2011; Dahl Reference Dahl1956, Reference Dahl1971; Doorenspleet Reference Doorenspleet2000; Gastil Reference Gastil1990; Gurr et al. Reference Gurr, Gurr, Gilles, Bishop, Ruttenberg and Dose1978; Munck and Verkuilen Reference Munck and Verkuilen2002; Przeworski et al. Reference Przeworski, Alvarez, Cheibub and Limongi2000; Vanhanen Reference Vanhanen2000).
In this article, I will argue that the existing measures tend to reinforce the idea that democracy is the domain of the state, with its procedures, institutions and political elites, while ignoring people’s views. The article will first describe and review the existing measures, which have been not only widely used, but also systematically scrutinized and criticized. One of the key criticisms is that the existing measurement instruments are limited in focus, dealing only with political elites and electoral procedures. I fully agree with this point, and as a result I believe we need to bring people to the heart of democracy measurement. The article will include proposals of new people-centred measures of democracy. These radically different measures of democracy take into account how people view democracy, and can be used in statistical studies.
People-centred measures are to be welcomed because they would mean that democracy will be measured in a less narrow, much broader, way. Existing measures have used indicators at the state and elite level, which does not necessarily correspond with how people view democracy, as will become clear later in this article. The second advantage is that developing people-centred measures would in itself be more democratic. So far, democracy measurement instruments have mainly relied on experts who determine what democracy is (and should be), while people’s views have been ignored. A final advantage is that developing new people-centred democracy measures could lead to more debate in the field of democracy studies. So far, quantitative studies have relied on only a few different measurement instruments which rate the countries and their democracies in similar ways. While there is much debate and contestation about different concepts of democracy, this is not reflected in the measures we use. More contestation in the form of more alternative measures of democracy would enrich our current understanding of democracy and move the debate in comparative politics about this theme forward.
THE DOMINANT MEASUREMENTS OF DEMOCRACY
The theme democracy has received a lot of scholarly attention since the end of the Cold War. On average, more than 20 per cent of original research articles published in the sampled journals in 2013 are related to the theme ‘democracy’. This includes both quantitative and qualitative studies (see Doorenspleet and Freyburg Reference Doorenspleet and Freyburg2014). There are major differences between the subfields of political sciences, with comparative politics journals publishing a considerably higher share of democracy-related articles than area studies journals. The study of democracy and democratization is at the core of the sub-discipline of comparative politics, with quantitative studies dominating the debate about explanations and the effects of democracy. Those studies cover a wide range of different questions and findings, enriching our knowledge and triggering debates among academics and policymakers. Despite this diversity and debate, there is hardly any variation in the way we have measured democracy. Quantitative studies of democracy and democratization have mainly relied on just two measures of democracy: those of the Polity IV Project and the Freedom House.2
Measuring Democracy: The Polity IV Project
The present Polity IV Project was initiated by US academic Ted Robert Gurr in the 1970s. It initially focused on state failure, risk and political instability (Eckstein and Gurr Reference Eckstein and Gurr1975; Gurr Reference Gurr1974; Gurr et al. Reference Gurr, Gurr, Gilles, Bishop, Ruttenberg and Dose1978). This work tried to detect and measure the characteristics of the more durable political systems, which were not necessarily democratic, focusing on stability and procedures, not on participation and people (Goldstone et al. Reference Goldstone, Bates, Epstein, Gurr, Lustik, Marshall, Ulfelder and Woodward2010). It has been sponsored by the Political Instability Task Force (PITF), which is funded by the Central Intelligence Agency (CIA).Footnote 3
The project covers all independent countries with a total population of 500,000 or more and provides data since 1800. Polity IV provides two aggregate indices, democracy (DEMOC) and autocracy (AUTOC), which are both measured on an ordinal scale using five different indicators, selected and coded by experts. In the Polity IV indices, the five indicators are weighted differently, and the scale most widely used is a 21-point scale which combines both indices and ranges from −10 (very autocratic) to +10 (very democratic).Footnote 4
Hence, democracy is measured by five indicators in total. Three indicators focus on structural characteristics by which chief executives are recruited. (1) The extent to which a country has institutionalized procedures for transferring executive power, so this indicator XRREG focuses on modes by which chief executives are selected. Countries in which ‘chief executives are determined by hereditary succession or in competitive elections’ get the highest score (Marshall et al. Reference Marshall, Gurr and Jaggers2013: 20–1) for this indicator, while countries in which ‘changes in chief executive occur through forceful seizures of power’ get the lowest score on the democracy scale. (2) The competitiveness of executive selection, so this indicator XRCOMP refers to the ‘extent that prevailing modes of advancement give subordinates equal opportunities to become superordinates’ (Gurr Reference Gurr1974: 1483). Countries in which ‘chief executives are typically chosen in or through competitive elections matching two or more major parties or candidates’ get the highest score (Marshall et al. Reference Marshall, Gurr and Jaggers2013: 21–2) for this indicator, while countries get the lowest score when their chief executives are determined by hereditary succession or during rigged, unopposed elections. (3) The openness of executive recruitment, so this indicator XROPEN measures the extent to which ‘all the politically active population has an opportunity, in principle, to attain the position through a regularized process’ (Marshall et al. Reference Marshall, Gurr and Jaggers2013: 22). Countries in which the ‘chief executives are determined by hereditary succession’ get the lowest score, while countries where ‘chief executives are chosen by elite designation, competitive election, or transitional arrangements between designation and election’ get the highest score (Marshall et al. Reference Marshall, Gurr and Jaggers2013: 22).
The fourth indicator (4) XCONST refers to ‘the extent of institutionalized constraints on the decision making powers of chief executives’ (Marshall et al. Reference Marshall, Gurr and Jaggers2013: 24). Countries in which ‘there are no regular limitations on the executive’s actions’ get the lowest score, while countries in which ‘accountability groups have effective authority equal to or greater than the executive in most areas of activity’ get the highest score (Marshall et al. Reference Marshall, Gurr and Jaggers2013: 25). The final indicator (5) PARREG focuses on the regulation of participation, so ‘the extent that there are binding rules on when, whether, and how political preferences are expressed’ (Marshall et al. Reference Marshall, Gurr and Jaggers2013: 25–6). The highest scores are for countries with ‘relatively stable and enduring political groups’, which ‘regularly compete for political influence and positions with little use of coercion’ (Marshall et al Reference Marshall, Gurr and Jaggers2013: 26).
The eventual measurement of democracy is the sum of the scores for the five indicators, leading to an overall scale from −10 to +10. The underlying concept of democracy as used in the Polity Project emphasizes the importance of free and fair competition among political parties, of political participation by the people, and of protection of civil liberties. In the end, however, the actual measure does not include elements of public political participation and civil liberties (see Coppedge et al. Reference Coppedge, Alvarez and Maldonado2008; Doorenspleet Reference Doorenspleet2005: 38–9, 49) and the focus is on stability of the political process, on rules and regulations, on political elites in general, and on the chief executives specifically (how they get into power, and once in power how their actions are checked and controlled). Hence, there is a gap between the concept and the actual measurement (see also Adcock and Collier Reference Adcock and Collier2001; Doorenspleet Reference Doorenspleet2000); moreover, the political elites and procedures are central in the Polity IV democracy scales and rankings, certainly not the people and their views.
Measuring Democracy: Freedom House Index
The Freedom House index was published for the first time in 1973 by Raymond Gastil and has been updated annually by the team of experts of the Freedom House (FH). The Freedom House defines itself as ‘an independent nongovernmental organization that supports the expansion of freedom in the world’ (Freedom House 2014). It was established in 1941, believing that ‘the spread of democracy was the best weapon against totalitarian ideologies’ and embracing ‘as its mission the expansion of freedom around the world’ while nowadays being ‘a strong voice for a U.S. foreign policy that places the promotion of democracy as a priority’ (Freedom House 2014). The Freedom House has strong financial and personal links with the US government and the CIA (see Giannone Reference Giannone2010: 73–6).Footnote 5
Freedom House rates the status of democracy and freedom in more than 190 independent countries (1972–now) and offers three different measurements. The first is a measurement and classification of electoral democracies, which includes countries with: (1) a competitive, multiparty political system; (2) universal adult suffrage for all citizens; (3) regular, secret, competitive and free elections; (4) significant public access of major political parties to the electorate through the media and through generally open political campaigning.Footnote 6
The other two measurements are based on checklists to measure political rights and civil liberties. Until 1990, it was unclear how Freedom House’s scale was constructed and how the scores were given. Gastil preferred contextual, political and cultural variables, and argued that ‘the deeper reason for avoiding numerical grading of subfactors is that every aspect we might wish to grade has a slightly different meaning within different countries’ (Gastil Reference Gastil1990: 31), which led to reliability problems. This changed in the early 1990s, though, when an entire team of experts started to score the countries, and introduced a more transparent and reliable way of coding. Based on dozens of different questions, the experts have constructed seven-point scales for both political rights and civil liberties.Footnote 7 The questions to measure ‘political rights’ are to find out whether the chief authority is recently elected by a meaningful process, whether the legislature is recently elected by a meaningful process, whether there are fair election laws, whether there are competing parties and significant opposition. The questions that measure ‘civil liberties’ explore whether there is freedom of expression and belief, if there are association and organization rights, and whether there is rule of law.Footnote 8
The underlying concept for Freedom House is freedom and not democracy, but the Freedom House data have been widely used by researchers as measures of democracy. The Freedom House indicators overlap greatly with the Polity IV indicators, and over the years ‘its author understood that the survey was essentially a survey of democracy’ (Gastil Reference Gastil1990: 26), although the word ‘freedom’ – instead of democracy – is still used to refer to those two measurements. While there is an attempt to measure the extent of political freedom and civil liberties in each country, the Freedom House data set relies quite heavily on institutions and procedures as surrogates for substantive democracy and a democratic society. For example, in attempting to evaluate the openness of a society to dissenting opinion, the survey team examines the kinds of laws and institutional protections for speech, but pays only limited attention to actual freedom of speech within the country. Formal rights and procedures are central, while the actual practice is less important. Liberty and political rights are central, while equality and socioeconomic rights are not. Negative freedom is central (for example, freedom from government intervention), while positive freedom is not (for example, social rights) (see Saward Reference Saward1994: 18–19). Thus, procedures, institutions and political elites prevail in the Freedom House scales.
Previous Measurements of Democracy: Old and New Critical Comments
Using quantitative measurements of democracy has become common, not only in comparative politics but also in other disciplines (international relations, economics, sociology and geography), and countless studies including such measurements have been published over the last 50 years (for overviews of those studies, see Doorenspleet Reference Doorenspleet2005; Møller and Skaaning Reference Møller and Skaaning2013; Shin Reference Shin1994).
Outside academia, measuring democracy and ranking countries is big business as well. The United Nations, the World Bank and national governments use those rankings to come up with policies regarding foreign aid and democracy promotion. Those rankings help policymakers and politicians decide which countries get aid and which countries do not get financial support. International democracy promotion has become an important industry (for example, see Burnell Reference Burnell2011, Reference Burnell2013). The fact that the measurements have been widely used and have received international recognition can be seen as a strong point. Moreover, the measurements facilitate the replication of findings in previous studies, which in turn has led to an accumulation of knowledge around the theme of democracy and democratization.
At the same time, these measurements have been systematically scrutinized and criticized by dozens of different studies (see, for example, Adcock and Collier Reference Adcock and Collier2001; Beetham Reference Beetham1994; Bollen Reference Bollen1980, Reference Bollen1986, Reference Bollen1993; Bollen and Paxton Reference Bollen and Paxton2000; Bowman et al. Reference Bowman, Lehoucq and Mahoney2005; Coppedge et al. Reference Coppedge, Alvarez and Maldonado2008, Reference Coppedge, Gerring, Altman, Bernhard, Fish, Hicken, Kroenig, Lindberg, McMann, Paxton, Semetko, Skaaning, Staton and Teorell2011; Doorenspleet Reference Doorenspleet2000; Gleditsch and Ward Reference Gleditsch and Ward1997; Gugiu and Centellas Reference Gugiu and Centellas2013; Munck Reference Munck2009; Munck and Verkuilen Reference Munck and Verkuilen2002; Pemstein et al. Reference Pemstein, Meserve and Melton2010; Plümper and Neumayer Reference Plümper and Neumayer2010; Treier and Jackman Reference Treier and Jackman2008). These studies have critically assessed many different aspects of the measures, such as their reliability, their aggregation procedures, the underlying definition, problems around transparency and replicability, the sources, the lack of specificity, limited coverage across time and space, the risk of conceptual stretching and the lack of rigour in its construction. It is worth noting that scholars have been far more critical towards the Freedom House scales than toward the Polity IV work. Furthermore, scholars have paid attention to political and ideological issues affecting the measurements. The Freedom House in particular has been accused of having crucial biases (Giannone Reference Giannone2010). For example, some scholars showed that the Freedom House scales are influenced by a conservative ideology (Scoble and Wiseberg Reference Scoble and Wiseberg1981) and by pro-market biases (Bollen Reference Bollen1986) giving leftist governments lower scores than can be justified (Mainwaring et al. Reference Mainwaring, Brink and Perez-Liñan2001). The Freedom House also seems to favour Christian and Western countries, while systematically giving low scores to Muslim and Marxist-Leninist countries (Bollen and Paxton Reference Bollen and Paxton2000).
Another fundamental point of critique, which is relevant for this article, is that the existing measures are limited, using a narrow set of indicators which focus mainly on political institutions and electoral procedures, political parties and elites. The vast majority of the measurements have taken a narrow view that democracy is mainly about the elections of political leaders, and – once in power – the checks and balances of the system. Some recent projects, such as the Varieties of Democracy (V-Dem) Project, acknowledge this narrow focus, develop alternative measures and treat the concept as a multidimensional one (see Coppedge et al. Reference Coppedge, Gerring, Altman, Bernhard, Fish, Hicken, Kroenig, Lindberg, McMann, Paxton, Semetko, Skaaning, Staton and Teorell2011). The V-Dem Project claims to cover ‘virtually all conceptions of democracy’, so not only electoral and liberal democracy, but also participatory (Pateman 1970), deliberative democracy (Dryzek Reference Dryzek1990, 2010; Fishkin Reference Fishkin1991; Mansbridge and Parkinson Reference Mansbridge and Parkinson2012), social democracy (Ake Reference Ake1991; Huber et al. Reference Huber, Rueschemeyer and Stephens1997; Sorensen Reference Sorensen1993: 16) and other models (see Held 2006). Although the new V-Dem Project is very impressive in its scope,Footnote 9 it still fits into the existing tradition of measuring democracy without taking people’s views seriously.Footnote 10
Based on the previous measures of democracy, we might think that democracy is not the domain of the people, and that people are rather beside the point. As we all know, the term ‘democracy’ originates from the Ancient Greek δημοκρατία (dēmokratía), which consists of the words δῆμος (demos, ‘people’) and κράτος (kratos, ‘power’ or ‘rule’), in contrast to the term ἀριστοκρατία (aristokratia, consisting of the words ‘elite’ and ‘rule’). Ironically, however, existing measures of democracy seem to prefer indicators at the state and elite level, ignoring how people view democracy. As will become clear in the next section, people-centred definitions of democracy are less narrow. Public opinion surveys have revealed that some people define the concept in a more conventional way by mentioning free and fair elections or democratic procedures or governance. Many people mention the importance of freedom. But a considerable number of people also mention more substantial elements and argue that socioeconomic development and equality, peace and stability are also part of the definition of democracy, or should be.
Existing measurements have ignored people’s views not only during the phase of conceptualization (What is democracy? What is the definition of the concept?), but also during the phases of operationalization (What are the main indicators by which we can measure the concept?) and measurement (‘To what extent are the indicators present in a particular country? What is the level of a country’s democracy, using the chosen indicators?’).Footnote 11 In other words, experts (political scientists) determine what democracy is and which indicators should be included, and thereafter experts (again, predominantly political scientists) rate the countries using those indicators and determine in this way how democratic these countries are. The current way of conceptualizing and measuring democracy in political science reinforces the idea that political elites – and not the people – are at the core of the concept and measurement of democracy. However, it is time to think about developing new people-centred measures as alternatives and using them in our studies of democracy and democratization.
WHERE ARE THE PEOPLE AND WHY DOES IT MATTER?
Quantitative studies have all relied on a limited set of measures of democracy which focus on the political elites, electoral procedures and political institutions, while people’s views have not been included. But does it really matter? Is it problematic? In my view, it certainly is.
It is problematic because there is a disconnect between how quantitative academics define democracy, and how people view democracy. There is only a very weak correlation between popular views of democracy and the Polity IV scores; there is only a moderate link between popular views and the Freedom House index (see Logan and Mattes 2010: 19–21, particularly table 6). These correlations are based on analyses of African countries, but similar findings are likely to be found in other countries of the world. Such findings indicate that there are clear discrepancies between views of experts on the one hand, and people’s views on the other hand.
Democracy has different meanings in different contexts. Not only African scholars but also political scientists in Asia and Latin America have argued that this is the case. For example, in scholars’ (expert) view of democracy, they mentioned elements such as ‘law and order’ and ‘the collective well-being of the community’ in the context of South-East and East Asia (Chan Reference Chan1997; but see also Sen Reference Sen1999), while socioeconomic development and economic equality are often mentioned in the African context. There has been a lively debate among scholars about the question of whether people in African countries understand democracy in a different way, namely not by focusing on procedural notions and political rights, but by understanding democracy in a broader way with substantive meanings which include socioeconomic elements (see, for example, Ake Reference Ake1993; Bratton et al. Reference Bratton, Mattes and Gyimah Boadi2005; Joseph Reference Joseph1997; Schaffer Reference Schaffer2000; Zuern Reference Zuern2009). As we now have the cross-national data from the surveys, we have the tools to investigate such claims empirically. It appears that in different countries people indeed have different views of democracy; this is not only clear in African countries (Bratton et al. Reference Bratton, Mattes and Gyimah Boadi2005), but also elsewhere in the world, and particularly in Asia and Latin America (see Canache Reference Canache2012; Dalton et al. Reference Dalton, Sin and Jou2007; Rose et al. 1999).
In their ground-breaking 2005 book which explores public opinion in Africa, Mike Bratton and his co-authors analysed and discussed the findings of surveys in 12 countries.Footnote 12 Because ‘people associate democracy with many diverse meanings’ (Bratton et al. Reference Bratton, Mattes and Gyimah Boadi2005: 87), respondents were asked to say which elements could be seen as essential for a democracy. It appeared that in seven of those countries ‘respondents report that substantive features are significantly more essential than procedural ones [. . .] At a minimum, therefore, expectations of what democracy should do also include material components of economic delivery and social equity’ (Bratton et al. Reference Bratton, Mattes and Gyimah Boadi2005: 87). In the other countries, procedural and substantive meanings exist side by side. Hence, many people indeed think economics (so not only political procedures) needs to be part of democracy itself. However, quantitative researchers keep looking at democracy in a formal and procedural way and carefully separate the spheres of economics from politics when conceptualizing and measuring democracy (Munck Reference Munck2009: 126–7; see also Saward Reference Saward1994).
Another interesting finding from public opinion surveys is that respondents who belong to the elites favour the procedural, minimalistic, elitist notion of democracy. Elites, who are only a relatively small proportion of the population and have good access to material goods on a daily basis, tend to understand democracy in procedural terms, while non-elites are more likely to understand democracy in substantive terms (see Bratton et al. Reference Bratton, Mattes and Gyimah Boadi2005; also Zuern Reference Zuern2009: 588). Such findings indicate that – within one single country – different people think differently about democracy, and that the discrepancies are the biggest between people who do not belong to the elite on the one hand, and the existing expert-based measurements on the other hand.
Scholarly attention has been paid to people’s views of democracy (Bratton et al. Reference Bratton, Mattes and Gyimah Boadi2005; Canache Reference Canache2012; Dalton Reference Dalton2004; Dalton et al. Reference Dalton, Sin and Jou2007; Diamond and Plattner 2008; Doorenspleet Reference Doorenspleet2012; Rose et al. Reference Rose, Mishler and Haerpfer1998; Shin Reference Shin2006) but this is often motivated by the scholars’ belief that democracy (as measured by experts) requires a certain level of support by its citizens and commitment to a particular (hence procedural) idea of democracy (see, for example, Almond and Verba Reference Almond and Verba1965; Dahl Reference Dahl1971: 30 and chapter 8; Diamond Reference Diamond1999). So, the scholarly interest in public opinion on democracy reinforces the minimal notion of democracy and is instrumental in the sense that only particular aspects of people’s views (that is, their support of democracy or satisfaction with the regime) are taken into account and are used as explanatory factors of the chance that a country will make a transition to democracy and/or consolidate its democratic regime (for example Inglehart and Welzel Reference Inglehart and Welzel2005). Other studies even suggest that people’s definitions of democracy are ‘incorrect’, as people still ‘confuse’ different elements and have ‘wrong’ ideas about democracy. In this view, people in democratizing countries still have a ‘lack of awareness’ and still need time to ‘learn’ what democracy is (see, for example, Dalton et al. Reference Dalton, Sin and Jou2007; Mattes and Bratton Reference Mattes and Bratton2007).
People’s views should be taken more seriously and treated as valuable. This is the more standard approach in qualitative studies, such as ethnographic, interpretative and participant observation, based on long-term in-depth field research. In such qualitative studies, scholars tend to pay more attention to diversity in people’s views about democracy and value them as insightful in themselves (see, for example, Schaffer Reference Schaffer2000; Van Wessel Reference van Wessel2010). By exploring what individuals in different countries think about democracy, scholars can genuinely learn from them. By investigating how people define democracy, we can discover what experts have overlooked and failed to include in their own way of measuring. Moreover, by including people’s views in the measures of democracy itself, we can make the measures less elitist and more ‘democratic’. Why would we keep using concepts and measures of democracy that delegitimize and denigrate people’s views of democracy and in the process continue omitting alternative indicators such as social and economic factors? Dahl (Reference Dahl1971) would argue that democracy should be valued not just as a means but also as an end in itself, but would he strongly defend that the minimalistic procedural notion has sufficient intrinsic value? What does it mean to support the idea of ‘democracy as an end in itself’ if democracy is defined in a very particular way? Is this procedural type really the type of democracy we would like to keep exploring in future research? Is it not time to explore different views of democracy, unearth new types and include them in measures of democracy?
DEVELOPING A NEW PEOPLE-CENTRED MEASUREMENT OF DEMOCRACY: NEW PROPOSALS
The conventional method of measuring has been to first ask experts to define democracy and to list the most important measurable indicators. Once there is agreement about those indicators and the range of the scale, the second step is to ask (other) experts to give a specific number to each of the indicators, which will eventually often be aggregated in one single score (by addition or multiplication). Not only the Polity IV Project, the Freedom House, but also the new V-Dem Project work in this way: expert judgement is crucial, not only during the first step of the process (What is democracy? What are the key indicators?) but also during the second step (What is the level of democracy in a specific country? What are the rankings?).Footnote 13 Instead, we could also ask people what they think democracy is, and then rate the countries based on those people-centred indicators. My proposals as described below are more people-centred, while still giving expert judgements a role. These are preliminary; the proposals are not yet fully and thoroughly developed, but are designed to give a flavour of the possible tracks that can be taken in future research.
First Proposal: Combining People’s Views (Step 1) with Expert Judgements (Step 2)
The first step in the process would be not to ask the experts, but to ask the people what they think democracy means. This could result in different answers, ranging from indicators of political institutions (‘the existence of competing political parties’) and electoral processes (‘the regular running of free and fair elections’), but also broader views (‘freedom of press’) and substantial thicker views (‘socioeconomic equality’ and ‘no unemployment’). Then, during the second step, the main indicators which are derived from how people define democracy can be measured by experts. This will result in one single score for democracy.
For the sake of illustration, let us focus on one country, and apply the proposed measurement to South Africa. Other countries can be measured and scored in the same way, following the procedures described below.Footnote 14 The open question is central: ‘What if anything does democracy mean to you?’ Using the Afrobarometer surveys, the people-centred findings of the first step for South Africa are shown in Table 1.
People’s Views of Democracy (South Africa)
Notes: aBased on the Afrobarometer surveys of 2005. The respondents could give up to three responses, but only the first response is taken into account in this particular variable (‘first meaning of democracy’).
bThis first category is based on answers to question Q. 35, pt 1 (Carter 2008): value labels 2, 3 and 20; in total 139, 167 and 15 respondents=321; N=321.
cThis second category is based on answers to question Q. 35, pt 1 (Carter 2008): value label 1; in total 791 respondents; N=791.
dThis third category is based on answers to question Q. 35, pt 1 (Carter 2008): value labels 4, 9, 10, 11 and 14; in total 106, 29, 26, 55 and 10 respondents=226; N=226; 3=Peace/unity/stability.
eThis fourth category is based on answers to question Q. 35, pt 1 (Carter 2008): value labels 5 and 6; in total 163 and 300 respondents=463; N=463; 4=Social/economic development, equality/justice.
fThis fifth category is based on answers to question Q. 35, pt 1 (Carter 2008): value labels 7, 8 and 15; in total 140, 16 and 15 respondents=171; N=171; 5=Governance/effectiveness/accountability/rule of law.
gPlease note that there are also missing values (category 6=Other meanings/don’t know/did not understand the question). See Carter (2008), the rest of the value labels in question Q. 35, pt 1. In total 428 respondents fit into this sixth category; those respondents are labelled as ‘missing’ and not included in the analyses and findings of this article (so total N=2400−428=1972). However, explaining this huge number of people who ‘don’t know’ certainly deserves more attention in future research.
These findings show that people in South Africa do not view democracy solely in a minimalistic, electoral way. Fewer than 17 per cent mentioned this indicator as their first choice, while 40 per cent believed freedom is most important, 11.5 per cent focused on peace and stability, almost a quarter (23.5 per cent) defined democracy in terms of socioeconomic development and equality, while fewer than 9 per cent mentioned more specific elements of a democratic system such as accountability and rule of law.
After this first step, we can combine these people’s views with expert judgements. Suppose we ask experts to give scores to the indicators as mentioned by the people, and then rate the country South Africa. We could, for example, rely on the expert judgements of the Polity Project and the Freedom House to measure indicator 1 and 2. For indicator 3, we can use data from the Global Peace Index. For indicators 4 and 5, we could use data from the World Bank (the WGI data) and from the UNDP (the HDI data). Then, the expert-centred findings (on a scale of 1 to 10) of the second step for South Africa could be as shown in Table 2.
Expert Judgements of Democracy (South Africa)
Notes: aFrom Polity IV: Regime Authority Characteristics and Transitions Datasets, see http://www.systemicpeace.org/inscrdata.html. In 2005, South Africa gets a score of 9, composed of 0 on the autocracy scale, and 9 on the democracy scale.
bFrom Freedom House, https://freedomhouse.org/report/freedom-world/2005/south-africa#.VIWzyTGsVXk In 2005, South Africa gets a score of 2 on the scale of civil liberties (running from 1=best to 7 =worst); a conversion of this score for a new scale from 1 (= low) to 10 (=high) gives South Africa a score of 8.5.
cFrom Global Peace Index, which measures national peacefulness according to 22 qualitative and quantitative indicators, see www.visionofhumanity.org/#page/indexes/global-peace-index/2008; on the scale of 1 (=less peaceful) to 5 (=more peaceful), South Africa gets a score of 4. A conversion of this score into a new scale from 1 to 10 gives South Africa a score of 7.5.
dFrom UNDP Human Development Reports, see http://hdr.undp.org/en/countries/profiles/ZAF. South Africa is ranked as 118th out of the 187 countries included in the Human Development Index, which measures poverty, income and inequality. A conversion of this score into a new scale from 1 (= lowest) to 100 (=highest) means South Africa would get a score of 37, as 63% of the countries have a higher score. Conversion of this scale into a scale from 1 to 10 would give South Africa a score of 3.7.
eSee Worldwide Governance Indicators (WGI) collected by the World Bank, www.govindicators.org. In 2008, South Africa gets a score of 64.42 for accountability, 69.42 for government effectiveness, 54.81 for rule of law and 62.62 for control of corruption. Hence, the average score is 62.82. Conversion of this scale into a scale from 1 to 10 would give South Africa a score of 6.3.
As a last step, we can give one single score for democracy, combining the people-centred measurement and the expert judgements:
$$\ \quad\qquad{\rm Democracy}\;{=}\;{\rm a1}\left( {{\rm x1}} \right){\rm }{\plus}{\rm a2}\left( {{\rm x2}} \right){\rm }{\plus}{\rm a3}\left( {{\rm x3}} \right){\rm }{\plus}{\rm a4}\left( {{\rm x4}} \right){\rm }{\plus}{\rm a5}\left( {{\rm x5}} \right)$$
where a1=proportion people’s view of indicator 1
where a2=proportion people’s view of indicator 2
where a3=proportion people’s view of indicator 3
where a4=proportion people’s view of indicator 4
where a5=proportion people’s view of indicator 5
where x1=expert score of indicator 1
where x2=expert score of indicator 2
where x3=expert score of indicator 3
where x4=expert score of indicator 4
where x5=expert score of indicator 5
This will result in a single score for democracy.Footnote 15 So in the case of South Africa:
$$\eqalignno{ {\rm Democracy}&{\;=\;}0.{\rm 163 }\left\,( {\rm 9} \right){\plus}0.{\rm 4}0{\rm 1 }\left\,( {{\rm 8}.{\rm 5}} \right){\plus}0.{\rm 115 }\left\,( {{\rm 7}.{\rm 5}} \right) \cr & {\plus}0.{\rm 235 }\left\,( {{\rm 3}.{\rm 7}} \right){\plus}0.0{\rm 87 }\left\,( {{\rm 6}.{\rm 3}} \right)\,{=}\,{\rm 1}.{\rm 467}{\plus}{\rm 3}.{\rm 4}0{\rm 85}{\plus}0.{\rm 8625} \cr & {\plus}0.{\rm 8695}{\plus}0.{\rm 5481}{\;=\;}{\rm 7}.{\rm 2} $$
Expert scores, which are conventional at the moment, would have been much higher for this particular case, as they would focus on indicators 1 and 2 with the Polity IV scale and the Freedom House scores; those indicators receive scores which are – with respectively 9 and 8.5 – higher than the 7.2 of the new people-centred measurement.
Second Proposal: Combining Expert Judgements (Step 1) with People’s Views (Step 2)
The second way of measuring would first rely on experts’ views of what democracy is (definition with the essential indicators), and then investigate people’s views with regard to those indicators (measurement with the scores for each country). The first step in the process would be to ask the experts, before turning to people’s views. We could, for example, rely on Dahl’s conventional definition of democracy, which focuses on competition and participation (see Dahl Reference Dahl1971; see also Coppedge et al. Reference Coppedge, Alvarez and Maldonado2008; Doorenspleet Reference Doorenspleet2000). Then, during the second step, those indicators as chosen by the experts can be rated by the people in large-N public opinion surveys.
For the sake of illustration, let’s again first focus on one country and apply the proposed measurement to South Africa. The two indicators of competition and participation (first step) are measured by a range of different questions (second step), such as ‘On the whole, how would you rate the freeness and fairness of the last national election?’ and ‘During the election campaigns in this country, how much do you personally fear becoming a victim of political intimidation or violence?’ and several others. The choice of questions is based on the interesting article by Logan and Mattes (Reference Logan and Mattes2012: 473–6), which also proposes to include public opinion data in the measurement of democracy,Footnote 16 and which is quite similar to the steps taken in my second proposal.
A measurement process following this second proposal would result in a score of 4.9 for South Africa (see Table 3, and see Logan and Mattes Reference Logan and Mattes2012 for more details of the questions and specific calculations). This score is quite low on the overall scale from 0 to 10. If we had relied entirely on expert judgements, the scores would have been much higher for this particular case, as they would focus the Polity IV scale and the Freedom House scores, which would lead to scores of respectively 9 and 8.5.Footnote 17
Combining Expert Judgements (Step 1) with People’s Views (Step 2) in South Africa
Notes: aAll indicators are first calculated on a 0 to 4 scale, from low to high, see table 3 in Logan and Mattes (Reference Logan and Mattes2012: 380). Logan and Mattes calculated a score of 1.1 for South Africa. If we recode this score for a scale from 1 to 10, it results in a score of 3.48 for South Africa.
bAll indicators are first calculated on a 0 to 4 scale, from low to high, see table 3 in Logan and Mattes (Reference Logan and Mattes2012: 380). Logan and Mattes calculated a score of 2.4 for South Africa. If we recode this score for a scale from 1 to 10, it results in a score of 6.4 for South Africa.
cThe average of the scores for participation and competition is 4.9.
If we move away from focusing on a single country and follow this second proposal for 19 African countries, it becomes clear that using people-centred measurement has a real impact on the democracy rankings of some countries. Using the data from the Afrobarometer and Logan and Mattes (Reference Logan and Mattes2012), my analyses show, for example, that TanzaniaFootnote 18 gets a very high score and that KenyaFootnote 19 gets a very low score using this second people-centred measurement of democracy. The expert judgements scales tell a different story for these countries, with Tanzania getting a very low score of 2 and Kenya getting a very high score of 7 on the Polity scale (ranging from 0 to 10). Hence, some countries have a much higher score using people-centred measurements of democracy compared with their expert coding (for example Tanzania), while others fall in the ranking (for example Kenya). Such findings raise questions about the experts’ assessments: do the experts give overly low scores for Tanzania because of its one-party dominant system, while Kenya gets high scores from the experts because it has a very competitive political system?
New Proposals, New Challenges
One could argue that people-centred measures are not comparable from country to country. However, this is not completely true, as it depends on the type of comparability (see, for example, Hui and Triandis Reference Hui and Triandis1985). For instance, my proposals follow precisely the same procedures, using the same questions and instruments in different countries, so there are no problems of comparability during the phases of operationalization and measurement. With respect to the conceptual comparability, it is unlikely that the concept of democracy has the same meaning across different countries. Countries have different histories, which might have a different impact on the political views of their citizens. So, people who fought against apartheid in South Africa might have different views of democracy from people who have lived in quite stable systems for several decades, such as Tanzania. This is why it is important to develop people-centred measures. Countries do differ, and by sticking to ‘objective’ abstract minimal criteria of democracy we ignore those differences. Moreover, we keep rating the countries in a ‘one-size-fits-all’ way, even though the shoe might pinch. The people-centred measures show where the shoe does not fit (see Logan and Mattes Reference Logan and Mattes2012: 471).
Instead of simply assuming that democracy is conceptually comparable, and working with a similar meaning in different countries (for example, ‘competitive elections’) as previous measures have done, these new proposals show that there are cross-national differences in how people define democracy (in South Africa, it is mainly about ‘freedom’ and ‘socioeconomic equality’). People in the US think in a different way about democracy compared with people in Scandinavian democracies, democracies in Asia or Africa, or elsewhere in the world. So why not embrace those differences and include them in the measurements themselves, instead of simply dismissing alternative ideas as unsuitable to be part of the concept and measurement of democracy? While people-centred measures might compare ‘apples’ with ‘oranges’, this approach at least acknowledges that there are different types of fruit; better to accept those differences than to focus only on the apples, or – even worse – believe that fruit consists only of apples, so that we study only those apples.
People-centred measures could have some interesting implications for the ranking of countries and evaluations of democracy more generally. Some countries might fall in the rankings, while others have a much higher score than that based on the expert coding. Exploring these differences would reinvigorate democracy studies and help us measure aspects of democracy that have not yet been included. If the public sphere and people’s views matter at all to democracy – and in this article I argue that they do – then including these aspects in our measures is an interesting new step in our understanding of democracy.
Another important critical comment could be that people living in very repressive countries, such as North Korea, cannot freely give their opinion and hence their views about democracy are not reliable. I fully agree, and therefore my proposals include expert judgements as well. Moreover, if we wish, we can always exclude the very repressive countries from the analyses. A final issue is there is still a lack of public opinion data to develop people-centred measures. We need more data with open questions asking what people mean by democracy, and it would be good to include such questions in future surveys on a regular basis. The proposals are not perfect, but they are intended to start us thinking about democracy in a different – less elitist – way. I do not want to argue that one way of measuring is ‘best’, but I simply would like to draw attention to the idea that it is not only expert judgements that are valid, but people’s views are also essential to fully understand a country’s democracy.
CALL FOR ACTION: REINVIGORATE DEMOCRACY STUDIES
Defining democracy is one of the fundamental tasks for political scientists; measuring democracy is a challenge as well. The means of measurement is extremely important, as it might affect our way of thinking about democracy in general, and restrict the range of definitions used in our research. It is crucial, not only because it is one of the core issues in our discipline as such, but also because it captures the limitations in the way the discipline has developed, namely the ivory tower in which political scientists determine how to define and measure democracy, while people’s views are not taken seriously. Democracy at a minimum requires an exchange of views between citizens about how they view democracy and their experiences of living in a democratic country, apart from particular democratic procedures and processes (cf. Stoker Reference Stoker2010). Democracy needs procedures, but democracy also needs people.
This article has shown that the existing measures of democracy still have a narrow focus on political elites and formal procedures, and people’s views do not yet have a genuine place in our current definitions and measures in quantitative comparative politics. This specific way of looking at democracy feeds elitism even further, and it makes attempts at democratic re-engagement even harder. ‘Political scientists don’t do politics, they seek to understand it,’ as Gerry Stoker convincingly states, ‘but I do think we should consider a blurring at the boundaries. Because – at the same time – we are implicated as foremost manufacturers of ideas about politics’ (Stoker Reference Stoker2010: 59). If we acknowledge this, then we need to pay much more attention to people’s views, and take them seriously when defining and measuring democracy.
Learning from those views, the real task now is to identify new ways to define and measure democracy in order to tackle the new puzzles in our field. I would like to urge scholars in comparative politics – not only when doing qualitative research but particularly when using quantitative or mixed-methods – to spend more time and effort in bringing the people back to the heart of democracy studies, thus reinvigorating democracy studies. More contestation would enrich our current understanding of democracy and move the debate forward in future studies of democracy and democratization. As scholars in comparative politics we can shape the future of the study of democracy and, for example, decide to shift our attention from elites to people and explore the theme of democracy in a different and more diverse way. The future of democracy is difficult to predict, but the future of the study of democracy is in our own hands.
ACKNOWLEDGEMENTS
A previous version of this paper was presented at the Government and Opposition workshop, ‘The Future of Democracy’ (Cambridge University, September 2014). I would like to thank all the participants for the lively discussions and the lovely atmosphere. My specific thanks go to Gerardo Munck, Lia Nijzink, the journal editors and the anonymous reviewers for their important and helpful feedback.
