Introduction
International human rights norms have, for many years, been studied as an important element of wider scholarly debates about international norm dynamics and political change. In terms of Martha Finnemore and Kathryn Sikkink’s norm life cycle, international human rights norms have clearly passed through the early stages of norm emergence and a form of ‘norm cascade’ has occurred.Footnote 1 This norm cascade, in the sense it is used here, refers to states’ commitment to human rights, through for instance ratifying human rights treaties and engaging explicitly in human rights discourse. Every state in the international community has ratified at least one human rights treaty. Early stages of norm emergence and acceptance are therefore considered to be no longer ‘particularly interesting’ when studying international human rights norms.Footnote 2 It is in the transition from commitment to compliance where the real interest lies; how to translate a set of human rights commitments on paper, to ‘sustained behaviour and … practices that conform to … international human rights norms’; what has been termed ‘rule-consistent behaviour’.Footnote 3
The transition from commitment to compliance is where scholarship on human rights norms becomes very contested. For a number of years, more sceptical scholars have undertaken empirical (and mostly quantitative) studies to support arguments that commitment by states to treaties on paper does not lead to better human rights protection in practice.Footnote 4 In some cases, it has been argued that commitment may even be a smokescreen, behind which governments can commit even worse abuses.Footnote 5 More optimistic scholarship argues that there is evidence that commitment can and does drive compliance.Footnote 6 This evidence is found mostly in studies in relation to violations of various civil and political rights where commitment combined with ‘a sustained spiral of increased information, pressure and sanctions, from linked internal and external actors’ can be seen to lead to positive change in multiple places.Footnote 7
While there is much that is contested about this research, there is widespread consensus on two issues that are of key importance to this article. One important finding that has emerged is that commitment to international human rights norms does not by itself lead to compliance. Bridging the gap between commitment and compliance is a complex process. Due to the UN’s weak institutional mechanisms for the enforcement of human rights protection, it largely has to rely on dialogue and persuasion. This has led key UN bodies to utilise what scholars have termed orchestration; the engagement of intermediaries to help with its monitoring and enforcement functions as a way of increasing compliance with human rights treaties.Footnote 8 There is widespread support for the idea that international and domestic civil society organisations are critical intermediaries in situations where states do improve their human rights performance.Footnote 9 This is because international and domestic NGOs are vital actors in sustaining the various techniques of dialogue and persuasion that might potentially lead to compliance with international human rights norms.Footnote 10 Their role is therefore scrutinised carefully in the case studies presented in this article.
A second important issue relates to the inherent problems and limitations of measurement of human rights performance in terms of both its scope and methodological shortcomings. In terms of its scope, most of the measurement that has taken place relates to compliance with a fairly narrow range of civil and political rights by state actors. There is very little systematic empirical research in relation to the majority of human rights norms, most notably economic, social, and cultural rights.Footnote 11 Nor is there significant research in relation to non-state actors, most importantly multinational corporations, where the literature has largely been concerned with the conceptual issue of who has human rights responsibilities, rather than how, and whether, those responsibilities are meaningfully enacted.Footnote 12 But recent scholarship has argued that ‘the relevance of human rights is no longer regularly challenged within the business sector’.Footnote 13 As a result, the move from commitment to compliance in relation to corporations has started to become the subject of serious scholarly debate. This article contributes to broadening the scope of the measurement literature by examining a broader set of rights, (with a particular focus on economic, social and cultural rights) and by comparing the dynamics of the transition from commitment to compliance with regard to states and multinational corporations.
In terms of methodothological shortcomings, there is acknowledgement from most scholars working in the field that there are limitations in the research that attempts to measure the links between commitment and compliance, for a variety of reasons. Obtaining reliable data sources in an environment where perpetrators are likely to hide abuses and those collecting data have limited resources is problematic, and leads to questions over the reliability of data that is collected.Footnote 14 Most major sources of human rights data are compiled by a limited group of Western organisations and governments, and this may undermine the credibility of the measurement process in the eyes of some actors.Footnote 15 When data is aggregated, compared and ranked, problems of reliability and comparability of different data sources are exacerbated, and questions are raised about the degree to which we can draw meaningful and robust conclusions about the performance of individual countries and the human rights system as a whole.Footnote 16 We therefore recognise that methodological approaches in any human rights measurement activity, including those examined in this article, require careful interrogation.
This article engages with the above debates about both compliance and measurement by analysing an under-explored aspect of the human rights measurement phenomenon. It examines under what conditions benchmarking and associated measurement initiatives within the UN system could and should play a role in promoting compliance with international human rights norms. It is organised around a comparative analysis of two cases studies. First, we analyse measurement and benchmarking initiatives promoted by key UN actors in relation to states’ human rights performance, with a particular focus on economic, social, and cultural rights. Second, we analyse a measurement and benchmarking initiative in relation to corporate human rights performance. This latter initiative flows directly from the UN Guiding Principles (UNGPs) created by John Ruggie, the UN Special Representative of the Secretary-General on Business and Human Rights. In both case studies, benchmarking emanates from measurement as a form of international governance (Type II, to use the typology of the editors of this Special Issue)Footnote 17 led by the relevant UN actors in relation to compliance with human rights norms. But in both cases, the subjects of the measurement process play a key role, as measurement is embedded within self-reporting processes whereby it is states and corporations respectively who have primary responsibility for providing the empirical data.
The method employed in both case studies is the same. First, we review official documentation and statements produced by key UN actors to understand the motivations and expectations of the measurement processes and benchmarking efforts that have been developed. We then present our own empirical analyses of those processes in order to evaluate the extent to which expectations have been realised. Throughout we engage with relevant scholarly literature that supports or contests our claims and those of relevant UN actors. We conclude the article by drawing comparisons between the two case studies.
In our first case study on states, we find benchmarking practices operating within a system of self-reporting and very much tied to the creation of an extensive range of quantitative and qualitative indicators. We find tensions here between the desire for indicators and benchmarking to strengthen compliance with international human rights norms, and technical and ideological misgivings about the role of quantitative measurement and comparative assessment in a human rights context. Ultimately we find that this tension has been resolved in favour of a very limited role for indicators and benchmarking in the overall monitoring of states’ human rights performance. But we also find that international and domestic civil society organisations have been an integral part of the measurement efforts that have occurred.
In our second case study on corporations, we find that there is a great discrepancy between the expectations of benchmarking and what has happened in practice. Ruggie has argued that the UNGPs themselves create a set of authoritative benchmarks against which progress in relation to corporate human rights performance can be assessed. Benchmarking is framed here in terms of a much more expansive and less quantitative process. A central component of the Guiding Principles is, again, self-reporting of human rights performance by corporations. We argue that the benchmarks associated with this self-reporting process are, to a far greater extent than for states, in danger of leading to superficial legitimation of corporate human rights performance, without evidence that substantive action has taken place. We find that a failure to realise the importance of domestic and international civil society organisations is a crucial flaw in the way the UN Guiding Principles have been operationalised.
States and human rights: Indictors and benchmarks
The development of indicators and benchmarks by UN Actors
Efforts to develop measurement tools in relation to international human rights norms within the UN human rights institutions started with a focus on state responsibilities for economic, social, and cultural rights (ESC rights). ESC rights were seen as marginalised within the UN system compared to civil and political rights. They did not have the same ‘infrastructure for their advancement’ (including a process for individual petition or even, for many years, a monitoring body), and were often viewed as ‘non-justiciable’ and lacking in normative strength.Footnote 18 There was a feeling that, in order to advance ESC rights, the international community would need a more robust measurement infrastructure using better statistical data in order to capture more accurately a state’s progression towards long-term goals on issues like health and education.Footnote 19 Indicators were seen as an important tool of measurement in this endeavour that would serve a number of useful functions, including the measurement of an individual state’s progress over time (benchmarking performance of a single state) and the generation of comparative data about the performance of different countries (comparative benchmarking between states).Footnote 20
Throughout the early years of these calls for indicators and benchmarking, it was very much up to the states themselves to create and apply them. The UN Treaty Monitoring Bodies (TMBs) were only involved in reviewing the practice that developed.Footnote 21 This was the result of a light-touch approach embedded in a system where the TMBs require states to submit periodic reports (every four or five years). Once reports are received, the TMBs, with input from NGOs, may request additional information from the state delegation during meetings in Geneva. The committees then issue ‘concluding observations’ in which they highlight areas of concern and make concrete recommendations for action.Footnote 22 It was primarily through this procedure that states were asked to make use of the indicators and benchmarks they had developed.
By the turn of the second millennium, however, the Committee on Economic Social and Cultural Rights (CESCR), which was charged with monitoring these rights, was arguing that a state’s failure to monitor could in itself amount to a ‘violation’ of human rights.Footnote 23 Over time, there was also recognition of the need to develop a more systematic approach to the development of indicators. The World Conference on Human Rights in Vienna in 1993 and the subsequent Vienna Conference Declaration and Programme of Action addressed this. It concluded that, ‘to strengthen the enjoyment of economic, social and cultural rights, additional approaches should be examined, such as a system of indicators to measure progress in the realization of the rights set forth in the International Covenant on Economic, Social and Cultural Rights’ (para. 98).
Attempts to create a systematic approach to the use of statistical information and indicators in the UN system can be officially dated from a request by the inter-committee meeting of treaty bodies in June 2006.Footnote 24 The Office of the United Nations High Commissioner for Human Rights (OHCHR) then formally started work on a more coherent conceptual and methodological framework through which human rights indicators could be created and utilised for measuring the performance of states in complying with their human rights obligations.Footnote 25 This work culminated in the production of a report in 2012 entitled ‘Human Rights Indicators: A Guide to Measurement and Implementation’.Footnote 26 This report is the key text in setting out current thinking about the use of statistical information, indicators, and benchmarking within the UN human rights system.
Between forty and fifty ‘structural, process and outcome indicators’ are elaborated in the report in relation to a range of rights protected by core UN treaties. Structural indicators capture the commitment or intent of the state in meeting its human rights obligations (for example, relevant international human rights treaties signed and ratified by the state, National Plans of Action created, etc.). Process indicators relate to the efforts that states have made to transform their commitments into protection and promotion of relevant rights on the ground (for example, coverage of targeted population groups under public programmes; human rights complaints received and the proportion addressed). Outcome indicators capture the results in relation to relevant rights (for example, reported cases of miscarriage of justice and proportion of victims who received compensation within a reasonable time; educational attainments by targeted population group).Footnote 27
Ultimately, the objective of this conceptual framework, the report argues, is to encourage ‘a practical, transparent and structured approach for a comprehensive translation of the human rights standards into concrete, well-defined, contextually meaningful indicators that help in the promotion and implementation of human rights’.Footnote 28 However, the report also makes it clear that the choice of which indicators to apply in any given scenario will be up to the users ‘in light of their objective and national context’.Footnote 29 This tension between comprehensiveness (between forty and fifty indicators for each human rights norm) and selectivity (states and other relevant actors can decide themselves which indicators to utilise) is symptomatic of the tensions at the heart of discussions about the proper place of measurement frameworks within human rights reporting processes. We return to this issue below.
Similar tensions are seen in OHCHR’s perception of the way in which indicators are used within the monitoring of state compliance with international human rights norms. OHCHR claims that indicators have ‘frequently’ been used in state party reports to the UN TMBs, as well as in the concluding recommendations of these bodies to states. On the other hand, there are currently no suggestions by any UN human rights body that indicators will play anything other than a minor role in the practice of reporting by states and monitoring by UN TMBs. The OHCHR 2012 report states that ‘qualitative and quasi-judicial assessments by independent human rights experts will continue to be the cornerstone of human rights assessment and monitoring, particularly for complex human rights issues’.Footnote 30
There are a number of interlinked reasons for playing down the role that indicators can and should play in the state reporting process. The most prominent within the OHCHR report is that indicators are unable to reveal the full complexity of a state’s human rights performance, even in relation to a specific right, and so remain a supportive and complementary aid to the process of making expert legal judgments about a state’s human rights performance.Footnote 31 Such arguments are echoed by commentators who fear that the use of indicators will create an audit culture that is inconsistent with the fundamental ethos of human rights and their protection.Footnote 32 Other commentators have raised related concerns that over-reliance on indicators in making a judgment about the human rights performance of states will lead to penalising countries which are concerned about human rights and are trying to tackle the problems, as opposed to countries which are determined to suppress them (and so would be less likely to report on them).Footnote 33
A second set of concerns arises from the resource constraints that many countries will face in measuring their own human rights performance. This is recognised within the OHCHR report, as well as in wider debates about the problems of the TMB process.Footnote 34 It is particularly problematic where evidence which must be collected to feed into the indicators requires extensive effort, for instance because it must be aggregated from a large statistically-representative sample of the population. This is very difficult for many developing countries to achieve, who, in addition to very limited budgets, will be affected by resource constraints such as poor rural infrastructure and lack of trained officials.Footnote 35
An even more limited role is envisaged with respect to benchmarking in relation to these indicators. There is no intention of using human rights indicators to create a composite measurement framework of indicators to rank or benchmark countries according to their human rights performance. It is argued that the complexity of human rights is such that the methodological issues faced in such a ranking exercise would be monumental.Footnote 36 There are also more deep-seated concerns that it would be inappropriate because human rights are ‘absolute standards that all societies have to strive towards; this aim cannot be diluted by creating relative performance benchmarks based on cross-country comparisons’.Footnote 37 Any comparison of state performance in the foreseeable future across countries would therefore be on the basis of comparison in relation to ‘a few specific human rights standards at a time, such as the right to education or the right to life or aspects of these rights (e.g. literacy rates, reported disappearances), and not the entire gamut of human rights.’Footnote 38
Despite the limited role envisaged for indicators and benchmarking within the UN TMB system, the use of indicators within the treaty monitoring process is envisaged by OHCHR as having a number of potential benefits. They could potentially streamline reporting processes, make them more transparent and effective, and enable NGOs to ‘exercise more effective oversight’ of human rights. This could lead to improved follow-up of the recommendations and concluding observations of the TMBs and other key human rights actors.Footnote 39 Could the conceptual framework of indicators created by OHCHR therefore play a significant role in increasing compliance with international human rights norms, or will the limitations, as envisaged by key UN actors, prevent this?
Self-reporting: Indicators and benchmarks in practice
The above analysis demonstrates the inherent tensions of OHCHR’s approach to the use of indicators within the UN system: comprehensiveness versus selectivity, frequent use versus technical limitations that constrain usage, and aspirations that indicators might increase compliance with international human rights norms versus ideological misgivings about the appropriate place of comparative measurement within the international human rights framework. We undertook an empirical study of how human rights indicators and benchmarking are currently being utilised within the UN TMB system in order to understand how these tensions play out in reality.
Our study analysed the reporting process in relation to African countries (where resources are likely to be scarcest) reporting to the Committee for Economic Social and Cultural Rights (CESCR) (where calls for measurement tools have been strongest) since 2006 (when OHCHR started its work aimed at trialling the use of indicators). It was based on an analysis of all relevant materials available through the OHCHR database with respect to the reporting process of CESCR.Footnote 40 We selected four rights (the rights to health, education, work, and social security) in order to analyse how indicators in relation to these rights are being utilised. A framework of between forty and fifty structural, process, and outcome indicators have been created for all four of these rights by OHCHR. Indicators for the right to health and education are more likely to complement other measuring activity required of states, whereas the right to work and social security are more likely to require bespoke action. In particular this can be seen from the fact that health and education have seventeen and six Millennium-Development-Goal (MDG)-related indicators, respectively, whereas the right to work and social security have only two and one, respectively.Footnote 41 The full results of our analysis are available online.Footnote 42
Twenty-one reports were produced by states over the period 2006–14.Footnote 43 In 86 per cent of cases, we found evidence that countries were reporting on all four of the rights that we were examining.Footnote 44 However, we found that states’ use of indicators varied significantly between the different rights. Reporting on the right to social security was almost exclusively undertaken in narrative form, with only isolated examples of indicators being utilised.Footnote 45 Indicators were used a little more often in relation to the right to work, occurring in 24 per cent of cases, but were mostly limited to three (the number of employed persons in the private sector and public sector; the number of jobs created annually; and the overall unemployment rate). Indicators were used more extensively in relation to the rights to health and education, but were still generally limited to five or six core indicators.Footnote 46 Maternal health indicators were most frequently used, with 76 per cent of states utilising at least one of these during the reporting process.
Unsurprisingly, benchmarking of performance was never undertaken by states in relation to other states’ performance. In our sample, 45 per cent of states benchmarked at least one indicator in relation to their own previous performance, but no state did this consistently. We were also interested in whether states were only benchmarking their performance when it was improving (a positive performance bias). However, this was not generally the case. For instance, in its report, Cameroon noted declines in the number of employers who observed health and safety requirements, granted leave to employees, encouraged the creation of unions or promoted the right to strike. Similarly, the Democratic Republic of Congo reported improvements in employment rates but declines in life expectancy, maternal birth mortality rates, and the number of students graduating.Footnote 47
CESCR engaged with states over their use of indicators, or failure to use them, in a number of different ways. In its formal requests to states for more information once state reports had been submitted, they often asked countries to provide more data in relation to the OHCHR indicators. This could involve asking countries for new data, for more information on indicators already submitted, or to explain how legislative and other changes were affecting data that had been provided. However, there was no comprehensive attempt by CESCR to address all the indicators created in the OHCHR framework. Rather, they engaged sporadically and often did not request more information in relation to any indicators associated with the all the four rights analysed (see Table 1). In their Concluding Observations, CESCR are often in a situation where they are restricted to demanding better data from states to allow any form of measuring and benchmarking of individual state performance to take place. For instance, in the concluding observations for Benin, the Committee questioned the lack of aggregated and updated statistical data, which prevented it from accurately assessing the state’s implementation of the provisions of the Covenant.Footnote 48 Unsurprisingly, given the wariness expressed in the OHCHR report with respect to benchmarking between states, we found no examples of benchmarking by CESCR of one state’s performance against another.
Illustrative table showing where CESCR has asked for more data in relation to indicators associated with the four rights analysed from 2012–14
Finally, we investigated the role of NGOs in relation to the reporting process. Based on the materials catalogued in the OHCHR database on the monitoring process, NGOs provided written reports for 59 per cent of the cases in our study.Footnote 49 However, the number of NGOs who provided reports in relation to each state varied significantly. For instance, in Egypt, where there is a long tradition of civil society engagement, 11 groups submitted some form of shadow report, while in relation to Angola’s 2008 report there was only a single factsheet from the Centre for Economic and Social Rights. In some states, such as Togo, a group of NGOs created a shadow coalition report.Footnote 50
It is widely recognised that NGOs lack resources and training and this reduces their capacity to produce credible data.Footnote 51 However, there were numerous examples of NGOs utilising a range of formal OHCHR indicators and other statistical data to augment, challenge and disaggregate the information provided by states. For instance, Kenyan NGOs made extensive use of statistical data relating to OHCHR indicators on evidence of sexual violence against children, and to challenge the state’s reporting on this issue.Footnote 52 In the concluding report on Kenya, the TMB then challenged the government and suggested improvements on several issues raised by civil society, especially with regard to the high rate of sexual violence that had been reported by the NGOs.Footnote 53 But overall, engagement with indicators by NGOs was sporadic and there was no attempt to hold states accountable for reporting on the full range of indicators developed by the OHCHR. Instead, the examples we found demonstrate how indicators can be utilised strategically by NGOs to engage with, and to challenge, individual and specific arguments made by the state concerning their self-assessment of their human rights performance. These arguments can then be taken up by the TMB in their concluding observations. These concluding observations are then taken up by civil society organisations to campaign for better compliance with human rights norms by States in the future.Footnote 54
Overall, our case study of African state reporting on economic, social, and cultural rights found that indicators and benchmarking play a limited role in the process. Indicators are utilised by most states in reporting to CESCR, but only in relation to a select number of indicators chosen by the states themselves. Instances of benchmarking are even more sporadic, and benchmarking is never used to compare one state’s performance with that of another state. CESCR and NGOs also engage with indicators in a selective manner, focusing on a very limited number of indicators, and CESCR’s role is often limited to demanding more information. Information provided in relation to indicators by NGOs is vital in that it is often an important mechanism allowing CESCR to challenge the state’s account of its own human rights performance in its concluding observations.
Evaluating the role of benchmarking
Ultimately, ideological misgivings and concerns about technical limitations by key UN actors currently triumph over aspirations that indicators and benchmarks might increase compliance with international human rights norms. While efforts to use indicators in a systematic fashion are still in their infancy and there is some evidence of state engagement with indicators increasing slightly, embedded dynamics of the system make it difficult to envisage a significantly enhanced role in the future. The limited application of indicators and benchmarking stems, more than anything else, from the fact that states are largely left to decide how to engage with the framework created by OHCHR. Resource constraints mean that it would be impossible for many states to engage with the full gamut of indicators provided. But self-selection by states means that it will be extremely difficult to induce states to converge to any form of common practice and to systematically use these measurement tools as a mechanism for driving compliance with treaties. So the system largely avoids the tendency observed in other global governance initiatives; for benchmarking to have the effect of ‘legitimating and promoting the recent histories and contemporary activities of Western states’ and producing an overall ‘performance score’ that may have significant legitimacy issues.Footnote 55 At the same time, selective engagement by states with indicators, together with limited oversight and comparative analysis by the TMBs, combine to make it very unlikely that this limited form of benchmarking will significantly induce compliance of States in the way that has been observed with other benchmarking initiatives.
To the extent that there is any influence on state behaviour, the role of NGOs is important, in providing indicators that augment, challenge and/or disaggregate the information provided by the state.Footnote 56 Given the limited resources and in-country knowledge of the TMBs, this role is absolutely vital to the reporting process, as highlighted in the broader literature on the UN TMBs, and in the literature on civil society engagement with global governance mechanisms more generally.Footnote 57 NGOs are also then of critical importance in utilising the recommendations made by CESCR – including for state improvement in relation to particular indicators, and in relation to the collection of additional data – to push governments in particular countries to take action and improve the underlying human rights situation.Footnote 58 CESCR are in effect therefore utilising NGOs to play a key orchestration role.
Human rights and transnational corporations (TNCs)
The UN Guiding Principles on Business and Human Rights (UNGPs)
The UN Framework for addressing human rights and business has a very different history from the state-based system described above. The UN Framework and Guiding Principles on Business and Human Rights were created by United Nations Special Representative of the Secretary-General on Business and Human Rights, John Ruggie, over the last decade. Ruggie created the UN Framework (henceforth, ‘the Framework’) to deal with the failures and inadequacies of existing international governance mechanisms for effectively addressing human rights and business. Numerous multi-stakeholder, self-regulatory initiatives, and cross-border accountability mechanisms have attempted to hold businesses accountable for their human rights conduct. But these initiatives have, for the most part, exhibited sizeable weaknesses and failings and, overall, have left significant accountability gaps in terms of overall human rights protection.Footnote 59 The UN Framework and Guiding Principles sought to tackle this issue.
The Framework addresses what states and business enterprises need to do to ensure business respect for human rights. It is augmented by the UN Guiding Principles on Business and Human Rights (the UNGPs), which work towards the implementation of the Framework and provide more detailed guidance as to how it should operate.Footnote 60 The Framework and UNGPs have been universally endorsed by the UN Human Rights Council, leading Ruggie to claim that they are ‘the authoritative global reference point for business and human rights’.Footnote 61
Instead of treaties, which states sign and ratify, Ruggie has instead created a series of principles, which require no formalised process of legitimisation by companies (other than a self-declaration of adoption in whatever form the corporation sees fit). The principles are intended to allow for measurement and comparison of the human rights performance of corporations, as well as the patchwork of different mechanisms for holding TNCs accountable for their human rights performance. In commentating on the UNGPs, Ruggie has repeatedly described them as ‘benchmarks’. For instance:
In short, where discord among states, businesses and civil society had paralyzed prior international efforts to encompass corporate conduct within the international human rights regime, there is now a common platform on which to build and authoritative benchmarks against which progress can be assessed.Footnote 62
The UN Working Group on Business and Human Rights (the UN body which is responsible for overseeing the implementation of the UNGPs) has also commented that they comprise, inter alia, ‘a set of benchmarks to enable stakeholders to assess and promote the respect of business, and its accountability, for human rights and engage with the issue’.Footnote 63
Ruggie and the UN Working Group appear to be using the term ‘benchmarking’ here in a manner that is very different from the way the term is used within the state-based system, and is more analogous to its use in scholarship relating to business management. In his discourse on the genealogy of benchmarking, Anna Yeatman described benchmarking as a process whereby companies undertake a comparative exercise to ensure consistency of products between national companies with a view to improving the quality of products and processes.Footnote 64 Benchmarking was taken to mean the improvement of an entity’s performance that is sought through comparison with an external point of reference or simply best practice.Footnote 65 Benchmarking, in this sense, is not necessarily linked to the development of statistical data or indicators but, rather, can be used in relation to a set of narrative principles that describe normative standards of conduct required by corporations.Footnote 66 Despite this fundamental difference in conception, Ruggie and the UN Working Group’s aspirations for their ‘benchmarks’ are analogous to those of UN actors within the state-based system: that they will provide a mechanism by which we can assess or measure progress being made by corporations in complying with international human rights norms.
The Framework and UNGPs are based on three underlying principles: the state duty to protect against human rights abuses by third parties, including business; the corporate duty to respect human rights; and the need for more effective remedies. In this article, we focus particularly on the second principle of the Framework, the corporate duty to respect human rights and the UNGPs that elaborate on the content of that principle. This is where data collection and measurement of performance in relation to corporations takes place. At the heart of this principle is the process of human rights due diligence (HRDD) which is, in effect, a self-reporting process analogous to the state-based system of reporting discussed above. Ruggie defines HRDD as ‘the steps a company must take to become aware of, prevent and address adverse human rights impacts’Footnote 67 and sets out his conception of what it entails in considerable detail in five of his Guiding Principles. These principles are effectively therefore the benchmarks that should be met in relation to the HRDD process.
The HRDD process requires that corporations consider their own potential and actual human rights impacts, and report on their findings. The principles include a prioritisation process so that companies undertake assessments where human rights risks are greatest (Principle 17); consultation with stakeholders and otherwise gathering evidence of potential or actual impacts (Principle 18); analysis of impacts in relation to recognised international human rights standards (Principle 18); integration of findings of HRDD within all relevant internal processes and functions of the corporation (Principle 19); monitoring or tracking the effectiveness of their response to impacts identified (Principle 20); and communicating with relevant stakeholders about how they are addressing their human rights impacts (Principle 21).Footnote 68
What we have through these principles is the sketching out of a process of investigation and reporting whereby the company investigates, analyses, and reports on its own performance in relation to human rights norms and standards. The sense that this is essentially a self-reporting process is reinforced by the primary value that Ruggie sees in the process. He argues that HRDD can transform a company’s relationship with human rights from one based primarily on ‘naming and shaming’ by external actors for corporate failures to respect human rights to one of ‘knowing and showing’ by companies regarding their own human rights performance.Footnote 69 The UNGPs therefore create a self-reporting process where the corporation is playing the same central role that the state is within the UN TMB system. But here we find a very different practice emerging, with a very different role for benchmarks and very different types of third party engagement with the system.
Self-reporting: UNGP benchmarks in practice
We reviewed the policies of 225 leading multinational corporations for this article. Companies were chosen on the basis of their size and their likelihood to be encountering human rights issues. We therefore included all members of the Equator Principles and Thun Group (banks), the International Petroleum Industry Environmental Conservation Association (oil and gas), and the International Council on Mining and Minerals (mining). Members of these associations were chosen because of the associations’ corporate social responsibility (CSR) influence in sectors that have potentially large human rights impacts worldwide. We also included the twenty largest companies ranked in the Forbes 2000 list in the fields of apparel, technology, and food and beverage, mining, oil and gas and finance. These are all areas of business where human rights concerns are particularly prevalent.Footnote 70
We searched the online resources of all the companies identified, using a variety of ways to ensure we found all human rights related commitments of the companies in question.Footnote 71 The full results of this review are available online.Footnote 72 We found that 46 of the firms had already explicitly adopted the Framework as part of their human rights policy, only three-and-a-half years after the publication of the UNGPs. This number can be expected to increase significantly in the coming years as a result of the adoption of the UNGPs in a range of international guidelines, standards, and frameworks. They are referenced or adopted, inter alia, in leading standards for responsible business conduct (for example, the OECD Guidelines for Multinational companies); key ethical investment indices (for example, the Dow Jones Sustainability Index and Microsoft’s ethical investment company, Maplecroft); important social standards for investment (for example, the European Investment Bank’s social and environmental performance standards); leading reporting initiatives (for example, the GRI Sustainability Reporting Guidelines); knowledge-sharing initiatives (for example, the Global Business Initiative of the Global Compact); and in sectoral codes and principles aimed at improving standards in particular industries (for example, the Voluntary Principles on Security and Human Rights, the International Code of Conduct of Private Security Sector Contractors, the Responsible Jewellery Council Codes of Practices, etc.).Footnote 73
Ruggie and a range of other actors have been keen to emphasise the success of the UNGPs as authoritative benchmarks.Footnote 74 Success in this sense is measured in terms of the adoption of the UNGPs, or reference to them, by corporations themselves, as well as by a wide variety of other bodies that are involved in setting standards and/or monitoring human rights performance by TNCs. But this extensive adoption of and/or reference to the Framework and Guiding Principles by a wide range of actors is not, in itself, any real measure of their success in terms of actual improvement of human rights performance; it is only proof of commitment not compliance.Footnote 75 In fact, our review found that, in most instances, publicly available references to the UNGPs by businesses and other key stakeholders was extremely superficial. The vast majority of corporate adopters limit themselves to a general statement of commitment with no detailed information about how they are being implemented. In terms of other stakeholders, public information relating to the incorporation of the guidelines is generally extremely sparse, and can even be limited to a link on the website to the Ruggie Framework.Footnote 76
When we focus on the narrower issue of HRDD, in our survey of companies, only 28 out of the 46 companies who commit to the Ruggie Framework (61 per cent) explicitly and unambiguously state that they are undertaking HRDD and of those companies, only one company (Nestlé) has any kind of report on the results of their HRDD process.Footnote 77 Many companies merely give vague statements that they are undertaking HRDD.Footnote 78 A number of other companies do provide some (mostly very limited) detail about their HRDD process, and some limited insights into the methods that they have adopted, but no actual reports of their human rights performance.Footnote 79 Where some detail is provided, there appear to be wide divergences in practice, with HRDD processes sometimes integrated into risk assessment processes, compliance monitoring, company audits, and/or impact assessment processes. The vast majority of practice is therefore occurring in private, with only the company itself, and, in some cases, an external evaluator chosen by the company, able to engage with the divergent practice that is emerging.Footnote 80
Nestlé’s report, produced in November 2013, represents the first time that a leading multinational company has publicly commented upon its HRDD process in its operations and supply chain. The report is mostly a commentary on the methodology of their HRDD process, with a summary of key findings and identification of key challenges. The report relates the results of the HRDD process that was undertaken with regard to its operations in seven countries where human rights risks were considered greatest. Even in these countries, the report makes clear that there were significant challenges, including the difficulties of adequately investigating all of Nestlé’s operations in each country, engaging with particular stakeholder groups (women, the elderly, disabled people, migrants and children), and tackling the full range of issues demanded by a comprehensive human rights approach.
Evaluating the role of benchmarking
Our study of practice in relation to the UNGPs raises some significant concerns. HRDD is clearly an integral part of the UNGPs. Adoption of the UNGPs means that companies are required to undertake the self-reporting process that is represented by HRDD. However, only 61 per cent of companies who have publicly committed themselves to the UNGPs even identify that HRDD is taking place. The publicly available information about the actual results of HRDD is limited to one company report, summarising their experience of the process, rather than directly reporting its results. We know from the state-based system that for a country to report on its overall human rights performance, even to a single treaty body, is a complex process that is resource intensive and produces results of very variable quality.Footnote 81 Nestlé identified significant challenges in fundamental aspects of the HRDD process in their seven-country study. But multinational corporations are able to claim, as a result of adoption of the Ruggie Framework, that they are undertaking HRDD with respect to all their operations globally, without providing any public information about what they are doing (for instance, Microsoft claim: ‘We assess the human rights impacts of all our operations on an ongoing basis’).Footnote 82 As a result, there is significant danger of superficial legitimation of corporate human rights performance, without evidence that substantive action has taken place.Footnote 83
On their own terms (that is, using benchmarking in the business management sense), the UNGPs appear to be failing as a set of benchmarks because they do not enshrine a process that, at the very least, creates conditions under which appropriate engagement of third-party actors can occur.Footnote 84 First, there is no equivalent to the TMBs in the corporate self-reporting process. The ‘guardian body’ of the UNGPs is the UN Working Group on Business and Human Rights, but they have not been given the powers or the resources to undertake any form of review of corporate performance. This means that the limited powers of the state-based system to orchestrate compliance with human rights norms have been entirely removed within the corporate one. Second, the UNGPs have not enshrined transparency as a fundamental ‘benchmark’ of the system.Footnote 85 As a result, and because the methodology and results of the assessments are not generally made publicly available, there is no way in which individual assessments can be scrutinised by NGOs in the way that they are in the state-based system.Footnote 86 The vital role that we know NGOs play in inducing compliance where there are weak institutional mechanisms has therefore been lost. Nor is there even the possibility of peer-to-peer learning, which is professed to be a strength of other UN initiatives on business and human rights, since the majority of practice is occurring within individual corporate silos.Footnote 87
At the same time, HRDD is increasingly being utilised by various actors to confer benefits on corporations who have undertaken it. Two examples are provided here. First, the Dow Jones Sustainability Index, which incorporates the Ruggie Framework and Guiding Principles, uses HRDD as part of its assessment process for determining sustainable investments, but only requires companies to make a brief statement pertaining to where they see HRDD as required in their operations.Footnote 88 Second, the US government requires HRDD to have taken place in relation to US companies that want to invest in Myanmar (Burma) with particular focus on the energy sector. In order to comply with the US Department of State Reporting Guidelines on responsible investment in Burma, corporations must disclose information on their policies and procedures, including those relating to human rights, worker rights, anti-corruption, land acquisition, grievance mechanisms, and the environment.Footnote 89 Only Coca Cola, of the companies who have so far reported, provided any detail of the HRDD process they undertook, and this was limited to a summary of findings that was insufficient to allow any kind of independent assessment of the robustness of the process undertaken.Footnote 90 These examples raise serious concerns. In the state-based system there was a refusal to engage in comparative benchmarking of performance between states, and yet in relation to corporations an analogous self-reporting process is used as the basis for the crudest possible form of ranking systems (companies that receive benefits, and those that do not from government bodies, ethical investment indices, etc.).
The failings of the HRDD process also highlight the problem of conceiving of benchmarking in the business management sense when addressing global governance challenges such as the human rights performance of corporations. Benchmarking efforts within self-reporting processes must create sufficient specificity to allow for confidence that corporate efforts are meaningful and worthy of comparison. Otherwise there is no way of gauging the extent to which corporations are actually moving towards compliance with human rights norms. Narrative principles that simply focus on process rather than provide guidance in relation to expected results do not seem well placed to capture these movements. Could more specific and quantitative indicators (such as those developed through the state-based reporting process) perform a role in directing corporations towards the type of human rights issues they should be measuring? Would benchmarking then stand on firmer foundations? Interestingly, it is clear from the limited information that is publicly available that companies who are making serious attempts to develop HRDD processes are creating their own sets of indicators against which to measure their performance.Footnote 91 However, the development of these indicators is happening in private, and the results are likely to be disparate and varied because insights about strengths and weaknesses will not be shared, and companies will make their own individual choices driven by internal factors. Such concerns are heightened by a survey of HRDD practice among multinational corporations, which found great variation between different companies.Footnote 92
To avoid this fragmentation and disorder, might there be a case for relevant UN bodies re(inter)nationalising the production of indicators, as happened in the state-based monitoring system? Such a process should not be undertaken lightly. Would corporations acquiesce to a framework of indicators that constrained their own measurement activities? Furthermore, in terms of feasibility, the creation of a measurement framework by which to seriously and holistically interrogate corporate human rights performance would be a very complex and multifaceted process. The non-profit research and policy organisation NomoGaia has been a pioneer in undertaking human rights evaluation of corporate performance. They have produced four human rights impact assessments of corporate footprint projects, which have, at times, produced transformative results on the ground.Footnote 93 In undertaking these assessments they have produced a toolkit, which contains over 300 ‘topics’ that need to be catalogued during the assessment process.Footnote 94 This type of measurement framework demonstrates the level of specificity and complexity involved in such endeavours, and should be seriously studied, as it could create the basis for a much more concrete shared understanding of what a self-reporting process should entail than the high level principles created by Ruggie that currently underpin corporate self-reporting practice.
The work by NomoGaia also reinforces the lessons that flow from the Nestlé study. It is clear that robust reporting on a corporation’s human rights impacts has only developed to a stage where it is suited to an assessment of particular activities in specific physical locations.Footnote 95 Assessing the overall human rights performance of a company is a completely different scale of endeavour, for which tools need to be developed and tested iteratively over time. As it is, HRDD processes in relation to aggregated human rights performance by large multinational corporations that are not subjected to public scrutiny are feeding into important decisions by states, banks, and a range of other actors about who is ethical or unethical, investing responsibly or irresponsibly. This is a dangerous phenomenon.
Conclusion
Our comparative examination of the aspirations and results of two different measurement initiatives within the UN system has demonstrated that benchmarking and related measurement activity are not currently being used by UN actors to seriously address ‘the compliance gap’ that has been identified in the academic scholarship. But the rationales for this limited impact are very different in our two case studies.
With respect to states, aspirations are limited as a result of an intentional decision to restrict the role of indicators and benchmarks because of ideological misgivings and technical concerns. The results are therefore largely to be expected. With respect to corporations, the aspirations for the benchmarks created by the UNGPs are much higher; they are intended to play a central role in increasing corporate compliance with human rights norms. The results are therefore of far greater concern. In the latter case, benchmarks, particularly in relation to HRDD, may be disguising rather than illuminating the steps that are required in order to translate commitment into compliance.
The absence of binding treaty commitments for corporations has meant that comparative analysis of the conditions under which states and corporations comply with international human rights norms has been very limited, albeit with some evidence this is changing.Footnote 96 Future academic research should involve much greater cross-fertilisation of ideas. For instance, the extensive academic research on the conditions under which states comply with international human rights norms highlights the importance of relevant UN bodies orchestrating domestic and international NGOs to help in the monitoring and enforcement of those norms. There is no reason to believe that this is any less true with respect to corporations. But our case study has demonstrated that NGOs are completely frozen out of the picture when it comes to the human rights self-reporting process undertaken by corporations in relation to the UNGPs. Comparative analysis clearly highlights this issue.
Future comparative scholarship should also consider the plurality of conceptions of ‘benchmarking’ that are in operation with respect to states and corporations. We have suggested that this plurality occurs as a result of the differing disciplinary origins of the term in the two case studies. A shared conceptual understanding of what the benchmarking process represents across the UN human rights system is an important bedrock for future endeavours in this field. Interestingly, the more quantitative conception of benchmarking, closely tied to the development of indicators, found within the state-based system has been the subject of much internal debate and scrutiny. This is driven, it seems, by fears of the ‘quantitative’ as failing to reveal the full complexity of human rights, as well as concern that such an approach is open to manipulation by actors who will try to ‘play’ the system. As a result, the role of ‘the quantitative’ is tightly circumscribed within a system that relies primarily on the judgment of experts expressed in narrative from.
On the other hand, the conception of benchmarks as narrative principles, unrelated to quantitative measurement, which is found in the corporate-based system has been subjected to very limited scrutiny within relevant scholarship. This is despite the fact that, as this article has demonstrated, the latter conception of benchmarking has been extremely problematic. The different levels of scrutiny and attention accorded to these two conceptions surely in part relate to the nature of the human rights community itself – a community of social scientists more comfortable with narrative than numbers. Does this suggest the need to better engage with a broader range of academic expertise (mathematicians, statisticians, economists, etc.) who might be able to unlock the potential of benchmarking and other quantitative measurement processes and thereby allow them to contribute more seriously to reducing the ‘compliance gap’ with international human rights norms?
Suggesting more serious engagement with quantitative measurement raises more fundamental questions about the proper place of benchmarking efforts within broader global governance initiatives. Concerns have been raised in this Special Issue about how benchmarking initiatives can be utilised by transnational actors to make comparative evaluations of performance which increase public and private power over distant entities, without due attention being given to the complexity of the underlying phenomena being measured, and to the limitations inherent in the benchmarking process.Footnote 97 Such concerns appear particularly acute where the benchmarked entity has limited control over the benchmarks which are utilised, and how data is collected and/or interpreted; at the same time the consequences of poor performance for that entity are often potentially serious (for instance, as in relation to credit ratings produced by international credit rating agencies or the World Bank’s Worldwide Governance Indicators).Footnote 98
In both of our case studies, however, the power dynamics of the system operate very differently. Attempts to measure human rights performance occur within self-reporting initiatives, where very little power lies with the UN actors who have instigated the measurement efforts. Benchmarking, in this context, is a mechanism by which attempts are made to constrain states and corporations; to make them report in a manner that allows their performance to be more effectively scrutinised. The greatest concern in this scenario is that indicators and benchmarks are ineffectual (or even act as a smokescreen for poor performance) because they are not sufficiently demanding, nor are they consistently and transparently applied.
Attempts to develop and expand robust benchmarking practice (tied to indicators which are actually capable of measuring performance, and appropriate financial assistance for actors who need it) look less troubling in relation to these self-reporting initiatives. This is because they are tools that can potentially be used to pressurise relatively powerful actors (states and corporations) into taking important obligations more seriously, and make it harder for them to ‘play’ the system. Development and application of robust and transparent indicators and benchmarks also potentially encourages more meaningful third party engagement. All of this suggests that views on the appropriateness of particular benchmarking efforts should be informed as much by the dynamics of the system in which they are operating as by the inherent limitations of the benchmarks themselves to operate as legitimate and appropriate tools of global governance.
