Benchmarking has proliferated transnationally as an instrument of governance. We are beginning to get a clearer understanding of its significance, its properties, and its variants. For instance, André Broome and Joel Quirk identify four distinctive components of benchmarking: simplification and extrapolation; commensuration (the development of common metrics); reification (the translation of complexity into stable quantifiable categories that are presumed to be universal); and symbolic judgment (which introduce assessments of relative performance or value into the process). They also identify four types of global benchmarking, each associated with different monitoring agents (national governments; international and regional organisations; profit-based institutions; and civil society actors).Footnote 1
This article aims to add to our understanding of global benchmarking by examining the relationship between benchmarking and the varying properties of the organisational contexts within which it operates. Benchmarking is often treated as a mechanism for ranking that is constructed by certain actors to alter the conduct of other actors. For instance, the four distinctive components of benchmarking identified by Broome and Quirk primarily refer to the components of the measuring device and the four types refer to variation in the actors which produce and manage benchmarks.Footnote 2 The connections these have with the contexts within which they operate are noted at times, for instance when the reification aspect of the benchmarking process leads to the neglect of cultural or historical context. They also point to the importance of third party users. However these are mostly treated as exogenous to the more central operations of the benchmarking process, and these and many other elements of the organisational contexts of benchmarking have not yet been analysed systematically enough.
Why should we care about the contexts of global benchmarking? I argue that understanding global benchmarking’s contexts is crucial to two issues that are most central to its significance. The first issue is whether it is effective. If a researcher sponsored by a powerful actor skillfully designs a benchmark, paying careful attention to constitutive elements such as the metrics, but then leaves these sitting in a notebook on the desk, this benchmark will be ineffective. Obtaining responses from targeted actors requires linkages between the benchmarking mechanism and the targeted actors. However unlike other types of governance, the targeted actors often cannot simply be commanded to comply with benchmarks. The benchmarks may need to activate complex networks of information, resources, and action to achieve their effects, often over long distances and numerous organisational boundaries. These networks are sustained or not sustained by the contexts of benchmarking. Where the networks are sustained by a sufficiently dense context we can say that they are embedded.Footnote 3 The importance of these networks and contexts is often obscured by an overemphasis on the power of the actors that create benchmarks, the weakness of targeted actors, or the formal properties and content of the benchmarking mechanism, such as the ranking.
The second crucial and related issue is its publicness. We should not only care about whether a governance instrument’s design allows it to achieve the outcomes associated with it, but also whether these are outcomes which contribute usefully to good global governance and are in the global public interest, solving global public policy problems, or instead are tools used by self-interested actors to enhance their own power or wealth. These self-interested actors can be public, such as when powerful states use benchmarking to coerce weaker ones, or private, such as when firms use benchmarking to gain competitive advantage. An example of the former is the creation in 2002 of the US Millennium Challenge Account, which set out 16 eligibility criteria that the world’s poorest countries were required to meet before being considered for US aid.Footnote 4 An example of the latter is Xerox Corporation, which pioneered business benchmarking in the 1970s in order to improve its performance relative to competitors.Footnote 5 I argue below that to evaluate the publicness of a benchmark it is necessary to analyse the degree to which the flows of information and action between the benchmark and its contexts are open and collaborative, with opportunities for targeted actors to shape benchmarks, or instead are marked by control, domination, and exclusion.
By identifying the importance of embeddness and publicness in benchmarking contexts it is possible to understand better the potential and limitations of benchmarking as an instrument in global governance. This article argues that there is an inherent logic promoting an expansion of publicness in benchmarking, but that this is continually limited by the difficulties associated with embeddedness. This challenges those who see benchmarks as neutral measuring devices that are helping observe target actors’ performance in a detached manner. A typical example of such ostensible detachment is the cross-national benchmarking of airports by Air Transport Research Society, which noted that ‘based on a rigorous analysis of airport cost and revenue data, the ATRS benchmarking study assessed the productivity and efficiency of nearly 200 airports and selected the most efficient airports in each of the regions, as well as the most efficient airport, overall’.Footnote 6 This article challenges those who minimise the agency of those striving to meet benchmarks and see benchmarking as primarily creating and obscuring relations of power. Tore Fougner, for instance, sees country competitiveness benchmarking by the World Economic Forum and the International Institute for Management Development as helping produce neoliberal states. Susanne Soederberg makes a similar argument about the California Public Employees’ Retirement System’s Permissible Country Index, which provides benchmarks for socially responsible investment.Footnote 7 It will be argued that a public impetus in benchmarking is evident in the history of business benchmarking, which displays ever-widening circles of engagement over time, but also in the two quite different cases of contemporary transnational benchmarking that are examined in the second part of this article.
The article draws upon actor-network theory (ANT) in analysing the organisational contexts of benchmarking. A distinctive feature of actor-network theory is the attention it devotes to the material aspects of extended chains of action. This emphasis on the characteristics of networks is not intended to suggest that interests are not important, but rather to bring out the types of connections that are needed for interactions to occur, whether these are associated with the self-interested exercise of coercive power, the positive-sum interests that rational choice theorists have extensively analysed, or quite different motivations, such as empathy.
To help identify the role of organisational contexts in benchmarking I have chosen two very different cases of transnational benchmarking to examine. The first is the benchmarking of the management of the uncertainties and risks associated with global disasters, such as tsunamis. The second is the benchmarking of performance in global supply chains. Both are analysed by examining documents associated with them, drawing on commentary about them in professional, business and scholarly publications. The cases have been chosen to maximise the variation in the types of benchmarking involved. We shall see that this variation is most crucially associated with variation in contexts rather than the properties of the benchmarking mechanisms or the power of the types of actors associated with them.
Conceptualising the organisational contexts of transnational benchmarking
Benchmarking and its contexts
Broome and Quirk have defined benchmarking as ‘an umbrella term for a wide range of comparative evaluation techniques – such as audits, rankings, indicators, indexes, baselines, or targets – which systematically assess the performance of actors, populations, or institutions on the basis of standardised measurements, metrics, and rankings.’Footnote 8 This emphasises the production of the benchmark itself. This leaves a number of other processes associated with benchmarking outside its implied boundaries in this definition. This includes the networks that carry the effects of the benchmark to the actors that it targets. Although one could use the term benchmarking to refer to the larger processes it involves, it is useful analytically to separate out the narrower properties of the benchmark itself from the larger processes that the benchmark draws on and activates. Therefore this article refers to these larger processes as the contexts of benchmarking, and retains the narrower conception of benchmarking consistent with Broome and Quirk’s specification above.Footnote 9
Rather than treating benchmarking as neutral measuring device or a tool used by powerful actors, this article treats benchmarking as a more active and autonomous relay in a set of networks that extend well beyond the interaction between the benchmark’s architects and targets. This emphasis on networks is consistent with the centrality of governance at a distance for benchmarking’s distinctiveness. One can certainly imagine a ranking system similar to benchmarking in a close hierarchical relation when a superior punishes a subordinate for not achieving a target that the former has created. However this is a very conventional form of governance, and benchmarking is only new, distinct, and meaningful as category if it involves something more: a significant degree of autonomy and discretion for those seeking to improve their performance, and governance at a distance, where the pressure to comply comes not from a superior’s direct threat of punishment, but from a pressure external to the direct relationship between the benchmark’s architect and target, such as market pressure, peer pressure, or the willingness of third parties to provide resources to the target actor.
Certain transnational benchmarking processes appear in practice to be more collaborative and horizontal than hierarchical.Footnote 10 Benchmarking in such processes is used by their architects to address their own collective problems. Benchmarking by the Organisation for Economic Cooperation and Development (OECD) can take this form when it involves the sharing of best practices among states of similar degrees of power. An example is the OECD’s well known Programme for International Student Assessment (PISA), which measures student performance in elementary and secondary grades.Footnote 11 These more democratic processes are circular, recursive, and reminiscent of the well-recognised stages of policy processes in democracies, such as agenda setting, problem identification, decision, implementation and evaluation, with the outcome of policy then feeding into the next round of the process.
In the case of this more horizontal benchmarking the stages start with the identification of the problem; followed by the identification of best practices; and then the formulation of metrics and scales (the part of the process that Broome and Quirk’s definition especially focuses on); the collection of performance data; the circulation of comparative performance data in a manner designed to elicit changes in performance; and finally revision of goals and measures used in this process. This then leads back recursively to the beginning stages of the process. In the more democratic and horizontal variants of this process each stage is accompanied by interaction between benchmarking and its contexts. The early stages involve channels or networks to bring information into benchmarking, and the later stages require networks to carry information out and activate pressures for compliance.
Analysing these interactions between benchmarks and the contexts through which the networks associated with them are sustained matches the historical shift in the study of domestic public policy to integrate analysis of policy networks outside the formal structures of government, and is consistent with the growing complexity and density of transnational governance mechanisms and the transnational communities they interact with.Footnote 12 This article’s emphasis on benchmarking contexts is complementary to Ole Jacob Sending and Jon Harald Sande Lie’s contribution to this Special Issue, which suggests that practices can precede benchmarks, and which highlights the authority relationship between benchmarks and those who are benchmarked.Footnote 13 However this article puts more emphasis on the materiality of the networks involved, rather than the more abstract qualities of relationships of authority. This recursive process is very similar to, and overlaps with, the processes that have been identified as experimentalist governance as developed and specified by Charles F. Sabel and Jonathan Zeitlin.Footnote 14 Their experimentalist governance model is more elaborate, defined, and complete as a form of governance than is benchmarking, which is a smaller and more mobile element of governance. Nevertheless both address the distinctive challenges of governance in fast-paced, horizontal, and complex settings that are common today, including in transnational settings.
Actor network theory
Actor network theory (ANT) provides a number of useful ways of understanding and highlighting the roles of these networks and how they can be embedded, which I briefly summarise here.Footnote 15
First, as previously noted, a distinctive feature of ANT is the attention it devotes to the material aspects of extended chains of action. While norms or ideas may be important, they cannot travel from one place to another without being carried physically, by a human, or a non-human object such as a paper document. Non-humans, whether constructed by humans or present independently of humans in nature, can have effects that are similar to those associated with human actors, as suggested by the ANT concept of ‘actant’ which can refer to both.Footnote 16 A key concept in ANT is translation,Footnote 17 which goes beyond the conventional meaning to include the ways that the humans and non-humans who constitute a network can modify ideas and actions as they travel through that network. The materiality of the networks and contexts through which the influence and action associated with a benchmark travel is crucial to understanding whether and how that benchmark will be effective.
The associations that must be established between actants in a network if action is going to travel over distances or humans are going to think or act in coordinated, structured ways are often fragile or unpredictable, but they can also be strengthened by being embedded in objects, such as written documents. An ‘immutable mobile’ is an object with knowledge embedded in it, such as a checklist or a schedule, which can circulate and organise humans as it travels. Digital technologies now can allow such objects to be updated and reconfigured in real time, creating ‘mutable mobiles’.Footnote 18 These features of networks are important in understanding how power can be reinforced or disrupted in them.
Bruno Latour has argued strongly against the common tendency to attribute human ideas or actions to a mysterious invisible social substance or structure, and has insisted on the importance of tracing the travels of ideas and actions in the manner noted above, which can then lead to the assembling of things seen as social.Footnote 19 As Latour puts it, ‘it is possible to remain faithful to the original intuitions of the social sciences by redefining sociology not as the “science of the social” but as the tracing of associations’. This is a contrast to diffusion models, where ideas are imagined to travel from one human mind to another often without consideration of the objects, such as documents or computer networks, which are required for this type of effect to occur.Footnote 20 It also differs from some uses of the concept of governmentality, which was developed by Michel Foucault. Governmentality is a decentred form of power that involves control at a distance, and the regulation of self-regulation, where subjects are produced who align their thinking and action with the imperatives of power.Footnote 21 While some governmentality approaches bring in mundane artefacts and objects in way that is compatible with ANT, in general the emphasis is on larger political rationalities which are often assumed to be implemented unproblematically, missing the complications that ANT highlights. As Peter Gibbon and Stefano Ponte have noted, ‘while rationalizations based on these models may be invoked in national or corporate economic strategies over long periods, it by no means follows that they are implemented on the ground or, where implemented, that they lead to material changes’.Footnote 22 Diffusion models and governmentality approaches are compatible with approaches that focus more narrowly on benchmarks without considering adequately the context of benchmarking. ANT is useful for highlighting the importance of these contexts.
In analysing power, ANT also looks to the capacity of actors to ‘enrol’ networks.Footnote 23 Power is therefore not a resource that is possessed by an actor to deploy at will. Rather it requires the alignment of existing networks towards the actor’s goal. These existing networks may have their own purposes, but may come together, sometimes by chance, to constitute a larger and more powerful assemblage. This decentralised conception of power and influence is consistent with the networks and contexts that render benchmarks effective. Barbara Czarniawski has argued for the concept of ‘action nets’, which are emergent patterns of activity that often precede in time the emergence of actors, networks, or organisations, challenging ANT’s assumption that actors and networks are preconditions for patterns of activity.Footnote 24 However, clearly actors in networks or organisations can also initiate actions. With regard to benchmarking, these points reinforce the importance of contexts for constituting successful benchmarking arrangements, while also showing that there are many opportunities for existing and emergent networks to be assembled into benchmarking arrangements, without expecting the architects of benchmarks to orchestrate everything.
Despite this decentralisation, key nodes in networks can at times exercise powerful effects over the networks as a whole. The concept of an ‘obligatory passage point’ is a point in the network through which actions must pass.Footnote 25 Examples include a scholarly reference that all articles must cite, a computer portal through which all electronic communications must travel, or an official office, through which all documents must pass. Such an obligatory passage point cannot be peremptorily assigned by a powerful actor, but instead depends on the properties of the network. A successful benchmark can be treated as an obligatory passage point.
ANT has devoted particular attention to the way that scientific laboratories, maps, or other technologies for measuring and observing, achieve their effects. The process of successive refinements, or reifications, to reduce the complexity of the world outside to a few small and manageable relationships on a piece of paper, allows those who possess this paper to rearrange these relationships and then bring them back out into the world, thereby obtaining leverage over it.Footnote 26 The flow of information in and back out is carried by networks, and the process is very similar to that associated with effective benchmarks.
Effectiveness and publicness
How can these three ideas from ANT be used to understand transnational benchmarking better? Together they strongly suggest that we need to examine the distinctive material properties of the networks and contexts to understand how benchmarks work, and more specifically to analyse the two crucial issues discussed earlier in this article.
The first of these issues was the effectiveness of benchmarking. ANT suggests that the effectiveness of benchmarks depends on the properties of their contexts. A key factor that contributes to effectiveness is whether the context of benchmarks is sufficiently dense to support the networks needed for benchmarks to be effective. If they are, we can say that they are embedded. Benchmarks that travel no farther than a file on their architect’s desk are not embedded, while ones that are integrated into dense distant networks with substantial material presence have a higher degree of embeddedness.Footnote 27 This embeddedness involves the type of material artefacts or objects highlighted by ANT. This goes beyond Mark Granovetter’s well-known use of embeddedness since his is a more abstract conception, and much of the research he has inspired does not consider the relevance of variations in the objects that carry or mediate network ties.Footnote 28 By itself effectiveness does not imply that a benchmarking arrangement is ethically good, since it may enable powerful actors to more effectively dominate weaker ones. However regardless of its ethical character, it is useful to know whether a benchmarking arrangement is effective or not.
The second issue, which has more ethical significance, is publicness, the degree to which benchmarks are helpful in addressing public policy problems or instead are tools used by self-interested actors to enhance their own power and wealth, perhaps at the expense of other actors or the public interest. Publicness is a useful criterion for assessing the normative qualities of a governance mechanism and the degree to which it addresses collective problems or instead is used to promote a particular interest.Footnote 29 In the past it was common to associate ‘public’ with governmental institutions and ‘private’ with business,Footnote 30 but this distinction has been undermined by the performance of public functions by private firms and by the capture of public institutions by private interests. As Jacqueline Best and Alexandra Gheciu have argued, such complications can be addressed by a practice approach which sees publics as assembled around matters that are recognised by the community involved as being of common concern, rather than as bounded spheres that have certain enduring social properties themselves.Footnote 31 This is consistent with ANT. As Latour has put it, ‘neither the public, nor the common, nor the “we” exists; they must be brought into being’. This is not incompatible with the large literatures on global publics which have been inspired by but have gone beyond Jürgen Habermas’s seminal work on the public sphere,Footnote 32 moving from his focus on a single national liberal public sphere to recognise multiple and often transnational publics, including counter-publics which resist dominant ones.Footnote 33
Typical practices that are likely to promote outcomes that are in the public interest include participation, transparency, accountability, and democracy.Footnote 34 These are procedural ways of identifying the public interest that help avoid the arbitrary character of judgements by researchers on whether outcomes are in the public interest – who are researchers to make such a judgment on behalf of the public? Such a procedural approach has its own challenges. For instance, as Daniel Naurin has argued, transparency may lead to an increase in self-interested rhetoric as those involved in governance opportunistically play to an audience in a way that they would not if they could formulate policies more collaboratively in private. However Naurin also points to the recurrent presence of policy rationales formulated with reference to the public interest, with and without transparency.Footnote 35 These complications can be mitigated by not focusing too narrowly, formulaically or precisely on one measure of the public character of benchmarking processes, but instead to look for clues that help assess whether a benchmarking process is primarily run for the benefit of its architects and at the expense of those it targets, or if instead it is a tool used to solve collective problems in a more democratic fashion. Who participates or is listened to in formulating and revising benchmarking, how open the process is, and how responsive it is to concerns of those it targets are all useful in comparing the publicness of benchmarking in different locations or points in time. With benchmarking, it is unlikely that the presence or absence of these characteristics can be assessed by only examining the relative power of the architects or targets of benchmarks, whether these are from the public or private sector, or the benchmarks’ formal properties. Instead it is necessary to explore the broader networks that carry flows of information and action between the benchmark and its contexts.
Greater publicness has intrinsic ethical benefits, helps ensure that a governance arrangement contributes to solving collective global problems rather than simply advancing particular interests, and is likely to improve effectiveness since the information flows it involves promote learning and accountability. In the theoretical literatures relevant to transnational benchmarking, Sabel and Zeitlin’s experimentalist governance modelFootnote 36 as discussed above, is a highly recursive and inclusive system of collective target-setting and identification of best practices. It exemplifies a type of benchmarking with high degrees of publicness. In contrast, the application of Foucault’s concept of governmentality to benchmarking highlights ways that it can reinforce relations of domination, since target actors are subject to a system of power and governance that they may resist but not create, reducing the publicness of the benchmarking mechanism, as does the presence of ‘capture’, where powerful business actors self-interestedly manipulate a governance system that is supposed to be public.
While it is easy to see how powerful actors or practices can incorporate benchmarking as an instrument which expands the control of subordinate actors, playing one target actor off against another as they vie to meet benchmarks that the powerful have formulated, I argue that the inherent logic of benchmarking pushes it more strongly in the direction of expanded publicness. In contrast to forms of governance based on command-and-control, benchmarking is enhanced by comparisons across greater numbers of actors; by more flows of information among them; by more active engagement by them; and by a greater willingness to rework existing structures in pursuit of collective goals. These are all elements of publicness as well. Therefore a decision to engage in benchmarking and the search for and adoption of best practices associated with it tends to incorporate a governance mechanism based on a logic that encourages expanding publicness.
This impetus towards publicness is consistent with the history of private-sector benchmarking, which has evolved from evaluating products and services to include learning and implementation of best practices.Footnote 37 There has been a shift of learning from the individual and group level towards more collective learning that changes organisational cultures. This has involved an increasing integration of benchmarking and strategic planning, change management, and process reengineering.Footnote 38 Over time benchmarking has also increasingly involved looking to other companies for process lessons, with a corresponding increase in the sharing of information.Footnote 39 Collaborative benchmarking where companies work jointly on their interactions is a very recent development.Footnote 40 In other words, the range of participants, the type of information shared, and the degree of structural change involved has continually expanded over time, shifting benchmarking from a quite narrowly focused tool used by managers to a form of governance, with expanding circles of participation.
However this expansion in publicness in benchmarking is certainly far from guaranteed. The challenge of embedding the necessary networks and flows of information can inhibit this logic. As well, powerful actors and practices can work to reinforce their control of subordinate actors by incorporating the aspects of benchmarking that assist this.Footnote 41 An analysis of the contexts of any particular case of benchmarking and the degree of embeddedness and publicness that this reveals is important in assessing the degree to which that benchmarking mechanism primarily works to enhance the power of dominant actors or instead to expand publicness and address collective problems. Despite this variation in the publicness of benchmarking, I hypothesise that the way that its inherent logic encourages publicness is sufficiently strong that it will be a prominent and visible tendency in transnational benchmarking. The cases presented in the next section provide support for this hypothesis.
Disaster risk reduction and supply chain benchmarking
Disaster risk reduction benchmarking: the problem
Over the past century there has been a dramatic increase in the number of natural and technological disasters.Footnote 42 During the 1990s, 188 million people per year on average were affected by disasters, six times the comparable number for conflict. Over 90 per cent of disaster fatalities occur in the developing world.Footnote 43 A strong theme in the disaster literature is the increasing recognition over time of the social and political dimensions of all disasters.Footnote 44 A related trend at the UN and elsewhere is the shift in emphasis from relief and recovery to disaster risk reduction and the fostering of resilience.Footnote 45 ‘Natural’ disasters can be socially constructed, such as when new human settlement changes the landscape and triggers floods or when people move into earthquake zones. Builders may have an interest in concealing risks and governments may prefer to postpone investments in infrastructure which might mitigate disaster risk. Benchmarking is a way to address these problems.
Disaster risk reduction benchmarking: the benchmarking design, architects, and target actors
There are many transnational benchmarking initiatives relevant to disaster risk reduction. I focus here on the most prominent one, the Hyogo Framework Self-Assessment, which is associated with the United Nations and national governments.Footnote 46 The Hyogo Framework Self-Assessment is part of a larger ‘Hyogo Framework for Action (HFA) 2005–2015: Building the Resilience of Nations and Communities to Disasters’ which was adopted by 168 countries at the World Conference on Disaster Reduction and endorsed by the UN General Assembly in 2005. The UN International Strategy for Disaster Reduction (UNISDR) was tasked by the conference with creating a systematic review mechanism. Five priorities for action were identified.Footnote 47 The HFA Review is a voluntary self-assessment of progress towards these priorities. An online monitoring and review tool was launched in 2007 and between by 2013, 146 countries had participated in a review, although not all reports were comprehensive. There are local, national, and regional reports, with the latter seeking to address trans-boundary disasters. The targets of the benchmarks are therefore all three levels of government, although ‘individual States are mainly responsible for the implementation of disaster risk reduction and for the development and application of national indicators’.Footnote 48 National reports are organised by ‘national focal points’ which produce the quantitative ratings by aggregating the subjective assessments of multiple stakeholders. The local assessments measure performance relative to ‘Ten Essentials’ for making cities disaster resilient. For instance Essential 7 is ‘Ensure education programmes and training on disaster risk reduction are in place in schools and local communities.’Footnote 49
The HFA ‘National Progress Reports’ webpage provides a ‘Query Builder’ that permits the visitor to ‘Create your own view of country national HFA progress by selecting, sorting and comparing the data and information you wish to see.’Footnote 50 For instance the Interim national progress report for 2011–13 provided by Kenya provides a score of ‘3’ for Kenya’s level of progress achieved on ‘Priority for action 1’ which is to ‘Ensure that disaster risk reduction is a national and a local priority with a strong institutional basis for implementation.’ Level 3 signifies ‘Institutional commitment attained, but achievements are neither comprehensive nor substantial.’ A set of responses to ‘Key Questions and Means of Verification’ are then provided to document the basis of this score.Footnote 51
In a November 2013 report, UNISDR set out further proposals for ‘a new system of indicators to measure progress in disaster risk management’. This was in response to a call for such indicators from the fourth session of the Global Platform for Disaster Risk Reduction, with 3,500 participants from 172 countries.Footnote 52 The indicator system would include measures of losses from disasters, together with resilience indicators, disaster risk management policy indicators, and underlying risk drivers indicators. The underlying risk drivers include 52 indicators in categories such as ‘economic and fiscal structure’, ‘poverty and social vulnerability’, and ‘overall governance’.
The contexts of disaster risk reduction benchmarking
The above discussion has provided an overview of the architects, targets, and design of transnational HFA disaster risk reduction benchmarks. Each of these is important in understanding the significance and functioning of the benchmark. For instance the inclusion of transnational regions and local governments in the HFA benchmarks indicates that an effort is being made to link the scale and locations of the target actors to the character of the problems the benchmark aims to address. The provisions for summarising, displaying, or reporting on national progress reports are an indicator of the importance of the reactions of the target actors to this benchmarking initiative. There are also important design questions that have been discussed. For instance the UNISDR has specified the ‘characteristics of good indicators’, such as ‘the indicator measurement should enable comparison over the different life-cycle stages of the policy or project, as well as between different policies or projects’.Footnote 53
Despite the importance of these elements of disaster risk reduction (DRR) benchmarking it becomes quickly evident in observing them that their functioning depends crucially on the contexts of this benchmarking initiative, including the degrees of embeddedness and publicness. The success of this benchmarking depends on the strengthening of linkages between the transnational benchmarks themselves, and the most local, specific, and material aspects of disaster risk mitigation and resilience. This strengthening requires that the linkages be embedded in a robust and sustainable way that brings together humans and objects in a recursive process of feedback loops from the benchmark to local activities. It also requires that levels of participation, flows of information, and relations of accountability be strengthened, expanding the reach of the networks, bringing in an expanding variety of governmental and non-governmental actors: that the publicness of the networks be increased. In other words, the UNISDR DRR benchmarks are configured to be an active relay in circular, robust, and open networks that span the stages set out in the conceptual section above.
The importance of these circular flows is readily apparent in the UNISDR documents. For instance the data that provide the inputs to the benchmarks are not easily available but depend on a chain of information and action: ‘obtaining measurements and maintaining reliable data series requires dedicated expertise and resources and can be costly. The data and methodologies upon which indicators and benchmarks depend are inevitably limited and imperfect.’Footnote 54 However the networks involve far more than this. The benchmark aims to solicit the active engagement of multiple stakeholders in a variety of ways and stages of the circle. This is evident for instance in the large numbers and variety of types of actors involved in the international UN-sponsored meetings that initiated, assessed and revised the HFA. In the implementation of the benchmarking in national or local contexts the goal similarly is to mobilise large numbers and varieties of stakeholders to alter their own conduct, to assess their own progress relative to the benchmarks, to help identify appropriate measures and ways to meet benchmarks, and to hold officials accountable, pressuring them to meet benchmarks. For instance, Phase I of the Local HFA Tool has five steps for convening stakeholders for the self-assessment that is its focus. Phase II then focuses on a review and revision process with stakeholders, and Phase III calls for the report to be posted to the web.Footnote 55
The importance of the networks that elicit local engagement is also apparent in the 2007–13 summary of the national HFA reports. This summary indicated that progress had been made in recognising the importance of DRR and related issues such as the importance of gender, and in establishing policy and legal frameworks. However, the report noted that such frameworks ‘do not automatically translate into effective DRR’ (p. 7). It calls for a new post-2015 framework for DRR which ‘should offer incentives for scaling up DRR activities to the point where they can make a perceptible material difference in peoples’ lives’ (p. 8). This will require greater local outreach, as well as overcoming resource and capacity challenges.Footnote 56 In other words, the networks had begun to extend as far as national officials and legislators, but not to citizens’ daily activities: more networks with more embeddedness and publicness are needed.
Online templates play an important role in increasing embeddness and publicness of networks at the national and local levels. Such electronic infrastructures embed the benchmarks and the interactions with them in computer software and hardware, establishing new links between the relevant stakeholders. They can create a degree of automaticity in the flow of information, enhancing publicness. For instance the English language version of the Local HFA Self-Assessment Tool discussed above is done through an on-line template, and the final step is ‘Make the report public by clicking “submit”.’
The outcomes that the benchmarking seeks to promote require the best practices that it identifies to be extended to very local and specific activities. For instance a report featured on the UNISDR website on Timor-Leste, one of the countries most exposed to extreme natural disaster risk, notes that disaster resilience there requires a mix of road improvements and changes in farming practices, since most of its people are subsistence farmers in villages cut off from other parts of the country. Changes in choices of crops and management of the soil are important in increasing the capacity for farming to recover after disaster. Changes at the political level are important, but these need to be combined with changes in the ongoing mix of human and natural activity on the land. The article on the UNISDR website creates a link in embedding networks needed for the benchmarking to connect with this type of change, but to be successful the network needs to extend in other ways to such local mixes of humans and natural actions.Footnote 57 This can include NGOs. For instance, a coalition of development and environmental NGOs, including the Netherland Red Cross, Care, Cordaid, the Red Cross/Red Crescent Climate Centre, and Wetlands International, has been engaged with the post-2015 Hyogo Framework in order to strengthen the links between local communities and ecosystems and the global HFA process.Footnote 58 This too involves strengthening the embeddedness and publicness of networks, and the way they engage with humans, technical artefacts, and other material aspects of the connections, such as the interactions of farmers, crops, and soil. The extension of the indicators to underlying risk factors such as poverty which was noted above is a further indication of the expanding circles of engagement of the HFA process.
Transnational supply chain benchmarking: the problem
As production has become more global the management of transnational supply chains has become extremely important.Footnote 59 Brands have been damaged by supply chain problems resulting in hazardous products, environmental damage or scandalous labour conditions. Supply chain management (SCM) has become more challenging as the supply chain has become more global, more complex, involving larger numbers of firms and products, more complicated products and production processes, faster manufacturing and delivery times, and leaner operations with less room for error. As an industry participant has noted, ‘Companies have systematically taken costs out of the system, but they have systematically added risks … Supply chains are much more fragile than before.’Footnote 60 In contrast to an earlier time when a single firm could impose a set of requirements on its own logistics employees or a few suppliers, there is a far greater need for collaborative standards that cut across multiple firms, functions, and industries. As well, efficiencies in supply chains are crucial for the competitive success of firms involved in them. Transnational supply chain benchmarking is designed to address these issues.
Transnational supply chain benchmarking: the benchmarking design, architects, and target actors
There are a number of benchmarking systems that have been designed for supply chain management. This case will focus in turn on the most widely used one, the Supply-Chain Operations References (SCOR) model, and a newer one that has been developed for faster retail supply chains, the Consumer Goods Forum Global Scorecard.Footnote 61
The SCOR model was developed by the Supply-Chain Council (SCC) and first released in 1997.Footnote 62 As one study noted, ‘it is widely acknowledged as the quasi-industry standard for supply chain management’.Footnote 63 The SCC was a non-profit organisation founded by two consulting firms in 1996, initially with 69 member-companies. In 2014, it had 406 members, including 89 large global corporations such as Dupont, Ernst & Young and Microsoft, 155 smaller firms, and 162 non-profits, mostly universities and research institutes. The business members were mostly industrial, business services and transportation services rather than retail.Footnote 64 In 2014, it merged with APICS, which is the leading professional association for supply chain management (SCM) and provider of SCM training and professional certifications, with nearly 40,000 members in more than 200 countries.Footnote 65
The SCOR model provides clear and detailed specifications of types of supply chain activities and processes. This facilitates benchmarking across processes, functions, firms, and industries. SCOR 11.0 distinguishes six standard processes: plan; source; make; deliver; return; and enable. The source-make-deliver sequence captures the flow of material in and out of each firm and can appear multiple times in a supply chain. ‘Return’ captures the reverse flow of defective products. ‘Plan’ involves the coordination of these bi-directional flows. ‘Enable’, which has been added more recently, is defined in the model as processes which ‘support the realisation and governance of the planning and execution processes of supply chains’, such as the management of rules, compliance, and contracts. It has standards for metrics in five categories: reliability; responsiveness; agility; cost; and asset management efficiency. The first three are ‘customer facing’ and the final two are ‘internal facing’.
SCORmark is a benchmarking system based on the SCOR model that is provided by APICS SCC, in alliance with a PwC subsidiary and that complements benchmarking efforts that individual firms can do with the SCOR model. SCORmark benchmarking assesses the performance of a firm relative to selected peer groups. The firm identifies targets, collects metrics data, and enters the data through a web portal. The firm is compared to peer firms in the five categories specified above.Footnote 66
The second supply chain benchmarking system that I examine here is the Global Scorecard that is sponsored by GS1 and the Consumer Goods Forum, and supported operationally by IBM Global Business Services. In contrast to the SCORmark, the Global Scorecard is more focused on integrating retail-oriented supply chains with consumer preferences.
One aspect of the type of integration that the Global Scorecard seeks to promote is the barcodes and Radio-Frequency ID (RFID) technologies that are used to track products through to the point of sale to consumers.Footnote 67 RFID technology has far greater capacity to store data and communicate with computer systems than do conventional barcodes – indeed there could be a unique code assigned to each individual item produced worldwide. Barcodes, RFID, and many other related standards are managed by GS1, a non-profit association with more than 1.2 million member companies in about 148 countries. As GS1 notes, ‘billions of times each day, the GS1 System of Standards connects physical things such as products, locations, logistics units, and assets with information about those things supplied by GS1 Communication Standards’.Footnote 68 An example of the integration of GS1 standards with the Scorecard is this performance measure: ‘% of sales with synchronized master data between trading partners via the GS1 global data synchronization network (GDSN)’.Footnote 69
The Consumer Goods Forum (CGF) is also a sponsor of the Global Scorecard. The CGF was created in 2009 through a merger between the Global Commerce Initiative (GCI), CIES (the food and consumer goods industry association), and the Global CEO Forum.Footnote 70 The CGF brings together senior executives and managers from seventy countries representing companies with combined sales of EUR 2.5 trillion.Footnote 71
The Global Scorecard describes its benefits as follows:
This set of tools allows companies to measure their current performance and capabilities using a globally standard set of measures. This allows the company to benchmark itself against others within the same global corporation and against peers in the same country, distribution channel or product category. Performance gaps can be identified and plans put in place to close the gaps or even break away from the crowd.Footnote 72
The Scorecard is an ‘on-line action planning tool’ that assesses the performance of companies with regard to collaboration within supply chains. It is associated with the ‘Efficient Consumer Response’ (ECR) initiatives to ‘develop a responsive, consumer-driven system in which manufacturers, brokers and distributors work together to maximize consumer value and minimize the supply chain cost’.Footnote 73 Different levels of the Scorecard are offered, with the Entry Level asking more basic questions that can be entirely addressed on-line by collaborators, and the Intermediate Level starts on-line and then requires a face-to-face workshop. The Intermediate Level is more detailed and can integrate action items to the assessment functions. A Full Global ECR Scorecard is for the most integrated and important relationships.
In addition to collaborative partners being able to assess and improve their own specific relationship, the initiative produces an annual Global Scorecard Survey which produces global benchmarks. From its start in 2004, the annual numbers of scorecards included in the survey have increased year by year to 9,099 in the 2011 survey, representing 8,790 companies from 62 countries with total revenues of US$ 2.25 trillion.Footnote 74
The contexts of global supply chain benchmarking
While there are many supply chain benchmarking frameworks, it is clear that each one’s effectiveness depends not only on its architects, targets and design, but on the degree to which it is embedded in the operations of supply chain networks. The types of widening circles of engagement that publicness involves are also important.
The importance of embeddedness in networks is most obvious in the case of the Global Scorecard, since this requires that firms work the standards it measures into the IT and logistics systems, along with the physical objects that are transported through the supply chains. An example is the creation of an RFID tag embedded in a package by a manufacturer’s supplier, which then seamlessly communicates with harmonised digitised systems as the package travels through one firm to another, towards the consumer. However the success of the SCOR model is also closely related to its ability to interact with the practical organisation of the flow of products through the supply chain, and integrating information systems with the SCOR model also provides significant benefits.Footnote 75 As the SCC’s chief technology officer has noted, the SCOR model ‘is a process framework that allows for rapid and reliable end-to-end descriptions of work, material, and information flows in supply chains. In a real sense, it’s a supply chain dictionary.’Footnote 76 Its benefits depend on the ability of the firms using it to actually rework all of these flows to improve performance.
The ever-widening circles of participation that publicness involves are also evident in both benchmarking systems. This is not uniform. Some of the challenges to the expansion of these systems are technical, such as how to integrate the standards with IT systems. Some are associated with the search for political or commercial advantage. For instance, access to key information about the SCOR model is restricted to paid-up members. Nevertheless, there are tendencies with both systems that match aspects of publicness. The comparisons involved in both the SCORmark and the Global Scorecard are more effective with larger numbers of firms, especially where a larger set of participants is needed to have enough firms in subcategories of particular types of supply chains for comparisons across these subcategories to be meaningful. In the case of the Global Scorecard, the benefits for each firm of adopting standardised GS1 communications technologies is increased with each additional firm that adopts them as well, since it expands the potential for automating transactions across multiple firms. Additionally, in both cases, the capacity to improve processes associated with the flow of goods through the supply chain is increased with the active engagement of everyone involved with that flow. In the case of the SCOR model, one of its big benefits is to focus everyone to work collaboratively on the standard processes that shape supply chain performance (plan; source; make; deliver; return; and enable) rather than organisational silos.Footnote 77
The expanding circles draw in customer preferences as well, and for retail supply chains these can include public goods such as environmental sustainability. Sustainability is a major focus of the CGF-sponsored 2016 Future Supply Chain project, which states ‘What does the current focus on sustainability have to do with on-shelf availability and costs in the physical supply chain for consumer goods? The answer is “everything”.’Footnote 78 The integration associated with the Global Scorecard increases traceability for individual products, with the potential for increased accountability in corporate social responsibility. However, as Genevieve LeBaron and Jane Lister have shown, corporate social responsibility benchmarking, which seeks to impose an external non-commercial purpose on the internal profit-oriented operations of corporations, has been less effective than the more collaborative business activities discussed in this article.Footnote 79
The expanding circles can also be seen in the pre-history of the Global Scorecard, which can be traced back to innovations pioneered by Walmart. Walmart built its success in becoming the world’s largest corporation in part on its aggressive and early integration of point-of-sale data with data warehousing and forecasting. Walmart has long been known for the most ambitious supply chain benchmarking effort in the private sector.Footnote 80 Its relationships with its thousands of global suppliers have been managed through its electronic data interchange (EDI) system. Wal-Mart’s ruthless management of its suppliers and its incessant demands to exceed previous performance measures are legendary.Footnote 81 In the past this system effectively set industry standards for supply chains, led by Walmart as the dominant firm.Footnote 82 However over time it has become more collaborative. Walmart was an early participant and leader in the Collaborative Planning, Forecasting, and Replenishment (CPFR) initiative sponsored by the Voluntary Interindustry Commerce Solutions Association (VICS) beginning in the late 1990s, which aimed for increased synchronised ‘e-collaboration’ on the management of barcode-related and other data.Footnote 83
The Global Commerce Initiative, which initiated the Global Scorecard and was subsequently merged into the Consumer Goods Forum, was launched in 1999 to further develop global supply chain standards. Its most prominent working group, on EDI, was co-chaired by Kraft Foods and Walmart (Nairn 2000). The goal of the standards was to create wider, more participatory, and more open forms of collaboration than had been the case when Walmart was pioneering the technology. By 2011, the GCI’s Global Upstream Supply Initiative (GUSI) had worked with GS1 standards to promote a standardised global model of close integration similar to the one that Walmart had developed, and to extend this back to the manufacturers’ suppliers. The adoption of standardised GS1 communications technologies across the supply chain allows increased automation of procurement, material forecasting, inventory management, dispatch, receipt, consumption of materials and financial settlement, with all the efficiency and other performance benefits that accompany this.Footnote 84 VICS merged with GS1 US in 2012.Footnote 85 The benchmarking associated with the Global Scorecard displays far more publicness than was the case with the earlier period dominated by Walmart. However this publicness mostly involves collaboration among firms in improving their profitability despite some incorporation of other public concerns such as sustainability.
Comparing the cases
The importance of context is evident in both disaster risk reduction and supply chain management. Their effectiveness depends on sustaining flows of information and action that work with the benchmarks but involve far more than an interaction of the architects and target actors with a benchmarking mechanism. The networks that sustain the flows need to be embedded in local contexts, technical artefacts, other objects and routine practices. In the case of disaster risk reduction this can extend to the interactions of a subsistence farmer with soil and crops, and in the case of supply chains this can extend to the placement of an RFID tag in a package and the configuration of the corresponding software in each firm’s digital systems that communicate with it.
The impetus towards expanding circles of engagement, which correspond to increased publicness, is evident in each case as well. In disaster risk reduction it is evident in the recognised importance of increasingly engaging civil society actors, citizens, and underlying risk factors, and in facilitating the circular flows of information that enhance the effectiveness of benchmarking. In supply chain management it is evident in the recognised importance for its effectiveness of engaging actors at all levels, in bringing in increasing numbers of actors, in promoting flows of information, and in continually reworking structures.
While the relative power of the architects and target actors and the character of the benchmark design are certainly significant in both cases, our ability to analyse and evaluate the benchmarking systems would be very limited if we were restricted to these variables. The performances of the benchmarks in the two cases are more closely related to the similar contextual factors that both face. In both cases, effective benchmarking requires a circular flow of information and action through networks that extend far beyond the benchmark itself, and engage with multiple actors in multiple locations. It still matters that the first case involves states and the second firms, since the degree of publicness is greater in the former case. However, the difficulty of establishing connections and soliciting engagement is similar across the two cases, even though this primarily involved citizens in the first case and employees in the second.
The character of the design of the benchmarks in the two cases is important, but only when considered in relationship to the contexts with which they interact. For instance both the HFA and Global Scorecard benchmarks have online templates, and it is significant that the former automatically publicises the submissions while the latter carefully preserves the confidentiality of the information submitted by companies. This feature of the mechanism is related to the public character of the architects of the HFA process and the private character of the architects of the Global Scorecard. However not all benchmarking information involving states is made public, and not all benchmarking information involving firms is kept private. The larger significance of these differences relates to how the benchmark interacts with its contexts, which in the case of disaster risks cuts more directly across all organisational boundaries than is the case with supply chains.
It is not possible to generalise from these cases to claim that transnational benchmarking in general promotes publicness. In other cases self-interest and power asymmetries may be more important than in these cases. However these cases do confirm that benchmarking can provide an impetus towards greater publicness, as well as showing the importance of the contexts of benchmarking for assessing the relative influence of this impetus and the factors that work against it.
Conclusion
This article has argued that it is important to go beyond analysis of the architects, target actors, and design characteristics of a benchmark to analyse more systematically the contexts of benchmarking, and whether and how these sustain the networks carrying flows of information and action that are needed if a benchmark is going to be effective. It drew on actor-network theory in identifying a method for analysing contexts of benchmarking. It identified embeddedness and publicness as two the key aspects of contextual variation. It also argued that the logic of benchmarking created an impetus towards increased publicness, although this was constrained by the challenges associated with embeddedness and the self-interest of the actors involved.
The two cases demonstrated the importance of contexts for the effectiveness of benchmarking, and for assessing their capacity to solve collective global problems. The need for benchmarking to establish robust connections with local contexts, so that the flows of information and actions are embedded, was clear, whether this was subsistence farmers and their crops, or the RFID tagged products moving through the supply chain. While disaster risk reduction and supply chain management involve very different problems and actors, both displayed an impetus towards greater publicness over time, although this was uneven. The two cases were striking for their similarities, despite the differences in the actors involved and the problems they sought to address. This confirms the distinctiveness of benchmarking as a mode of governance, where the interactions between the benchmark and its contexts display certain similarities despite variations in the types of actors that constitute its architects and targets or how the benchmark itself is designed. While the two cases examined here were illustrative, there is much more valuable research which can be done on other cases of interaction between benchmarks and their contexts, more systematically identifying the conditions that contribute to the effectiveness and publicness of benchmarking.
Transnational benchmarking is inserted into an increasingly dense, complex, and accelerated global governance setting where it has to interact with a diverse set of actors spread out over multiple locations if it is to be effective. Rather than a measuring instrument, or a tool used by one set of actors in their interaction with another set, benchmarking, in its most distinctive form, is best conceptualised as a partially autonomous relay and accelerator in a circular and reflexive flow of actions and information. Benchmarking can be a collaborative mode of governance that can be used by a set of actors to solve their own collective problems. However, benchmarking can also still be used by powerful actors to advance their particular interests. The balance between this and publicness in any particular benchmarking system can only be assessed by analysing the way that the benchmark interacts with its contexts, and whether or how circular and reflexive networks carry information and actions from the benchmark through those contexts and then back again. There are many opportunities to further link the analysis in this article to related or complementary literatures and approaches, many of which have been noted above, including the literatures on experimentalist governance, transnational communities, global publics, global governmentality, and transnational value chains, among others.
