When I was appointed United Nations Special Rapporteur on Contemporary Forms of Racism, Racial Discrimination, Xenophobia and Related Intolerance in 2017, it had never crossed my mind that of all the global challenges confronting racially marginalized groups, emerging digital technologiesFootnote 1 would present themselves as among the most personally terrifying fronts. I am a scholar of international migration, borders, and racial governance, but I was an international human rights lawyer first. Some of my most professionally formative experiences have involved bearing witness to soul-crushing systems and regimes of oppression and the testimonies of those subject to them. I think, for example, about someone I interviewed to prepare for an asylum hearing who had been forced to give birth alone in a forest as she fled ethnic cleansing, or someone else who was part of an incarcerated group I worked with at a maximum-security prison in California, destined to live the rest of her life in a tiny box.
The horror of spectacular violence, and increasingly that of institutionalized violence, is within easy reach for most people, even though, as the ongoing, widely broadcast Israeli genocide of Palestinians shows with each passing day, it takes more than widely held revulsion to bring an end to such violence. Whatever we might say about emerging digital technologies in 2024, while concerns with their societal ramifications are more widespread than they were in 2017, it seems a stretch to say these technologies are widely perceived as warranting a sense of exceptional unease. In this brief epilogue, I reflect on the nonetheless persisting sense of unease, verging on terror that I developed in my UN work on emerging digital technologies, and how the contributions in this edited volume have helped me make sense of these emotions, and the circumstances that provoked them.
I am neither a technology nor a data expert of any sort. Within the human rights framework I have devoted most attention to equality and nondiscrimination rights, rights that typically have not been at the fore of international legal debates on the global governance of emerging digital technologies, and that arguably remain a side show, at best. Within the field of human rights, concerns with emerging digital technologies have traditionally been framed as implicating the right to privacy and to a lesser extent the right to freedom of expression and others. Historically, the inclusion of equality and nondiscrimination concerns has been on narrow terms that have resulted in impoverished mobilization of these rights. From the very beginning of my time in my UN mandate I received invitations to participate in global policymaking and related processes at the intersection of technology and human rights. Yet almost invariably, these invitations asked me to speak specifically to the role of equality and non-discrimination rights in the adjudication of hate speech online, often rehearsing broader debates within the human rights system that have seen states – with the United States at the helm – pit speech and expression rights against equality and nondiscrimination rights.Footnote 2
The tech and human rights debate within and outside the UN system has shifted in important ways, but at the time – even with limited human rights exposure to the tech world – I found the narrowing of the relevance of equality and nondiscrimination norms to the realm of hate speech concerns unsettling, if familiar. Unsettling because the equality and non-discrimination effects of any phenomena that quite literally transform and remake societal relations on the scale of emerging digital technologies could hardly only or mostly be confined to online hate speech. As a Special Rapporteur the sheer (supra-)sovereign force of some of the big Silicon Valley tech companies was palpable. Whereas bilateral meetings coordinated by the Office of the High Commissioner for Human Rights for me in my UN capacity had solely been with diplomatic representatives of nation-states, I recall being stunned one year when such a meeting had been arranged for me with the “diplomatic corps” of one such company. More was always obviously at stake than content moderation.
Yet the reductive framing of the stakes, as I mentioned, was familiar. The tendency of the field of international human rights and the broader global governance terrain of multilateral institutions has been to marginalize racial justice and equality and to adopt a reductive approach to their stakes.Footnote 3 Where they have received the most attention, these causes have typically been reduced to issues of combating the explicitly racist conduct of individuals while completely disregarding (but reifying) structural and systemic racism operationalized through liberal law and policy, including at the international level.Footnote 4 Global governance within the United Nations, even through initiatives such as the Sustainable Development Goals that are intended to address global inequality adopt what Critical Race Theorists have termed a “color-blind” approach.Footnote 5 A purported even-handedness that entails ignoring racially disparate conditions in favor of treating all individuals and groups the same, even if these individuals and groups are differently situated, including because of historical structures of intentional discrimination such as European colonialism. It is thus not surprising that the human rights preoccupations of the primary global governance forum – the United Nations – would effectively narrow the relevance of racial equality and nondiscrimination norms mostly to influencing social media content moderation policies, and even then, doing so on terms seemingly set by social media platforms.Footnote 6 Meta’s politically and economically motivated recent abandonment of its independent fact-checking policy illustrates the pitfalls. On the one hand, content moderation matters for racial justice and for human rights more broadly. On the other hand, without approaches that target the worldmaking capabilities of social media companies, these companies remain unconstrained. The problem of global corporate predation is, of course, a much deeper one than can be addressed within the tech and human rights frame, although it implicates the voluntary business and human rights regime that so poorly mitigates this predation. But the problem might be materially easier to tackle within the context of a global discourse that begins from recognition and repudiation of the structural and systemic account of this problem, as opposed to accounts that render the problem two dimensional.
It was against this backdrop that I produced two thematic reports focusing on racial equality and emerging digital technologies, one of which dealt specifically with the context of immigration and border enforcement. My first report attempted to bring racial equality and nondiscrimination principles to bear on the structural and institutional impacts of emerging digital technologies, relying on the research of critical scholars of technology and on the experiences of racially marginalized communities subject to these impacts in the realms of employment, education, health care, and criminal justice among others.Footnote 7 At the heart of the report was an examination of how emerging digital technologies exacerbate and compound existing inequities, many of which exist along racial, ethnic, and national origin grounds. The examples highlighted in the report raised concerns about different forms of racial and intersectional discrimination in the design and use of emerging digital technologies. In some cases, this discrimination is direct, and explicitly motivated by intolerance or prejudice. In other cases, discrimination results from disparate impacts on groups according to their race, ethnicity, or national origin, even when an explicit intent to discriminate is absent. And in yet other cases, direct and indirect forms of discrimination exist in combination, and can have such a significant holistic or systemic effect as to subject groups to racially discriminatory structures that pervade access to and enjoyment of human rights in all areas of their lives. The second report detailed the ways that “digital borders”Footnote 8 – international borders increasingly reliant on machine learning, big data, automated algorithmic decision-making systems, predictive analytics, and so on – enhance the scope and precision of already racially discriminatory borders.
The modest ambition of these two reports was to move the human rights debate on racial discrimination and tech beyond the narrow confines of content moderation concerns. In some sense, I aimed to help broaden the scope of the global governance debate to encourage policymaking attention to the structural reification of racial hierarchies across and within societies, and to the means of that reification. By the time the reports were done, I had the nagging sense that though worthwhile, an analysis that treated global governance and those governed as independent variables, itself reproduced a myopia not unrelated to fixating on online hate speech as the racial equality concern of emerging digital technologies. This edited volume, and the nuance and brilliance of its framing by the editors confirm that intuition. Global Governance by Data illustrates how we might ask the right questions across disciplinary silos to make sense of our world as it, and we are iteratively remade by emerging digital technologies. Some of the contributors, for example, push us to think more deeply about how global governance by data reconstitutes racial meaning (Phan and Wark) even though, it seems we cannot quite say yet whether this entails a thoroughgoing reconstitution of the nature and targets of racial injustice.
Phan and Wark argue that AI and machine learning calculate and construct populations premised “not on discrete, shared characteristics – on ‘identity’ – but on measures of alignment determined using complex, automated machine learning techniques: on what’s referred to as the ‘proximity’ between individuals” possibly resulting in “racialisation and racial discrimination premised on logics of proxification.” They offer a compelling account in the shifting techniques of classification and construction and this will surely impact racialization, and this is something we can know based on our experiences with what prior technological innovations have done. Consider, for example, the work that international borders do to classify and construct, and how as their technologies for doing so have shifted over time, so too, have the forms and outputs of their racializing power. The category “non-Western migrant” for example is a twenty-first-century construction made possible by border institutions, including innovations such as the EU’s Schengen visa regime.Footnote 9 What remains to be seen, and what I am not yet convinced of (perhaps due to a failure of imagination) is that proxy logics can fundamentally, or even significantly dislodge the structural, material, and especially colonial grooves that are constitutive of race as a global, historical structure of injustice. Although new, the category non-Western migrant enacts racialized injustice that is most pronounced for immigrants and their descendants who are not White and Christian. Can global governance by data, without a whole lot more, dislodge White supremacy as a global institution?Footnote 10 Phan and Wark are surely right that the “politics of proxies” will demand different political techniques of resistance, in particular I think they increase the urgency for building solidarity across different groups of racially or otherwise marginalized people. Even in the absence of “associative” identities,Footnote 11 solidarity across communities subjected to the same systems of domination but on different terms remains difficult work. Unless AI and machine learning can fundamentally remake the colonial, capitalist DNA of the contemporary world order, the core of our racial paradigm, and the nature of racial injustice within it will remain largely intact.
The provocation of the title – global governance by data – to my mind, calls attention to, the de facto and by design shared consciousnesses that are shaped by the ascendance of data. Data, the digital technologies of their manipulation and enlivening, and the corporate entities firmly in the driver’s seat, all sit at the very heart of our imagination and our connections, two vital foundations for any real hope of emancipatory futures. For example, climate disorder – a catastrophe on a planetary scale – has become a growing site of techno-chauvinistFootnote 12 fever dreams actualized in the name of justice, in some cases in ways that compound the racial injustices that have structured and enabled the extraction, industrialization, and consumption driving the disorder.Footnote 13 The editors of this volume seek to train our attention on “what data does as an active force in global governance: the ways it recomposes and shapes new subjectivities, including legal subjectivities, forms of material agency and sociality; draws people, places and things together into novel relations to be valorised, known and ruled through calculative and representational techniques, and the ways it reconfigures institutional routines, legal and regulatory practices.”Footnote 14 What data does is not merely a matter of scope, degree, or kind with respect to what is governed globally. This volume makes the compelling case that what is at stake in novel or at least remarkable ways is the terms on which we make sense of our internal and external worlds, who we are, and what governance is.
Inquiries along the lines pursued in this volume are critical. They probe the “logics” of big data analytics – the promises, assumptions, beliefs, and rationalities at their heart – and how they are put into effect. As one contribution highlights the ontological condition of complicity for life under fascism (Arvidsson, Gandorfer, and McQuillan citing Massumi), so, too, I think, the entire volume serves to remind that there may no longer be an “outside” of global governance by data.
The volume leans into the often-neglected reality that “frames” – the paradigms we mobilize to make sense of the world – in effect constitute our multiple worlds. Where borders specifically are concerned, the volume includes contributions exploring how global governance by data in the realm of border infrastructure is reconstituting state agencies (Raso). Raso offers an account I found haunting of how digital border infrastructure contributes to the intensification of state power even as it diffuses state agencies in ways that push accountability further from reach. As a legal scholar, I was unnerved anew by Raso’s implicit reminder that liberal proceduralism will not save us – not that it ever could. She writes, “conventional remedies cannot prevent infrastructural elements from focusing authority into the future, allowing traces of riskiness to remain in circulation potentially indefinitely.” In my own work I have observed that so-called digital or virtual borders are digital racial borders. The contributions to this volume, and the work of its editors offer an even deeper and more urgent cut: the “digital” may be remaking the nature of the “racial.” As mentioned above, the extent of this shift remains to be seen. But it seems different, data-associated logics are firmly in the game as shifting not only the nature of the “racial” but also of the “border.” As one of the editors noted elsewhere, the “virtual border” can be a “a system of discrimination and division where the standards of hierarchy or inclusion are always kept in play.”Footnote 15
To return to the sentiment of unease/borderline terror which I described at the start, it has never been anchored in an understanding of emerging digital technologies as inherently or singularly “bad” or “dangerous.” Rather, I have come to think after reading this volume that what I experienced and continue to experience as deep unease is the most tractable of a range of other as yet intractable emotions. A suite of emotions gradually triggered by a growing awareness of how central these technologies and their related infrastructures have become (not exclusively of course) to the constitution of human subjectivities globally, and how for now, the power to dictate the trajectory of these technologies, at least as a matter of political economy, is concentrated in too few incredibly wealthy individuals. Private corporations wield monumental influence in the design and use of emerging digital technologies. Among digital platforms, at the time I published my report seven “super platforms” – Microsoft, Apple, Amazon, Google, Facebook, Tencent, and Alibaba – accounted for two thirds of the total market value of the world’s seventy largest platforms.Footnote 16 In 2019, Google has 90 percent of the global market for Internet searches; occupying two thirds of the global social media market, Facebook was the top social media platform in more than 90 percent of the world’s economies, and Amazon had an almost 40 percent share of the world’s online retail activity. These corporations are not only quasi-sovereign, but they are also world-makers. Contributions in this volume by Yeung and Cohen offer a granular account of how commercial logics, data logics, and governance logics interact in sometimes unexpected ways. Yeung’s work, for example, explores the transposition of tools tailored to low-stakes commercial consumer decision-making to high-stakes public governance decision-making contexts. She calls attention to the specific trajectories of injustice that are triggered with this shift, which is difficult to do without the more granular political economic study she conducts.
The tendency in the forums where the global is governed is to keep our analyses tethered to the dominant and preferred categories and modes of analysis. These are the funded, platformed conversations within the United Nations and other multilateral governance forums, but they are also the academic mainstream in fields adjacent to policy. How can we make AI fairer? How do we eliminate bias? And so on. The opportunity to engage more critically, more open-endedly, and in the registers exemplified by the contributors to this volume are essentially nonexistent. I cannot help but wonder what it would mean or do if global policymaking could encounter (not incorporate) the sort of transdisciplinary exchange exemplified by the volume, perhaps most pointedly by Arvidsson, Gandorfer and McQuillan?
Whereas it is the form of the UN report to conclude with tidy recommendations or “best practices,” the heart of the editors’ project – it seems to me – includes an invitation to dive headfirst into the unknown (and even the unknowable) of emerging digital technologies; to develop new ways of seeing and knowing and connecting; and to begin from the premise that nothing has changed, and everything has changed.