This commentary begins at the end. For all chapters in this section conclude with a reflection on violence and power. From a discussion of platform power,Footnote ¹ the violence that results from state surveillance and deportation to an analysis of racialised biometric borders,Footnote ² and an honest reflection on being complicit in and resisting fascist politics,Footnote ³ all authors agree that there is no escape from the racist, sexist and classist structures that shape how populations are governed using digital technologies. A critical engagement with notions of being, belonging and citizenship are central in all chapters, illustrating empirically how digital technologies are at the heart of exclusionary practices. Conceptually, all three chapters speak to a growing debate at the intersection of Science-and-Technology Studies, Feminist Theory and Critical Surveillance Studies. While the chapters differ greatly, in terms of empirical focus, methodology and style, this commitment to a critical analysis of digital technology and violence brings out a few common themes that deserve more attention. In this commentary, I focus on three of these themes and engage with the chapters, bringing out their similarities and differences. Furthermore, following the themes of this Part, I attempt to further the insights of the chapters by connecting to Critical Race Theory. In doing so, I took up Ardvidsson et al’sFootnote ⁴ invitation to think: what can a chapter do? There is no need to overestimate what this commentary can do, yet one can have a modest ambition to foster a generous debate, in solidarity, with care.Footnote ⁵

Truth

An important element of algorithmic violence that we observe in the different chapters, is how digital politics and algorithmics impact our understanding of ‘truth’, and how truth-telling via data can effect epistemic violence. Epistemic violence refers to the violent ways of producing or erasing knowledge in the production of the Other.Footnote ⁶ In Tazziolli and Valdivia’s chapter ‘truth’ is unpacked in the context of biometric surveillance of migration and the use of algorithmic calculation at the border. In these algorithmic calculations the ‘truth’ is shaped by what is considered the ‘ground truth’. This ‘ground truth’ is a process where the model is shaped through an initial labelled dataset that should mirror reality. Tazzioli and Valdivia show how algorithms are trained to find matches based on biometric traits, connecting images to actual faces. To avoid mismatches in this identification process, it is necessary to train the algorithm with the ground truth improve its robustness: the ability to carry on with calculations and prediction on a new dataset or data that contains outliers. In making these mathematical calculations, algorithms are inscribed with a form of neutrality, pretending their decisions are not political but the outcome of a neutral mathematical process. Tazzioli and Valdivia contest this assumption of neutrality and show how biometric databases are a historical continuation of colonial governance practices using biometrics to sort people into different categories.Footnote ⁷ The (bad) quality of the images, or the discrimination embedded in the training data, however, have disproportionally impacted racialised populations in facial recognition models. In other words, the ‘ground truth’ with which many algorithms at the border are trained is already filled with racial hierarchies and histories of violence. Considering this history, ‘biometric methods and methodologies can only produce dehumanizing mathematical results’.Footnote ⁸ In other words, algorithmic truth entrenches historical asymmetry, administering epistemic violence. Truth, just as neutrality, fits into a sanitised and quantified view by system designers on improving accuracy of the socio-technical systems at work, without attention to the broader socio-political contexts in which these systems work.Footnote ⁹

Aradau’s chapter shifts our focus from algorithmic to legal truths, that is: the establishment of facts through legal decision-making. Studying how digital data is presented in asylum cases, Aradau shows how courts decide what kind of legal truths can be attributed to asylum applicants based on their social media activities. Social media platforms, however, exercise a form of platform power, which obfuscates the distinction between surveillance and data extraction. Therefore, courts’ assumptions that more data will lead to a better assessment of what really happened ignores how platforms access, extract, monetise and curate data in multiple ways. Nevertheless, this assumption and associated demands by courts for more and more data place ‘a burden to know’ on ordinary citizens to know exactly what happens with their data and what consequences this data might have.Footnote ¹⁰ The argument in this chapter is that platform power, like biometric data collected at the border, obfuscates the boundaries of public/private if one makes statements on social media, and even of what is true/false. Despite this complexity, courts need to come to a verdict: a legal truth. Asylum or deportation. Violence or care.

Arvidsson, Gandorfer and McQuillan’s chapter helps us to understand the broader implications of the way algorithmic calculations impact what we understand as the ‘truth’. Algorithmic calculation often confirms what is already known and accepted as “true”. This concurs with Tazzioli and Valdivia’s description of ‘ground truth’ which takes a particular representation of the world as the truth to begin with. Yet, as Ardvisson et al write: ‘nothing starts without a history, or with a clean slate. Beginning can thus only ever mean speaking from the middle’.Footnote ¹¹ Indeed, ‘truth’ in algorithmic governance, as evidenced in these chapters, is a particular way of pretending that there is truth in the way algorithms fixate the past as (datafied) evidence. Through algorithmic practices, the meaning of some (datafied) evidence is sedimented, rather than a product of interpretation or potential change. While scholars have paid attention to the fluid nature of socio-technical systems,Footnote ¹² it seems that racial hierarchies are fixed rather than evolving. Fixation, as BenjaminFootnote ¹³ describes, or an ‘image that does not change’ lies at the core of stereotypes and at the heart of racist practices. Fixation, stereotyping, othering: a beginning of violence. In sum, what the chapters therefore illustrate is how algorithms create a distinction between ‘true’ and ‘false’ representations of reality. The chapters show us that such binary understandings produce algorithmic violence as they close off other ways of interpreting or knowing the world.

Visibility

The making of truth, violence and visibility are intimately connected in (algorithmic) surveillance. The three chapters in this Part engage with current debates in (algorithmic) surveillance studies and illustrate what legal or political becomes visible, obfuscated, foreclosed or illuminated, and how these optics and apertures can open onto violence.Footnote ¹⁴ Looking closely at the term ‘algorithmic surveillance’, it indeed means nothing more than having a set of rules that determines what to see or watch over.Footnote ¹⁵ Critical scholarshipFootnote ¹⁶ reminds us that surveillance practices are not neutral processes of ‘watching over’, but a way of producing racialised, gendered and classist accounts of living bodies. Browne argues that surveillance technologies work through the assumption that they can reveal ‘the truth’ on the racial identities of bodies. Identifying bodies through fingerprinting is an example that finds its origin in the colonial context where race-making and scientific method worked together to ‘identify untrustworthy colonial subjects in India and Egypt’Footnote ¹⁷ and justified racial violence.Footnote ¹⁸

To date, academic conversations on algorithmic surveillance have focused on questions of (in)visibility and privacy to protect individuals from violence. There is a shared concern, however, across the chapters to move beyond issues of privacy and attune to (epistemic) violence. In the chapter on biometric surveillance by Tazzioli and Valdivia, we learn how the debates around ethical use of technologies have rendered existing racial borders invisible. The authors argue ‘[t]he use of data-based systems and algorithms to enact exclusionary laws and policies contributes to invisibilise consolidated hierarchies of mobility as well as the ongoing transformations of racialised borders’.Footnote ¹⁹ In other words, discussions on ‘ethical use of data’ might focus on what data is extracted from migrants at the borders, but they obfuscate the racist structures that govern migration in the first place. Similarly, Aradau presents data wrangling as a conceptual tool that is used to describe the practices of cleaning data and making it presentable. The concept ‘carries the meaning of dispute, of struggle and contestation over what data migrants can produce, how they can then collect it, and how they need to transform it into legal evidence’.Footnote ²⁰ Aradau observes that courts demand a disclosure of data by applicants with a disregard of potential violent outcomes of this making-public. She cautions that issues of privacy are not sufficient to understand the full effects of platform power, as ‘[h]aving all the data is not only a question of privacy and the exposure of personal and often sensitive information to courts. The request for all the data intensifies the asymmetries of power and exposes asylum seekers to a specific form of epistemic violence’.Footnote ²¹ What we observe here is again a dual enactment of (in)visibility: while privacy and transparency might be understood in the eyes of the court as binaries, to understand the interactions between platforms, citizens and data, we need to move beyond (in)visibility as an issue of privacy alone.

Together, the chapters push current debates on privacy/transparency, by unpacking the power structures that are embedded in the gaze of algorithmic surveillance, and their complicity in violence. As such, they commence a conversation towards future change and critique. Arvidsson et al propose that visibility, like the ‘truth’, is not disembodied nor objective; there is no seeing everything from nowhere. Algorithmic vision from nowhere is just another ‘eye that fucks the world’.Footnote ²² Tazzioli and Valdivia suggest indeed that: ‘Further research should interrogate how does it look like to see AI at the border like a migrant: indeed, from such a perspective, claims around consent and privacy would be decentred in favour of claims for undoing racialised borders’.Footnote ²³ Algorithmic surveillance, if we follow the critical approach that is present in all the chapters, becomes more than only the set of rules that determines who or what to see. Rather, we gain an important insight that writing from a situated place rather than an objective one is a starting point to resist the racist gaze that continues to dehumanise. It is through this gaze that racialised bodies (or other marginalised bodies) become visible, while pre-existing racial hierarchies remain invisible as ‘blind spots’ of the algorithm.Footnote ²⁴ Prior to algorithmic (in)visibility, however, Fanon already discussed how dehumanisation is the result of a white gaze, where whiteness is the prototypical way of seeing. Fanon writes that: ‘already I am being dissected under White eyes, the only real eyes. I am fixed. Having adjusted their microtomes, they objectively cut away slices of my reality’.Footnote ²⁵ As Fanon shows such dehumanising gazes are not just conceptual tools, they are lived experiences of becoming fixated epistemologically. Again: fixation, stereotyping, othering: a beginning of violence.

Fixation

Thinking through the different chapters, I propose that we should stay (just a little bit longer) with the trouble of fixation. Recall Fanon’s statement: ‘I am fixed’, signalling both a racial fixation of the body and an epistemic closure of being. Connecting to the chapters and the aforementioned themes, epistemic closure and (racial) fixation is also at the heart of algorithmic violence. Ruha Benjamin writes about this as ‘a racial fix is itself a social technology that creates vertical realities – freedom, security and happiness above and bondage, precarity and misery below – all the while masking and distorting the lives of the latter from those who experience the former’.Footnote ²⁶

To describe algorithmic calculations as fixations might seem counterintuitive. After all, the three chapters illustrate a continuous desire of companies and governments to collect and share more data. In this desire, we see the opposite of fixation. Data should flow. Data should improve our knowledge. Data should enable innovative policies. Whether this is in the case of asylum proceedings in courtFootnote ²⁷ or at the borderFootnote ²⁸ or in realising a fascist desire.Footnote ²⁹ However, all authors resist the idea that more data is better or more revealing or enabling. Amidst the fluid and ever-growing ways that data can be shared, collected and stored, the extraction of data from vulnerable and marginalised populations continues to fixate persons and things into specific categories that justify political action. More data essentially reproduces the same epistemic closure: I am fixed.

As the moments of injustice and violence that are addressed in these chapters have shown us, there is no quick-fix for these issues. Furthermore, the chapters agree that a technical fix cannot address the social and political context that is (re-)produced through algorithmic calculations. We are therefore not in need of better apps or more accurate data to form ground truths. Perhaps we are in need of unfixing global data infrastructures.

Conclusion?

In the previous three sections I have examined common themes that help us understand how algorithms produce violence and complicate accountability. The chapters in this section have critiqued the assumption that algorithmic decision-making is disembodied and neutral, not troubled by a racist and sexist history or the social realities in which it operates. Such assumptions amplify a binary vision of the world and its inhabitants: visible/invisible, guilty/innocent, truth/false. Nevertheless, what possibilities are left for those who want to resist or push back against algorithmic violence and want to move beyond the binaries? Arvidsson et al suggest that we should turn to post-human feminist alliances: ‘posthuman feminist transversal alliances are de-territorial, moves along lines of flights, are Deleuze-Guattarian “packs-swarms”. Yet, these are not necessarily “bro-packs”, rather they are feminist, sense-active, de-de-selectionary and becoming’.Footnote ³⁰ Yet, what does that mean concretely? What practical or conceptual tools do such statements give us?

Tazzioli and Valdivia propose a deeper interrogation of racist borders as a site where the differential treatment of migrants becomes apparent. Their suggestion for further research is to see AI at the border like a migrant to fully understand what an alternative future could look like. Epistemic openness is furthered by shifting our perspective from the powerful to the marginalised. Similarly, Aradau’s chapter critiques the way a ‘burden to know’ is placed on ordinary citizens, while powerful institutions have limited understanding of the way data extraction and surveillance functions. Arvidsson et al similarly suggest ‘speaking is speaking up against, from below’.Footnote ³¹ Further research is needed to formulate better protection against algorithmic violence to those who are marginalised and fixed. As such, research can move from a ‘quick-fix’ to an ‘epistemological unfixing’. Such unfixing requires opening up research to Fanon’s lived experiences of racialised and marginalised communities, unsettling interpretations of the past, and making space for ‘a mode of being constantly ongoing (becoming), always shifting its relations’.Footnote ³²

McKittrick writes that ‘the sharing of ideas (no beginnings, no ends) enables a terrain of struggle, through which different futures are imagined’.Footnote ³³ As the chapters in this section show, we are in dire need of imagining and realising a different algorithmic future. Until now, algorithmic tools and governance have provided a repetition of the past. Past knowledge, hierarchies and politics re-appear through novel configurations. Much of the violence remains the same. We are fixed. To move out of this epistemic closure, perhaps sharing of critical ideas and novel suggestions of accountability are the way to envision a new future. As I find myself struggling with words to conclude, perhaps what this commentary can do, is to share and bring together ideas (no beginnings, no ends).