These three chapters offer critical diagnoses of how algorithms govern through their interactions with and beyond data. Each focusses on a particular technical practice of algorithmic governance, provides an account of its logics, and seeks to reveal the consequences of embedding such practices in assemblages of governance. For Phan and Wark, the technical practice is ‘proxification’: the ‘substitut[ion of] what we want to measure with what we can measure, allowing us to act upon data about the world as if it represented the world itself’. They provide an account of the ways in which ‘proxy logics’, and associated practices of ‘clustering’, transform racial categories and racialised governance. For Goldenfein, the relevant practice is that of the ‘human in the loop’ (HitL), understood as both a ‘regulatory strategy’ and a ‘highly malleable legal technology’. His chapter explores the ways in which the reconfiguration of the role and nature of the ‘human’ in novel socio-technical governance assemblages both redistribute accountability and preclude critical interrogation. Yeung’s focus is on the use of commercial techniques of machine learning (ML) – specifically risk profiling and citizen-scoring – in public administration and management. Through their misleading promise of accurate prediction, she suggests, these techniques ‘redistribute power, benefits and burdens in ways that are hidden in plain sight’ and at the same time ‘foreclose politics by side-stepping debate and deliberation about how collectively the state should act’.

In this short intervention, we offer, first, some reflections on what these chapters share along three distinct axes – the analytical, the methodological and the political – before, second, seeking sympathetically to extend them, including by identifying novel questions raised by their analyses.

Analytically, all three chapters are concerned with the constitutive effects of algorithmic governance on – and perhaps, for Phan and Wark, as – the collective life of a polity or social body. Phan and Wark discuss the ways in which technological routines associated with algorithmic proxification, including ‘clustering’, rework the substance and forms of race as a socially constitutive category. Their primary interest, in this chapter, is in commercial firms’ practices of consumer profiling, marketing, price differentiation and so on (without necessarily identifying these practices with ‘private’ action). By contrast, the other two chapters examine the reconstitution of the polity through the reconfiguration of routines of public administration. For Goldenfein, a HitL constitutes a ‘public’ as a political entity that is only in conversation with ‘humans’ in algorithmic systems – which in turn produces institutional and political occlusions, and concomitant lack of accountability. For Yeung, it is precisely the use of private strategies of risk management by public administrations that is the problem. In her account, the ‘public’ and ‘private’ are already constituted and distinct spheres organised around fundamentally different forms of political relationship. She traces the deleterious transformative effects on those political relationships of transposing ML from private to public, via the promise of ‘prediction’, ‘actionability’ and ‘early intervention’.

As such, the chapters share a core analytic: thinking about the relationship of algorithmic governance to institutional and organisational routines that make up day-to-day governing, such as the identity and functioning of the HitL, or the interactions between ML and public sector organisations. The chapters thus also share in a more-or-less explicit effort to expand critical approaches to algorithmic governance: to expand the aperture of critical analysis to bring into focus the relationship between algorithmic routines and organisational routines, especially in the projection of public power.

This has real merit, including at a methodological level. For one thing, this is an approach which demands attentiveness to the specificities of the relation between technological practices and organisational routines – including the particular ways in which the ‘new’ is articulated with the ‘old’. It follows that algorithmic governance is not one thing but many and varied things, demanding fine-grained analysis of institutions and organisations alongside technologies in order to manifest and understand these changes. For another thing, this is a methodological approach that allows a nuanced appreciation of continuity and change. All three chapters note that much of what is happening is at some level reminiscent of what has gone before – the history of analogue and early digital governance by proxy; the enduring presence of token humans in increasingly automated or mechanistic governance; the long history of depoliticisation through the embedding of private sector practices – including ones of prediction, actionability and early intervention – in public management. This enables the chapters to give us a sense (without being too prescriptive) of what might be different today, and whether it is a matter of degree or kind.

The chapters offer perspectives on two related changes in particular. The first is a set of transformations in the speed, scale and degree of automation – specifically, the increasing use of technologies of measurement, analysis, inference and prediction which operate semi-independently of, and rework, directed human cognition. The second is a (possibly) consequent set of changes to the relationship between governing, data, and truth claims, which result in practices of governance that both produce and are produced by knowledge claims which are constitutively provisional, unsettled and partial. Phan and Wark note, for example, the way that proxification both enables semi-automation through the use of AI and ML systems to draw inferences and make determinations, on the one hand, and at the same time produces modalities of decision-making predicated upon rapid and iterated approximation, on the other. These two developments are connected: it is precisely because proxies are always necessarily imperfect and provisional measures of ‘the real world’ that algorithmic governance emerges as a form of governance predicated on an ‘as-if’ and shifting construction of the social world. Similarly, Goldenfein draws attention to the practice of ‘red flagging’ in his description of the HitL. ‘Red flags’ arguably have a similar epistemic character: not a final determination but a provisional and uncertain indicator of a state of affairs, establishing a provisional and ‘good enough’ reality until specifically disproven. This epistemic shift does real work: it mobilises wilful or strategic ignorance of a range of contextual facts as a resource of an administrative system; and it redistributes resources by inverting the burden of proof. Finally, Yeung’s chapter shows the flip side of provisionality. As she notes, one can observe in many contexts an unthinking trust on the part of public administrators, especially in the capacity of ML to perform something called ‘prediction’, that they presume to be ‘accurate’, and a source of truth about the social world. But ‘prediction’ here means something different from its ordinary usage – ‘accurately predicting outcomes from a portion of data that formed part of [an] original data-set from which the training data was taken’ – with the consequence that accuracy (in the sense of the ‘truth to life’ of predictions) is ultimately unmeasurable.

Finally, the three chapters offer a shared sense of political disquiet. They allow the reader to explore how algorithmic governance shapes the conditions of possibility for individual and collective politics through describing the political transformations that it effects. Phan and Wark, for example, trace a shift from a politics of identity to politics of proximity (although whether it is a politics of proximity tout court, or a politics of identity made through algorithmic approximation, is not yet clear). Goldenfein similarly points to an accountability politics around ‘orphaned’ – or perhaps ‘not-quite’ – decisions. Yeung points to new distributions of power and practices of domination which bypass democratic debate and even escape public awareness. All three also encourage the reader to imagine the other forms of politics that these transformations foreclose. Their political spirit is thus not programmatic, prescriptive or policy-oriented. Rather, by refocusing analytic and methodological lenses of critique, they call for broader political reengagement or reinvestment.

We now turn to our reflections on the chapters in the same vein – to extend them analytically, methodologically and politically. Analytically, at least two of the chapters rely on the possibility and existence of some sort of constitutionalised or constitutionalisable collectivity in order to generate new politics and to produce the potential for the accountability of algorithmic governance regimes. This is most explicit for Yeung, who sees the use of ML as mapping onto an existing public/private divide (where ML travels from private to public, with negative effects). Here, each side of the divide has different logics and political and economic imperatives. It is also explicit for Goldenfein, where the ‘human’ in the loop is – and is not – answerable to an unproblematised general citizenry. Both of these chapters, then, in their own way suppose the possibility not just of a politically salient collectivity, but of one which in some fundamental sense can have a degree of autonomy from algorithmic governance itself. Phan and Wark are slightly differently oriented. Rather than focusing on a political ‘public’, they worry about the ways in which the socio-technical reconstitution of social relations through proxification generates new hierarchies, which might in turn shape possibilities for individual and collective political imaginations. And because these imaginations are crucial to ‘the task of resisting new modes of racialisation and racial discrimination premised on logics of proxification’, it will be necessary ‘to learn a new set of political techniques equal to a politics of proximity’.

Yet what are the conditions of possibility for any such ‘public’ – or sociality – to be potentially external to or a check on algorithmic governance? One of the consequences of the transformation of governance into ML routines might be to erode the political distinction between private and public subjecthood, into streams of actionable data. In other words, the movement of ML from private to public might be an effect of how ML transforms political subjectivity – a very different situation from one in which ML causes a transformation of political subjectivity as it moves from private to public. If the former is so, it would require an analysis which brings the mechanisms by which people are transformed into data into that very analysis, along with the use of data by ML. Furthermore, what does algorithmic governance do to the epistemic conditions of possibility for collective politics as traditionally understood? We noted above that algorithmic governance deals not in shared ‘truths’ but ‘as-if’ realities, and that the social collectivities it produces (e.g. ‘clusters’) can take the provisional and unsettled form of quasi-digital artefacts. To what extent, in what form, and in which spaces can social relations of solidarity survive this transformation of the epistemic foundations of the political and civic sphere?

Methodologically, we emphasise the importance of dwelling on the organisational routines of the ‘public’ sector – especially as a means of understanding how public–private relations are reconfigured through algorithmic governance. From time to time (and with good reason) the chapters assume the practices and logics of these routines. Yeung, for example, notes that for ‘digital platforms, the promise of actionable insight rests on the ability to pre-empt and personalise services offered to users, given that the size of a platform’s user-base is typically directly proportional to its market valuation. But for public administrators, actionable insight from risk prediction tools promises to optimise the allocation of public resources’. For Goldenfein,

The human and its routines are however less of sociological and political interest here, than they are of interest in terms of the ‘distribution of cognitive tasks and the reconfigurations of accountability’.

But understanding how algorithmic and governance routines couple is crucial. These chapters show us that the logics of algorithmic governance emerge not just from digital techniques themselves, but their operationalisation in an all-too-familiar and predictable context of human and institutional ‘limitations’: data gaps, fallible practices of approximation, a desire to avoid responsibility or reward allies, impossible time and resource constraints and so on. And more can be done to attend to their articulation in specific contexts. As we have noted elsewhere, the politics of big data governance (a fortiori algorithmic governance) is found not only in its internal operations, but also in the relationship between it and ordinary state institutions and routines; in how different practices imagine reworking that coupling; and in how different imaginaries of remaking organisational routines legitimated the application of big data, algorithms and the like, to state planning.Footnote ¹ So, for Goldenfein, ‘automation does not break legality, it merely rehearses and amplifies the rule of law’s unresolved tensions, which are then papered over by symbolic human oversight rules’. What, exactly, is contained in the act of papering over these tensions is enigmatic, but of real political import.

We also argue for centring reflexivity and reflexive practices further in the analysis of algorithmic governance. By ‘reflexivity’ here we mean the capacity – whether individual, organisational or institutional – for systematic self-criticism and responsive adaptation (whether in the mode of recursion, or the ‘learning’ and rewriting of pattern matching algorithms and so on). Importantly, each of the chapters identify particular forms of reflexivity as crucial to algorithmic governance: after all, one of the central attractions of algorithmic governance is its promise to enable a governance regime to adjust or adapt swiftly and at scale to the world through an apparently endless capacity to collect, analyse and process more data. At the same time, these chapters also share a critique of algorithmic governance as lacking in reflexivity in certain ways – each criticises algorithmic governance as producing unwarranted faith in algorithmic prediction, and ignorance of its flaws, limitations and weaknesses. How comfortably do such critical claims fit with the epistemic shifts noted above – in which algorithmic governance is associated with truth claims that are in some sense always already unsettled, and ways of making choices which are only ever on the basis of ‘as if’ realities?

It may not always be the case that algorithmic governance is associated with an absence of scepticism. In other contexts (and in line with the idea that there are continuities as well as changes in the novelty of algorithmic governance), we can see how some systems that have the same features that these papers diagnose have reflected on and adapted to some of the critiques. For example, development practice suffers from technologisation, and with it, an emphasis on prediction, actionability, early intervention and so on – just as Yeung’s ML does. But we also see work from within development critiquing the transposition of technical and standardised operational regimes into the public sector.Footnote ² Going back to Yeung’s terms, this shapes the political consequences of the application of algorithmic governance, in the sense of producing or foreclosing political horizons – by engaging with, reflecting on and critiquing the ways that algorithms and governance relate, from within and as part of bringing algorithmic and governance routines together. Framed in this way, enquiries into algorithmic governance might encompass questions and related approaches around the forms that reflexivity takes in algorithmic governance; how algorithmic governance shapes what counts as reflexive practice, and its distribution in governance arrangements; and how these arrangements might be historicised.

Finally, politically, we turn to whether and how the politics of algorithmic governance is found not only in transforming individual and collective politics, but also underdetermining them, thinning them out, and deferring them. In other words, and in an echo of some of the reflections above, it appears from the chapters that algorithmic governance is powerful not only for how it changes the forms of collective existence, but also how it reworks its substance, and potentially even erodes its conditions of possibility. In particular, it seems to us that algorithmic governance at least as often brackets and displaces the resolution of questions of truth and value, as it purports to resolve them. For Phan and Wark, for example, proxification constitutively involves looking away from the (unmeasurable) real world to a second-best world of available (but incomplete and poor quality) data, and measurable (but ambiguous) outcomes or behaviours. In continually approximating – and reworking its own techniques of approximation – is technology ‘opaque’ and politically neutralising, or could it be described as a means of bracketing and politically provisionalising ultimate questions of truth and value? Similarly, for Goldenfein, the human decision-maker is a spectral presence – the point of decision-making always taking place somewhere else within a complex and mysterious-by-design regime of distributed cognition.

This recalls the politics of governance projects committed to the desacralisation of the collective:Footnote ³ the enduring suspicion of determining shared matters of value in a stable fashion, and with it, the relentless critique and reconstitution of its expressions.Footnote ⁴ And as many have noted, political responses to these projects of resistance and reinvestment risk being bracketed and coopted along with all the others – becoming yet another competing expression of ultimate value.Footnote ⁵ In other words, the chapters remind the reader that critiquing algorithmic governance requires ‘breaking brackets’, as much as a resistance to ‘political foreclosure’.