I. Introduction
Public decision-makers are increasingly using artificial intelligence (AI) systems that can exercise a comparatively high degree of autonomy and aim to substitute, rather than complement, the decision-making of human actors.Footnote 1 From administering law enforcement and criminal justice to determining the provision of public health and 1 welfare services, autonomous AI systems play a growing role in government decision-making.Footnote 2 The use of this technology by public actors raises distinctive problems that do not arise in the case of private actors.Footnote 3 While legal scholars have written extensively on the practical consequences of these developments, including their impact on the rights and interests of persons directly affected by algorithmic decisions,Footnote 4 few have grappled with the public use of autonomous systems from a noninstrumental perspective.Footnote 5
By employing an analytic lens that characterizes the collective role of citizens in understanding and contributing to public decision-making, we develop a conceptually novel and institutionally actionable noninstrumental analysis of the use of advanced AI technology in government. Drawing on the work of legal theorists critiquing the privatization of government functions,Footnote 6 we argue that outsourcing certain decisions of public institutions to autonomous AI systems can be analogized to privatization and, accordingly, amounts to an impermissible delegation of public authority and abdication of public responsibility.
Our main claim is that the decisions of autonomous AI systems cannot generally be considered public decisions, that is, decisions made in the name of citizens as a whole, as opposed to decisions of particular individuals or a group of individuals. Public institutions, on this view, are not simply entities that act in our interests. Rather, they are public by virtue of being able to speak in our name and act on our behalf.
This concept of the “public” has previously been harnessed to oppose certain forms of privatization on noninstrumental grounds. Dorfman and Harel maintain that the delegation of government functions to private entities is problematic not (only) because it may have adverse consequences, but because certain decisions should as a matter of principle only be made by public institutions that represent and engage the political community as a whole.Footnote 7 For example, they illustrate that privatizing criminal punishment undermines the distinctive value of punishment as expressing public condemnation, that is, condemnation made collectively on behalf of a political community. In such cases, the legitimacy, and indeed the underlying purpose, of decision-making is rooted in the public status of the decision-maker.Footnote 8
This article extends that prior work in three main ways. First, we develop the concept of the “public” from a predominantly negative concept, which prohibits certain private decision-making, to a positive concept, which requires the active participation and engagement of citizens in the decision-making of public institutions. Second, we translate the abstract concept of the “public” into a set of concrete conditions—transparency, participatability, and challengeability (developed in detail in Section III.A)—that must be met in order for a decision to be considered genuinely public.Footnote 9 Third, we analyze how these conditions constrain and structure the use of AI by government so as to protect citizens’ collective right to understand, participate in, and contest the decisions of public institutions.
We thus break new ground by identifying and grappling with a concern that has been largely neglected to date: that the use of AI systems in public decision-making can distance and ultimately alienate the resulting decisions from their rightful author, namely the public. When an advanced AI system is entrusted with independently implementing a sufficiently broad decision-making mandate, citizens may be deprived of the opportunity to engage in meaningful deliberation, such that they can no longer be considered the author of the resulting decisions.Footnote 10 By refraining from exercising their own discretion and forgoing open discourse and debate, public actors that delegate their decision-making responsibilities to AI systems compromise the public character of hitherto public decisions and disempower the constituents of a political community.
What is the scope of this concern? That is, in which contexts is it important that decisions made by the state satisfy the conditions of transparency, participatability, and challengeability and thus count as genuinely public decisions? We identify two main contexts. The first, building on prior work in legal theory, involves goods that are inherently public, i.e., goods whose legitimacy necessarily requires that they be provided by a public actor, such as criminal punishment.Footnote 11 The second, novel context, involves goods that are not inherently public but become public by virtue of a political community incorporating them into public life and thereby imbuing them with a new social and normative status. For example, while marriage is not an inherently public good that the state must provide, if the state decides to provide that good, then it incurs an obligation to facilitate public participation in defining citizens’ rights and obligations in connection with marriage. In other words, once the state renders marriage a public practice, it must satisfy the conditions for publicness.
Applying this noninstrumental perspective to algorithmic decision-making in public institutions has important implications. Existing instrumental critiques of AI, such as concerns relating to algorithmic fairness,Footnote 12 can arguably be remedied through technical interventions in the design of AI technology. That is, if we can build more accurate and less biased algorithmic systems, then public actors should be permitted to outsource decision-making to those systems. Our account rejects this conclusion. Technical fixes, while necessary for tackling instrumental concerns associated with the use of AI in government, do not address more fundamental problems facing such delegation. Outsourcing the decision-making of public institutions to AI systems displaces human engagement and deliberation, thereby severing the connection between a political community and decisions that should express citizens’ collective judgment.
The argument we advance, to be sure, is not the only noninstrumental account of algorithmic decision-making in public institutions. John Tasioulas argues that human deliberation is necessary for sustaining “rational autonomy,” which he characterizes as “the capacity of human beings to step back in thought from their desires, and inclinations, or from the influence of established patterns of behavior, peer pressure, and so on in order to assess the pros and cons of any given course of action.”Footnote 13 Aziz Huq raises the concern that an individual exposed to an algorithmic decision may feel “disempowered and enervated by her exclusion from any effectual role in the [decision-making] process.”Footnote 14 Huq also suggests that individuals may have a participatory entitlement grounded in autonomy and moral responsibility.Footnote 15 Other theorists maintain that “inscrutable algorithms’ inability to provide adequate explanations for their predictions in specific cases may amount to a failure of the normative duty to provide reasons for decisions in individual cases.”Footnote 16 These claims, however, largely focus on the individual actions of decision-makers or the rights of citizens as individuals, rather than as members of the polity.Footnote 17
Departing from this well-trodden path, Neli Frost offers a different perspective: the integration of algorithmic decision-making in public administration could produce an “impoverished publicness” in which the dialogical processes that enable citizens to participate in governance are undermined and eroded.Footnote 18 Our account builds on this critical, yet neglected, perspective by describing the specific conditions under which citizens as a collective are empowered to partake in public decision-making, articulating the precise mechanisms through which the use of advanced algorithmic systems can undermine these conditions, and exploring potential pathways for AI to satisfy those conditions and, at times, even bolster citizens’ ability to participate in public decision-making.
The remainder of the article is organized as follows. Section II describes the features of AI systems that are increasingly used in public decision-making. Section III develops the three conditions of publicness (transparency, participatability, and challengeability), discusses the circumstances in which they must be met, and illustrates that contemporary uses of AI in government typically fail to satisfy these conditions. Section IV addresses the longer-run implications of our claims, as well as several objections to our claims, including arguments that the conditions of publicness we establish are too demanding. Section V explores institutional and technological avenues for using AI systems in a manner that protects and potentially enhances the democratic character of public decisions.
II. AI in Public Institutions
The potential benefits of using increasingly advanced AI systems in public institutions are manyfold. These systems can support high-speed, large-scale, and low-cost decision-making, including in the administration of public services, such as welfare and health care.Footnote 19 In addition, advanced AI systems can sometimes produce higher-quality decisions than human actors, especially where decisions need to be informed by large quantities of data, such as in tax enforcement.Footnote 20 Finally, AI systems can potentially mitigate human bias in decision-making, including in immigration processes and asylum claims.Footnote 21
The range of AI systems used in public institutions is vast. On one end of the spectrum are rule-based systems that explicitly encode “if-then” rules designed for a particular task.Footnote 22 On the other end of the spectrum are machine learning systems, which are not explicitly programmed to perform particular tasks but instead learn to do so from training data and optimization algorithms. While both rule-based systems and machine learning play important roles in public settings, the latter has dramatically grown in significance in recent decades. Modern machine learning systems, especially those based on neural networks, can perform a myriad of economically valuable activities, including data analysis and content generation tasks,Footnote 23 which public institutions are keenly exploiting (as we discuss below). Given the prevalence of machine learning in public administration and the normative challenges they present, our analysis focuses on this kind of AI system and, specifically, on advanced AI systems that can exercise a comparatively high degree of autonomy and purport to substitute, rather than complement, the decision-making of human actors.Footnote 24
We now turn to examine four aspects of machine learning that, from both a technical and institutional standpoint, warrant close attention. These are (1) the reliance of machine learning on large quantities of training data, (2) the treatment of decision-making as a prediction problem, (3) the opacity of machine learning, and (4) the extent to which machine learning systems can operate autonomously, that is, perform tasks independently with only limited human supervision and intervention. We address each aspect and its application in public decision-making.
A. Training Data
The performance of a machine learning system is highly dependent on the data used to train it. Desirable properties of training data include the size of the dataset, the diversity of the dataset, and the suitability of the dataset to a system’s intended use.Footnote 25 A shortcoming in any one of these properties can adversely affect a system’s performance. The importance of training data size is readily apparent in the case of image recognition. Breakthroughs in the ability of AI systems to classify images accurately arrived only when training datasets grew from thousands of images to millions of images.Footnote 26 Due to the vast quantity of data that governments collect from citizens, whether through mandatory disclosure (such as tax returns) or large-scale surveillance (such as CCTV cameras), governments often possess sufficiently large datasets to train highly performant AI systems.
Diversity of training data is a thornier issue. Continuing with the example of image recognition, a training dataset that is insufficiently diverse along race or gender lines can yield a biased AI system that reproduces problematic patterns contained in the training data.Footnote 27 In addition, a lack of diverse training data can yield brittle AI systems, that is, systems that fail to generalize to new, unforeseen contexts.Footnote 28 For example, in a canonical case illustrating a failure of generalization, an image recognition system designed to distinguish between huskies and wolves—whose training dataset included many images of wolves against snowy backgrounds—learned to mistakenly classify huskies against snowy backgrounds as wolves.Footnote 29 Although governments may possess large quantities of data, such data may be insufficiently diverse, giving rise to AI models that are disturbingly brittle or contain harmful biases.
Finally, the performance of AI systems is highly context-dependent. A system may perform well in one context (that resembles the training data) but perform poorly in another context (that does not resemble the training data).Footnote 30 For example, image classifiers trained to recognize faces of people belonging to particular races or genders may fail to recognize faces of people belonging to other races or genders.Footnote 31
B. Prediction Problems
Machine learning is, at its core, a prediction technology. AI systems based on machine learning are designed to make predictions on the basis of patterns in their training data.Footnote 32 For traditional prediction problems, such as inventory forecasting, the use of AI is fairly intuitive. Data scientists collect historical inventory data and train a model to learn to predict future inventory supply and demand. To the extent machine learning is more accurate or cheaper than other prediction tools (e.g., in fingerprint analysis), the case for using AI is relatively uncontroversial.Footnote 33
The effectiveness of machine learning has, however, prompted users of the technology to broaden the scope of tasks that they (implicitly) treat as prediction problems.Footnote 34 For example, translating between languages was traditionally a linguistic activity that involved expositing rules about language.Footnote 35 With the advent of powerful AI systems, translation has been recast as a prediction problem: models learn statistical patterns in language and can produce accurate translations by predicting which words are most likely to follow other words.Footnote 36 A similar transformation has occurred in medical diagnosis. Expert human evaluation of a patient’s characteristics and symptoms can be replaced with a pattern recognition tool that predicts which diagnosis best reflects the historical diagnoses of a similarly situated patient.Footnote 37
Although not altogether new, this shift toward casting policy problems as prediction problems has spread to public institutions, including the courts.Footnote 38 Richard Re and Alicia Solow-Niederman describe the development as a transition from “equitable justice” to “codified justice.”Footnote 39 For example, criminal sentencing, which traditionally relied on considerations such as mercy, retribution, and personal responsibility,Footnote 40 now places increasing weight on recidivism predictions.Footnote 41
Apart from the well-documented inaccuracies and biases of machine learning, the growing use of prediction tools presents a more fundamental question: Is it appropriate to treat public decision-making exclusively as a prediction problem amenable to machine learning? If so, under what conditions? The problem in adopting such an approach is that typically prediction involves uncertainty, and uncertainty raises moral and political concerns that ought to be resolved on moral and political grounds. While prediction may in some circumstances guide the decision-maker, including by providing critical empirical information, it must not dictate their decision. Before attempting to more fully address these concerns, we consider two further aspects of machine learning.
C. Opacity
Deep neural networks, on which most advanced AI systems are based, are notoriously opaque. While we understand in general terms how neural networks operate—namely, by learning to map a set of inputs to a set of outputs through an optimization processFootnote 42—we cannot typically explain why a neural network produces the particular outputs that it does.Footnote 43 For example, although we understand in detail the design of language models and have some insight into which components of the model are responsible for particular outputs,Footnote 44 we have only a very limited understanding of the reason a model produces particular outputs.Footnote 45 This issue affects algorithmic decision-making more broadly, including in the context of public decision-making. For example, AI systems used by governments to make decisions concerning immigration cases rarely communicate the reasoning behind particular decisions.Footnote 46
The opacity of contemporary AI systems has significant implications for the use of these systems, especially by public institutions subject to civic and democratic duties. First, the opacity of AI systems suggests that they should not be deployed in settings where explanations are required or expected, whether as a matter of law or as a widely accepted social norm.Footnote 47 Second, even where explanations are not required or expected, the opacity of AI systems can obscure indications of whether a system is aligned with the values of its user, the persons it affects, or broader societal interests.Footnote 48 Consider the following examples. In judicial deliberations, it is crucial that the reasoning of decisions be explicit, both to enable aggrieved persons to challenge decisions and to foster the development of socially responsive judge-made law while protecting judicial legitimacy.Footnote 49 Similarly, in administrative contexts, it is critical that decisions be made demonstrably on the basis of appropriate considerations.Footnote 50 Black-box AI systems render this impossible.
D. Autonomous Operation
The final feature of advanced AI systems we consider here is the degree to which they can operate autonomously, i.e., perform tasks independently with only limited human involvement.Footnote 51 This feature is particularly critical to scrutinizing the legitimate uses of advanced AI systems in public institutions because the ability to operate autonomously can encourage public institutions to delegate more complex, consequential, and broadly scoped tasks to AI systems, repose greater trust in those systems, and exercise weaker oversight over them.Footnote 52
While many algorithmic systems currently serve as decision support tools that assist human actors in making decisions, there is a growing trend in deploying advanced AI systems that can operate with a greater degree of autonomy and can, from a technical (but not necessarily normative) perspective, make decisions with only limited human involvement. For example, language models, once regarded as “stochastic parrots”Footnote 53 that merely reproduce text contained in their training data, can now perform complex tasks, particularly in digital environments, such as government computer networks or databases.Footnote 54
Public institutions have already experimented with these systems. For example, in early 2025, the US Department of Government Efficiency (DOGE) used language model–based AI systems to evaluate the so-called “job justifications” of federal employees.Footnote 55 Looking ahead, governments could integrate these systems into other high-stakes domains, such as welfare services decisions and even emergency response planning.Footnote 56 As we discuss below, our perspective on the use of increasingly autonomous AI systems in public institutions is as follows: the greater the degree of autonomy exercised by or afforded to an AI system, the greater the risk of distancing and alienating the resulting decisions and actions from their rightful authors, namely the public.
III. The Public Character of AI
The following section has two parts. In Section III.A, we identify and characterize three conditions that must be met in order for a decision of a public institution to be considered genuinely public, namely transparency, participatability, and challengeability. In Section III.B, we specify the classes of decisions that ought to be public and, consequently, are required to meet the three conditions of publicness. In both parts, we illustrate that contemporary AI systems deployed by public institutions typically fail to satisfy these conditions.
* * * * *
At the most abstract level, a public decision is a decision that is made in the name of citizens as a whole rather than by a particular individual or a distinct group. Public institutions are not simply entities that act for us. Instead, they are public by virtue of being able to speak and act in our name. This requires that their decisions be subject to public scrutiny and shaped through public deliberation involving diverse segments of the population.
This characterization of publicness calls for further specification. To be genuinely public, a decision must meet certain conditions. First and foremost, the public must be able to understand and participate in the decision-making process.Footnote 57 Prior to a decision being made, citizens must be afforded the opportunity to play a role in engaging with the decision-relevant considerations and reasoning, and deliberating about them. Without such an opportunity, citizens cannot regard the decisions of public institutions as their own. This is a demanding requirement because it necessitates providing opportunities for actual engagement by members of the public, as opposed to merely reaching a decision that happens to correlate with the public’s interests.
What is the nature of the public engagement that is at stake here? Is such public engagement feasible or even desirable? Consider the normatively charged domains in which public institutions might, and sometimes already do, use AI systems: pretrial detention, asylum decisions, child abuse investigations, welfare fraud detection, and many others.
In these domains, while it is unfeasible and often inappropriate to involve members of the public in particular decisions, this does not imply that such decisions ought not be subject to some form of public deliberation. Specifically, it is critical that at least some decisions are exposed to the public and give rise to shared reflection—for it is through public contestation that latent questions of justice become intelligible, that the moral sensibilities of citizens are formed and refined, and that the normative contours of collective life are crystallized. Decisions concerning particular individuals have indeed become focal points for such public engagement.Footnote 58 Even if citizens do not (and should not) have the formal power to make such decisions, their involvement and participation are prerequisites for democratic legitimacy.
One important reservation should be made before we develop the conditions of publicness. There are decisions that, although public in an important respect, cannot satisfy these conditions. Consider the case of a democratically elected leader who declares war on the basis of evidence that is not disclosed, claiming that a surprise attack is about to happen. This decision is indeed a public decision that can be attributed to citizens and it is made in the public’s name. Yet, the conditions of publicness that are presented below are not satisfied. Such deviations may be justified when there are compelling reasons for them, but they should be understood as exceptions that stand in need of justification. The deviation is justified because disclosure would defeat the purpose of the decision itself. This example illustrates that at times there are decisions made by the state that count as public in a different sense than the one developed here.Footnote 59
Last, let us raise an additional challenge. One may think of nondemocracies with a strong sense of national pride where decisions made by a dictator are supported by the public and there is a sense that such decisions can count as public. We do not preclude the possibility of using the term “public” in a different context and for different purposes. We defend our characterization simply on the basis of the way the term is often understood in liberal democracies and we also believe that this understanding is one that has important normative implications.Footnote 60
While our concerns arguably portray an idealized account of public decision-making that is rarely fully satisfied in practice, they nevertheless capture an important point: the greater the public engagement in decision-making, the greater the extent to which citizens can be considered the authors of the resulting decisions and the stronger the public character of those decisions. Participation is not binary but scalar, varying along a continuum. Accordingly, although the public cannot be involved in each and every decision made by public institutions, its involvement in at least some of these decisions is key to maintaining their public character. We now turn to the specific conditions comprising that public character.
A. Conditions of Publicness
Certain decisions are considered binding because they are made by public officials or public entities. For instance, a judge’s pronouncement that a person should be punished is binding because it is made by a judge or a court. A similar pronouncement made by a private individual, even if that individual has comparable skills or experience, would not have the same effect. Judges are public officials; private individuals are not. But what is at the heart of this public–private distinction? What imbues certain decisions with a public character?
While the notions of “public decisions” and “public officials” can be understood in many ways, we approach the terms with a normative purpose in mind: delineating the conditions for the legitimate use of AI systems in public decision-making. Accordingly, we characterize public decisions as decisions that can normatively be considered decisions of the citizens, that is, decisions in which citizens can (and should) be regarded as the authors of those decisions. This, of course, does not imply that citizens must in practice be involved in actually making all decisions. Rather, it suggests that citizens must be able to understand and evaluate the underlying reasons for decisions and, where appropriate, contest them. In other words, public decisions are decisions that can, at least in principle, be shaped by citizens’ actions and values, be attributed to the public, and for which the public can ultimately be held responsible.
Clearly, the conditions we establish can be satisfied to varying degrees and depend on the particular institutional context. All things being equal, the greater the extent to which a decision-making process exhibits transparency, participatability, and challengeability, the more “public” the resulting decision and the more that citizens can be considered authors of the decision.
1. Transparency
The first condition that must be met for a decision to be considered a public decision is transparency. Given that the term is highly contested and the subject of a vast literature, it needs to be carefully unpacked. Our starting point is that the legitimacy of decisions made by public institutions turns on their public character. The decisions must be considered or count as if they have been made by the public. Accordingly, citizens must have an opportunity to participate in the decision-making process such that they can regard the resulting decisions as their own and not merely as decisions that happen to align with their interests or values.
Under this account, a decision cannot be regarded as a public decision if the considerations underlying the decision are opaque. To be an author of a decision, a person needs to understand the reasoning behind that decision. This is because authorship requires not merely that the decision is one that the author agrees with or supports. It requires that the author engages with the reasoning underlying the decision. It is this active engagement that makes the decision one that can be properly attributed to an actor. Transparency, on this view, is not merely a means to improving the substance of public-impacting decisions, but an institutional feature without which a decision cannot be considered to have been authored by the public.
Can AI systems satisfy this demanding standard? To answer this question, we need to tease apart the notion of transparency in the context of AI systems. According to some scholars, algorithmic transparency falls neatly within the good governance principle of reason-giving, which is needed to facilitate judicial and administrative review.Footnote 61 Transparency, including as facilitated by an individual’s “right to explanation,”Footnote 62 can shed light on deficient reasoning and assist in challenging the resulting decisions. According to other scholars, algorithmic transparency supports developing a new science of machine interpretabilityFootnote 63 and even radically reengineering the technology to become more legible to humans.Footnote 64 Regulations in major jurisdictions are beginning to enshrine various, and sometimes conflicting, notions of transparency.Footnote 65
As we argue above, these approaches to transparency share one thing in common: they treat transparency as instrumentally valuable.Footnote 66 That is, they consider transparency a useful tool for mitigating or preventing defective algorithmic decisions. Transparency, understood in this way, is not inherently important. It is instead a means to achieving socially desirable ends.
Our approach is different: we side with those who regard transparency as intrinsically valuable.Footnote 67 More specifically, we see transparency as a precondition for producing decisions that are genuinely public. Accordingly, satisfying existing instrumental notions of transparency is not necessarily sufficient or appropriate for our purposes. A more robust form of transparency is needed.
To make this concrete, consider the following scenario: citizens know that a particular AI system is very likely to make a decision that correlates with their interests or judgments. Citizens even understand in detail how the system operates. These facts, we suggest, do not imply that citizens necessarily become the author of decisions made by the AI system. To become the author of the decisions, citizens need to understand and engage with the normative reasons underlying the decisions and not only assess the desirability of the substantive outcome of the decision.Footnote 68
This requirement is especially important in the context of processes or institutions designed to promote public participation. When transparency is required merely to protect the interests of an individual affected by a decision, that individual presumably has some decision-relevant knowledge acquired independently of any transparency mechanism. After all, the decision concerns their personal rights and interests. By contrast, when transparency is required for the purpose of facilitating broader public participation in decision-making, citizens are less likely to possess highly granular decision-relevant knowledge because the decision does not (directly) concern their personal rights and interests.
Accordingly, in these cases, a robust form of transparency would need to tailor the nature and scope of information it provides to the particular processes and institutions that facilitate public participation in decision-making. Although decisions of public institutions do not consistently satisfy this demanding standard, we suggest that public officials can satisfy it, at least in principle, and often strive to do so in practice.Footnote 69 In particular, public officials often satisfy this robust form of transparency in the case of normatively controversial decisions, by specifying the reasons for their decisions.Footnote 70 Administrative law in several major jurisdictions, including the United States, requires that government agencies provide reasons for certain decisions.Footnote 71
Presently, decisions of advanced machine learning–based systems cannot satisfy this demanding standard. Despite efforts to make such systems more transparent, our understanding of the reasons for the decisions they produce usually remains very limited. For example, mechanisms that enable language models to communicate their internal reasoning processes (known as their “chain of thought”) have been demonstrated to be unreliable—i.e., the outward explanation of their reasoning is not in fact responsible for the system’s outputs. (In experiments, researchers can corrupt or tamper with the system’s ostensible reasoning, yet the system nonetheless reaches the same outcome.)Footnote 72 Accordingly, the problem in this case is not merely a lack of transparency, but a façade of transparency that offers false assurance of understanding the reasoning behind algorithmic decision-making.
2. Participatability
The second condition that must be met for a decision to be considered a public decision is participatability: the decision-making process must facilitate broad public participation, especially in shaping the considerations of the public actor making the decision.Footnote 73 At first glance, it may seem that participatability does not necessarily require any actual participation on the part of citizens. The mere opportunity to participate might be sufficient, provided citizens can access and understand the decision-relevant considerations and decide whether or not to endorse the decision. On this view, transparency guarantees participatability. If citizens can understand and deliberate regarding the considerations underlying a decision, then citizens should count normatively as the authors of the decision.
But this understanding of participatability is, in our view, too narrow. Participatability necessarily implies some degree of actual participation. Specifically, it requires that citizens acquire and retain the capacity and motivation to actually participate in decision-making. For decisions to be participatable, citizens must, in fact, participate in the making of at least some decisions. If citizens were to never participate in decision-making, their deliberative capacity would degrade, and consequently, the prospect of public participatability would be foreclosed. Seen in this light, actual (if only occasional) participation is vital to maintaining an engaged citizenry capable of partaking in the deliberative processes inherent in public decision-making.Footnote 74
Needless to say, the foregoing analysis does not imply that every citizen must participate in the making of every public decision. We suggest that participatability requires that all (or most) citizens actively engage in public discourse, be attuned to the considerations that underlie public decisions, and be afforded the opportunity to express their opinions on public affairs. If these conditions are met, citizens can be regarded as having participated.
The conditions we establish here are not as utopian as they may seem. Legal systems contain numerous rules and doctrines that facilitate and even mandate public participation. One familiar example is juries.Footnote 75 Another example, at least in the United States, is the principle of open courtrooms, which is a fundamental aspect of the legal system rooted in the First Amendment and the Sixth Amendment to the US Constitution.Footnote 76 A further example is notice-and-comment rulemaking procedures, which bring proposed regulations to the public’s attention, enable the public to submit comments, and require government agencies to consider those comments.Footnote 77
Can participatability, understood in this way, be achieved in the context of algorithmic decision-making? One response is that the public can participate in forming the legal rules governing the use of AI systems, particularly in government contexts.Footnote 78 Because citizens shape the rules regulating these systems (e.g., through notice-and-comment rulemaking), they can perhaps be regarded as participating in the decision-making processes subsequently undertaken by the systems.
We, however, argue that this is not sufficient. While participating in the framework governing the use of a decision-making system is important, it is not synonymous with participation in the actual use of that system. According to Henry Richardson, “discerning morally relevant features of a case, marshalling facts, analyzing arguments and ideas, citing examples and comparing the case at hand with a range of related cases … are generally indispensable means to productive discussion and deliberation about concrete ethical issues.”Footnote 79 For public deliberation to flourish, it needs to engage with concrete facts and real-world scenarios. John Tasioulas puts it simply: “It is one thing to take responsibility for the design and general adoption of an automated system that yields often unanticipated decisions …, it is quite different actually to make a decision in such a case and stand behind it.”Footnote 80 In sum, it is the participation in concrete decision-making, as opposed to abstract rule-making, that triggers moral sensibilities and stimulates fruitful political discourse.
This observation helps address a possible objection to our argument, namely that citizens could be regarded as participating in decision-making if the decisions of an AI system accurately reflect their considered judgments. According to this objection, active participation becomes unnecessary where there is sufficiently close correlation between the decision of the AI system and the judgments of the public. We reject this objection as it overlooks the dialogical process prompted by actual public participation in decision-making. That is, what gives citizens authorship over public decisions is not (only) that those decisions reflect what they would have chosen, but that they actively engaged in the reasoning that produced them.
Another perspective suggests that participatability could potentially be facilitated through human oversight of algorithmic decisions.Footnote 81 In our view, this would depend on, among other things, the timing of the oversight. If human oversight takes the form of post-fact audits, it could not be said to foster participatability. Such oversight serves to evaluate the quality of the resulting decisions; it does not constitute meaningful public participation in the decision-making process because it takes place after the decision has been made. If, however, human oversight takes the form of real-time human scrutiny of decision-making processes, there is a far stronger case for satisfying the condition of participatability because such oversight could actually impact the relevant decision. The degree of participatability thus turns on a combination of the decision-making context, the AI system being used, and the rigor and timing of human oversight.Footnote 82 Given this, it is worth noting that more expansive and resource-intensive human oversight could increase the cost of deploying AI systems in practice and, arguably, undermine the rationale for using them in the first place.Footnote 83
3. Challengeability
The third condition that must be met for a decision to be considered a public decision is challengeability: the considerations underlying a decision must be sufficiently comprehensible such that even citizens who have not actively participated in making the decision can scrutinize the decision and, if they wish, challenge or question the decision.Footnote 84 Importantly, we use the term challengeability rather than contestability because the latter typically refers to a right reserved for persons directly affected by a decision.Footnote 85 Our discussion of challengeability, meanwhile, refers to the right of citizens more broadly to be regarded as authors of public decisions and to challenge those decisions.
As with participatability, challengeability relies on engaged citizens who have the capacity and agency to deliberate over and, if necessary, challenge public decisions. Yet there are important differences between participatability and challengeability. The former requires significant involvement on the part of citizens. Challengeability, in contrast, is a less demanding requirement. It can be fulfilled through the involvement of broader (and perhaps less engaged) segments of the public. That is, decisions made by public authorities need to be justified in ways that enable many citizens to review the decision and challenge it even if they do not (and perhaps cannot) participate in making the decision itself. Notably, the challenge need not be a formal or institutional challenge; it can simply consist of citizens forming a critical opinion on a particular decision of a public official or agency and perhaps sharing this critical opinion with others.
In sum, there are two factors that differentiate participatability from challengeability: the depth or intensity of the engagement (or the potential for such engagement) and the scope of persons involved. Moreover, as with participatability, it is not sufficient that public decisions be challengeable only in principle. Individuals must actually engage in challenging public decisions, at least occasionally. To be clear, this does not require that every citizen regularly mount challenges against public decisions. Rather, it requires that citizens actively engage in public debate and have meaningful opportunities to challenge public decisions. Without this, citizens cannot be regarded as the authors of a public actor’s decisions.
How can challengeability be achieved in the context of algorithmic decision-making? One approach is to rely on an individual’s right to contest algorithmic decisions after the fact, as enshrined in EU law.Footnote 86 However, this approach is problematic for several reasons. First, legal mechanisms instituting the right to contest algorithmic decisions are relatively limited.Footnote 87 Second, it is unclear whether there exist technical mechanisms to give practical effect to this right, including due to the opaqueness of commonly deployed AI systems.Footnote 88 Third, the right to contest algorithmic decisions is, at least in its current form, fundamentally individualistic.Footnote 89 That is, the right serves the particular person affected by a particular decision, rather than the public more broadly.
4. The Three Conditions in Action
The case of Loomis Footnote 90 vividly illustrates how the three core conditions of publicness—transparency, participatability, and challengeability—come under strain in practice. Eric Loomis’s six-year prison sentence was determined in part by the COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) algorithmic system, which provided a prediction of his likelihood of reoffending. Although the judge formally retained discretion, the algorithmic system played a central role in determining the sentence. The controversy that followed the case revealed not only issues regarding the accuracy and fairness of algorithmic systems, but deeper concerns regarding whether such systems can be squared with basic procedural ideals—which can be observed through the three conditions of publicness.
First, the case of Loomis exposed a profound deficit of transparency. Neither Loomis nor the public could access the model’s source code, variables it used (and their weighting), or internal logic because they were protected as trade secrets. As a result, neither the defendant nor the public could understand how legally salient categories—such as dangerousness or recidivism risk—were operationalized in the system. Transparency here is not merely epistemic; it is normative. When the grounds of a coercive decision are hidden, the state cannot plausibly claim to govern through reasons that are publicly intelligible.
Second, the case illustrates how algorithmic sentencing undermines participatability. Meaningful participation presupposes that citizens (as well as affected individuals) can engage with the reasons offered against them—by explaining, contextualizing, or contesting the factual and normative premises on which those reasons rely. In this sense, algorithmic decision-making risks hollowing out the idea that individuals are coauthors, rather than mere objects, of the institutional processes that govern them.
Third, and most sharply, Loomis foregrounds the problem of challengeability. A right to challenge a decision presupposes the ability to identify potential errors, biases, or inappropriate normative assumptions behind a decision. Where the operative reasoning is inaccessible, challenges become speculative or formalistic. The defendant, as well as citizens reviewing the case, could allege unfairness in general terms but could not point to specific flaws in how the algorithm assessed the particular defendant. The case thus illustrates how publicity and contestation of individual cases are not merely safeguards against mistaken outcomes, but structural preconditions for sustaining a legal order committed to reason-giving, participation, and the possibility of principled contestation.
* * * * *
The foregoing analysis examined the conditions required for a decision to be considered a genuinely public decision. But why does this matter? Why and when should we be concerned about whether or not certain decisions maintain their public character? We argue that the need for transparency, participatability, and challengeability is not uniform, and does not apply to all decisions. Rather, it varies depending on the type of decision, the epistemic demands involved, and the moral status of the decision.
B. Types of Decisions That Must Satisfy the Conditions of Publicness
We identify two general categories of decisions that need to satisfy the conditions of publicness: (i) decisions concerning the provision of intrinsically public goods (such as criminal punishment), and (ii) decisions that become public by virtue of the state incorporating a domain of action (such as marriage) into the public sphere and, thereby, imbuing that domain with a degree of publicness that triggers the conditions of transparency, participatability, and challengeability.
1. Intrinsically Public Goods
Intrinsically public goods are goods whose value depends on the status of the actor that provides them. What is important with respect to these goods is not only the provider’s technical competence, that is, their ability to reach a correct or just decision, but also the provider’s identity or status. Specifically, the provider must be of a public character.
This view is legislatively entrenched in some jurisdictions. For example, the US Federal Activities Inventory Reform (FAIR) Act of 1998 describes an “inherently governmental function” as “a function that is so intimately related to the public interest as to require performance by Federal Government employees.”Footnote 91 The primary reason for this requirement is a principled one, namely that public officials (and only public officials) speak in the name of the public.
The example of criminal punishment is instructive. Criminal sentencing is often perceived to be an intrinsically public good.Footnote 92 Although private actors can potentially assist in accomplishing various goals of criminal justice, such as running more cost-effective private prisons that strengthen deterrence, these actors are defective in one key respect: they lack the public status and authority to punish. Punishment, as Dorfman and Harel argue, is an expressive or communicative act of public condemnation and must, therefore, be carried out by a public actor.Footnote 93 A private individual has no right to impose a criminal punishment because her view concerning the wrongfulness of the act has no greater weight than the view of the alleged wrongdoer. Seen in this light, the resulting good—public censure—is inherently tied to the actor that provides it, namely, the state.Footnote 94
Identifying certain functions as “intrinsically public” has important doctrinal implications. When striking down a law that sought to establish private prisons, the Supreme Court of Israel held that decisions affecting the fundamental rights of inmates can only be made by public institutions:
The special constitutional status of the right to personal liberty and the fact that it constitutes a condition for exercising many other human rights mean that the legitimacy of denying that liberty depends to a large extent on the identity of the party that is competent to deny that liberty and on the manner in which that liberty is denied.Footnote 95
Delegation of decision-making to AI systems, we thus suggest, can be roughly analogized to privatization.Footnote 96 Because AI systems, like private corporations, do not qualify as public actors, they cannot provide intrinsically public goods such as criminal sentencing. While AI systems may be instrumentally useful in achieving certain criminal justice goals, their use is limited. For example, to the extent that punishment is an intrinsically public good, designed to communicate public condemnation, the determination of criminal sanctions cannot be delegated to an AI system, however powerful its predictive capacities. Decisions resulting from such delegation could not be regarded as public decisions attributable to the citizens and would arguably amount to an illegitimate use of violence.
Unlike a judge or other public official,Footnote 97 AI systems cannot generally be said to act in the name of the public. While the technology may promote efficiency or reduce noise in criminal sentencing,Footnote 98 it typically lacks the public character, and therefore legitimacy, to provide intrinsically public goods.
2. Incorporation into Public Institutions
Intrinsically public goods, however, are not the only domain in which decisions must satisfy the conditions of publicness. We argue that, at times, if the state decides to incorporate a domain of action into public institutions, then those institutions incur an obligation to comply with the conditions of publicness.
This position reflects a foundational judgment that the state’s choice to perform a particular function is not merely an administrative choice of bureaucratic convenience, but the adoption of a normative stance concerning which social practices are deemed to have collective significance and therefore warrant collective decision-making. When a polity asserts that certain decisions be governed by public institutions—such as marriage, the formation of companies, or changes in civil status—it signals that these decisions have significance that extends beyond the particular individuals affected and impacts the political community as a whole.
The example of marriage is illuminating. At one level, marriage is an intensely personal commitment between two individuals, grounded in affection, intimacy, and mutual choice. Yet virtually all modern legal systems require marriage to be formally registered, recorded, and publicly verifiable. Unlike intrinsically public goods, the decision to incorporate marriage into public institutions is a discretionary decision on the part of the polity. By making decisions concerning who can marry, what duties spouses have to one another, and what rights are granted by the state to married couples, the state shapes key aspects of the social and economic status of intimate relations.
We argue that once the state incorporates such a social practice into the public sphere, it transforms that practice into a process of public decision-making that must comply with the conditions of publicness. At a minimum, such decision-making must be inclusive, giving voice to those whose interests, identities, and ways of life are directly affected; transparent, so that the grounds for the decisions are publicly articulated and contestable; and deliberative, so that competing conceptions of value can be confronted rather than suppressed. Absent such conditions, the public incorporation of social practice risks degenerating into an exercise of coercive state power rather than an expression of shared public authorship.
IV. Taking Stock
The following section has three parts. The first addresses the longer-run implications of our argument for civic engagement. The second considers whether our argument falls prey to the nirvana fallacy by imposing unreasonably demanding conditions on public decision-making. The third explores the tradeoffs implied by our argument for productively using AI systems in public institutions.
A. Civic Engagement
Our argument thus far has emphasized the importance of civic engagement. Building on the view that the privatization of government functions “cuts off the link between the decisions and the citizens and, therefore, erodes political engagement and its underlying notion of shared responsibility,”Footnote 99 we have argued that the delegation of public decision-making to AI systems that exhibit inadequate transparency, participatability, or challengeability could sever ties between public institutions and members of the polity.
The concerns with the delegation of public decision-making to AI systems are also tied to the nature of the technology itself and the market structure in which it is developed. Produced by private corporations, the most advanced AI systems, including language models, suffer from serious transparency problems.Footnote 100 Limited information is provided regarding their training data, architecture, and other important features.Footnote 101 In addition, these AI systems are often sold by for-profit corporations committed to business interests rather than civic duties or responsibilities.Footnote 102
If this trajectory continues, then more acute problems will arise in the longer run. Declining civic engagement stemming from the delegation of public decision-making to AI systems could compound in a dangerous feedback loop. If citizens withdraw from public decision-making, they may gradually lose the capacity and motivation to partake in political life, prompting a recursive process of political disempowerment and exclusion.Footnote 103 In time, citizens may be increasingly alienated from the political processes that govern society. The issue is not only that the automation of public decision-making will reduce current levels of civic engagement but that automation may altogether foreclose the possibility of meaningful civic engagement.
The case of criminal sentencing is once again instructive. Criminal sentencing concerns more than the accomplishment of particular social outcomes (e.g., deterring n prospective criminals by x units of deterrence). It is bound up in public discourse and engagement on fundamental social questions (e.g., balancing the role of mercy, deterrence, and retribution). But criminal sentencing is hardly the only example. Civic engagement is also necessary with respect to decisions concerning the provision of other goods, such as welfare or public health services. More generally, civic engagement is particularly important with respect to decisions that involve normative tradeoffs, such as between equality and efficiency or between domestic justice and cosmopolitan justice. The reliance on algorithmic systems for making these value-laden decisions, even if they ultimately accomplish socially desirable outcomes, is likely to come at the expense of civic engagement. Public institutions, we suggest, should ask themselves whether this is a price worth paying.
B. The Nirvana Fallacy and Double Standards
Before exploring the implications of our analysis for governing and circumscribing the use of AI systems in public decision-making, let us briefly examine a potential objection to our analysis, namely whether our account implicates the nirvana fallacy—that is, the presentation of an idealized and unrealistic portrait of current public decision-making practices.Footnote 104 More precisely, do the decisions of public actors presently satisfy the conditions of transparency, participatability, and challengeability? And if not, can we expect AI to satisfy these conditions? The concern, at its core, is that we risk holding AI systems to an unreasonable double standard.Footnote 105
Consider the issue of transparency in judicial decision-making. While we typically expect judges to reach decisions on the basis of transparent and impartial deliberation, critical theories have challenged this account, demonstrating (or, at least, purporting to demonstrate) that judges’ sectarian views and personal biases can affect judicial decisions.Footnote 106 To the extent this occurs, judicial decision-making can hardly be said to foster transparency. Consequently, Huq’s observation that “machine-learning instruments may be roughly as opaque as human decisions” offers bittersweet comfort.Footnote 107 Neither human decisions nor algorithmic decisions can be genuinely public. Both are impenetrable black boxes.
We, however, have reservations about this view. Our position draws on the kind of transparency we expect from all decision-makers, whether human or machine. We are not interested in the inner workings of decision-makers—which cannot be fully understood in either humans or AI systemsFootnote 108—but in decision-makers communicating legible reasons for the decisions they make. Cary Coglianese and David Lehr describe this as “reasoned transparency,” as distinct from “fishbowl transparency.”Footnote 109 Reasoned transparency requires the decision-maker to provide arguments such that her decision could be reviewed by others who could then judge whether these arguments justify the decision. One need not establish that the arguments are ones that, as a matter of fact, guided the decision-maker. Instead, it is enough that these arguments can be reviewed by a third party who can evaluate their cogency and normative force.
It is, we suggest, reasoned transparency that offers meaningful insight into public decision-making and enables members of a political community to participate in and, where necessary, challenge public decisions. Without reasoned transparency, public institutions lack democratic legitimacy. Seen in this light, the condition of transparency does not discriminate between human and algorithmic decision-makers. The condition is neither novel nor unique to AI systems. It is a core aspect of procedural due process that already applies and is met in a wide range of administrative contexts.Footnote 110
Similarly, the conditions of participatability and challengeability that we develop are not unique to AI systems. Participatability and challengeability derive from deeply rooted notions of civic engagement and inclusive democracy and existing due process rules,Footnote 111 but notably seek to broaden the beneficiaries of these conditions to include all members of a political community—not only persons directly affected by a decision. This extension draws inspiration from jurisdictions in which a broad range of actors can challenge public decisions even if they are not directly affected by those decisions.Footnote 112
C. Revisiting the Tradeoffs
A further objection to our account is that we arguably understate the potential benefits of using advanced AI systems in the public sphere, including their capacity to support prosocial activities at a large scale and relatively low cost.Footnote 113 This is not our intention. We recognize and value these benefits. We also recognize the practical limitations and risks of these AI systems, as explored in prior literature.Footnote 114 Finally, we recognize that AI systems are sometimes introduced in order to remedy democratic deficits in decision-making, including to support civic engagement in public decision-making.Footnote 115
How then should policymakers respond to our noninstrumental critique of using AI systems in public institutions? In short, we propose that our critique should inform the tradeoffs involved in choices concerning the use of AI systems in public decision-making. While policymakers currently weigh the efficiency gains for governments against the concrete harms to individuals and communities, we suggest that policymakers should also consider questions of democratic legitimacy.Footnote 116 For example, do the decisions to be delegated to autonomous AI systems involve the provision of inherently public goods or social practices that the state has incorporated into its sphere of action? Do the decisions give rise to value-laden choices that warrant broad civic engagement? To what extent can institutional and technical safeguards protect the public character of the resulting decisions, and importantly, will those safeguards undermine the benefits of using AI systems?
In response to these questions, we argue that decisions in some domains should (almost) never be delegated to AI systems. For example, criminal punishment is such a domain as punishment conveys public condemnation and public condemnation requires public engagement. It is no accident that some decisions concerning pretrial detention have been delegated to AI systems, while the determination of punishment remains an exclusively human responsibility. The former is often characterized as a technocratic procedure that rests upon the prospects that the defendant commits a crime or escapes, while the latter is characterized as a matter of moral or political judgment. Note, however, that we do not necessarily wish to endorse this distinction.
The implications of this characterization extend beyond cases in which AI systems have no legitimate role to play. Generally speaking, we suggest that the more a decision can be characterized as a moral or political judgment as opposed to a technocratic procedure, the more rigorously it must comply with the conditions of publicness. At the risk of analytic circularity, a state’s delegation of certain decisions to AI systems, at least where there is broad public engagement and support for such delegation, arguably suggests that the decisions in question are comparatively technocratic in nature and thus need not satisfy the conditions of publicness especially rigorously.
V. Pathways to Genuinely Public AI
Our final contribution explores how AI systems could protect and even enhance the democratic character of public decisions, subject to the conditions we established above. For AI systems to genuinely enfranchise citizens, it is not sufficient that the values embedded in an algorithmic system align with broader societal interests.Footnote 117 AI systems must, to the extent required by the context in which they are deployed, fulfill the conditions of transparency, participatability, and challengeability. As we argued earlier, there are at least two important features that characterize these requirements: (1) citizens must engage in reasoning; (2) the reasoning must be concrete reasoning, i.e., reasoning about particular cases, as distinct from reasoning (only) about abstract rules. We will now briefly examine whether recent developments in AI technology can address these concerns, organizing our analysis along three different pathways.
A. Avoiding Black Boxes
The first pathway we consider involves avoiding opaque black-box AI systems wherever possible. Cynthia Rudin has challenged the widespread assumption that “black box systems are a necessary evil, because they have a supposed performance advantage over simpler … systems,” describing the tradeoff between transparency and performance as a “myth.”Footnote 118 For example, a simple algorithm relying only on age, gender, and prior criminal record could predict rearrest just as accurately as the proprietary COMPAS algorithm that supposedly used up to 137 inputs.Footnote 119 With respect to language models, which are perhaps the quintessential black-box system, attempts to understand their internal reasoning (known as their “chain of thought”) have been met with mixed results.Footnote 120 While it is possible to instruct language models to provide detailed explanations for their outputs and measure whether those explanations, in fact, account for the resulting output, these techniques are not currently robust.Footnote 121 If, however, these techniques improve, then language models would arguably exhibit reasoned transparency.Footnote 122 By rendering the reasoning behind their outputs more legible, such techniques could enable AI systems to better fulfill the condition of transparency and, in doing so, also increase the prospects of participatability and challengeability.Footnote 123
B. Designing AI for (and by) the Public
The second pathway to protecting the public character of algorithmic decisions involves expanding the role of citizens in AI development and better incorporating citizens’ values into the design of AI systems.Footnote 124 Given that state-of-the-art language models are trained (in part) on the open internet, one could argue that these models already incorporate and represent citizens’ interests and perspectives.Footnote 125 This view, however, endorses a highly reductionist understanding of participation.Footnote 126 Aside from the fact that AI training data are typically collected without people’s permissionFootnote 127 and encode many harmful biases,Footnote 128 the inclusion of data originating from members of the public does not imply that the resulting AI model actually reflects their values, let alone acts on their behalf or requires them to intentionally reason about public decisions. After all, it is one thing for a person to actually deliberate about a particular decision; it is quite another to infer from data how that person might deliberate if they were afforded such an opportunity in practice.
One potential solution is to establish public “data trusts” whereby citizens can collectively decide how their data are used in the development of AI models so as to better reflect their values and interests.Footnote 129 This could improve the transparency of the resulting models and, thereby, enhance citizens’ ability to participate in and challenge decisions made using these AI models. A more practical alternative is to train AI models on texts that already reflect deep-seated societal values. For example, in the original “Constitutional AI” technique, language models critique and retrain themselves based on a set of principles contained in a so-called “constitutional” text comprised of, among other things, principles derived from the Universal Declaration of Human Rights.Footnote 130
Another approach involves engaging members of the public to select the rules and principles that guide the operation of AI models.Footnote 131 If implemented successfully and adopted by public institutions, these methods could support algorithmic decisions that better advance citizens’ values and interests. Such methods, however, are still a far cry from citizens directly participating in and challenging particular decisions made by public institutions. Even if these methods can produce AI models that accurately reflect people’s convictions, they do not enable people to actually participate in or challenge an AI model’s decisions. As illustrated in Section III.A.2, the making of decisions that (happen to) reflect public opinion and genuine public participation in decision-making processes are not one and the same. While the former might be instrumentally valuable, the latter remains key to democratic legitimacy.
C. Using AI to Support Civic Engagement
The third pathway aims to reenvisage the purpose of AI in public decision-making. Thus far, we have considered the use of AI in government as a form of delegation, roughly analogous to privatization. But this need not be the case. Instead of replacing human decision-makers or democratic institutions, AI systems could be used to facilitate large-scale deliberative processes that afford citizens meaningful opportunities to participate in public decision-making.Footnote 132 For instance, AI systems could help aggregate, analyze, and communicate viewpoints in online deliberative platforms that can serve as valuable inputs into public decision-making.Footnote 133 Several governments have experimented with these platforms, including in the United Kingdom, United States, and Taiwan.Footnote 134
Bolder proposals include aggregating the preferences of large numbers of citizens, training AI models on those preferences, and using the resulting models to simulate human deliberation.Footnote 135 While this approach might facilitate transparent deliberative processes that members of the public can freely inspect and critique, it remains unclear whether simulated deliberation fulfills the condition of genuine participatability.Footnote 136 This is because, as discussed earlier, the public character of decisions does not only hinge on their correlation with the public’s interests. To count as decisions made by the public, decisions require the opportunity for actual public participation. Without this, citizens cannot be regarded as authors of the resulting decisions.
Moreover, reliance on algorithmic systems that successfully facilitate preference aggregation or simulation could decrease citizens’ motivation and, eventually, impair their ability to participate in public decision-making.Footnote 137 In the longer run, decreased participation in public decision-making could degrade the quality of the underlying data needed to build AI systems that can reliably aggregate or simulate societal values. Without thoughtful institutional design, a dangerous feedback loop may ensue.
VI. Conclusion
In an era of Asimovian technological advances, it is fitting to conclude by consulting the prophet of progress himself. Asimov’s Franchise describes a society in which conventional elections are replaced by a computer that, in consultation with a statistically representative “Voter of the Year,” selects candidates to fill public office.Footnote 138 While the system’s choices may be computationally impeccable, a society so dependent on that technology raises deep concerns. Flourishing democracies require robust public institutions, which themselves rely on informed and engaged citizens. Outsourcing democratic responsibilities to algorithmic technologies could disempower both citizens and public institutions and sever the vital connection between them. Averting this danger—without stifling the benefits of automation—is a delicate dance.
Open access
