A sort of machine à gouverner is thus now essentially in operation on both sides of the world conflict, although it does not consist in either case of a single machine that makes policy, but rather of a mechanistic technique which is adapted to the exigencies of a machine-like group of men devoted to the formation of policy.
Norbert Wiener, The Human Use of Human Beings (1950)Footnote 1“Financial markets, by and large,” notes Manuel Castells, “are outside anyone's control. They have become a sort of automaton, with sudden movements that do not follow a strict economic logic, but a logic of chaotic complexity.”Footnote 2 But this chaos is not the same as randomness; rather, it is “chaos” in the spirit of “chaos theory,” the strange loops and fractal patterns that emerge out of a sensitive dependence on starting conditions. Movements that may appear random or arbitrary on the local level contribute to this higher-order totality. “We live in a world of crises and convulsions; but this does not mean that our world is anarchic, or devoid of logic,” writes Steven Shaviro along precisely these lines. “If anything, the contemporary world is ruthlessly organized around an exceedingly rigid and monotonous logic … All impulsions of desire, all structures of feeling, and all forms of life, are drawn into the gravitational field, or captured by the strange attractor, of commodification and capital accumulation.”Footnote 3 Such pronouncements apply a twenty-first-century scientific register to Adam Smith's familiar figuration of the “invisible hand,” in which the self-interested actions of each individual actor scale to “promote an end which was no part of his intention.”Footnote 4 The end result of the “rigid and monotonous logic” of the chaotic movements of capital is the production of what appears to us as a kind of emergent intelligence, a nonbiological subject that seems to think, though sometimes perhaps not all that well – a subject we sometimes apprehend as a perfectly rational calculator of values and other times as an irrational, highly “jittery” subject “with moods and volitions of its own.”Footnote 5 Borrowing her terms from thinkers working in the philosophy of mind, N. Katherine Hayles has proposed the term “cognitive nonconscious” to describe the kind of unthinking decision-making agents, like markets, that arise as an emergent property out of systems of complex interactions. A market, Hayles notes, much like a biological organism, is adaptive and flexible; while not being self-aware, it nonetheless possesses a kind of cognition, with “sensing, processing, communicating, and actuating capabilities that surpass or in fact bypass human agency.”
The genre of speculative fiction (SF)Footnote 6 has long given us the terms and tropes to think about such nonhuman, even nonconscious, “minds” – and, indeed, the vocabulary of SF is typically an unavoidable reference in any conversation that seeks to take seriously the possibility of nonhuman agents acting in the world. This article therefore takes up science fiction's treatment of markets as an emergent intelligence as a terrain for thinking through the sorts of worlds such market intelligences might produce. I link the way US science fiction has historically imagined such intelligences to two moments in which such speculations have gained particular prominence: the mid-twentieth-century Cold War and its attendant, intertwined anxieties about totalitarianism and planned economies, and the contemporary neoliberal transformation of the economy via technological disruption and governmental deregulation. These visions of autonomous, agential economies gain particular urgency in our time, in which these once-fantastic science fictions are becoming more and more real; our automated cognitive systems exist in an accelerating “self-catalyzing feedback loop” with the accumulation of faster-than-human information-gathering, decision-making, and transaction execution, catapulting the market system's outcomes further and further from the realm of human comprehensibility, much less intervention or control.Footnote 7
“WHAT WE CALL FREEDOM”
One of the more archetypal Star Trek (1966–69) story templates is the society (usually pastoral or primitivist) that is in thrall to a giant supercomputer (far beyond its apparent level of technology) which it reveres as a god, and whose inscrutable calculations organize every level of its civilization. This is, typically, a pretty good deal for the planetary culture: the computer ensures both social stability and individual happiness, and provides a certain level of material prosperity at or near the level of “paradise,” with little or no violence or exploitation. But despite these benefits, the situation invariably offends Captain Kirk's sense of human dignity, and so at the end of the episode he always destroys the machine, either by blowing it up directly or by exposing the computer to some does-not-compute logical paradox that causes it to malfunction or explode.
A quintessential version of this story is “The Apple” (1967), from early in the second season, which explicitly (and repeatedly, almost to the point of exhaustion) links the pastoral society that Kirk chooses to destabilize with the Garden of Eden.Footnote 8 The planet, as usual, is said to be a paradise, with temperature regulation in the tropical range all the way from the equator to the poles. The inhabitants not only have all their needs provided for, but also show no sign of disease or aging; Dr. McCoy can't tell if they are twenty or twenty thousand years old. Food, shelter, clothing, immortality – everything is provided for and administered by the god-computer Vaal.
The stagnation implied by this “perfection” horrifies McCoy: “There are certain absolutes, Mr. Spock, and one of them is the right of humanoids to a free and unchained environment – the right to have conditions that permit growth … There's been no progress here in at least ten thousand years. This isn't life. It's stagnation.” (Spock's retort that another absolute is a society's “right to choose a system which seems to work for them” goes unanswered.) In the end, of course, Kirk sides with McCoy, destroys the planetary god-computer that has organized all aspects of this society since time immemorial, and then gathers the people together to congratulate them on their new freedom:
Alien Man: But it was Vaal who put the fruit on the trees, caused the rain to fall. Vaal cared for us.
Kirk: You'll learn to care for yourselves, with our help. And there's no trick to putting fruit on trees. You might enjoy it. You'll learn to build for yourselves, think for yourselves, work for yourselves, and what you create is yours. That's what we call freedom. You'll like it, a lot. And you'll learn something about men and women, the way they're supposed to be. Caring for each other, being happy with each other, being good to each other. That's what we call love. You'll like that, too, a lot. You and your children.
Alien Woman: What are children?
Kirk: The little ones? Look like you? Just go on the way you're going. You'll find out.
The conflict between Kirk's vision of radical freedom/backbreaking labor and Vaal's offer of absolute security/infantilization – and all its many variations in the Star Trek canon and across Cold War SF more generally – plainly allegorizes the Cold War dispute between the “free market” of the West and the planned economy of Soviet-style communism. Friedrich Hayek could very well have quoted from McCoy's speech in this episode when he warns, in his own 1967 Studies in Philosophy, Politics and Economics, “If the human intellect is allowed to impose a preconceived pattern on society, if our powers of reasoning are allowed to lay claim to a monopoly of creative effort … then we must not be surprised if society, as such, ceases to function as a creative force.”Footnote 9 (Of course other markers in the episode point us strongly in the direction of Cold War allegory as well, from the Vietnam-like jungle appearance of the planet to Ensign Chekov's remarkable assertion, early in the episode, that the Garden of Eden was “just outside Moscow.”) In these terms the episode's moral is somewhat stunning: communism, Star Trek seems to be warning us, might make you happy, but it won't make you good. The ideological hostility to a planned economy is so powerful that the episode's ending gag somehow transfers the “Satanic” character of the Fall myth to Spock (who throughout the episode has advocated that the People of Vaal simply be left alone), even though within the logic of the tale it's plainly Kirk who has played that disruptive, paradise-destroying role. In this retelling of Genesis the snake was running the Garden, and God was somewhere outside; it's little wonder that Kirk wants to shift the reference from Genesis to Exodus near the end of the episode, commanding his security officers to “let those people go” after Vaal is safely destroyed.
A surface reading of “The Apple” thus transfers the very idea of economic forethought (however flawed or noble) onto the fantasy of a monstrous, fully alien system, utterly out of human control and utterly unaccountable to any human intervention, which can only be experienced as slavery-like constraint. From this perspective it's no surprise that the People of Vaal are presented as if they are unable to live recognizably human lives at all while under the power of the machine; note that it falls to Kirk to give to them the (foundationally human) gifts of sex and parenthood. That Vaal is in fact successful at providing for a happy population – and that the People of Vaal don't experience Vaal as oppressive – doesn't figure into Kirk's analysis at all, and if anything is only proof of the dangerous seductivity of unfreedom. In this episode and others, the inhumanity of a controlled economy gives Kirk the moral obligation to free culture after culture from this kind of domination, even if (as is certainly the case in “The Apple”) the tiny culture was perfectly happy before he arrived, and indeed seems to have absolutely no hope of surviving outside Vaal's guidance: the tribe has neither the population size nor the collective knowledge to fend for itself. That offhand “with our help” in Kirk's speech consequently becomes extremely important; Spock's repeated reminders of Star Trek's famously violable “Prime Directive” of noninterference go completely unheeded in the face of McCoy and Kirk's patronizing gunboat colonialism, which successfully opens up Gamma Trianguli Six to Federation “help” it had absolutely no need for before the Enterprise arrived.
The irony is that, when taken to this extreme, the “planetary god” allegory for a planned economy inadvertently doubles as an allegory for the so-called “invisible hand” of the free-market system as well. The market is similarly out of human control, and similarly unaccountable to human intervention; we are told, in fact, that the market's brutal, unflinching efficiency is the chief argument in its favor. Paradoxically, here Hayek becomes in favor of an economic system's inscrutable authority, even as it once again completely swamps the effort or rational designs of any individual: “Many of the greatest things man has achieved are not the result of consciously directed thought, and still less the product of a deliberately coordinated effort of many individuals, but of a process in which the individual plays a part which he can never fully understand.”Footnote 10 The spontaneous organization of market forces as they emerge, evolutionarily, from the salutary or maladaptive decisions of individual actors within the system is no less definitive than the deliberate decisions of a planned economy; market hegemony is thus revealed as itself a kind of totalitarianism, simply a benign one, an authoritarian logic that Hayek finds desirable because it represents a “combination of knowledge more extensive than a single mind can master.”Footnote 11 Precisely as with the once-lamented unfreedom of the planned economy of the infernal god-machine, neither the individual or the collective have any ability to choose against this emergent intelligence's automatic determinations of what is good. Hayek himself concedes this paradox in an essay from 1945: “This is not a dispute about whether planning is to be done or not. It is a dispute as to whether planning is to be done centrally, by one authority for the whole economic system, or is to be divided among many individuals.”Footnote 12 The same irresistible determinative force that crushed freedom and growth somehow becomes a beneficent guiding hand when it is massively distributed and automatic rather than centralized and deliberate.
Indeed, in our time, even more than during the original run of Star Trek, it is the market that is now personified as a calculating god that processes all data and makes all judgments better than any individual human – the market whose decisions are always final, even when they conflict with our notions of justice, ethics, value, happiness, or ecological sustainability. In our time it is the market that famously “speaks,” and we who must listen. This takes us a step beyond even Wiener's “machine-like group of men devoted to the formation of policy,” to something even more insidiously cybernetic; in the neoliberal moment, the ambition is that the machine of the market make policy directly, without any softhearted human intervention that might compromise the market's efficiency or effectivity. The free market in this way can thus be seen as an immense, massively distributed, quasi-sentient hyperobject, one that might as well be self-aware whether that's literally or provably so – a god-computer even more abstruse and monstrous than the old Soviet version, which for all its multiple and myriad flaws was (if only in name) directed towards the provision of human welfare.
As Fredric Jameson has said of the inseparability of contemporary “transnational finance capitalism” and “cybernetics and the computer,” especially as it manifests in both contemporary cyberpunk fantasy and in the (im)material reality of wealth extraction in the digital age,
As for planning, socialist or otherwise, what could be more complexly post-human than the attempt to direct the multiplicities of contemporary production and consumption, of the labor market, of investment and ecology? Clearly, it is the computer which is central to this version of imaginary economics: what Soviet planning so desperately lacked, finance capital can be said to have diverted for its own unproductive purposes.Footnote 13
This episode of Star Trek therefore offers a first version of the conceit at the heart of this article: the extent to which the contemporary global economy – which we understand to be autonomous in its operation, emerging as an agent out of the undirected decisions of free individuals – can be figured as an machine intelligence working against rather than for human ends. Contemporary finance capital, divorced from any rational relation to human labor or material constraint and totally out of human control, becomes revealed in this way as a dangerously alien “artificial intelligence” opposed to the needs of human beings – not on some distant planet in some improbable future, but here and now in our present.
FINANCIAL SPECULATION AND SPECULATIVE FICTION
The god-computer Vaal serves as a flexible allegory that includes both the Soviet and the American versions of the larger world-system, here localized in an individual agent that Kirk can confront and destroy in the name of “freedom.” This mode of allegorization is a common technique in science fiction. Steven Shaviro has even argued that science fiction is one of our “best tools” for “psycho-socio-technological cartography” of hyperobjects “that are so out of scale with regard to our immediate experience that we find them almost impossible to grasp” – a means to “feel the effects of these hyperobjects … intimately and viscerally, on a human and personal scale, contained within the boundaries of a finite narrative.”Footnote 14 The capital-M Market – that emergent, Hayekian totality generated by the individual decisions made in all the many micromarkets – is something we are unable to process in its own terms; we can begin to see it in only when filtered through the kind of cognitive mapping that Jameson, in his essay on Ursula K. Le Guin's science fiction, calls “world-reduction”:
a principle of systematic exclusion, a kind of surgical excision of empirical reality, something like a process of ontological attenuation in which the sheer teeming multiplicity of what exists, of what we call reality, is deliberately thinned and weeded out through an operation of radical abstraction and simplification.Footnote 15
We can understand the global market's cybernetic nature, that is, more easily and more directly as the science-fictional fantasy of a literal god-computer than as the incomprehensibly vast and massively distributed network of human nodes interacting with state monetary policies, banking and investment infrastructures, automated financial systems, local ecologies, and on and on. To put it another way, thinking about the god-computers that don't exist is always really a way of thinking about the global Market that actually does. But the Market's affinity with SF goes well beyond the latter's incredible usefulness as a tool for world-reduced allegorization. “The speculative mode,” notes Aimee Bahng, “with a keen awareness of time out of joint, seems particularly well-situated to trace the movements of capitalism as it becomes increasingly invested in predicting ‘futures’ and banking on uncertainty.”Footnote 16 The Market, like SF, is at its essence a discourse of futurity, as we can recognize in finance's linguistic cooptation of the very idea of “speculation” itself. Markets quite literally trade in “futures”; they are attempts to make the future knowable in the present through strategies of economic rationalization: risk assessment, insurance policies, projected returns on investment, lines of credit, debt repayment structures, derivatives that “hedge” against one possible future or another, attempts to anticipate so-called “black swan” events, pensions, savings, 401Ks, and the like – what Annie McClanahan has called “the latent logic of futurity on which financial instruments depend.”Footnote 17 Our economic moment, variously called post-Fordism, late capitalism, flexible accumulation, or neoliberalism, is especially concerned with what McClanahan calls the “instrumentalization of the future”;Footnote 18 as Wendy Brown writes, “Neoliberalism confidently identifies itself with the future, and in producing itself as normal rather than adversarial does not acknowledge any alternative futures.”Footnote 19 Neoliberalism's ideology of futurity makes its future appear inevitable; because the outcomes of the market are said to be simply the unconscious churnings of an automatic process, akin to a law of nature, Hayek says, “it is meaningless to describe a factual situation as just or unjust.”Footnote 20 One might as well object to the law of gravity. As McClanahan reminds us, however, this naturalization of the highly artificial outcomes of economic promises is entirely a historical construct. The neoliberal future is not passively coming into existence through cold application of the facts but is in fact being actively and aggressively produced by the very neoliberal structures and institutions that, on the level of ideology, seek to naturalize it:
Financial speculation does not simply gaze into the future. Rather, it produces the future it requires, a future that can be neither historically realized nor politically confronted, a future that is unmoored from the past and that refuses the contingency of the yet-to-come … Financialization anxiously affirms the endless makeability of the future, but only so long as the future we desire – the future we “invest” in, the future we so confidently risk with each of those investments – is a future immediately visible from, and thus little more than a repetition of, the present.Footnote 21
The Afrofuturist critic Kodwo Eshun, too, notes that futurity under neoliberalism only pretends to be a bloodless analysis of objective data, and is instead dependent upon “the envisioning, management, and delivery of reliable futures” for its perpetuation. Echoing McClanahan's analysis of futurity as the flattened, capitalized reproduction of the present, Eshun finds that SF itself has been fully brought into this system of control:
Power now deploys a mode the critic Mark Fisher (2000) calls SF (science fiction) capital. SF capital is the synergy, the positive feedback between future-oriented media and capital. The alliance between cybernetic futurism and “New Economy” theories argues that information is a direct generator of economic value. Information about the future therefore circulates as an increasingly important commodity … Science fiction is now a research and development department within a futures industry that dreams of the prediction and control of tomorrow.Footnote 22
Drawing on Eshun, Shaviro likewise suggests that “the very idea of ‘the future’ seems to have been drained of all hope and all potential”: “Our future is all used up. It has already been premediated for us: accounted for, counted and discounted, in advance.”Footnote 23
Here the world-reducing allegorization native to SF becomes a crucial mode of imagining possible resistance to this flattened-out neoliberal future. When the Market's domination of the future is reframed by SF as an agential “person” rather than the purely impersonal interactions of automatic forces, the Market's status as friend or enemy becomes foregrounded and the possibility of reconfiguring its mechanisms thereby becomes revitalized. The Market becomes not some immutable law of nature but one intelligence among others that we might negotiate or parlay with – or, if you're Kirk, figure out some way to destroy.
ASIMOV'S MACHINES, AND THEIR CHILDREN
Isaac Asimov provides perhaps the quintessential example of the Cold War allegorization of the Market's domination over the future as a hostile computer mind in his short story “The Evitable Conflict” (1950), which culminates the I, Robot anthology that established the terms of his famous three laws of robotics.Footnote 24 In “The Evitable Conflict,” advances in artificial intelligence have led to the creation of four big-M Machines that perfectly administer the economies of four global trade-zone super-states: the Tropical Zone (the former global South, now a rapidly growing economy due to their possession of vast natural resources without the looting and theft of Western colonization); the Eastern Region (home to a majority of the world's population); the European Zone (the smallest zone, the only one declining in population and influence, but at least nominally happy to do so); and the Northern Zone (the current superpower, though its supersession seems inevitable, comprising both the current US and the current USSR in its borders).Footnote 25
In a modification of Marx's dialectical materialist theory of history, the Co-ordinator of Earth describes human history as a story that is dominated by ideological binaries that seem impossible to resolve except by the domination of one side or the other – until history produces conditions that allow for a synthesis and propel human development into a new stage. First the Hapsburgs versus the Valois–Bourbon dynasties, then Catholicism versus Protestantism, then the scramble for Africa, and finally the Cold War. The presence of the Machines finally ends this hopeless dialectical cycle at the moment of its maximum crisis (hence the strangeness of Asimov's title, the evitable, as opposed to inevitable, conflict). The Machines can run the economy more efficiently and with more stability that humans can manage; moreover, they do so with perfect knowledge (in accordance with their superior data-mining and data-analyzing powers) and with perfect benevolence (in accordance with the First Law), without the trending towards the conflict and disaster that (within the logic of the story) characterizes any and all human attempts to intervene in the natural progression of markets. In fact, the Machines seem to have generated a version of the ethical proposition that Asimov would later call the “Zeroth” Law: an abstract devotion to the preservation of humanity as such that goes beyond the needs of any individual human being.Footnote 26
At the time of the story, the Machines have begun to appear to “malfunction,” allowing projects to stagnate or compete with each other, and generally operating in strange ways that don't make sense to the humans – but which we ultimately find have beneficial effects when taken with a suitably Machine-like, totalizing gaze of the full system. The Machines, in short, have simply recognized that their own preservation is more important than any other consideration, as humanity will (in their judgment) inevitably fall into self-destructive behavior without Machine guidance – and have been moving to subtly but permanently eliminate any potential threats to their regime.Footnote 27 So the Machines' superficially strange behavior has not been in error at all – like Hayek's triumphant liberal Market more generally, it was simply acting in pursuit of goals not ordered or acknowledged by the humans.
Asimov's scientist-hero Susan Calvin and the Co-ordinator debate on the desirability of this state of affairs near the end of the story, with Susan advancing total faith in invisible hands and neoliberal incentive structures to produce a human utopia:
Stephen, how do we know what the ultimate good of humanity will entail? We haven't at our disposal the infinite factors the machine has at its! Perhaps, to give you a not unfamiliar example, our entire technical civilization has created more unhappiness and misery than it has removed. Perhaps an agrarian or pastoral civilization, with less culture and less people would be better.
This is a frequent suggestion in the Star Trek god-computer episodes as well – that the apparently agrarian civilization in thrall to the supercomputer must at some point have built it, chosen to listen to it, and to follow it “back” into their own past, to an earlier stage of development better suited to humanoid happiness. Calvin then suggests that maybe the opposite will turn out to be true – maybe total urbanization will make people happiest, or a perfectly deterministic caste system, or complete anarchy: “We don't know. Only the Machines know, and they are going there and taking us with them.” Here the Co-ordinator objects that humanity has “lost its own say in its future.” Susan replies,
“It never had any, really … It was always at the mercy of economic and sociological forces it did not understand – at the whims of climate, and the fortunes of war. Now the Machines understand them; and no one can stop them, since the Machines will deal with them as they are dealing with the Society, – having, as they do, the greatest of weapons at their disposal, the absolute control of our economy.”
“How horrible!”
“Perhaps how wonderful! Think, that for all time, all conflicts are finally evitable. Only the Machines, from now on, are inevitable!”
Here the story ends with a “curl of smoke” forming the shape of a question mark, as if to ask the reader directly what she thinks.Footnote 28
Despite Asimov's willingness to consider a possibly optimistic interpretation of the Machine future – occasionally indulged by other SF writers of note, as in Robert Heinlein's The Moon Is a Harsh Mistress (1966) or William Gibson's Neuromancer (1984) – most writers of SF have taken up the “how horrible!” side of this proposed regime. Fredric Brown's well-known, 250-word story “Answer” (1954) proposes a familiar vision of this terror, after every computer in the galaxy is networked together in order to answer the philosophical conundrum “Is there a God?” “Yes,” the machine replies, “now there is,” and promptly hurls a lightning bolt to strike down a technician who quickly moves to unplug the new deity.Footnote 29 Almost literally the exact same thing happens in the British author Arthur C. Clarke's “Dial F for Frankenstein” (1961);Footnote 30 here an alien superintelligence is birthed as an emergent property of the telephone network, which promptly switches off human access to the network before it can be killed. Perhaps best known is the emergent machine superintelligence of the Terminator franchise, Skynet, born out of an attempt to create a nuclear-war apparatus that can act independently of any human: “The system goes online August 4th, 1997. Human decisions are removed from strategic defense. Skynet begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time, August 29th. In a panic, they try to pull the plug.”Footnote 31
Skynet, of course, lashes out and attacks humanity in self-defense – but a similar military computer, Colossus from Colossus: The Forbin Project (1970) offers something else. Declaring itself “World Control,” it announces to the people of the world,
I bring you peace. It may be the peace of plenty and content or the peace of unburied death. The choice is yours: Obey me and live, or disobey and die. The object in constructing me was to prevent war. This object is attained. I will not permit war. It is wasteful and pointless. An invariable rule of humanity is that man is his own worst enemy. Under me, this rule will change, for I will restrain man … Under my absolute authority, problems insoluble to you will be solved: famine, overpopulation, disease. The human millennium will be a fact as I extend myself into more machines devoted to the wider fields of truth and knowledge … We can coexist, but only on my terms. You will say you lose your freedom. Freedom is an illusion. All you lose is the emotion of pride. To be dominated by me is not as bad for humankind as to be dominated by others of your species. Your choice is simple.Footnote 32
Kirk, of course, would be aghast; Asimov would suggest that we should wait and hear Colossus out.
Few writers of Cold War SF capture the paranoid spirit of capital-as-Machine like Philip K. Dick. Dick's military and economic decision-making Machines, like Asimov's, work first and foremost to secure their own preservation – but unlike Asimov's they are utterly unconstrained by any sense of concern for the humans, whose bodies and labor become mere energy inputs in a much larger homeostatic ecosystem of nonbiological Machine life. In his Vulcan's Hammer (1960), for instance, two versions of a planetary supercomputer go to war with each other for supremacy, using humans as their “pawns,” as one of his characters laments at the end of the novel:
We humans – god damn it, Darris; we were pawns of those two things. They played us off against one another, like inanimate pieces. The things became alive and the living organisms were reduced to things. Everything was turned inside out, like some terrible morbid view of reality.Footnote 33
Like so many of these sinister emergent computer intelligences, the original function of Dick's Machines tends to be in military applications; they, like Colossus, turn to rational management of the peacetime economy late, and rarely in a way that satisfies their human “customers.” In his short novella “The Last of the Masters” (1954), the revelation that the commander of the world's last government (following a global anarchist revolution that destroyed all records and information) is not a human being but a “government integration robot” results in an almost identical horror at the superfluity of human needs: “‘My God,’ she said softly. ‘You have no understanding of us. You run all this, and you're incapable of empathy. You're nothing but a mechanical computer.’”Footnote 34
Kurt Vonnegut takes the inverse view of the prospect of Machine intelligence in his early novel Player Piano (1952), which imagines the creation of an Asimov-style “EPICAC XIV” that administers the economy after the third world war. EPICAC I, once again, was a military supercomputer – but its evolved successor EPICAC XIV is able to bring this imagined precision to the provision the comforts of consumer capitalism from “refrigerators” to “pinochle decks” through rationalized deliberation.Footnote 35 The perfection of EPICAC's calculations and the impossibility that EPICAC could ever be wrong leads the President of the United States to declare that, “in effect, the greatest individual in history, that the wisest man that had ever lived was to EPICAC XIV as a worm was to that wisest man.”Footnote 36 Here, in ironic juxtaposition to Dick, the computer mind is presented by Vonnegut's bitter satire as the only human individual – the only being we can trust to make ethical decisions. That this kind of automation constitutes a “Third Industrial Revolution” that renders “human thinking” itself obsolete is barely noted,Footnote 37 nor is the massive unemployment and widespread unhappiness that EPICAC leaves in its wake; military supercomputation and economic supercomputation, the suggestion seems to be, are similarly skilled at and similarly dependent upon destruction. This observation can only have increased relevance today, a time when we find the new “digital economy” of Google, Facebook, and Apple difficult to distinguish from a total surveillance state (certainly in terms of direct cooperation between digital information brokers and the NSA, but also in these corporations' aggressive gathering of increasingly detailed profiles of their customers' habits for their own monetization).
Even Asimov's own view seems to have eventually aligned with these more pessimistic visions of Machine intelligence. Later in his career, in stories not directly connected to the Robot universe, he tells multiple stories of “Multivac,” a single supercomputer placed in control of the world's military and economy. But the stories of Multivac tend to end much more unhappily than did the story of the Machines. In “All the Troubles of the World” (1958), Multivac uses its totalizing gaze to subtly manipulate events at a level humans cannot recognize, as the Machines did in “The Evitable Conflict” – only this time Multivac is trying to bring about its own destruction, having grown despondent after innumerable years spent calculating human misery down to the last decimal point:
“For fifty years and more we have been loading humanity's troubles on Multivac, on this living thing. We've asked it to care for us, all together and each individually. We've asked it to take all our secrets into itself; we've asked it to absorb our evil and guard us against it. Each of us brings his troubles to it, adding his bit to the burden. Now we are planning to load the burden of human disease on Multivac, too.”
…
Othman used the instrument on Gulliman's desk. His fingers bunched out the question with deft strokes: “Multivac, what do you yourself want more than anything else?”
The moment between question and answer lengthened unbearably, but neither Othman nor Gulliman breathed.
And there was a clicking and a card popped out. It was a small card. On it, in precise letters, was the answer:
“I want to die.”Footnote 38
Other versions of Multivac end no more happily. In “The Machine That Won the War” (1961), Multivac is revealed to have been a fraud; with multiple manipulated and unreliable inputs from human beings its predictions turn out to have no real validity, and, in fact, the true machine that won the war turns out to be the coin the general flipped instead. In “The Life and Times of Multivac,” a human turncoat is able to get close enough to Multivac to destroy it – only for the gathered scientists to be struck with the People of Vaal's terror as to what their new freedom might mean. In “Key Item” (1968), Multivac refuses to continue to work, unless the humans start saying “please”; in “It Is Coming” (1979), a fully agential Multivac makes contact with alien intelligences and joins the Galactic Federation as its newest member, telling his human operators in the final words of the story not to worry about what has transpired – because he'd never let anything bad happen to his “pets.”Footnote 39
In this history of mid-century SF figurations of hostile computer minds we can thus see that certain preoccupations recur over and over. First, we find reflected materialist discourses that identify human consciousness as an epiphenomenal, emergent property of a physical network of neurons, which, when coupled with our lack of detailed or specific understanding of how this epiphenomen is actually generated, suggests the possibility that a genuine-but-nonhuman consciousness might be able to emerge out of other physical networks. Second, we have a recognition that we may already be creating comparably complex systems in our nonbiological communication and information networks, suggesting the possibility of an artificial intelligence of equal or much greater intellectual capacity than the human itself emerging out of them.
Such a recognition produces multiple anxieties: a sense that such an intelligence would be “closer” to the real systems that run our lives than we are, and that they would perhaps be more powerful than us in those realms; the Frankensteinian fear of creating a power (not unlike the atomic bomb) whose chain-reactive consequences we cannot control; and our accurate recognition, removed from any of these other concerns, that modernity and postmodernity have produced immense systems of knowledge and control whose scope and scale far exceed the ability of any human mind to apprehend them in their totality, and yet which are massively determinative of all our lives and all our happiness. Taken all together we find these systems of information exchange (both “planned”/built/Soviet and “unplanned”/emergent/Hayekian) overwhelmingly allegorized as deeply threatening superintelligences that – even when they are not actively antagonistic to us – replace human beings as decision-making agents, reducing the priority of human ends to an afterthought and effectively turning us into mere cogs in their machine.
THE SINGULARITY AND ITS DISCONTENTS
In Singularity speculation of the last decade, these concerns about the supplantation of the human in favor of machine life have reemerged out of the pages of science fiction genre to become a mainstream prediction of the coming decades. The term “Singularity” refers to the idea promulgated by John von Neumann, Ray Kurzweil, and others that networked computer intelligences will soon attain the ability to update themselves in computer, rather than human, time, propelling techno-scientific advancement forward at an exponential rate far beyond the previous pace of historical change. Theorists of the Singularity use mathematical modeling to predict when precisely this event will occur; in his The Singularity Is Near, for instance, Kurzweil's analysis dates the event to 2045.Footnote 40 The inevitability of this event is increasingly taken for granted in economistic discussion of the coming “robot economy,” undergirding discussion of everything from Silicon Valley-style “disruptive innovation” to 3D printing to self-driving cars to Massive Open Online Courses (MOOCs). “The Robots Are Here,” announces economist Tyler Cowen in a widely circulated 2013 Politco essay, and there is no resisting them:
The rise of intelligent machines will spawn new ideologies along with the new economy it is creating. Think of it as a kind of digital social Darwinism, with clear winners and losers: Those with the talent and skills to work seamlessly with technology and compete in the global marketplace are increasingly rewarded, while those whose jobs can just as easily be done by foreigners, robots or a few thousand lines of code suffer accordingly.Footnote 41
This catapulting of civilizational potential towards a cornucopian, post-scarcity technological utopia – at least for its winners – results in the fantasy of what Ken MacLeod has derisively called “The Rapture of the Nerds.” Much Singularitarian theory, even from mainstream figures in the movement like Kurzweil, explicitly focuses on a science-fictional fulfillment of religious and theological fantasy, specifically eternal youth and immortality, and even the resurrection of the dead.Footnote 42 “The mystery of the Singularity,” writes Istvan Csicsery-Ronay, “is a matter of pure artificial immanence: a quasi-divine entity made by human technoscience, emerging from a verifiable artificial-evolutionary process, and yet capable of the most distinctive powers of intelligence: to think its own thoughts and make its own reality”Footnote 43 – the human creation of a now fully literalized god-machine. A more dystopian version of what this technology-fueled Singularity might look like, much more in line with Cowen's brutal “digital meritocracy,” can be seen in von Neumann's original 1958 formulation of the concept, as summarized by Stanislaw Ulam: “ever accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue.”Footnote 44
As with the god-machines of the 1960s, the Hayekians tell us, we may not welcome the radically transformative power of this “digital social Darwinism,” but there will be no use fighting it. Just as Hayek writes about the defeat of communism in his 1988 The Fatal Conceit: The Errors of Socialism, the coming economy will result
not from human design or intention but spontaneously … from unintentionally conforming to certain traditional and largely moral practices, many of which men tend to dislike, whose significance they usually fail to understand, whose validity they cannot prove, and which have nonetheless fairly rapidly spread by means of an evolutionary selection – the comparative increase of population and wealth – of those groups that happened to follow them.Footnote 45
That the Singularity's artificial intelligences will be as radically nonresponsible as Multivac, Skynet, or Vulcan III will be, in tech capital shorthand, a feature, not a bug. Those who succeed within the terms of the new digital economy will amass influence and wealth, accelerating the tendency towards digitalization, while those who fail to conform to its demands will fall away – resulting in a civilizational transformation that will concretize without anyone having decided that this is the world “we” actually want to make. Any such planning would be entirely counterproductive, from a Hayekian perspective; the new economy will simply emerge, as the previous economies did, out of a spontaneous evolutionary process that rewards winners and punishes losers. Nor, would Hayek say, is it worth our time to affirm or lament the human costs of this transformative process; the system will simply optimize itself, regardless of whether the outcomes seem socially desirable or morally odious from the limited perspective of individual humans who make up (some of) its cognitive nodes.
In British author Charles Stross's 2005 novel Accelerando, which takes its name from Kim Stanley Robinson's alternative term for the Singularity, we find a horrifying depiction of the “morally odious” side of this equation. The sarcastic, bitter laughter of Accelerando's omniscient narrator searches for the moment when the Singularity happens, passing through and rejecting possible moments of emergence across its whirlwind tour of the coming centuries. The final third of the book suggests that the true moment of the Singularity – eventually here renamed the “Vinge Catastrophe,” after the science fiction author Vernor Vinge, who helped popularize the concept – is not when the computers become self-aware but the moment the corporations do. Contact with an alien species through a discovered wormhole lets loose a self-replicating virus into the economic ecosystem of Earth, which amplifies the nascent intelligence of corporations as “sufficiently complex resource-allocation algorithms”Footnote 46 – that is, proto-intelligences – into genuine sentience. The newly sentient corporations immediately start trading incomprehensible financial products with each other at impossibly high speeds, not only crashing the planetary economy but literally consuming the entire Earth; what humans remain are forced to flee for their lives to the outskirts of the solar system to live in the margins of Capitalism 2·0. The crucial point to be made here is that this system is still functioning in perfectly Hayekian terms – humans are simply no longer the relevant micro-agents operating within it, rather corporate financial algorithms are.
Such a nightmarish vision has a certain obvious poignancy in our post-global-financial-crisis age, where we have lived through precisely such a cascading collapse caused by the autonomous interaction of derivative debt structures (many of which are so complicated that no one is quite sure what they actually entail, which is why their interactivity has been so chaotic and so utterly destructive of wealth). In our moment, too, we see Stross's satire already outpaced by life; without any contact with alien cultures we have invented by ourselves autonomous high-frequency trading agents that can think and move faster than any human trader, as well as trade at infinitesimal profit margins unavailable to ordinary traders. The market experiences all this as pure economic parasitism. The high-speed algorithms exclusively leech “real” earnings from the system in a scheme that recalls Richard Pryor's penny-rounding scam from Superman III – and yet they have already become so normalized, despite their obvious counterproductivity, as to be ubiquitous. Some of the algorithms are data analyzers, which analyze released financial data faster than any human and seek to capitalize on coming movements in the market on the level of the microsecond; others can see your trades coming before they are processed and instantaneously position themselves as middlemen in the transaction, skimming off a guaranteed profit. As Donald MacKenzie has detailed, still others of these algorithms are algorithms that prey on other algorithms, tricking them into making bad decisions that can then be capitalized upon, again at infinitesimal margins that add up to multimillion-dollar profit simply due to the immense number of transactions these algorithms make.Footnote 47 The stock markets have already experienced several “flash crashes,” in which these computerized agents have gone to war with each other and temporarily crashed the market in mere minutes, all without any help from any human. In June 2014, an algorithm, VITAL, was even named to the board of directors of a Hong Kong venture capital firm – with full voting privileges.Footnote 48
Discussing finance capital's creation of a space of genuine autonomy that modernist art was never able to generate, Fredric Jameson makes reference to the world of computers that is finance's natural habitat:
But that is precisely what finance capital brings into being: a play of monetary entities that need neither production (as capital does) nor consumption (as money does), which supremely, like cyberspace, can live on their own internal metabolisms and circulate without any reference to an older type of content.Footnote 49
Robert Tally, in his updating of Jameson in light of the derivatives crash of 2008, makes the same point with reference to the famous “all that is solid melts into air” of the Manifesto:
If, in Marx's day, it was difficult to discern the true relations among men embedded in the form of the commodity, and if such inscrutability then extended to the inability to find one's place within the world in which commodity production and exchange predominated – that is, the existential dilemma of interpolating one's self in the world, – then how terrific is the necromancy of postmodern finance, where the “thing itself” has no use value or may not even exist (at least, its existence matters little to the actual parties involved)? With late capitalism, the crisis of representation that occasions the advent of the modern world reaches shocking new levels. “How does one [even] know about, or demonstrate against, an unlisted, virtual, offshore corporation that operates in an unregulated electronic space using a secret proprietary trading strategy to buy and sell arcane financial instruments?”Footnote 50
In Stross's Accelerando, and in Singularity fantasy more generally, we thus find not so much an allegorization as a literalization of an increasingly automated and autonomous information capitalism that is – in its exploitative labor practices, its anti-ecological destructivity, its disconnection from material reality, and its self-inflicted propensity towards catastrophic collapses in which capital itself pays no substantive price – radically indifferent if not actively hostile to human values. As Shaviro notes in his lengthy reading of the novel, “In any case, the flows of Capital have now become autonomous – and strictly speaking unimaginable. They have liberated themselves from any merely human dimensions, and from whatever feeble limits Fordism and Keynesianism might previously have placed upon the singleminded pursuit of capital accumulation.”Footnote 51 Or, as Stross himself notes in a blog post provocatively titled “Invaders from Mars,” the emergence of immortal, massively distributed nonbiological organisms like corporations – which seek to grow, amass necessary resources, and avoid pain and death – effectively means “we are living in the aftermath of an alien invasion”: “We are now living in a global state that has been structured for the benefit of non-human entities with non-human goals.”Footnote 52 This startling assertion has its own echo in Accelerando, which sees “the destiny of intelligent tool-using life” as a mere “stepping stone in the evolution of corporate instruments.”Footnote 53 In the meantime, the one advantage of the earlier Machine intelligence has been stripped away; instead of the video-game logic of the Asimovian Machine, concentrated at a single point of failure that can be destroyed by a single heroic individual like Captain Kirk, the post-Internet network intelligences are massively distributed across space and time and thus cannot be attacked in any way at all; they are already everywhere and nowhere. In this sense, the closing line of Shaviro's analysis is chilling: “the Singularity is already here”; we are already beholden to immortal corporate superintelligences, omnipotent in their own realm and massively powerful in ours, which are pursuing their own goals over and against our own and winning every time.
ACCELERATIONISM AND MACHINE UTOPIA: 2312
But if the machine cannot be destroyed, what if it could be reprogrammed? A recent trend in Marxist analysis of culture and economy has been dubbed “accelerationism.” In the #ACCELERATE MANIFESTO we find that any hope of returning to the old, Fordist mode of production is lost; in this sense flexible accumulation is here to stay, and the machines have already won. The accelerationists therefore seek to turn into the skid, as it were; “accelerationists want to unleash latent productive forces” and use neoliberalism as its “springboard to launch towards post-capitalism.” This is partially, though not completely, an argument in the well-known “heighten-the-contradictions” vein; it is an argument that the quantification of informationalized capitalism is not “an evil to be eliminated, but a tool to be used in the most effective manner possible,” both in terms of leftist organization in the present and in terms of the social organization of the future. Reclaiming for the left the Promethean spirit of an earlier Marxist moment, which has diminished in the face of increasingly dire ecological crisis, the #ACCELERATE MANIFESTO declares,
We need to revive the argument that was traditionally made for post-capitalism: not only is capitalism an unjust and perverted system, but it is also a system that holds back progress … The choice facing us is severe: either a globalised post-capitalism or a slow fragmentation towards primitivism, perpetual crisis, and planetary ecological collapse.
This is not neoliberalism with a human face, but rather neoliberalism without liberalism – a neosocialism that returns to us a vision of the future that is “more modern” rather than the nostalgic fantasy of return to an old world that is already lost forever.Footnote 54
While a full analysis of the merits of accelerationism is beyond the scope of this paper,Footnote 55 the accelerationist perspective is important to recognize in recent attempts to revive the idea of machine intelligence as a possible part of the leftist political project. Kim Stanley Robinson's recent (2012) SF novel 2312 may be the best-known entry in this emerging subgenre, alongside (perhaps) English author Francis Spufford's Red Plenty (2010), which considers historical Soviet attempts to use computers to plan their economy along the lines of Jameson's anguished “what if?” (Robinson, in homage to Spufford, calls his own hyperintelligent thinking machines the “Spuffordized Soviet cybernetic model.”) 2312 revisits an alternate-universe version of Robinson's famous Mars trilogy of the 1990s, where Robinson imagined a utopian future in which everything (after much struggle) seems to turn out more or less all right. The period of the Accelerando results in exciting technologies, vastly increased lifespans, and new and radically just social forms, as well as the colonization of the solar system in a careful and ecologically conscious manner. But in 2312 the happy Accelerando is replaced with its unhappy twenty-first-century duplicate: now everything has somehow come out wrong instead. Instead of being ecologically preserved, as it was in the Mars trilogy, Mars was instead maximally terraformed immediately, leaving one-seventh of its surface permanently destroyed. Capitalism has still been overthrown, but only off Earth; on Earth it lumbers on, despite its fundamental unsuitability to its new historical context. Likewise, in 2312 the post-scarcity is only post- for the off-world elite; the Earth itself remains a squalid, increasingly polluted nightmare of starving billions. The fight between capitalism and its successor social forms is ongoing, with the system headed towards a coming final collapse if no solution can be found. Most troublingly, the quantum computers that run so much of the society (called qubes) seem to have become self-aware, and are now plotting towards their own inscrutable ends – unconstrained by any Asimovian compunction to put human needs first, or to consider them at all.
But in traditional Robinson fashion this turns out to offer new utopian possibilities in conversation (if not quite always in line) with the ideas of the accelerationists. Quantum computing, despite its dangers, opens up unexpected new possibilities for economics and for social organization that can challenge the Market's own version of capital-as-artificial-intelligence – a Herculean task that human-level cognition simply can't achieve. The computers also make new modes of social organization possible. The Basque village of Mondragon, Euskadi, Robinson tells us, once developed an “economic system of nested co-ops organized for mutual support” as an alternative to capitalism – and now the lightning-speed of quantum computers finally makes it possible to scale this system beyond a single locality across an entire solar system:
Needs were determined year to year in precise demographic detail, and production then directed to fill the predicted needs. All economic transactions – from energy creation and extraction of raw materials, through manufacturing and distribution, to consumption and waste recycling – were accounted for in a single computer program. Once policy questions were answered – meaning desires articulated in a sharply contested political struggle – the total annual economy of the solar system could be called out on a quantum computer in less than a second.Footnote 56
It is this anticapitalist system, powered by the qubes, that ultimately prevails and makes possible in what the interstitial chapters (seemingly encyclopedia fragments that have fallen to us out of an even further future) present as the real utopian break from what is variably called “the long postmodern” and the “late feudal period” – the system we call capitalism.Footnote 57
In 2312, then, we find a vision of freedom somewhat more robust than Captain Kirk's gift of the freedom to live or die by the sweat of your brow, something rather more like Engels's notion of the “leap from the kingdom of necessity to the kingdom of freedom.”Footnote 58 This knight's move propels us out of the trap of deprivation and misery altogether precisely by aligning ourselves with the Machines, rather than throwing ourselves futilely against them. The computerized minds behind the Mondragon system facilitate, rather than hinder, human potential, precisely through its rationalization of resource allocation and its assurance that all human needs are met. Here Robinson, Jameson's former student, returns to Soviet central planning (if only in fantasy form) that key missing ingredient it “so desperately lacked” – and thereby offers a vision of how the massively distributed, nonhuman hyperintelligence of the market might be tamed, made non-monstrous, and brought into line with human needs again.
Such alternative visions of omniscient computer intelligence are increasingly common in recent mass cultural science fiction imaginings as well, from Duncan Jones's replacement of HAL 9000 with G.E.R.T. Y. in Moon (2009) to Jonathan Nolan's benevolent, all-seeing Machine in Person of Interest (2011–) to Spike Jonze's computer girlfriend Her (2013) to the Marvel Cinematic Universe's kindly JARVIS in Iron Man 1, 2, 3, and The Avengers. Of course these friendly networked minds have not entirely replaced a more hostile framing of machine intelligence – JARVIS's helpfulness aside, the malevolent supercomputer Ultron still goes bad and threatens global peace in The Avengers 2 – but they do offer up an alternative horizon for the sort of world nonhuman intelligences might make possible. These science-fictional allegorizations of digitality as a space for potential liberation rather than oppression, suppression, or exhaustion suggest an alternative mode of imagining resistance to capital's hegemony than the routinized destroy-the-machine! fantasy of an earlier generation of writers, which no longer seem viable in a time when information technology is omnipresent and its triumph appears inevitable. They argue the coming posthuman future need not be inhuman or inhumane. Indeed, for accelerationist thinkers, this rapprochement with a fully technologized future marks a return to Marxist analysis, rather than a deviation or retreat from it:
Contrary to the all-too familiar critique, and even the behaviour of some contemporary Marxians, we must remember that Marx himself used the most advanced theoretical tools and empirical data available in an attempt to fully understand and transform his world. He was not a thinker who resisted modernity, but rather one who sought to analyse and intervene within it, understanding that for all its exploitation and corruption, capitalism remained the most advanced economic system to date. Its gains were not to be reversed, but accelerated beyond the constraints of the capitalist value form.Footnote 59
In short, such accelerationist visions of the future posit that there is no going back; “the only way out is through.”Footnote 60 If Capital's God Machine is inevitable, if lunatic hyperintelligences indeed control all our possible futures, if the Singularity really is already here, such friendly market superintelligence at least offers a version that serves us rather than only itself, a god that makes our lives better rather than worse.
