Introduction
In discussing the perspectives opened up by the developing convergence between nanotechnology and the emergent technologies of biotechnology, information technology, and cognitive science, known as NBICFootnote 1, Allhoff and colleagues, in their report “Ethics of Human Enhancement”, forecast that human beings will become increasingly efficient and intelligent through the incorporation into the human body of machines (nanobots; electronic implants). They write, “Together with ever-shrinking computing devices, we appear to be moving closer to cybernetic organisms (or ‘cyborgs’), that is, where machines are integrated with our bodies or at least with our clothing in the nearer-term”Footnote 2. This development is deemed acceptable by transhumanists and rejected by humanists.
On one hand, transhumanismFootnote 3 can be defined as an ethics of using the development of these converging and emerging technologies to enhance human beings; currently, it goes by the designation ‘humanity plus’ or ‘H+’, as a way of expressing its positive view of the impact of human enhancement. The humanity plus perspective, as represented by scientist Ray KurzweilFootnote 4, is based on the hope that in the near future human beings will be able to surpass the limitations imposed by their biological condition, thus optimizing the enhancement of human performance in sports, work, and the military. More than mere science fiction, hopes like these constitute the dreams that, from here on in, will drive faith in nanotechnology research and development for the purpose of human enhancement.
On the other hand, humanism, represented by thinkers in the humanities and social sciences such as FukuyamaFootnote 5, could be defined as an ethics of the human condition. The conception of the human condition that underlies humanism is essentially based on our biological humanness and could be designated as ‘H’: “[I]s humankind a species in process of disappearance, will it soon yield to a new biological species: the posthuman”Footnote 6? The word “posthuman” means something that comes after humanity—seemingly a paradox, and if so, an untenable oneFootnote 7: “How can we believe that we may, as humans, come to witness the end of our reign”Footnote 8? From a humanist perspective, this loss of humanness could be symbolized as ‘H–’, to convey its negative impact on the human condition.
The purpose of this is to demonstrate the difficulties of conducting interdisciplinary work between transhumanism and humanism on the question of the risks associated with the ethical and social acceptability of human enhancement through the development of NBIC. These difficulties emerge in the context of the debate between the principal defenders of transhumanism, who have backgrounds in the natural sciences, and the principal defenders of humanism, who have backgrounds in the social sciences and the humanities. Our aim is to demonstrate that transhumanists and humanists essentially differ on these questions: 1) the identification of risks and impacts; 2) the assessment that serves as the foundation for the acceptability or unacceptability of the risks and impacts; and 3) faith in the capacity of science to overcome the identified risks to human beings.
The present comparison of the transhumanist and humanist stances is based on a framework for the analysis of the ethical and social acceptability of risks and impacts which, briefly, comprises these steps: 1) stating clearly what is under discussion; in this instance, stating clearly what is referred to when we speak of ‘human enhancement’; 2) identifying the risks and impacts (a. determining the stakes likely to be affected by a given unforeseeable event; b. determining the likelihood in principle of the probability that the unforeseeable event will arise; c. estimating the intensity of the negative or positive consequences for the stakes identified); 3) assessing these impacts and risks individually by appealing to a specific valued state; and last, 4) assessing all the individual assessments in a single final judgmentFootnote 9.
Our presentation of the divergences that exist in the debate between transhumanism and humanism constitutes a necessary first step in intervening in the debate in an interdisciplinary manner.
1. What Is It We Are Talking About?
What are we talking about when we refer to NBIC development for the purpose of human enhancement? We are talking about the convergence of nanotechnology, biotechnology, information science, and cognitive science as forming part of a complex process of incorporating into the human body fully developed nanoproducts (e.g., nanobots introduced into the bloodstream to cure cancer; organic implants; neural implants), for the purpose of creating cyborgsFootnote 10. Philosophers, Armin Grunwald and Yannick Julliard, for example, view the convergence of these technologies with humanity as the wholly natural end goal of the desire to surpass humanity’s biological condition and continue towards a world of cyborgs: “The convergence of humanity and technology seems to be the natural end of moving this reasoning into the realm of speculation. The concept of ‘cyborgs’ as technically enhanced humans or as humanly enhanced technology can be raised”Footnote 11.
Kurzweil presents a more detailed version of the progression of our human condition towards a cyborg state. He predicts that what he terms “human body 3.0” will arrive by the 2040sFootnote 12. By that time, increasing progress in incorporating implants and nanorobots will have rendered the human body more nonbiological than biological:
[The] most compelling, scenario involves the gradual but inexorable progression of humans themselves from biological to nonbiological. That has already started with the benign introduction of devices such as neural implants to ameliorate disabilities and disease. It will progress with the introduction of nanobots in the bloodstream, which will be developed initially for medical and antiaging applications. Later, more sophisticated nanobots will interface with our biological neurons to augment our senses, provide virtual and augmented reality from within the nervous system, assist our memories and provide other routine cognitive tasks. We will then be cyborgs, and from that foothold in our brains, the nonbiological portion of our intelligence will expand its powers exponentiallyFootnote 13.
Kurzweil distinguishes two types of cyborg. The first, which he designates “human body version 2.0”, consists of restored humans. The second, that of “human body version 3.0”, just presented above, consists of enhanced humans.
Prostheses, in general, supplement lost or damaged body parts with the integration of mechanical devices; bionic implants allow model organs or body parts to mimic the original function more closely. Kurzweil writes, “Nanotechnology promises the tools [nanocomputers, nanobots] to rebuild the physical world—our bodies and brains included—molecular fragment by molecular fragment, potentially atom by atom” to restore lost function, organs, and limbsFootnote 14. The primary purpose of this restorative cyborgization is the repair of broken or missing processes so the individual can enjoy an improved level of function.
Yet it could be argued that the human body version 2.0 scenario is already problematic, since it is difficult to distinguish therapy from enhancement:
Certain developments—even if mostly involving physical interventions intended to offer therapeutic benefits—raise a number of fundamental questions with regard to the personal and social aspects of human identity: what we understand and consider to be human, what is deemed normal (or acceptable), and what is not. Furthermore, the boundary between therapy (healing, treating, restoring to normal) and the optimization of human performance (improving, boosting beyond the norm) can be tenuous and hard to define. What is the difference between an intervention meant to enhance performance and a therapeutic intervention designed to correct an acquired or innate disability? How is a disability definedFootnote 15?
Kurzweil shares the view of Mihail Roco and William Bainbridge that the complex process incorporating of nanoproducts into the human body will inexorably bring us to the scenario of human body version 3.0:
The compelling benefits of overcoming profound diseases and disabilities will keep these technologies on a rapid course, but medical applications represent only the early-adoption phase. As the technologies become established, there will be no barriers to using them for vast expansion of human potentialFootnote 16.
When humanists Francis FukuyamaFootnote 17, Jean-Michel BesnierFootnote 18, and Jean-Pierre DupuyFootnote 19 engage with the transhumanist position and in particular the conception of the cyborg, they speak of a “posthuman future”. Besnier is concerned primarily with the nature of implants:
The bionic human—or the cyborg, as the being that emerges from the hybridization of the organic with the cybernetic is commonly called—if it were in fact to develop, following increased recourse to prostheses and electronic implants, would do a bit more damage to the image that humanity would like to have of itselfFootnote 20.
Fukuyama, for his part, is concerned with nanobots: “molecular-scale self-replicating machines capable of reproducing out of control and destroying their creators”Footnote 21. And finally, Dupuy is preoccupied with the development of those forms of nanotechnology that tend towards the creation of a world of cyborgs: “Let me repeat that the nanotechnology effects that concern us here are not only the effects of the technology in itself, but also the effects of the ideas [the purposes] that draw technology on, whether the technological implementations see the light of day or not”Footnote 22.
2. Divergent Identification of Risks and Impacts
Transhumanists and humanists do not work with the same kinds of analysis of the risks and impacts (the unforeseeable and uncertain events) associated with NBIC development as regards the fundamental ethical concern about what is at stake: the identity and quality of life of the human being.
2.1 Positive Impact on Quality of Life: Transhumanism
Kurzweil calls for progress in the development of artificial intelligence (AI) at the heart of NBIC as a way of achieving as early as possible the expected benefits of the coming of nanobots for introduction into the human bloodstream and organic and neural replacement implants. For him, these constitute solutions that will transcend numerous human biological malfunctions that entail diseases such as cancer. The AI explosion that will accelerate the process of incorporating nanoproducts into the human body thus constitutes, for Kurzweil, the more realistic scenario for responding to a critical question asked by scientists, which has been framed as follows: “Setting aside hype, what are the realistic scenarios for progress with emerging technologies that have the potential to make us all smarter, stronger, healthier, longer-lived, kinder, more fulfilled, and more sociable”Footnote 23? For Kurzweil, the incorporation of AI nanobots is necessary to achieve the best quality of life possible.
2.2 Positive Impact on Human Identity: Transhumanism
Kurzweil believes we are approaching a time when computers will become not just intelligent but more intelligent than humans, in order to accelerate and control the development of nanobots for incorporation into the human body. Come the AI explosion, there will result rapid progress in the development of nanobots and electronic implants, such that humanity—our bodies, our intelligence, and our spirits—will be wholly and irreversibly transformed. That is the most realistic scenario for accelerated NBIC development with a view to enhancing the performance of the human body and brain (programmable blood; ‘optional’ hearts; organic and neural implants devised to be incorporated into the human body in order to overcome disease and prolong human life by transforming humans into cyborgs). At this stage, we come to the human body version 3.0 scenario: the human body will become more nonbiological than biological. At the end of the process:
We can expect that the full realization of the biotechnology and nanotechnology revolutions will enable us to eliminate virtually all medical causes of death. As we move toward a nonbiological existence, we will gain the means of “backing ourselves up” (storing the key patterns underlying our knowledge, skills, and personality), thereby eliminating most causes of death as we know itFootnote 24.
2.3 Risk to Quality of Life: Transhumanism
In the context of this human body version 3.0 scenario, Kurzweil acknowledges different risks. The first is the risk of the loss of control over self-replicating nanobots (“gray goo”) that could destroy the biomass: “The point is that without defenses, the available biomass could be destroyed by gray goo very rapidly. As I discuss below…, we will clearly need a nanotechnology immune system in place before these scenarios become a possibility”Footnote 25.
The second risk is the possibility of activism or terrorism using undetectable nanobots in the water and food supplyFootnote 26.
The third risk concerns AI robots and nanobots whose intelligence will rival that of humans. “Such robots may make great assistants, but who’s to say that we can count on them to remain reliably friendly to mere biological humans”Footnote 27?
2.4 Risk to Quality of Life: Humanism
Humanists such as Bill Joy and Fukuyama mention the risks of humanity’s self-destruction as a consequence of loss of control over technological developments. For example, in Why the Future Doesn’t Need Us, Joy specifies risks associated with AI machines:
If the machines are permitted to make all their own decisions, we can’t make any conjectures as to the results, because it is impossible to guess how such machines might behave. We only point out that the fate of the human race would be at the mercy of the machines.Footnote 28
For Fukuyama, “Observers like Bill Joy have worried about nanotechnology—that is, molecular-scale self-replicating machines capable of reproducing out of control and destroying their creators”Footnote 29. And whereas Kurzweil believes these threats can be avoided by science, Fukuyama reasons, “such threats are actually the easiest to deal with because they are so obvious. If you are likely to be killed by a machine you’ve created, you take measures to protect yourself”Footnote 30.
2.5 Risk to Human Identity: Humanism
Like other humanists, Fukuyama believes that if the human body becomes more nonbiological than biological, there is a risk of losing human identity. Even if human nature is not part of a fixed order, it is not limitlessly plastic:
Human nature is very plastic, and we have an enormous range of choices conformable with that nature. But it is not infinitely malleable, and the elements that remain constant—particularly our species-typical gamut of emotional responses—constitute a safe harbor that allows us to connect, potentially, with all other human beingsFootnote 31.
Furthermore, in a posthuman world, shared humanity is at risk: “It could be one in which any notion of ‘shared humanity’ is lost, because we have mixed human genes with those of so many other species that we no longer have a clear idea of what a human being is”Footnote 32.
For the humanist Dupuy, the loss of our biological humanness directly affects our notion of human identity by affecting the whole meaning of the human condition:
If we had to be the cause of the closing of the door of the future [of biological humanness], the very meaning of the whole human adventure would be destroyed forever, and indeed retrospectively. I fear that a philosophy of happiness or pleasure is simply not adequate to the magnitude of the stakes involvedFootnote 33.
What are we to conclude from this brief overview of the two divergent analyses of risks and impacts that characterize the debate between transhumanism and humanism? Both humanism and transhumanism focus on the same stakes: the quality of life and human identity. The transhumanist analysis can be characterized as a comprehensive analysis of both positive impacts and risks. The humanist’s analysis centres on risk and thus does not consider the positive impacts of implants and nanobots. Both analyses consider the risk of the loss of human control over machines (gray goo, terrorism, AI machine superiority). The fundamental difference between the two risk analyses concerns human identity: for the transhumanists, there is a gain for human identity (humanity +) when the human body becomes more nonbiological than biological; for the humanists, the loss of human biological nature brings a loss of our human identity.
We can conclude that both analyses deal with theoretical risks and impacts: their probability has not been gauged. But even if these of risks and impacts are theoretical, it remains rational to contemplate them. Our sense of their existence is based on the idea that we have no proof that they are impossible—thus we can conclude they are possible. To put it another way, since we cannot demonstrate the impossibility of these risks and impacts arising in the future, they are possible. This is a scientific problem; yet it is at the mercy of the uncertainty, the ignorance, the rift, the confrontations that subsist between transhumanism and humanism.
If we look at the complexity of this scientific problem in the light of the debate that pits conflicting analyses against each other—the transhumanist analysis of projected positive impacts versus the humanists’ assessment of estimated negative risks—the need for assessment becomes clear. The transhumanist approach could, even with the best of intentions—that of achieving the positive impacts—result in unacceptable risks. Should we base our judgment of the acceptability or unacceptability of these risks and impacts on an ethical assessment? If so, the risks’ acceptability, that is, our decision to assume or reject them, would be founded on an assessment based on a criterion. As well, the acceptability of the positive and negative impacts, that is, our decision to assume them or not, will be founded on a final assessment based on a balancing of positive and negative consequences. For example, which consequences weigh more heavily in the balance? And based on what criteria will we assign such-and-such a weight to such-and-such a consequence? And is it not difficult to answer this question using divergent assessments of the ethical and social acceptability of the risks and impacts, assessments based on criteria as uncertain as that of ‘human nature’—a concept with two contradictory conceptions in the debate between transhumanism and humanism?
3. Divergent Risk and Impact Assessments
The acceptability of human enhancement through NBIC development must be based on an assessment of the analyses of two potential impacts: namely (a) loss or gain of human identity; and (b) loss of control over nanobots and AI machines. In what divergent ways are these two impacts being assessed, such that transhumanists find them acceptable and humanists find them unacceptable? (Clearly this issue includes the problem of ignorance of the risk.)
3.1 Assessment of the Risk of the Loss of our Human Identity
How are we to assess the potential loss of our biological nature? Fundamentally, it is through a value judgment about the importance one assigns to the nature of the human being as a foundational ethical criterion that an individual will be able to arrive at a judgment about whether this very first risk is acceptable, that is, whether it should be assumed. And to whom can the individual turn when the parties to the debate—the transhumanists and the humanists—seem unable to engage in a true dialogue, one involving the co-development of the meaning of ‘human nature’, in order to arrive at a sound judgment? For example, transhumanists and humanists may agree that the assessment must be based on the nature of what it means to be human, of what it means to live as a human being, of what is deemed acceptable; but their ethical assessment of risks and impacts will nevertheless necessarily diverge because of the gap between their ways of understanding this criterion. On one hand, a transhumanist like Kurzweil will consider that the potential loss of our biological nature by incorporating nanobots and organic and neurological replacement implants into the human body, thus transforming human beings into cyborgs, is a positive and morally acceptable impact—a value judgment based on the importance of fulfilling our human nature. But his criterion for what constitutes human nature resides not in our biological limitations, but in our ability to surmount those limitations. Kurzweil expresses, as follows, his disagreement with the humanist Bill Joy, for whom the risk of losing biological humanity is morally unacceptable:
Then perhaps our basic disagreement is over the nature of being human. To me, the essence of being human is not our limitations—although we do have many—it’s our ability to reach beyond our limitations. We didn’t stay on the ground. We didn’t even stay on the planet. And we are already not settling for the limitations of our biologyFootnote 34.
“Will we still be human?” Kurzweil expands on this fundamental question as a way of developing an argument for eliminating the word “posthumanity”, because, he reasons, this word leads to the humanist misconceptionFootnote 35 that the process of incorporating nano machines is accompanied by the risk of killing our essential biological humanness by producing a cyborg or of turning biological humanity into an endangered species that will soon give way to posthumanity (that is, after-humanity, rather than a period of humanity plus):
Still Human? Some observers refer to the post-Singularity period as “posthuman” and refer to the anticipation of this period as posthumanism. However, to me being human means being part of a civilization that seeks to extend its boundaries. We are already reaching beyond our biology by rapidly gaining the tools to reprogram and augment it. If we regard a human modified with technology as no longer human, where would we draw the line? Is a human with a bionic heart still human? How about someone with a neurological implant? What about two neurological implants? How about someone with ten nanobots in his brain? How about 500 million nanobots? Should we establish a boundary at 650 million nanobots: under that, you’re still human and over that, you’re posthumanFootnote 36?
For their parts, humanists like Fukuyama and Dupuy maintain that any change to our essential biological humanness must be deemed a morally unacceptable risk to society as a whole, because, in adopting such change, we lose our human identity. Why is our biological nature fundamental to our human identity? As Fukuyama notes, for Christians, “the answer is fairly easy: it comes from God”Footnote 37. But for Fukuyama and Dupuy, our biological nature is not “infinitely plastic” and can only vary between certain limits that must not be transgressed.
On this view, biological nature is a necessary condition for living as a human, with limitations such as aging and dying that determine our human condition. For Fukuyama, these biological factors are at the basis of our connection to others and of our shared identity. As well, our notions of equality, dignity, freedom, and democracy are embedded in our biological natureFootnote 38.
“Fukuyama maintains that human nature must be considered fixed even if it isn’t, because the consequences of extreme human plasticity would be the disappearance of democratic values” such as those of equality and autonomy as cherished in the humanist and religious spheres that stand opposed to the process of the shift towards the nonbiologicalFootnote 39. Democracies can and must restrict these consequences for human nature that are deemed unacceptable: “True freedom means the freedom of political communities to protect the values” associated with this human natureFootnote 40.
Fukuyama’s and Dupuis’ concept of human nature converge with that of Paul Ricoeur:
Most humanists are in the habit of opposing the acceptance of finiteness to the immoderate desire for infiniteness, a boundless desire that the Greeks denounced as hubris and that causes humanity to descend into self-destruction and the failure represented by despair. In this perspective, the true happiness of the human being as found in historical and concrete existence consists not in acting out the immoderate desire to conquer human finiteness (limitations, aging, fear of death) in order to achieve infiniteness (the joy of being infinite and immortal), but in the act of accepting suffering and finiteness: “Man is the Joy of Yes in the sadness of the finite”Footnote 41.
In 2008, some “ethicists” deemed that Oscar Pistorius should not be allowed to run in the Beijing Games nor in any competitions organized within the International Association of Athletics Federations. It was argued that the design and ergonomics of his carbon fiber prostheses placed Pistorius in the embarrassing position of being a cyborg. This athlete’s case represented a real problem for members of the International Olympic Committee: clearly they persuaded themselves that his prostheses were more than restorative and ultimately amounted to making him a human plus (H+). They feared that his victory would drag down the humanist ideal enshrined in the Olympic Games:
They went as far as imagining the kind of folly that might lead some athletes to want to mutilate themselves in order to use similar prostheses. For these reasons, Pistorius was not allowed in the group of men with no artificial aids. Some people suspect some calamitous denaturings likely to set bad examples and put human beings on the slippery slope to becoming the opposite of humanFootnote 42.
In 2012, however, Pistorius’s was allowed to run in the London Games. The judgment of the Beijing Games was overturned: Pistorius’s prostheses were not deemed to give him an unfair physical advantage.
3.2 Assessment of the Risk of the Loss of Human Control over Machines
Transhumanists and humanists make the same analysis of the risk of the loss of human control over machines (gray goo, terrorism, AI machine superiority). Their assessment of quality of life is also similar. The principal difference between them resides in the means of control of this risk.
For KurzweilFootnote 43, the risks of losing control are acceptable in light of benefits achieved for human health and life; but the risk’s acceptability is based on the urgent recommendation for increased investment in the creation of defensive technologies and the proper orientation of the process of NBIC development towards the creation of feasible nanobots capable of incorporation into the human body. In this scenario, “gray goo” will be countered with “blue goo” (“‘police’ nanobots that combat the ‘bad’ nanobots”). These police nanobots could act as sentinels endowed with high levels of AI for the purpose of protecting humanity from bad or enemy nanobots, which could act like poisons or weapons of war during an arms race: “Thus, assuring a ‘constructive’ Singularity (one that avoids degenerate outcomes such as existential destruction by gray goo or dominance by a malicious AI) could be the best course to prevent the simulation from being terminated”Footnote 44.
For Fukuyama, the threat of loss of human control over machines is best dealt with by human regulation: “Like nuclear weapons or nanotechnology, these are in a way the easiest to deal with because once we have identified them as dangerous, we can treat them as a straightforward threat”Footnote 45. On this view, the criterion for judgment resides in the fact that human beings, by virtue of their free and responsible nature, thrive better by making the kind of technoscientific progress that does not risk killing life on Earth: “True freedom means the freedom of political communities technological progress to protect the values they hold most dear, and it is that freedom that we need to exercise with regard to the biotechnology revolution today”Footnote 46.
Kurzweil rejects Fukuyama’s final assessment: the need for the strict regulation of nanotechnology. In defense of this position, he argues the need for a counterbalancing perspective. On one hand, we have the risks associated with the loss of human control over the machines; on the other hand, we must consider the risk of not developing proper technologies for human development. For Kurzweil, then, the event most closely linked with serious risk (to human life and health; to equity between present and future generations; to the human rights of those subjected to the event) is any delay in the process of NBIC development that will enable us to survive as superintelligent cyborgs:
Even small delays in implementing emerging technologies can condemn millions of people to continued suffering and death. As one example of many, excessive regulatory delays in implementing lifesaving therapies end up costing many lives. (We lose millions of people per year around the world from heart disease alone.)Footnote 47
To conclude this section, if we compare the final transhumanist and humanist assessments of NBIC, the fundamental difference lies in the two parties’ conceptions of human nature. For the humanist, human nature based on biological complexity is the value that we must preserve. For the transhumanist, human nature based on overcoming our biological limitations is the value we must preserve.
As well, there is the question of the risk, following many years of scientific development, of loss of control over AI machines. Faith in the capacity of science to solve the problem of loss of control is an implicit issue in assessing the acceptability of this risk. This implicit issue must be made explicit and addressed if we are to expose the full difficulties of interdisciplinary work between transhumanism and humanism.
4. Divergence of Faith in the Ability of Science to Overcome the Problem of Risk
Can we rationally believe that science will help us overcome all risk of loss of control over the two-edged scientific and technological processes associated with NBIC? Assuming such faith in science is warranted, it is then understandable that the transhumanists find these risks acceptable. But in a sense, it has not been proven—and indeed could not be proven by present-day human beings—how it will be possible to scientifically overcome that panoply of purely theoretical risks of the destruction of humanity that flow unceasingly from the new powers associated with NBIC. Speculatively speaking, these new powers give rise to risks of toxicity, loss of control over freedom, and nefarious use by nanobots for the purposes of creating cyborgs: “Who is controlling the nanobots?” and “Whom are the nanobots talking to?”Footnote 48 It remains the case that, while both transhumanists and humanists are aware of this problem of the twofold nature of NBIC development that makes it difficult to assess the risk of losing control over nanobots, they nevertheless diverge as regards faith in the capacity of science to solve the problem in a manner that will keep pace with scientific developments.
On one hand, Kurzweil bases his whole hope on the idea that the scientific solution to problems of risk will emerge from the process of the development of nanotechnology. He stakes everything on the notion of this development being based on AI, trusting that this will reassure people about science’s capacity to eliminate the problem in question:
If you can swallow that idea, and Kurzweil and a lot of other very smart people can, then all bets are off. From that point on, there’s no reason to think computers would stop getting more powerful. They would keep on developing until they were far more intelligent than we are. Their rate of development would also continue to increase, because they would take over their own development from their slower thinking human creatorsFootnote 49.
Many authors agree with Kurzweil that our best hope consists of placing our confidence in the scenario of a strong AI at the heart of NBIC development in order to become superintelligent cyborgs who will defend our human values (human nature; freedom; and the good life we desire) such that we will not drift into confusion, abuses, and war. Nanobots will offer a friendly intelligence—‘friendly strong AI’—incorporated into our bodies as protection against ‘unfriendly strong AI’. Kurzweil is not asking us, though, to assume the risk of humanity’s destruction based on a moral assessment derived from a black-and-white weighting of the positive and negative consequences of ‘friendly strong AI’ (the good) and ‘unfriendly strong AI’ (the evil)Footnote 50. He cannot count on humanists, whom the precautionary principle renders powerless, to come up with a proper balance for controlling the powers of nanobot superintelligence and the accompanying risks of loss of control: “As Bostrom, Freitas, and other observers including myself have pointed out, we cannot rely on trial-and-error approaches to deal with existential risks”Footnote 51.
On the other hand, the majority of humanists like Fukuyama, Dupuis, and Edgar Morin have maintained their conviction that faith in the capacity of science to transcend such existential risks is blind. As Dupuy writes, “[T]here is in fact neither knowledge nor mastery. [Disenchantment] derives, paradoxically, from belief and the act of faith!”Footnote 52 The reason is that the development of cybernetics is a “collective and anonymous mechanism, a process without a subject, blind and unreflecting—what Heidegger summed up in the aphorism ‘Science does not think.’” As Morin explains:
• It has been very difficult and has taken a very long time to arrive at the notion that science, which had become identified with reason, progress, and the good, could be profoundly ambivalent in its very nature. Awareness of this ambivalence began to spread especially during the 1908s, when it came to be seen that the two greatest catastrophes threatening humanity, nuclear and ecological catastrophe, would both have been inconceivable without scientific development.
• Since modern science is, by its very nature, indifferent to any ethical considerations other than the ethics of knowledge and the ethics of respect for the rules of the scientific game, many scientists are blind to ethical problems raised by scientific activity. This blindness is itself inherent in objective knowledge.
• There is blindness in science as to what it is, what it does, what it is becoming, and what it could do or could become.
• The only barrier to human experimentation is rooted in humanist and religious moralityFootnote 53.
Conclusion
Our tools for analyzing the risks and impacts associated with NBIC development have enabled us to bring out those fundamental differences that leave us subject to the problem of scientific uncertainty, ignorance, conflict, and confrontation between transhumanism and humanism: divergent identification of the risks and impacts associated with the process of incorporating nanobots and implants into the human body with the creation of cyborgs as an end goal; the divergent assessments that underlie the acceptability or unacceptability of these risks and impacts; and divergence as to faith in the capacity of science to overcome identified risks to human beings.
How can we face divergences like these and determine the moral acceptability of the risks associated with human enhancement by means of NBIC? The following steps should be taken:
1. The debate should not deal with theoretical risks, as it does at present. All theoretical risks are possible and cannot be challenged in a reasoned debate. Inquiry should at least deal with probable or hypothetical risks.
2. It is necessary to conduct a debate on the validity of the conceptions of human nature that are being deployed in the final balancing. Absent this, the moral debate lies outside philosophical rationality. Both transhumanists and humanists have bases for justifying their conceptions of human nature. Can we find a philosophical discussion in the literature that demonstrates the superiority of the basis for the claims of one conception over the other? If so, in what way would the conception of human nature relied on by transhumanists be superior to the conception relied on by humanistsFootnote 54?
3. If it proves impossible to give the conceptions of human nature an a priori grounding, they must be established a posteriori, in an interdisciplinary dialogueFootnote 55.
