Two Landmark Legal Cases

In 1884, a landmark case in England established a precedent throughout the common law world that necessity is no defense to a charge of murder. In this case, two shipwrecked sailors, Dudley and Stephens, when a third survivor (the cabin boy Richard Parker) fell into a coma, decided to kill him for food to save their own lives and that of a fourth survivor.^{Footnote 8} Dudley and Stevens were convicted and sentenced to hang, but were reprieved and in fact served 6 months in prison.^{Footnote 9}

One hundred and sixteen years later in the year 2000, the questions of when and why might it be permissible to kill one human individual to save another, of how life and death choices between individuals can be justly made, were questions debated over many months in the United Kingdom, both in public and through the courts. These are the issues raised by the case of the so-called “Manchester conjoined twins” which captured the public imagination not only in the United Kingdom but internationally.

The drama turned essentially on the legitimacy of killing one of the twins so that the other might be saved, in conditions in which it was impossible to save both, and as to who should decide such an issue. I studied and wrote about^{Footnote 10} this case at the time, met the consultant surgeon involved, and followed the court proceedings and the appeal.

The twins were born on the 8^th of August 2000, their bodies fused from the umbilicus to the sacrum,^{Footnote 11} and the lower ends of their spines and spinal cords also fused.

Jodie seemed neurologically normal, whereas Mary had a number of severe brain malformations and abnormal neurological responses.

An elective procedure to separate the twins involved an estimated mortality risk of around 6%. Any separation operation would lead to the death of Mary. It was believed that, although Jodie would have to undergo a series of operations through childhood to correct her congenital malformations, she would eventually be able to lead a substantially normal life if separated from Mary.

On the 18^th of August, 10 days after the birth, the hospital initiated proceedings in the High Court under the Children Act 1989 seeking:

This declaration was granted on 25^th August.^{Footnote 13} Both the parents and Official Solicitor acting on behalf of Mary appealed. On 22^nd September, the Court of Appeal dismissed the appeal and upheld the declaration. On 6^th November, the elective separation operation was performed, Mary died in an operating room, and, Jodie was expected to enjoy a relatively good quality of life with her family thereafter.^{Footnote 14}

This is what it takes in a mature democracy to decide just one such case. Such decisions involve a careful weighing of the evidence and of all the circumstances, the issue of consent and what, absent consent, can be imposed on innocent citizens.

The Moral Machinists purport to carry out this ‘process’ in advance, using superficial sampling in the form of an online “game,” and believe this could yield results that will enable owners, manufacturers of, and passengers in, driverless cars to settle in advance the legal and ethical ramifications of any deaths resulting from the programming of the vehicles. This is naïveté of heroic proportions!

What is Public Morality?

Accepting for a moment, that the holy grail is to find out how to obtain cognizance of public morality and then program driverless vehicles accordingly, the following four steps are offered by the Moral Machinists as crucial:

1. 1) Find out what “public morality” will prefer to see happen in a range of scenarios.

2. 2) On the basis of this “discovery” claim both popular acceptance of the preferences and persuade would-be owners and manufacturers that the vehicles are to be programmed with the best and most acceptable solutions to any survival dilemmas they might face.

3. 3) Citizen agreement, thus characterized, is then presumed to deliver moral license for the chosen preferences.

4. 4) This yields “permission” to program vehicles to spare or condemn those outside the vehicles when their deaths will preserve vehicle and occupants.

Unfortunately, they fail on all counts. Here’s why: The Moral Machinists do not demonstrate the first idea of what morality is, or might be, and hence what being “cognizant of public morality” might amount to.

Ronald Dworkin, one of the greatest Jurisprudential and Constitutional lawyers and philosophers of recent times, drew a distinction which is of crucial relevance here. In the context of a discussion of the famous debate between two other leading lawyers, Lord Patrick Devlin and H.L.A. Hart, concerning the enforcement of morality, Lord Devlin had made great play of the importance of respecting public opinion and community values. Dworkin’s telling rebuke was: “What is shocking and wrong is not his [Lord Devlin’s] idea that the community’s morality counts but his idea of what counts as the community’s morality.”^{Footnote 15}

So-called autonomous vehicles do not solve moral dilemmas, if they did, if they only could, they would be ‘persons,’ properly so-called (super-intelligent AI persons), and would therefor necessarily have rights, interests, duties, and votes, like the rest of us persons. Much more important, their ‘lives’—we should more appropriately say their ‘existences,’ would matter, and would count equally with that of humans.^{Footnote 16} Alas, they would, in the world of the Moral Machinists, have preferences and make decisions without solving the dilemmas that makes those preferences moral.

“Exactly as not just any judgment about things in which science is interested is ‘scientific’”^{Footnote 17} or a part of science, so not just any judgments about things with which morality is concerned are moral judgments.^{Footnote 18} The solving of a moral dilemma involves much, much more than having a preference for one possible outcome of a moral dilemma, just as the resolution of a scientific problem requires much more than simply opting for (stipulating) a particular solution! It has to show how the circumstances which make it a moral dilemma, have been weighed carefully one against another, and morally persuasive reasons, facts and/or justifications found for having a moral preference for one outcome rather than another. Majorities are not necessarily right; neither science nor ethics is produced by casting votes for particular ‘answers’; happy though such a possibility might seem to some! The Moral Machinists are proposing the moral equivalent of deciding whether the world is flat by finding out what people would prefer the answer to be.^{Footnote 19}

Tossing a coin, to be sure, selects an outcome, but not for moral reasons.^{Footnote 20} Neither coin tossing, nor algorithm ‘obedience,’ nor the methods described in the Moral Machine Experiment paper, are methods or processes of moral deliberation! Nor do they seem to have resulted from much deliberation of any kind.

The Moral Machinists work is “useless,” as they claim that of ethicists would be, in the paragraph quoted above. They have not discovered whether citizens and corporations are, or are not, prepared to accept the moral and legal consequences of the decisions of the autonomous vehicles they might either own, or travel in, or have manufactured. Among which will be the consequences of deliberately causing the deaths of innocent bystanders and other road users, consequences which involve much more than operationalizing a principle of minimizing the immediate harm of directing a vehicle. Proper consideration of the possible harms of the preferences delivered by the Moral Machinists must surely include their effect, for example, on due process, the principle “that no person should be condemned unheard,” on constitutional protections for freedom of the individual, and on justice, including criminal justice. All these are considerations of which the authors of The Moral Machine, and their experimental subjects, exhibit no awareness whatsoever!

Having a preference for killing (or sparing) one person rather than another (even a preference shared with thousands or even millions) doesn’t make it moral. The preferences of a certain “Bohemian corporal,”^{Footnote 21} it is salutary to remember, came to be shared by millions.

Laws Arrived at Democratically, and Over Time, are One Indicator.

The helpful drawing provided by the Moral Machinists (1. b. in their paper) for their subjects, which shows a driverless car with the option of mowing down three old people on a crossing or driving into a solid wall, is a proverbial “no brainer.” A human driver should surely steer for the solid barrier relying on the cars collision technology, crumple zones, air bags etc., to keep the occupants safe, or, if not safe, then at least alive. They should hope for the best, rather than self interestedly arguing to themselves that it doesn’t matter that they save themselves because the ‘old codgers’ on the crossing have ignored a ‘do not cross’ sign, and so deserve everything they get!

In any event, there could be no certainty that the passengers in the driverless vehicle would be killed or even injured; and they certainly would be at less risk in their vehicle, than the unprotected old people on the crossing. I can see no rational, nor any moral basis in this example for a choice (human or machine) to do anything but try to avoid the old people.

Surely the right thing to do is to design better safety cages in driverless cars, rather than programming the AI controlling them to select convenient, cheap, lazy and ‘soft’ targets derived from a computer game, for sacrifice.

We have no space here to talk further about the totally trivializing setting of the questions, in which the moral machinists are inviting respondents to make their life or death selections, as simple expressions of what they “prefer.”

We also need to recall that the law, both civil and criminal, is also an expression, if incomplete, of public morality; and if not necessarily more reliable, it is at least more soberly arrived at, than the snap expression of preferences relied on by the Moral Machinists in their “experiment.” The Moral Machinists should have been aware that they also need parliaments, legislators, the courts, human rights conventions and many others (not simply a crude vox pop) not only to be cognizant of their lethal plans, but have debated them, consulted about them, and legislated to accommodate them, possibly in contravention of considerations to which we will now return.

Capital Punishment for Jaywalkers?

The Moral Machinists ask (paragraph 4):

Any decent citizens would immediately ask: What gives you the right to condemn to death innocent people, even ones so wicked as jay-walkers,^{Footnote 22} without examination of the question as to whether anyone has acted illegally, a due process, which commands public respect, which has heard their defense and convicted them, especially in a jurisdiction (unlike the U.K.) which has not abolished the death penalty?

The Moral Machinists seem unaware that most such matters of life or death have not been left to the bare preferences of people who have shown no evidence of deliberation. All juries for example, are cautioned about their responsibilities, required to hear and pay attention to evidence and argument from both sides, (prosecution and defense and also from the judge) and deliberate. I find no evidence of consideration in the Moral Machine Paper, nor any evidence of deliberation by those consulted. “Public morality,” as they crudely and mistakenly understand it, requires only ill informed, unconsidered preferences, given instantly and thoughtlessly, as if playing a computer game!

Due Process

Rather, it is the case that public morality expressed through laws, civil and human rights conventions, and in many other publicly accessible ways in most civilized societies, has not simply been left to individual ‘preference,’ let alone to popular preferences or to popular prejudice, nor yet to psychologists, nor even to philosophers! It has evolved over a lengthy period, often painfully; informed by history, art, literature, culture, personal experience, and much more. It cannot just be bolted on by a Q and A: ‘Do you prefer picture A or picture B?’ There is, for example, a difference between a ‘preference’ and a ‘prejudice,’^{Footnote 23} let alone a moral judgment and a prejudice. Consideration of that difference is nowhere evident in the prejudices (or is it simply the preferences?) of the Moral Machinists or their experimental subjects.

If the answers to the questions chosen by the Moral Machinists are remotely relevant, why are not answers to the questions as to whether or not citizens prefer to spare white people, or women, or priests, or those in military uniforms, not equally crucial? The answer is that these machinist questions have been put through a tendentious ‘moral’ filter, not a filter of data about popular morality or knowledge of ethical reasoning, but a question-begging filter of their own devising. Would the authors accept that if asked they should accept an overwhelming preference for “sparing” only white men (or only black women for that matter) as a part of public morality which required respect, let alone implementation by a machine, without any checks or balances? And what about their apparent embracing of the idea of summary ‘justice’ meted out without any due process on innocent victims? What does/did that tell the respondents about the moral seriousness, let alone the credentials of the questions and the questioners? What does it tell us about the moral seriousness and the scientific credentials of the experimenters and of the journal which published their paper?

The idea that it might be open to individual citizens or corporations to decide who shall be “spared” and who condemned to death, and that this might be a matter of mere individual “preference,” made on the basis of the sorts of sampling described in their paper, whether of vehicle, or owner, or vehicle programmer, or population sampling, is outrageous in the extreme. The question “do you prefer to spare jay walkers or innocent passers by” beggars belief. Why not “do you prefer to spare Jews or gypsies, migrants or citizens, politicians or nurses, sports stars or vagrants?” It is an invitation to approve the summary execution of jaywalkers for something which, even if it constitutes a minor misdemeanor, is not something which the so-called jaywalkers have been either charged with or of which they have been convicted.

It is not, in most jurisdictions, simply up to individuals (whether those individuals are people or even driverless vehicles), to take the law into their own hands and refuse to “spare,” that is, to ‘deliberately execute,’ an innocent fellow individual for the ’crime’ of jaywalking, particularly in circumstances in which the self interest of the vehicle and passengers is obviously paramount.

Oates Law

I have a modest proposal^{Footnote 24} to make on this subject. No one should deliberately kill the innocent without an excuse of overwhelming, plausible, and judicially approved necessity, as in the Manchester conjoined twins case. If a vehicle is directable, but out of control, the driver should not deliberately kill others rather than put herself^{Footnote 25}/itself, its passengers and vehicle at risk. If we are to program driverless vehicles, the road would be much safer if the following was the first law of vehicle robotics. We may call it “Oates Law”.

On 1st November 1911 Captains Scott and Oates, and 14 other members of Robert Falcon Scott’s Antarctic expedition, set off from their Cape Evans base camp for the South Pole. “On 15 March, Oates told his companions that he could not go on and proposed that they leave him in his sleeping-bag, which they refused to do. He managed a few more miles that day but his condition worsened that night… On the morning of 16 March (or possibly 15 March – Scott was unsure) Oates walked out of the tent into a… blizzard and to his death. Scott wrote in his diary: “We knew that poor Oates was walking to his death, but though we tried to dissuade him, we knew it was the act of a brave man and an English gentleman.” According to Scott’s diary, as Oates left the tent he said, "I am just going outside and may be some time…" Oates most likely died on March 17^th.^{Footnote 26} All of Scott’s party eventually died on this journey.

Oates legendary self-sacrifice to try to save the lives of his colleagues in the face of diminishing supplies of food and the need to make enough speed to reach further supplies, sets a moral example that might daunt most humans, let alone driverless cars. What we might expect of an autonomous vehicle, but probably not perhaps, simply of a driverless one is a question for another occasion?^{Footnote 27}

Whatever its priorities, it will surely be conscious that its solution to the problem will ‘say’ something about what sort of creature it is, and will influence, as does all decisionmaking, the sort of creature it will from thenceforth be, both in its own ‘mind’ and in the minds of others. But which ‘other minds’ (if any) will/should it care about most? And of course, an analogous consciousness of what advocating or acting on the principles of the Moral Machine will say about the morality of those who have seriously proposed such a scheme for deciding how, ethically, we might program driverless cars.

Perhaps the most obvious answer to the spurious questions put by the highly amoral and indeed immoral, Moral Machine Experiment, is that driverless vehicles should always risk themselves and their occupants, relying on the safety built into the structure of their vehicles, rather than choose between different groups of innocent bystanders, not least because of the obviously corrupting self-interest involved. And all bystanders must be presumed innocent unless due process has found them guilty, and even then, it is not up to men or machines to devise their own additional penalties. True, not many people would want to ride in, or own, a driverless car with these priorities. But perhaps that is for the best; until, that is, we have real Autonomous Vehicles who can take their own moral responsibilities seriously, and who have had a proper education, not least in law and ethics.