The technological advances that have enabled the deployment of autonomous and remote weapons systems raise a range of significant legal issues that are exacerbated by the priority of advanced weapons systems in research and funding programmes. The complexity of these legal issues will be compounded by increasing technological sophistication and greater proliferation of advanced weapons systems.
These new technologies of war appear, at first flush, to offer the capacity to reduce incidental injury and collateral damage in armed conflict through their potential to offer a more stringent adherence to the principles of distinction and proportionality. While such ability should be welcomed, what has been surprisingly absent from the legal consideration surrounding their use are the logically anterior questions as to whether both autonomous and remote weapons systems can remain meaningfully categorized as ‘weapons’, whether the current legal categorization is adequate to regulate their use, and how their use may challenge the existing legal regime.Footnote 1 Far from stimulating an exclusively theoretical discussion, posing these questions is fundamental to understanding the nature of these technological advances in situations of armed conflict and other complex violent environments, which is essential for the formulation of appropriate legal regulation.
The emergence of technologically advanced military platforms challenges current notions of what weapons and the ‘means and methods of warfare’ are because of their capacity to filter and analyse information, to draw conclusions, and to reach decisions. In short, both autonomous and remote weapons systems possess characteristics associated with autonomy. While this is clearest with autonomous weapons systems, which currently influence human decision-making and which may make decisions over the use of lethal force in the near future, contemporary remote weapons systems are capable of acting with varying levels of independence from direct human control that concomitantly decrease the necessity and relevance of human oversight.Footnote 2 Indeed, the operational independence of contemporary remote weapons systems can relegate the role of the human supervisor only to suspending or aborting attacks once they have been deployed.
These capacities place such technologically advanced military platforms in a distinctly separate category from all preceding forms of military equipment. Throughout history, from the arrow to the ballistic missile, weapons have been the passive implements and inert tools that human agents have directly manipulated in order to inflict violence, damage, and injury. With the advent of autonomous, and to a lesser extent remote, weapons systems, however, the application of force and ensuing military destructiveness may require minimal, if any, human decision-making or oversight. Autonomous and remote weapons systems appear to subsist between the existing legal categories of ‘weapons’ and ‘combatants’. Classification as mere weapons fails both to acknowledge that these systems do not inflict violence in a direct manner but rather serve as intermediary platforms from which weapons are deployed, and to capture their varying levels of autonomy over the use of force. Conversely, the humanitarian protections afforded to the category of combatant imply the exclusion of machines. This suggests that there are significant conceptual and practical barriers that prevent autonomous and remote weapons systems from being classified as combatants. Regulating autonomous and remote weapons systems simply as weapons will result, at best, in partial, and therefore inadequate, mechanisms that fail to account for the real challenges that they pose.
In this context, the legality of both autonomous and remote weapons systems will be evaluated in light of three current uses and challenges: the targeted killing of ‘terrorist’ suspects within the context of armed conflict; the civilianization of military force; and their potential to extend cyberwarfare beyond the virtual world into the physical world. These fall within the broader context of the challenges posed by the new technologies of warfare addressed elsewhere in this edition of the Review.
Finally, there is the need to address the concomitant questions of responsibility that accompany this autonomous capacity in the deployment of military force. Responsibility in law is a concept that has several disparate dimensions.Footnote 3 Thus, although it may be possible for a machine to be responsible in a strictly causal sense for the production of specific results or outcomes,Footnote 4 these are not necessarily accompanied by legal or moral responsibility in a role, liability, or capacity understanding of responsibility that usually attaches to human action.Footnote 5 This disparity in the capability for legal responsibility between humans and machines leads to serious ramifications concerning the accountability for the use of force that arise from the use of autonomous and remote weapons systems, in turn raising the spectre for allegations of impunity.
This article concludes that international humanitarian law in its current manifestation is insufficient to regulate the growing use of autonomous and remote weapons systems. While this is partially due to the permissive nature of IHL in according primacy to military necessity, its failure predominantly arises from its structural inability to cope with the challenges raised by this novel means and method of armed conflict. That the source of the problem is rooted in the question of categorization, however, is simultaneously its source of hope. This is because the current system of IHL is capable of accommodating autonomous and remote weapons systems provided that a method for their categorization in law is negotiated, accepted, and legitimated, and provided that a system for allocating and attributing responsibility for their use can be agreed upon. At this watershed in the development and deployment of autonomous and remote weapons systems, it is particularly timely to undertake a rigorous critical consideration of these issues. Addressing these issues now would contribute to avert similar allegations of impunity that have plagued the modern private military company industry due to the questions regarding their legal categorization and the mechanisms of accountability.Footnote 6
This article begins by sketching out the contemporary state of development for autonomous and remote weapons systems and by providing some historical context. The article then moves to outline the pressures that will push for greater automation of these advanced weapons systems that will result in their increasing deployment in the near future. Then, for the purposes of reviewing their legality, the article critically analyses whether autonomous and remote weapons systems can be appropriately classified as weapons or, more broadly, as means or methods of warfare. In this vein, the article continues by illustrating various issues that arise from applying current weapons laws to autonomous and remote weapons systems. The legality of remote weapons systems will then be tentatively situated within the context of their current use in the targeted killing campaigns of ‘terrorist’ suspects, and within the context of some legal implications that may arise with the greater proliferation of both autonomous and remote weapons systems in the near future. Finally, the article will address the claim that advanced weapons systems may become superior to human agents in the battlespace in their adherence to humanitarian principles, while highlighting the persistent responsibility gap with respect to autonomous and remote weapons systems and the potential for impunity they will create. This article concludes that IHL in its current manifestation is insufficient to regulate the growing use of autonomous and remote weapons systems.
The current state of autonomous and remote weapons systems
Due to the secrecy shrouding military technology, it is difficult to ascertain precisely the current cutting-edge capability of military robotics. Furthermore, even if it were possible to capture their contemporary capacity, the rapidity with which these technologies develop would quickly render this picture obsolete.Footnote 7 For the purposes of this article, it will only be necessary to provide a brief factual sketch, which can be formed from the numerous examples of autonomous and remote weapons systems that are, or have recently been, deployed in the battlespace. For example, Ronald Arkin describes a range of weaponized unmanned military vehicles produced by a number of different companies that are currently available for service on land, sea, and air.Footnote 8 On land, available weaponized systems range from the Samsung Techwin SGR-A1 intelligent surveillance and security guard robot, which is equipped to deliver lethal or non-lethal force either with or without human decision-making, to the iRobot Packbot and TALON SWORDS platforms that are not autonomous.Footnote 9 In the air, the most well-known weaponized unmanned aerial vehicles (UAVs) are the MQ-1 Predator and the MQ-9 Reaper of the US Air Force.Footnote 10 These have gained notoriety for their role in the targeted killings of suspected ‘terrorists’ and were reported to be responsible for over seven hundred deaths in the eighteen month period from the beginning of the Obama Administration to the end of June 2010 in Pakistan alone.Footnote 11 Furthermore, P. W. Singer adds that outer space may soon be a potential zone of conflict opened up to robotic warfare.Footnote 12 The combined picture is one where unmanned military vehicles are fulfilling the full range of military roles and are fast becoming ubiquitous in the battlespace.
To date, these weapons systems have been more remote than autonomous: they are teleoperated by humans rather than being capable of autonomous operation. Teleoperated weapons systems have a long lineage that pre-dates the First World War,Footnote 13 and are relatively uncontroversial from the perspective of IHL because they are ultimately under the full control of human operators.Footnote 14 In other words, remote weapons systems, in the strict sense, are unlikely to engage any additional dimension of IHL in relation to other conventional weapons systems. Instead, it is the rising levels of autonomy that categorically differentiate autonomous from remote weapons systems and their predecessors. In the United States, this process took place in the 1980s to offset the Soviet threat with conventional weapons systems.Footnote 15 Despite the subsequent lull in the pace of development arising from the 1990s peace dividend, interest in advanced weapons systems was soon reignited with the maturation of these military technologies combined with a growing appreciation of the range of roles that they may play in future armed conflict and other complex environments. As a result, research, development, and deployment have surged in the new millennium as these advanced military systems are proving their utility and tentative steps towards weapons autonomy are being made.Footnote 16
In contradistinction to the relatively long history of remote weaponry, the technological developments that have enabled the possibility of increasingly autonomous weapons systems have taken place only recently. Rather than simply constituting a small step in the same direction, the introduction of autonomy into weaponized systems, however, poses many unique challenges to IHL. This is because the hitherto human monopoly over the decision to deploy or inflict violence is challenged by autonomous weapons systems. Furthermore, the capacity for autonomous decision-making may elevate advanced weapons systems from the category of passive military materiel towards that of the active combatant. It should be emphasized at this point that it is not the independent capacity to kill or maim that is the objection being raised here,Footnote 17 but rather that the weapons system itself is able to decide, or significantly influence the decision, whether or not to inflict violence. This decision-making capacity is, however, accompanied by neither the prospect of responsibility nor accountability, thereby eroding the incentives to comply with the rules on the conduct of hostilities. This questions the adequacy of IHL in its current state because its categories have not yet been adapted to accommodate non-human decision-making entities capable of inflicting violence. These advanced weapons system developments also raise challenges under international criminal law insofar as the allocation and attribution of responsibility for unlawful harm is concerned.Footnote 18 The difficulty of categorizing autonomous weapons systems in particular is evident in the terminological confusion that plagues this topic, and is reinforced in attempts to apply the current state of law to the category of weapons systems addressed below. It is, however, important first to briefly illustrate some of the pressures that drive the trend towards autonomy in order to show that this is unlikely to be a temporary phenomenon.
Trajectories for future development
In 2001 the United States (US) Congress mandated specific developmental goals for significant proportions of combat vehicles to be unmanned in the near future.Footnote 19 Later, in 2007, Congress stipulated a strong policy preference for unmanned systems in Department of Defense acquisition programmes by reversing the onus of proof: the development of manned programmes now requires justification through a certification scheme that unmanned systems would be incapable of fulfilling system requirements.Footnote 20 The Department of Defense subsequently (in 2007) devised a coordinated plan to develop and deploy an increasingly sophisticated array of unmanned systems over the next twenty-five years.Footnote 21 These policy incentives complement military utility, which together provide a clear practical driving force behind the desire to field autonomous weaponry: not only are advanced weapons systems cheaper to produce, operate and maintain, but they are perceived to be more capable and efficient than their low-tech, directly human-operated counterparts.Footnote 22 Furthermore, it has been claimed that both autonomous and remote weapons systems enable an increase in the projection of state power despite declining military recruitment figures and, in decreasing the exposure of friendly forces to danger, will significantly lower the number of casualties and remove the democratic resistance to military deployment.Footnote 23
There are clear pressures towards automation. Armin Krishnan points, for instance, to the force multiplier effect gleaned from automating even basic processes within remote systems, whereby one person will be capable of controlling several remote weapons systems.Footnote 24 This push towards automation is reinforced by the perceived performance superiority of such systems that may be capable of enhancing the abilities of human combatants. Ronald Arkin has suggested that advanced systems may be able to analyse and collate large amounts of information thereby enabling a speedier and better informed reaction, and has further pointed out that autonomous systems maybe capable ‘of independently and objectively monitoring ethical behaviour in the battlefield by all parties and reporting infractions that might be observed’.Footnote 25 These perceived benefits may be enhanced by the reduction of military budgets in many Western states.Footnote 26 Militaries may scramble towards advanced technological systems to compensate for lost capabilities.
Finally, although this does not directly affect considerations of legality, it should be noted that the development of both autonomous and remote weapons systems is not unique to the traditional militarily advanced states such as the United States, or even Western NATO countries more broadly.Footnote 27 For instance, Iran has recently unveiled its first unmanned bomber, and China has also showcased a new fleet of drones that raise questions of broader strategic significance.Footnote 28 This revolution in military technology not only upsets the current balance of military capabilities, but may also have subtler legal effects. This is because the lack, or inadequacy, of legal regulation over both autonomous and remote weapons systems would be more difficult to rectify once these technologies have proliferated.
Terminological hurdles: weapon, means or method of warfare
While the terminology applied to this topic has thus far been used consistently, it is important now to elaborate upon these terms and the applicable legal definitions. This will illustrate that IHL, as currently conceived, is incapable of coherently categorizing both autonomous and remote weapons systems.
It must be noted at the outset that the terms ‘weapon, means or method of warfare’Footnote 29 have not been exhaustively defined in IHL or applicable legal instruments. In lieu of a legal definition, reliance is placed instead on the constellation of stable and identifiable characteristics that shape these terms. This is evident in the plain linguistic meaning of ‘weapon’; the dictionary definition for which is primarily ‘a thing designed or used for inflicting bodily harm or physical damage’ and secondarily as ‘a means of gaining an advantage or defending oneself in a conflict or contest’.Footnote 30 Thus, the conflation between the physical implements through which violence is inflicted, and the techniques by which it is used for these purposes, share early etymological roots that imply a fundamental connection. Indeed it would be nonsensical to consider the characteristics of a weapon isolated from the context of its use, which is reflected in the ICRC Guide to the Legal Review of New Weapons, Means and Methods of Warfare.Footnote 31
That the understanding of what a ‘weapon’ is has been assumed to be commonly held may be inferred from the fact that this term has not been defined in the conventions and provisions that are directly relevant.Footnote 32 Yet, commentators have alluded to the meaning of ‘weapons’ under international law. William Boothby suggests that weapons are ‘tools of warfare, of killing, maiming, and destruction’,Footnote 33 while Justin McClelland suggests that the term ‘connotes an offensive capability that can be applied to a military object or enemy combatant’.Footnote 34 According to the ICRC Guide, the ‘terms “means and methods of warfare” designate the tools of war and the ways in which they are used’.Footnote 35 The Guide refers to national military documents to further illuminate the term, with those from Australia and the United States in particular providing definitions that are not self-referential. The Australian Instruction provides that a ‘weapon’ is ‘an offensive or defensive instrument of combat used to destroy, injure, defeat or threaten. It includes weapon systems, munitions, sub-munitions, ammunition, targeting devices, and other damaging or injuring mechanisms’.Footnote 36 The US Department of Defense's Law of War Working Group differentiates between the terms ‘weapon’ and ‘weapon system’. The former refers to ‘all arms, munitions, materiel, instruments, mechanisms, or devices that have an intended effect of injuring, damaging, destroying or disabling personnel or property’ while the latter is more broadly conceived to include ‘the weapon itself and those components required for its operation, including new, advanced or emerging technologies’.Footnote 37 The present article adopts this distinction between a weapon and a weapons system. This is because autonomous and remote weapons systems cannot be narrowly categorized as only weapons because they do not inflict damage or harm in a direct manner, as a mine or a cruise missile would. Instead, they are appropriately categorized as a weapons system because they serve as an intermediary platform from which the actual weapons are deployed. Finally, the US Law of War Working Group's definition is limited to the military systems that are integrally associated with the use of force; this has the merit of appropriately excluding non-violent support systems, such as surveillance platforms, from the purview of a weapons system.
Turning next to means and methods of warfare, it is clear that these terms are conflated with the weapon itself, at least insofar as the review of the legality under Article 36 of Additional Protocol I is concerned. Kathleen Lawand writes: ‘A new weapon – that is, a proposed means of warfare, cannot be examined in isolation from the way in which it is to be used – that is, without also taking into account the method of warfare associated with it’.Footnote 38 The interconnectedness of these terms arises from the expansive nature of Article 36, which does not clearly distinguish between ‘weapons’ and ‘means of warfare’; in other words, Article 36 may be tautological in order to cast as broad net as possible. The ‘method’ of warfare, on the other hand, ‘is usually understood to mean the way in which weapons are used’.Footnote 39 Justin McClelland usefully suggests that the terms ‘means’ and ‘methods’ should be read together in order to ‘include those items of equipment which, whilst they do not constitute a weapon as such, nonetheless have a direct impact on the offensive capability of the force to which they belong’.Footnote 40 While the example he gives is a mine clearance vehicle, when read together the terms means and methods should be extended to autonomous and remote weapons systems in order to ground rigorous legal review. This is because advanced weapons systems may deploy existing conventional weapons in novel ways that might otherwise circumvent a holistic approach to the review of legality, as discussed below. This is clearly a useful approach with which to address autonomous and remote weapons systems because in ‘military technological thinking and research, atomistic ontologies are being replaced by thinking in terms of systems, networks, and swarms’.Footnote 41 In other words, adhering to strict divisions between armed and unarmed systems or between autonomous and remote systems may become untenable due to the close interconnectedness of these systems.
Leaving aside the terminological questions that hang over ‘weapon, means and method of warfare’, the capacity for autonomous decision-making pushes these technologically advanced systems to the boundary of the notion of ‘combatant’. Confusion between these categories is evident in the range of approaches by commentators in a recent special issue of Philosophy and Technology. For example, Ugo Pagallo uses the term ‘robot soldier’Footnote 42 in a clear departure from the established categories of IHL, although clearly alluding to the potential for autonomous weapons systems to mirror the capability of combatants, while other authors in that special issue consider these as only weapons.Footnote 43 The German military manual, which provides that ‘combatants are persons who may take a direct part in hostilities, i.e., participate in the use of a weapon or a weapon-system in an indispensable function’, indicates potential for the confusion between means and methods of warfare and combatants.Footnote 44 Although this characterization was used in the context of differentiating categories of non-combatants who are members of the armed forces, the circularity of this definition illustrates precisely the difficulties associated with defining ‘weapon’ and ‘weapons system’. The point is, however, that aside from the explicit reference to ‘persons’, the definition of a combatant as an operator of a weapon or a weapon system illustrates the potential for classifying an autonomous weapons system as a combatant, at least in theoretical terms.
This article will not seek to consider autonomous weapons systems as combatants because of the profound implications this would entail for IHL. Rather, the point is to highlight the potential ontological impact of autonomy on weapons systems, questioning their categorization as strictly ‘weapons’. The use of autonomous and remote weapons systems that possess autonomous capacities clearly poses challenges to contemporary IHL.
Applying current weapons laws to autonomous and remote weapons systems
While the previous section tackled the terminological questions associated with the categorization of autonomous and remote weapons systems, this section analyses their compliance with currently applicable laws governing weaponry.Footnote 45 It should be noted at the outset that there is currently neither explicit prohibition of autonomous and remote weapons systems nor any international regulation for their deployment in situations of armed conflict per se. There was the potential for unmanned combat aerial vehicles (UCAVs) to breach specific Treaty-based restrictions because they share some characteristics both with cruise missiles and with bombers. For example, ground launched cruise missiles within certain mass parameters were prohibited under the Intermediate-Range Nuclear Forces Treaty 1987.Footnote 46 UCAVs, however, could be distinguished from cruise missiles because they were designed to return to base and because they possessed flight control capable of altering the route to the target. Similarly, UCAVs could be excluded as a bomber under the Strategic Arms Reduction Treaty (START)Footnote 47 because of differences in both range and payload. These distinctions led the US authorities to consider that UCAVs did not generally violate these specific Treaty obligations.Footnote 48
Legal review of new weapons, means, and methods of warfare
As alluded to above, however, the lack of directly applicable regulation does not absolve legal considerations surrounding the intrinsic characteristics of the weapons themselves, or their use in ‘some or all circumstances’, because all new means and methods of warfare must be subjected to legal review. Although this requirement is most recently expressed in Article 36 of Additional Protocol I of 1977 to the Geneva Conventions, its roots may be traced back to the 1868 St Petersburg Declaration that is regarded as the first major international instrument to prohibit the use of a specific weapon in armed conflict.Footnote 49 That ‘the use of means and methods of warfare’ may be subject to legal consideration is considered to be customary IHL.Footnote 50 These criteria were elaborated upon by the International Court of Justice in its 1996 Advisory Opinion:
The cardinal principles contained in the texts constituting the fabric of humanitarian law are the following. The first is aimed at the protection of the civilian population and civilian objects and establishes the distinction between combatants and non-combatants; States must never make civilians the object of attack and must consequently never use weapons that are incapable of distinguishing between civilian and military targets. According to the second principle, it is prohibited to cause unnecessary suffering to combatants: it is accordingly prohibited to use weapons causing them such harm or uselessly aggravating their suffering. In application of that second principle, States do not have unlimited freedom of choice of means in the weapons they use.Footnote 51
Since Article 36 of Additional Protocol I is considered to embody the customary law obligation of reviewing weapons, it forms a useful starting point. While the text of the Article itself does not elaborate upon the scope or meaning of the phrase, the Commentary to the Article provides that:
The words ‘methods and means’ include weapons in the widest sense, as well as the way in which they are used. The use that is made of a weapon can be unlawful in itself, or it can be unlawful only under certain conditions … However, a weapon that can be used with precision can also be abusively used against the civilian population. In this case, it is not the weapon which is prohibited, but the method or the way in which it is used.Footnote 52
Similarly, the ICRC Guide provides that ‘the legality of a weapon does not depend solely on its design or intended purpose, but also on the manner in which it is expected to be used on the battlefield’.Footnote 53 These sources suggest that a weapon which is prima facie lawful, or which has previously passed legal review, may subsequently be used in a manner that is deemed unlawful. This raises significant implications for the legal review of both autonomous and remote weapons systems. These advanced weapons systems cannot strictly be categorized as weapons because they generally serve as an intermediary platform from which existing weapons, which have previously passed legal review, are deployed. Yet, the way these conventional weapons are used has been drastically altered when deployed by autonomous or remote weapons systems; consequently, a new legal review should be required that takes into account these new means and methods of warfare from a holistic perspective.
William Boothby has provided an initial analysis of the legality of autonomous and remote weapons systems.Footnote 54 His primary characterization, however, concerns the ability of an unmanned system to deploy or control weapons. Although there is a requirement under Article 36 of Additional Protocol I to review the legality of systems that do not control weapons, he considers it is unlikely that these systems will contravene any of the relevant considerations.Footnote 55 Moving on to what he terms ‘unmanned combat vehicles’, he draws attention to relevant legality considerations. Where the prior decision concerning an attack remains with a person, he sees no relevant issues being raised.Footnote 56 It is the autonomous decision-making with relation to an attack that ‘must be considered by the weapons reviewer in the light of the precautions which are required by international law before an attack is launched’.Footnote 57 In such instances, the legal reviewer must consider the ability of the system to adhere to the discrimination requirement to distinguish between civilians and combatants. He emphasizes that:
human decisions, some of them taken in advance of the UCV [unmanned combat vehicle] mission, can suitably constrain the timing, location, objective, and means of any UCV attack, the algorithms, depending on their sophistication and reliability, may be able effectively to restrict attacks to objects recognized by the software as legitimate military objectives.Footnote 58
While William Boothby may be strictly correct with relation to determinations of legality under IHL, he does not go on to consider the question of responsibility for the actions of a remote weapons system.
Finally, Noel Sharkey has drawn attention to the slippery slope engendered by the atomized nature of weapons review, especially in the area of autonomous and remote weapons systems:
Take the case of the MQ1-Predator UCAV. JAG [Judge Advocate General's Corps] first passed it for surveillance missions. Then when it was armed with Hellfire missiles, JAG said that because it had previously passed both the Predator and the Hellfire missiles, their combination did not require a review … If arming robots keeps soldiers out of risk and the weapons are already legal, then there might be no legal opposition to deploying robots with weapons.Footnote 59
While this type of reasoning may be appropriate for the legal review of other combinations of weapons and weapons systems, applying such an approach to both autonomous and remote weapons systems fails to recognize the potential for radical transformation in the conduct of armed hostilities raised in this specific context.Footnote 60
At a minimum, new means and methods of warfare must satisfy the two principles of unnecessary and superfluous injury,Footnote 61 and distinction.Footnote 62
Superfluous injury or unnecessary suffering
The prohibition of means and methods of warfare that are of a nature to cause superfluous injury or unnecessary suffering under IHL is found in Rule 70 of the ICRC Study.Footnote 63 William Boothby considers this issue irrelevant in the present context because the legality of the weaponry that autonomous and remote weapons systems deploy is independently reviewed.Footnote 64 Where the weapons system itself does not inflict superfluous injury or unnecessary suffering, this is certainly correct. While there may be some exceptions, such as where an autonomous or remote weapons system is itself the weapon (such as the US military's Switchblade),Footnote 65 it is indeed unlikely that remote and autonomous weapons systems will challenge this principle.
Discrimination
The customary IHL basis for the principle of discrimination is encapsulated in Rules 11 and 12 and supported by Rule 71, which prohibits the use of weapons that are by nature indiscriminate.Footnote 66 With regard to current technological capabilities, roboticist Noel Sharkey writes that ‘no autonomous robots or artificial intelligence systems have the necessary skills to discriminate between combatants and innocents’.Footnote 67 The poor record of autonomous and remote weapons systems in distinguishing threats was poignantly illustrated by the shooting down of the civilian Iran Air Flight 655 by USS Vincennes in July 1988 resulting in the deaths of all 290 on board.Footnote 68 The warship was equipped with an automated Aegis system which marked the civilian passenger jet as an ‘assumed enemy’ prior to the crew authorizing weapon launch.Footnote 69 During the course of the 2003 invasion of Iraq, an almost identical scenario resulted in the downing of two allied planes when US Patriot missile batteries classified the aircraft as Iraqi rockets.Footnote 70 Thus, there is a strong case against the capacity of an autonomous and remote weapons system to fulfil the discrimination requirement, even in instances where humans ultimately make the final decision to strike.Footnote 71
Precautionary requirements
The principle of discrimination is further supported by the distinct requirement embodied within Rule 17 that requires that parties to the hostilities take precautions in the choice of means and methods of warfare in order to avoid or minimize incidental injury to civilians and collateral damage to civilian objects.Footnote 72 As indicated by William Boothby, precaution is likely to be the most relevant ground for considering the legality of autonomous weapons systems. This will require human involvement in the decision-making process either ‘in the loop’ or by constraining ‘the timing, location, objective, and means’ of an attack such that the weapons system would be capable of restricting attacks only to legitimate military targets.Footnote 73 These are, however, two very different situations: in the former, there is continuous human monitoring in contradistinction to the latter where human decision-making is only involved during the initial stages of an attack. This difference becomes important in the context of Rule 19:Footnote 74 in the latter situation, as Boothby acknowledges, it is likely that human decision-making will be required unless the initial set of constrains remain valid throughout the entire operation.Footnote 75
The principle of proportionality
A further consideration concerns the principle of proportionality. While some commentators seek to analyse proportionality in the context of weapons causing superfluous injury or unnecessary suffering, Yoram Dinstein, among many others, criticizes such an approach because proportionality is a principle that arises with the consideration of incidental injury to civilians and collateral damage to civilian objects in relation to the military objective pursued.Footnote 76 Thus, the question of proportionality may arise independent of discrimination considerations. Although the term proportionality may not be specifically mentioned in Additional Protocol I, it does find expression in Article 51(5)(b) which prohibits expected incidental injury and collateral damage to civilians or civilian objects excessive in relation to the military objective anticipated. This obligation is reiterated in Article 57(2)(a)(iii) of Additional Protocol I and Article 8(2)(b)(iv) of the Rome Statute which establishes such ‘clearly excessive’ loss of life, injury, or damage as a war crime. Thus, proportionality is an important consideration where there is potential for unjustifiable effects for civilians.
With regard to autonomous and remote weapons systems in relation to this criterion, Noel Sharkey writes: ‘there is no sensing or computational capability that would allow a robot such a determination [of proportionality], and nor is there any known metric to objectively measure needless, superfluous or disproportionate suffering. They require human judgement’.Footnote 77 This raises an issue specific to autonomous and remote weapons systems; presently a person must be sufficiently involved in the decision-making loop to satisfy the proportionality criterion.
Combining discrimination and proportionality with regard to the current methods of warfare in which autonomous and remote weapons systems are embedded raises significant questions of legality. The inability to comply with the rules of discrimination and proportionality is particularly apparent in recent instances of targeted killings conducted with remote weapons systems. Similarly, autonomous and remote weapons systems may have difficulties in recognizing hors de combat status as a result of poor sensing and computational ability; this may result in further violence being inflicted upon individuals who are hors de combat (or seek to be, by surrender) and, therefore, in violation of the IHL prohibition of the Denial of Quarter.Footnote 78
Situating the legality of autonomous and remote weapons systems
In order to fully address the legality question of autonomous and remote weapons systems it is essential to consider how they are currently used.Footnote 79 While autonomous and remote weapons systems may not be inherently unlawful, the ways they are used may be. If such weapons systems are persistently implicated in legally controversial practices, however, it may justify a reconsideration of the legality question. Three especially pertinent uses and challenges are raised below.
Targeted killings and remote weapons systemsFootnote 80
Despite the controversy surrounding the practice of targeted killings, there is currently no commonly accepted definition. A former Legal Advisor for the International Committee of the Red Cross suggested that ‘targeted killing’ denotes ‘the use of lethal force attributable to a subject of international law with the intent, premeditation and deliberation to kill individually selected persons who are not in the physical custody of those targeting them’.Footnote 81 The controversial legality of this practice was highlighted by Philip Alston, the former UN Special Rapporteur on extrajudicial, summary or arbitrary executions, in his challenge to the US government to provide the legal basis upon which these take place:
Targeted killings carried out by drone attacks on the territory of other States are increasingly common and remain deeply troubling. The US Government should disclose the legal basis for such killings and identify any safeguards designed to reduce collateral civilian casualties and ensure that the Government has targeted the correct person.Footnote 82
Less than a year later, Harold Koh, the Legal Advisor to the US Department of State, replied with the unequivocal position held by the Obama Administration:
What I can say is that it is the considered view of this Administration – and it has certainly been my experience during my time as Legal Adviser – that US targeting practices, including lethal operations conducted with the use of unmanned aerial vehicles, comply with all applicable law, including the laws of war.Footnote 83
It is beyond the scope of this article to consider the validity of Harold Koh's assertion regarding US targeting practices.Footnote 84 Rather, the pertinent question concerns the review of the legality of using autonomous and remote weapons systems in a sustained campaign to kill suspected terrorists extraterritorially.Footnote 85 In other words, while autonomous and remote weapons systems may generally be capable of fulfilling the legality requirements for new weapons, their use in the context of targeted killings campaigns may not have been considered during the initial legal review. While Article 36 of Additional Protocol I requires States ‘to determine whether its employment would, in some or all circumstances, be prohibited’ the Commentaries specify that ‘the article is intended to require States to analyse whether the employment of a weapon for its normal or expected use would be prohibited under some or all circumstances. A State is not required to foresee or analyse all possible misuses of a weapon, for almost any weapon can be misused in ways that would be prohibited’.Footnote 86 However, the position of this current author is that insofar as remote weapons systems can be considered as weapons, and have become inextricably associated with the policy of targeted killings, Article 36 would require a reappraisal of their legality because ‘[their] normal or expected use’ would be transformed when used as implements that enable a legally controversial practice.
There are three specific legal questions that, although inherent to the practice of targeted killings, are made more complex by the use of autonomous and remote weapons systems. The first is the requirement of distinction between military and civilian targets, which is challenged by the labelling of the targets as ‘suspected terrorists’. In an armed conflict, persons in this perceived category are prima facie civilians and are protected as such except when and for as long as they participate directly in hostilities; in non-international armed conflict specifically, individuals considered as members of organized armed groups having a ‘continuous combat function’ can be targeted at all times.Footnote 87 The uncertainties surrounding the ability of remote weapons systems to discriminate between legitimate military targets and non-military targets raise serious concerns about the erosion of the protection of civilians.
The second is the related question of proportionality in the use of force to prevent excessive force from being directed at civilians or civilian objects. This consideration is all the more important in this context because the suspected ‘terrorists’ being targeted are likely to intermingle with civilians or to be in the midst of civilian objects. Targeted killings by remotely controlled UAVs have reportedly been responsible for large numbers of casualties.Footnote 88 While the official Central Intelligence Agency (CIA) statistics claim a clean record with zero civilian casualties, this claim is the subject of considerable dispute.Footnote 89 This provides some factual basis for the contention that the use of autonomous and remote weapons systems in pursuit of a regime of targeted killings is unlawful on the grounds that it fails the requirements imposed by discrimination and proportionality.
The third legal question is the inevitability of disproportionate force associated with the denial of surrender or hors de combat status of the target.Footnote 90 There are difficulties inherent in attempting to surrender to remote weapons systems, but these may be overcome, as in an example provided by P. W. Singer where Iraqi combatants effectively surrendered to an American remotely controlled UAV in the first Gulf War.Footnote 91 The question rather is whether the intention to surrender or hors de combat status would be recognized by more autonomous weapons systems where human attention becomes increasingly alienated and removed. Finally, the question of surrender to an autonomous or remote weapons system raises a technical legal issue: the institution of surrender is one between rival combatants. As a combatant cannot surrender to the weapons of the opposing side alone, this raises questions as to the possibility of surrendering to such a system, or conversely, whether either autonomous or remote weapons systems can coherently be categorized as mere weapons.
The civilianization of military force
Military and strategic history has been characterized by attempts by the state and its military establishments to monopolize the material and technological means through which organized force is deployed.Footnote 92 The ambiguous status of autonomous and remote weapons systems is starkly illustrated by the lack of control in its development, production, and distribution in contradistinction to this monopolizing tendency; indeed, the rapid technological advancement in the area of military robotics has not, paradoxically, been accompanied by attempts to regulate its use or distribution. In part this may be due to the fact that civilian research and development underlies much of the relevant technology and the production of the actual machinery occurs in civilian facilities. Furthermore, civilians are intricately involved in the maintenance and actual use of these systems.Footnote 93 Thus, attempts by the military establishment to monopolize the associated technology or capability may be futile.
The heavy involvement of civilians in all stages of autonomous and remote weapons system development, production, maintenance, and use may significantly widen the category of legitimate military objective. First, this creates a potentially wide category of dual-use facilities where autonomous and remote weapons are being designed, or produced, for both civilian and military functions. The text of Article 52(2) of Additional Protocol I may be understood to enable the classification of all dual-use facilities as legitimate military targets.Footnote 94 Second, civilians operating autonomous and remote weapons systems are likely to lose their immunity from direct attack for the duration of their direct participation in hostilities and will thus be susceptible to domestic prosecution.Footnote 95 It is also important to note that this legalistic consideration regarding legitimate military targets may overshadow the tendency towards civilians bearing the brunt of the adverse effects of contemporary armed conflict in reality. This may in turn create pressure to target civilians and civilian objects in lieu, especially taking into account the increased direct participation in hostilities of the former and the dual-use of the latter.
Aside from purely IHL considerations, the infiltration of military force into the civilian sphere may engender the creep of militarization into law enforcement and policing.Footnote 96 Furthermore, the ready availability of this technology enables broad access: anti-whaling campaigners, for example, have deployed surveillance drones to spot a Japanese whaling fleet, and ‘drones and other types of unmanned aerial vehicles […] are being sent on civilian missions such as crop inspections or marine mammal surveys’.Footnote 97 Indeed, civilian applications for drone technology are seen in the United States as inevitable, and the Federal Aviation Administration is to propose new rules to integrate small drones into national airspace.Footnote 98 While it must be emphasized that these systems are not currently weaponized, and therefore not within the ambit of autonomous and remote weapons systems, it should also be noted in this context that the unmanned military systems were initially deployed for surveillance purposes and were only weaponized subsequently. It may therefore not be surprising if drones in the civilian sphere, such as those to be used by police, were to be weaponized at a later stage, leading to the subtle militarization of the civilian sphere.
Cyberwarfare: blurring the lines between the virtual and real worlds
The deployment of autonomous and remote weapons systems may allow the conduct of cyberwarfare to have very concrete and real-world effects. An emerging field,
cyberwarfare is the conduct of military operations by virtual means. It consists of nation-states’ using cyberspace to achieve essentially the same ends they pursue through the use of conventional military force: achieving advantages over a competing nation-state or preventing a competing nation-state from achieving advantages over them.Footnote 99
Currently, this form of military conflict exists primarily in ‘information warfare units to develop viruses to attack enemy computer systems and networks, and tactics … to protect friendly computer systems and networks’.Footnote 100 Thus, cyberwarfare is correctly termed; it cannot yet be considered a form of armed conflict because it remains exclusively within the virtual realm of cyberspace but bears many of the other hallmarks of warfare. This is because the aim of cyberwarfare is ‘to get the upper hand of the enemy in a war under conditions of informatization … whether or not we are capable of using various means to obtain information and of ensuring the effective circulation of information’.Footnote 101
While conduct in cyberspace may have significant and pervasive effects in the real world, it is the emergence of autonomous and remote weapons systems that may directly incorporate cyberwarfare (along with its lower-threshold counterparts of cybercrime and cyberterrorism) into armed conflict in the physical world. This is because the computer systems underlying both autonomous and remote weapons systems are especially vulnerable to cyberattack, which may in turn mean that these weapons systems may be hijacked for the purposes of perpetrating a physical attack in the real world. This has already been revealed in the recent ‘Keylogger’ computer virus infection at Creech Air Base in Nevada.Footnote 102 While it is uncertain whether or not this was actually a directed attack, and while it appeared benign, the vulnerability of both autonomous and remote weapons systems was clearly demonstrated. With the specific vulnerability of these systems to cyberattack in mind, the recent Iranian claims to have brought down an advanced US stealth drone by hacking into its systems underline the potential dangers of cyberattacks having real world repercussions.Footnote 103
These cyberspace issues are further compounded by difficulties surrounding responsibility for the actions of autonomous and remote weapons systems. There are two main limbs to the challenge of responsibility. The first is simply that the control of even a strictly remotely controlled weapons system may constantly be under doubt because of the possibility that its information systems have been compromised. This raises questions about whether it will be possible to definitively attribute responsibility over such a system to its controller.Footnote 104 The second difficulty stems from the nature of cyberwarfare itself. Leaving aside the additional difficulties associated with cybercrime and cyberterrorism, unlike ‘the physical world, when a country is at war, it knows it is at war and, most likely, with whom’ when it comes to cyberwarfare it may be impossible to ascertain ‘who was responsible for the attacks or why they were launched’.Footnote 105 While this engenders serious concern for cyberspace, the potential for real world violence to be unleashed through the anonymity of cyberspace is likely to create impunity for potentially grave violations of international humanitarian and human rights law.
Superiority of autonomous and remote weapons systems and the responsibility question
As the potential failings of autonomous and remote weapons systems have been addressed, their potential to outperform their human counterparts in discrimination and proportionality tasks must equally be considered. The roboticist Ronald Arkin is optimistic that robotic lethality may be suitably governed to the point that both autonomous and remote weapons systems may outperform humans.Footnote 106 Similarly, in relation to the discrimination criterion, Justin McClelland writes:
One area that will need careful consideration is the application of the criteria of distinction to the employment of ‘autonomous’ weapons. Such weapons have the capability, to varying degrees, to make decisions without any human involvement on the identification and attack of targets. This absence of what is called a ‘man in the loop’ does not necessarily mean that the weapon is incapable of being used in a manner consistent with the principle of distinction. The target detection, identification and recognition phases may rely on sensors that have the ability to distinguish between military and non-military targets. By combining several sensors the discriminatory ability of the weapon is greatly enhanced.Footnote 107
The precise determination of legality with regard to discrimination will depend upon the characteristics of the system itself, but it should be noted here that the potential for increased sensory ability may be irrelevant if the computational system evaluating the data input is incapable of making an appropriate analysis, as Noel Sharkey has suggested.Footnote 108
Ronald Arkin notes further that autonomous and remote weapons systems may be capable of better adherence to IHL compared to human combatants.Footnote 109 For example, these weapons systems may be able to combine the input from an array of sensory data to assess the threat and confirm the target, and it may be possible to programme the weapons system to refrain from attack despite risk to itself until a higher degree of certainty is ascertained to meet the principle of discrimination. Similarly, autonomous and remote weapons systems may be equipped with non-lethal weaponry in combination with discrimination precautions to rebalance the proportionality consideration. Finally, such systems will be resilient to adverse psychological effects that underlie the perpetration of some unlawful acts by human actors.
Thus, while Ronald Arkin may be correct that machines may possess a greater capacity to adhere to IHL, which may also in turn incentivize human soldiers to respect IHL, this approach neglects the fundamental consideration of responsibility for their breach. Peter Cane writes that responsibility looks in both temporal directions, historic and prospective; the former concerns notions of accountability for actions after the fact, while the latter serves to delineate obligations and duties before the fact.Footnote 110 Applying this idea of responsibility to the current discussion, it is clear that the greater capacity for adhering to IHL by autonomous and remote weapons systems is exclusively prospective in outlook. This is because only the obligation or duty to adhere to the relevant legal requirements can be programmed into the weapons system. Notions of historic responsibility are simply inapplicable under current legal understandings. In other words, while a higher level of obligation or prospective responsibility to adhere to IHL may be programmed into autonomous and remote weapons systems, it will be difficult if not impossible to attribute historic legal accountability if this law is breached. The potential for impunity arising from the use of such weapons systems is thus readily apparent.
The difficulty associated with historic responsibility is further compounded by possible attempts to attribute such responsibility. Although there is the potential that artificial decision-making may elevate its ontological level, the concomitant questions raised for responsibility have not been settled. Clearly, both the purpose and appropriateness of punishing a machine are questionable. Relocating the locus of punishment to natural persons with the closest nexus to these machines, however, runs the risk of scapegoating those persons: the possession of autonomous decision-making capacity may break the causal chain that would justify the attribution of responsibility to those persons. Thus, autonomous and remote weapons systems may have a higher capacity to adhere to IHL, but will inevitably have much lower levels of responsibility for any breaches. This leads to impunity for conduct in armed conflict.
The problems associated with responsibility are further compounded by the atomized approach of the law to questions of responsibility; that is, that it seeks to attribute responsibility to a concrete and definable entity for the creation of some specified effect. This runs contrary to the development of networks and swarms.Footnote 111 This has implications for responsibility for autonomous and remote weapons systems, as Mark Coeckelbergh explains:
In a network, (military) activity is not about single, atomistic agents exercising their agency in single actions. Instead, agency (if this is still the adequate term at all) is distributed, collective, and emergent. It cannot be reduced to the level of the parts (systems metaphor), nodes (network metaphor), or – why not – ‘bees’ (swarms metaphor). None of the parts, nodes, or bees control the action (in this sense they are not agents), but the system, network, or swarm as a whole acts.Footnote 112
Thus, current conceptions of legal responsibility may be wholly inadequate to address the questions raised by the rise of autonomous and remote weapons systems. This inadequacy becomes all the more important when the outcomes of the use of these weapons systems rise to the level of international crimes.
Conclusion
While this article has focused on the many potential pitfalls arising from the emergence of autonomous and remote weapons systems, it is necessary to emphasize their potential to reinforce humanitarian principles and enable a closer adherence to IHL. Ronald Arkin is certainly correct in highlighting the potential superiority of autonomous and remote systems vis-à-vis human frailties in situations of armed conflict. Moreover, the utility of these systems continues to be demonstrated, which guarantees their future in the battlespace.
It is, however, easy to be blinded by the combined apparent superiority and inevitability of autonomous and remote weapons systems so that IHL fails to fully realize the categorical departure that is signalled by their arrival. As David Kennedy has pointed out, ‘humanitarian rules may well criticize too little – relying for their implementation on the agreement of the military and political establishments which collectively promulgate them. Waging war within the rules may so little constrain the use of force that adherence to humanitarian rules will do more to legitimate than contain force.’Footnote 113 As discussed above, the continued applicability of IHL to these novel weapons, means and methods of warfare is apparent. The problem, however, is that IHL provides guiding principles rather than clearly defined rules and regulations. Again, David Kennedy speaks of the problem that this creates: ‘Humanitarian standards seem too vague to restrain those determined to use force, too manipulable to embody humanitarian commitments. In the chaos of war, it seems unlikely that anything other than a clear rule will function’.Footnote 114
The development of autonomous and remote weapons systems is currently in its infancy, so Kennedy's critique based on international humanitarian standards need not apply. It is still possible at this stage to articulate rules that are directly applicable to the use of these weapons systems, as well as to delineate the boundaries of permissibility for their future development. The important questions of responsibility need neither be ignored nor neglected until a watershed catastrophe compels their consideration, and establishing the limits and the modalities to attribute responsibility now will limit the scope for future impunity. Finally, the pronouncement of applicable rules needs to be accompanied by monitoring and enforcement mechanisms. This will present a significant hurdle because states are unlikely to impose limitations upon themselves in instances where they possess, or are in the process of developing, means and methods of warfare that are likely to confer military superiority. As Theodor Meron warns: ‘The tremendous progress in the humanization of the law of war brings into sharp relief the stark contrast between promises made in treaties and declarations … on the one hand, and the harsh, often barbaric practices actually employed on the battlefield’.Footnote 115 The humanity of IHL is jeopardized not only by the emergence of autonomous and remote weapons systems, but also by the failure to recognize these as being categorically different from preceding weapons and by not recognizing their potential to drastically alter the existing legal category of means and methods of warfare.
This article has argued that contemporary IHL is insufficient to regulate some technologically advanced weapons systems, and that the current legal categorization is challenged by the emergence of autonomous weapons systems that possess autonomous capabilities. Insofar as both autonomous and remote weapons systems do not adequately fit into current categories of IHL, it may be that these systems should constitute a novel common category. Both autonomous and remote weapons systems do not fit squarely within the legal understanding of a weapon, and create subtle, yet fundamental, changes to the current legal understanding of means and methods of warfare. It may therefore be inappropriate to expand the existing categories to encompass these advanced weapons systems. Instead, it is likely that new rules will need to be developed to ensure that the potential superiority, in humanitarian terms, of these advanced weapons systems is harnessed and that concomitant responsibility for their use is firmly established to incentivize compliance and to forestall allegations of impunity. The need to establish an architecture of responsibility for the use of autonomous and remote weapons systems becomes especially acute where their use leads to allegations of international crimes.
