Killer Robots
Robert Sparrow, School of Philosophy and Bioethics, Faculty of Arts, Monash University, Victoria 3800, Australia. E-mail address: Robert.Sparrow@arts.monash.edu.au
Abstract
abstract The United States Army's Future Combat Systems Project, which aims to manufacture a ‘robot army’ to be ready for deployment by 2012, is only the latest and most dramatic example of military interest in the use of artificially intelligent systems in modern warfare. This paper considers the ethics of the decision to send artificially intelligent robots into war, by asking who we should hold responsible when an autonomous weapon system is involved in an atrocity of the sort that would normally be described as a war crime. A number of possible loci of responsibility for robot war crimes are canvassed: the persons who designed or programmed the system, the commanding officer who ordered its use, the machine itself. I argue that in fact none of these are ultimately satisfactory. Yet it is a necessary condition for fighting a just war, under the principle of jus in bellum, that someone can be justly held responsible for deaths that occur in the course of the war. As this condition cannot be met in relation to deaths caused by an autonomous weapon system it would therefore be unethical to deploy such systems in warfare.
Citing Literature
Number of times cited according to CrossRef: 187
- Lambèr Royakkers, Sjef Orbons, Design for Values in the Armed Forces: Nonlethal Weapons and Military Robots, Handbook of Ethics, Values, and Technological Design, 10.1007/978-94-007-6994-6, (1-22), (2021).
- John Forge, Erik Cuevas, Fernando Fausto, Adrián González, Weapons Research is Morally Wrong, New Advancements in Swarm Algorithms: Operators and Applications, 10.1007/978-3-030-16860-5_3, (29-46), (2020).
- Craig Webster, Stanislav Ivanov, Robotics, Artificial Intelligence, and the Evolving Nature of Work, Digital Transformation in Business and Society, 10.1007/978-3-030-08277-2, (127-143), (2020).
- Joshua G. Hughes, The Law of Armed Conflict Issues Created by Programming Automatic Target Recognition Systems Using Deep Learning Methods, Yearbook of International Humanitarian Law, Volume 21 (2018), 10.1007/978-94-6265-343-6_4, (99-135), (2020).
- Matthew Breay Bolton, Cayman C. Mitchell, When Scientists Become Activists: The International Committee for Robot Arms Control and the Politics of Killer Robots, Global Activism and Humanitarian Disarmament, 10.1007/978-3-030-27611-9, (27-58), (2020).
- Minao Kukita, Robot Ethicsロボットの倫理, Journal of the Robotics Society of Japan, 10.7210/jrsj.38.18, 38, 1, (18-22), (2020).
- Michael Kühler, Technological Moral Luck, Technology, Anthropology, and Dimensions of Responsibility, 10.1007/978-3-476-04896-7_9, (115-132), (2020).
- Thomas Grote, Ezio Di Nucci, Algorithmic Decision-Making and the Problem of Control, Technology, Anthropology, and Dimensions of Responsibility, 10.1007/978-3-476-04896-7_8, (97-113), (2020).
- Jan-Hendrik Heinrichs, Artificial Intelligence in Extended Minds: Intrapersonal Diffusion of Responsibility and Legal Multiple Personality, Technology, Anthropology, and Dimensions of Responsibility, 10.1007/978-3-476-04896-7_12, (159-176), (2020).
- Michael Nagenborg, Can We Forgive a Robot?, Technology, Anthropology, and Dimensions of Responsibility, 10.1007/978-3-476-04896-7_11, (147-157), (2020).
- Nir Douer, Joachim Meyer, The Responsibility Quantification Model of Human Interaction With Automation, IEEE Transactions on Automation Science and Engineering, 10.1109/TASE.2020.2965466, 17, 2, (1044-1060), (2020).
- Catrin Misselhorn, Maschinenethik und Philosophie, Handbuch Maschinenethik, 10.1007/978-3-658-17484-2_3-2, (1-23), (2020).
- Russell Belk, Mariam Humayun, Ahir Gopaldas, Artificial Life, Journal of Macromarketing, 10.1177/0276146719897361, (027614671989736), (2020).
- Changrok Soh, Daniel Connolly, New Frontiers of Profit and Risk: The Fourth Industrial Revolution’s Impact on Business and Human Rights, New Political Economy, 10.1080/13563467.2020.1723514, (1-18), (2020).
- Michael Skerker, Duncan Purves, Ryan Jenkins, Autonomous weapons systems and the moral equality of combatants, Ethics and Information Technology, 10.1007/s10676-020-09528-0, (2020).
- Daniel W. Tigard, Niël H. Conradie, Saskia K. Nagel, Socially responsive technologies: toward a co-developmental path, AI & SOCIETY, 10.1007/s00146-020-00982-4, (2020).
- Dorna Behdadi, Christian Munthe, A Normative Approach to Artificial Moral Agency, Minds and Machines, 10.1007/s11023-020-09525-8, (2020).
- Sadjad Soltanzadeh, Jai Galliott, Natalia Jevglevskaja, Customizable Ethics Settings for Building Resilience and Narrowing the Responsibility Gap: Case Studies in the Socio-Ethical Engineering of Autonomous Systems, Science and Engineering Ethics, 10.1007/s11948-020-00221-5, (2020).
- Fabio Tollon, The artificial view: toward a non-anthropocentric account of moral patiency, Ethics and Information Technology, 10.1007/s10676-020-09540-4, (2020).
- Leonie Seng, Mein Haus, mein Auto, mein Roboter?, Maschinenethik, 10.1007/978-3-658-21083-0_4, (57-72), (2019).
- Oliver Kirchkamp, Christina Strobel, Sharing Responsibility with a Machine, Journal of Behavioral and Experimental Economics, 10.1016/j.socec.2019.02.010, (2019).
- Raul Hakli, Pekka Mäkelä, Moral Responsibility of Robots and Hybrid Agents, The Monist, 10.1093/monist/onz009, 102, 2, (259-275), (2019).
- Catrin Misselhorn, Autonome Waffensysteme/Kriegsroboter, Handbuch Filmtheorie, 10.1007/978-3-658-17484-2_20-1, (1-16), (2019).
- Catrin Misselhorn, Maschinenethik und Philosophie, Handbuch Filmtheorie, 10.1007/978-3-658-17484-2_3-1, (1-23), (2019).
- Andrea Spagnolo, What Do Human Rights Really Say About the Use of Autonomous Weapons Systems for Law Enforcement Purposes?, Use and Misuse of New Technologies, 10.1007/978-3-030-05648-3, (55-72), (2019).
- Daniele Amoroso, Benedetta Giordano, Who Is to Blame for Autonomous Weapons Systems’ Misdoings?, Use and Misuse of New Technologies, 10.1007/978-3-030-05648-3, (211-232), (2019).
- Bertram F. Malle, Stuti Thapa Magar, Matthias Scheutz, AI in the Sky: How People Morally Evaluate Human and Machine Decisions in a Lethal Strike Dilemma, Robotics and Well-Being, 10.1007/978-3-030-12524-0_11, (111-133), (2019).
- Michael Decker, Autonome Systeme und ethische Reflexion, Die Maschine: Freund oder Feind?, 10.1007/978-3-658-22954-2, (135-158), (2019).
- Tassilo Singer, Tassilo Singer, Kapitel XIV Völkerstrafrecht – Individuelle Verantwortlichkeit, Dehumanisierung der Kriegführung, 10.1007/978-3-662-57856-8, (501-529), (2019).
- Catrin Misselhorn, Maschinenethik und Philosophie, Handbuch Maschinenethik, 10.1007/978-3-658-17483-5, (33-55), (2019).
- Karsten Weber, Thomas Zoglauer, Maschinenethik und Technikethik, Handbuch Maschinenethik, 10.1007/978-3-658-17483-5, (145-163), (2019).
- Bertram F. Malle, Matthias Scheutz, Learning How to Behave, Handbuch Maschinenethik, 10.1007/978-3-658-17483-5, (255-278), (2019).
- Catrin Misselhorn, Autonome Waffensysteme/Kriegsroboter, Handbuch Maschinenethik, 10.1007/978-3-658-17483-5, (319-334), (2019).
- Ilse Verdiesen, Filippo Santoni de Sio, Virginia Dignum, Moral Values Related to Autonomous Weapon Systems: An Empirical Survey that Reveals Common Ground for the Ethical Debate, IEEE Technology and Society Magazine, 10.1109/MTS.2019.2948439, 38, 4, (34-44), (2019).
- Michael Decker, Von den three laws of robotics zur Roboterethik, Mensch-Maschine-Interaktion, 10.1007/978-3-476-05604-7, (345-351), (2019).
- Simon Burton, Ibrahim Habli, Tom Lawton, John McDermid, Phillip Morgan, Zoe Porter, Mind the Gaps: Assuring the Safety of Autonomous Systems from an Engineering, Ethical, and Legal Perspective, Artificial Intelligence, 10.1016/j.artint.2019.103201, (103201), (2019).
- George Lucas, The deadly diffusion of lethal technologies, Science, 10.1126/science.aaz4545, 366, 6467, (810-810), (2019).
- Ezio Di Nucci, Should we be afraid of medical AI?, Journal of Medical Ethics, 10.1136/medethics-2018-105281, 45, 8, (556-558), (2019).
- Lambèr Royakkers, Peter Olsthoorn, Lethal Military Robots, Unmanned Aerial Vehicles, 10.4018/978-1-5225-8365-3, (394-411), (2019).
- Jai Galliott, Tim McFarland, Autonomous Systems in a Military Context (Part 2), Unmanned Aerial Vehicles, 10.4018/978-1-5225-8365-3, (433-451), (2019).
- Klaus Kornwachs, Smart Robots – Smart Ethics?, Datenschutz und Datensicherheit - DuD, 10.1007/s11623-019-1118-2, 43, 6, (332-341), (2019).
- Robert Sparrow, Rob McLaughlin, Mark Howard, Naval robots and rescue, International Review of the Red Cross, 10.1017/S181638311800067X, (1-21), (2019).
- Jilles Smids, Sven Nyholm, Hannah Berkers, Robots in the Workplace: a Threat to—or Opportunity for—Meaningful Work?, Philosophy & Technology, 10.1007/s13347-019-00377-4, (2019).
- Sven Nyholm, Jilles Smids, Can a Robot Be a Good Colleague?, Science and Engineering Ethics, 10.1007/s11948-019-00172-6, (2019).
- Mark Coeckelbergh, Artificial Intelligence, Responsibility Attribution, and a Relational Justification of Explainability, Science and Engineering Ethics, 10.1007/s11948-019-00146-8, (2019).
- Roos de Jong, The Retribution-Gap and Responsibility-Loci Related to Robots and Automated Technologies: A Reply to Nyholm, Science and Engineering Ethics, 10.1007/s11948-019-00120-4, (2019).
- Manuel Dietrich, Thomas H. Weisswange, Distributive justice as an ethical principle for autonomous vehicle behavior beyond hazard scenarios, Ethics and Information Technology, 10.1007/s10676-019-09504-3, (2019).
- Hendrik Huelss, Norms Are What Machines Make of Them: Autonomous Weapons Systems and the Normative Implications of Human-Machine Interactions, International Political Sociology, 10.1093/ips/olz023, (2019).
- Giulio Mecacci, Filippo Santoni de Sio, Meaningful human control as reason-responsiveness: the case of dual-mode vehicles, Ethics and Information Technology, 10.1007/s10676-019-09519-w, (2019).
- Johannes Himmelreich, Responsibility for Killer Robots, Ethical Theory and Moral Practice, 10.1007/s10677-019-10007-9, (2019).
- Steven R. Kraaijeveld, Debunking (the) Retribution (Gap), Science and Engineering Ethics, 10.1007/s11948-019-00148-6, (2019).
- Ryan Kiggins, Robots and Political Economy, The Political Economy of Robots, 10.1007/978-3-319-51466-6, (1-16), (2018).
- William Boothby, Dehumanization: Is There a Legal Problem Under Article 36?, Dehumanization of Warfare, 10.1007/978-3-319-67266-3, (21-52), (2018).
- Bertram F. Malle, Matthias Scheutz, Learning How to Behave, Handbuch Maschinenethik, 10.1007/978-3-658-17484-2_17-1, (1-24), (2018).
- Karsten Weber, Thomas Zoglauer, Maschinenethik und Technikethik, Handbuch Maschinenethik, 10.1007/978-3-658-17484-2, (1-19), (2018).
- Heather M. Roff, David Danks, “Trust but Verify”: The Difficulty of Trusting Autonomous Weapons Systems, Journal of Military Ethics, 10.1080/15027570.2018.1481907, 17, 1, (2-20), (2018).
- Seumas Miller, Seumas Miller, Cyber-Technology, Dual Use Science and Technology, Ethics and Weapons of Mass Destruction, 10.1007/978-3-319-92606-3_7, (91-104), (2018).
- Aiden Warren, Globalization, Security, and Drones, Global Encyclopedia of Public Administration, Public Policy, and Governance, 10.1007/978-3-319-20928-9, (2732-2740), (2018).
- Torben Swoboda, Autonomous Weapon Systems - An Alleged Responsibility Gap, Philosophy and Theory of Artificial Intelligence 2017, 10.1007/978-3-319-96448-5_32, (302-313), (2018).
- Ariel Guersenzvaig, Autonomous Weapon Systems: Failing the Principle of Discrimination, IEEE Technology and Society Magazine, 10.1109/MTS.2018.2795119, 37, 1, (55-61), (2018).
- Sven Nyholm, The ethics of crashes with self‐driving cars: A roadmap, II, Philosophy Compass, 10.1111/phc3.12506, 13, 7, (2018).
- Amitai Etzioni, Amitai Etzioni, Pros and Cons of Autonomous Weapons Systems (with Oren Etzioni), Happiness is the Wrong Metric, 10.1007/978-3-319-69623-2_16, (253-263), (2018).
- Lambèr Royakkers, Peter Olsthoorn, Lethal Military Robots, The Changing Scope of Technoethics in Contemporary Society, 10.4018/978-1-5225-5094-5.ch006, (106-123), (2018).
- Marcus Schulzke, Drone Proliferation and the Challenge of Regulating Dual-Use Technologies, International Studies Review, 10.1093/isr/viy047, (2018).
- Filippo Santoni de Sio, Jeroen van den Hoven, Meaningful Human Control over Autonomous Systems: A Philosophical Account, Frontiers in Robotics and AI, 10.3389/frobt.2018.00015, 5, (2018).
- Patrick Taylor Smith, Just research into killer robots, Ethics and Information Technology, 10.1007/s10676-018-9472-6, (2018).
- Sven Nyholm, Attributing Agency to Automated Systems: Reflections on Human–Robot Collaborations and Responsibility-Loci, Science and Engineering Ethics, 10.1007/s11948-017-9943-x, 24, 4, (1201-1219), (2017).
- John Danaher, Nehal Bhuta, Susanne Beck, Robin Geiß, Hin-Yan Liu and Claus Kreß (eds). Autonomous Weapons Systems: Law, Ethics, Policy, Ethical Theory and Moral Practice, 10.1007/s10677-017-9830-1, 20, 4, (931-933), (2017).
- Maaike Harbers, Joachim de Greeff, Ivana Kruijff-Korbayová, Mark A. Neerincx, Koen V. Hindriks, Exploring the Ethical Landscape of Robot-Assisted Search and Rescue, A World with Robots, 10.1007/978-3-319-46667-5_7, (93-107), (2017).
- Sean Welsh, Clarifying the Language of Lethal Autonomy in Military Robots, A World with Robots, 10.1007/978-3-319-46667-5_13, (171-183), (2017).
- Filippo Santoni de Sio, Killing by Autonomous Vehicles and the Legal Doctrine of Necessity, Ethical Theory and Moral Practice, 10.1007/s10677-017-9780-7, 20, 2, (411-429), (2017).
- David Danks, Alex John London, Regulating Autonomous Systems: Beyond Standards, IEEE Intelligent Systems, 10.1109/MIS.2017.1, 32, 1, (88-91), (2017).
- Aiden Warren, Globalization, Security and Drones, Global Encyclopedia of Public Administration, Public Policy, and Governance, 10.1007/978-3-319-31816-5, (1-9), (2017).
- Bartosz Brożek, Marek Jakubiec, On the legal responsibility of autonomous machines, Artificial Intelligence and Law, 10.1007/s10506-017-9207-8, 25, 3, (293-304), (2017).
- Jürgen Altmann, Zur ethischen Beurteilung automatisierter und autonomer Waffensysteme, Handbuch Friedensethik, 10.1007/978-3-658-14686-3, (793-804), (2017).
- Niklas Schörnig, Rüstung, Rüstungskontrolle und internationale Politik, Handbuch Internationale Beziehungen, 10.1007/978-3-531-19918-4, (959-990), (2017).
- Stephanie Carvin, Normal Autonomous Accidents: What Happens When Killer Robots Fail?, SSRN Electronic Journal, 10.2139/ssrn.3161446, (2017).
- Markus Christen, Thomas Burri, Joseph O. Chapa, Raphael Salvi, Filippo Santoni de Sio, John Sullins, An Evaluation Schema for the Ethical Use of Autonomous Robotic Systems in Security Applications, SSRN Electronic Journal, 10.2139/ssrn.3063617, (2017).
- Jakub Złotowski, Kumar Yogeeswaran, Christoph Bartneck, Can we control it? Autonomous robots threaten human identity, uniqueness, safety, and resources, International Journal of Human-Computer Studies, 10.1016/j.ijhcs.2016.12.008, 100, (48-54), (2017).
- David J. Gunkel, Mind the gap: responsible robotics and the problem of responsibility, Ethics and Information Technology, 10.1007/s10676-017-9428-2, (2017).
- Hin-Yan Liu, Karolina Zawieska, From responsible robotics towards a human rights regime oriented to the challenges of robotics and artificial intelligence, Ethics and Information Technology, 10.1007/s10676-017-9443-3, (2017).
- J. Marshall Beier, Short circuit: retracing the political for the age of ‘autonomous’ weapons, Critical Military Studies, 10.1080/23337486.2017.1384978, (1-18), (2017).
- THOMAS SINCLAIR, Preventing Optimific Wrongings, Utilitas, 10.1017/S0953820816000388, 29, 4, (453-473), (2016).
- Migle Laukyte, Artificial agents among us: Should we recognize them as agents proper?, Ethics and Information Technology, 10.1007/s10676-016-9411-3, 19, 1, (1-17), (2016).
- Ryan Jenkins, Duncan Purves, Robots and Respect: A Response to Robert Sparrow, Ethics & International Affairs, 10.1017/S0892679416000277, 30, 03, (391), (2016).
- Ben Tran, Ethical Concerns in Human Enhancement, Leadership and Personnel Management, 10.4018/978-1-4666-9624-2, (119-144), (2016).
- Hartwig Steusloff, Michael Decker, Human Action in the Loop, Effective Standardization Management in Corporate Settings, 10.4018/978-1-4666-9737-9.ch016, (352-368), (2016).
- John Danaher, Robots, law and the retribution gap, Ethics and Information Technology, 10.1007/s10676-016-9403-3, 18, 4, (299-309), (2016).
- Robert Sparrow, Robots and Respect: Assessing the Case Against Autonomous Weapon Systems, Ethics & International Affairs, 10.1017/S0892679415000647, 30, 1, (93-116), (2016).
- Mark Coeckelbergh, Drones, Morality, and Vulnerability: Two Arguments Against Automated Killing, The Future of Drone Use, 10.1007/978-94-6265-132-6_12, (229-237), (2016).
- Marcus Schulzke, Rethinking Military Virtue Ethics in an Age of Unmanned Weapons, Journal of Military Ethics, 10.1080/15027570.2016.1257851, 15, 3, (187-204), (2016).
- Robert Sparrow, Robots as “Evil Means”? A Rejoinder to Jenkins and Purves, Ethics & International Affairs, 10.1017/S0892679416000289, 30, 3, (401-403), (2016).
- Bertram F. Malle, Matthias Scheutz, Jodi Forlizzi, John Voiklis, undefined, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 10.1109/HRI.2016.7451743, (125-132), (2016).
- Spyros G. Tzafestas, Spyros G. Tzafestas, War Roboethics, Roboethics, 10.1007/978-3-319-21714-7_9, (139-153), (2016).
- Brandt Dainow, Key dialectics in cloud services, ACM SIGCAS Computers and Society, 10.1145/2874239.2874247, 45, 3, (52-59), (2016).
- Thomas W. Simpson, Vincent C. Müller, Just War and Robots’ Killings, The Philosophical Quarterly, 10.1093/pq/pqv075, 66, 263, (302-322), (2015).
- Hironori Matsuzaki, Gesa Lindemann, The autonomy-safety-paradox of service robotics in Europe and Japan: a comparative analysis, AI & SOCIETY, 10.1007/s00146-015-0630-7, 31, 4, (501-517), (2015).
- Duncan Purves, Ryan Jenkins, Bradley J. Strawser, Autonomous Machines, Moral Judgment, and Acting for the Right Reasons, Ethical Theory and Moral Practice, 10.1007/s10677-015-9563-y, 18, 4, (851-872), (2015).
- Tjerk de Greef, Alex Leveringhaus, Design for responsibility: safeguarding moral perception via a partnership architecture, Cognition, Technology & Work, 10.1007/s10111-015-0329-z, 17, 3, (319-328), (2015).
- James Igoe Walsh, Political accountability and autonomous weapons, Research & Politics, 10.1177/2053168015606749, 2, 4, (205316801560674), (2015).
