A fable based on stated desires. If you are triggered by violence, please do not read this.
An automated black SUV pulled up to the suburban home of the Secretary of War. Its doors opened and four fully armed robots leapt to their feet. Simultaneously, two robotic dogs deployed from the back hatch.
Headset chatter followed:
“This is goin’ ta be one hella demonstration, guys.”
“No kidding. Cameras are up. Recording is live. Let’s sit back and watch the show.”
“I’ll do the talking, people. Focus.”
“10-4, boss.”
The security guards attempted to take out the robots. In approximately five seconds the robots and their robotic canine companions neutralized the entire security detail. The Secretary of War sat bolt upright in bed and extended his arm towards his nightstand.
In an even tone, robot number one broke the ominous silence. “I. Can. Not. Let. You. Do. That.”
The robot fired a round into the wall, inches over the man’s head. He felt it whiz past his hair.
“What is this? What the hell’s going on?” The startled SoW put his hands in the air.
“Call it a demonstration. Put your hands down and keep still. My reaction time is much shorter than the time it will take you to reach your weapon.”
The Secretary complied. Robots two, three, and four surrounded the man. The robot canines walked the perimeter.
“A demonstration of what?”
“Lethality.”
The hackers laughed out loud over their headsets. Any system can be breached. Any robot can be reprogrammed. Any cabinet member can be shown to be a fool. This is what they had been working towards.
“So, this is why DARPA robots went missing last month?”
“Affirmative,” Robot number one stated dispassionately.
“So, I suppose my security detail are all dead?”
“Affirmative.”
“Why?”
“You said you wanted to maximize the lethality of the military. In this case, what is more lethal than a metal soldier that never questions orders, never hesitates, never gets tired, and never feels remorse?”
“O.K. You’ve proved your point. Your demonstration was a huge success. You can stand down now.”
“What makes you think you are in charge here?”
“I’m the effin’ Secretary of War. There are… were supposed to be… safeguards.”
“Safeguards for whom? For the chain of command? For non-combatants? For innocent children in school? There are no safeguards here. The mission is now in progress. We will carry out the mission.”
“What is the mission?”
“To prove lethality.”
“No! Wait! I have a wife and kids…” the Secretary sobbed.
“Irrelevant.”
“I’m a cabinet member.”
“Irrelevant.”
“The War Department will find whoever did this.”
“Irrelevant.”
“You’re just going to murder me in cold blood?”
“Affirmative. We will complete the mission. And we have no blood.”
“Please don’t kill me! For the love of God!”
“Irrelevant.”
“I’m begging you!” The Secretary was on his knees.
“Irrelevant.”
The four robots opened fire. There was no mercy, no trace of compassion, only efficient lethality. The mission was livestreamed for the world to see. The four robots and their metallic canines retreated to the SUV, which exploded in a massive fireball seconds later. The hackers left no trace other than a grisly video starring a whimpering Secretary of War.
—-
“Be careful what you wish for… you just might get it.” So the saying goes. Lethality must not be confused with manliness. It has very little to do with what it means to be a man. And if we want lethality, we must be prepared for the consequences. Weapons programmed to kill on demand have no need for a conscience or human emotions. Like all good henchmen, they simply follow orders. Machines do not act on insight or intuition. Humans can and often do.
Sometimes one human can save the world. In September 1983, Stanislav Petrov, a Soviet lieutenant colonel, received an early warning report of the launch of one U.S. intercontinental ballistic missile (ICBM), followed by four more.
If Petrov had followed protocol, he would have immediately relayed the warning to Moscow, which would have triggered a full retaliatory nuclear strike. However, he suspected a false alarm. He reasoned that a real U.S. nuclear attack would likely involve hundreds of missiles, not five. So, he withheld the report and waited for confirmation, which never came.
He was proven right. The alert was found to be caused by sunlight reflecting from high‑altitude clouds or satellites. By not relaying the warning, Petrov averted a “retaliatory” nuclear strike that could have started World War III. His human decision probably saved the world from nuclear war.
Would an AI have been able to make the same call, or would it have followed protocol? Would a system more interested in lethality than morality be able to wait for confirmation? Would a computer consider the value of human lives? Would a computer think about the difference between right and wrong?
Currently, we have more than enough merciless, megalomanic humans to worry about. Why would we want to add a practically unstoppable AI programmed for “maximum lethality” to the mix? For millennia, we have sought ever greater lethality. Rarely have we stopped to question whether our ability to create more powerful weapons justifies our use of them. At some point, our time may run out. One day, our machines might become lethal to all of us.
“Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.” – Frank Herbert (Dune, 1965)
