{"slug": "the-human-is-becoming-latency", "title": "The Human Is Becoming Latency", "summary": "Representatives from 128 countries meeting in Geneva last week agreed on a non-binding document that could eventually form the basis for international rules governing lethal autonomous weapons, following more than a decade of negotiations. The definition of \"autonomous weapon\" was reportedly weakened during the final stages of the talks, with the United States and Russia favoring national guidelines over legally binding international restrictions. The agreement leaves unresolved how much human control must remain in targeting decisions as militaries and defense companies field systems that operate far faster than human operators.", "body_md": "[Artificial Intelligence](https://www.gonzocapital.net/tag/ai/)\n\n# The Human Is Becoming Latency\n\nHumanity would very much like to keep a human in charge of the killer robots. Unfortunately, humans are increasingly becoming the slowest part of the system.\n\n*The world wants humans to remain in control of autonomous weapons. Modern warfare may reward everyone who removes them.*\n\nHumanity finally reached a glorious agreement on dreaded killer robots last week.\n\nIn my own backyard over in Geneva!\n\nI think this maybe, just maybe, qualifies as significant news. Don’t know, though; could just be all the Sci-Fi I read…\n\nAfter more than a decade of negotiations, 128 countries meeting in Switzerland last week agreed on a document that could eventually, possibly, form the basis for international rules governing “lethal autonomous weapons.”\n\nBroadly speaking, these are weapons capable of selecting and engaging targets with varying degrees of autonomy once activated, although getting governments to agree on exactly where that definition begins and ends has proven to be part of the problem.\n\nThe text is unsurprisingly non-binding, fairly typical of bureaucrats who have a knack for emulating snails.\n\nSadly, the definition of [__‘autonomous weapon’ was reportedly weakened__](https://www.reuters.com/world/states-reach-agreement-autonomous-weapons-talks-geneva-2026-09-05/?ref=gonzocapital.net) during the final stages of those negotiations.\n\nStrange bedfellows emerged, with the United States and Russia continuing to favor national guidelines over legally binding international restrictions.\n\nSo after ten whole years of impressively slow deliberation, humanity has finally managed to agree that perhaps we should maybe take the bold step of continuing discussions about whether or not machines should be allowed to decide who dies.\n\n‘Who’ refers to our fellow humans if that wasn’t clear enough.\n\nIn fairness, this is how diplomacy is known to work. Or not work?\n\nNuclear arms accords weren’t merely decided over a quick and pain-free lunch either.\n\nBut there is something almost poetic about our beloved diplomat friends debating the precise meaning of \"human control\" while defense companies and militaries are busy building super weapons designed to operate an order of magnitude faster than humans can.\n\nEveryone seems quite comfortable with allowing a human to remain somewhere in the process of taking a life. Or many.\n\nSo that means we’re allowed to have artificial intelligence. We can still make use of our autonomous drones.\n\nWe can automate targeting, surveillance, navigation, and threat assessment, and most of the decision-making too.\n\nBut somewhere, somehow, there should still be a human exercising judgment.\n\nThe trustworthy adult in the room.\n\nThe problem is that the pesky human grown-up is slow. Which presents a problem. And modern warfare has famously never been very sentimental about bottlenecks.\n\n## **PLEASE CONFIRM YOU WOULD LIKE TO KILL THIS PERSON**\n\nImagine for a moment that a drone spots an SUV moving through a ‘contested’ area.\n\nSoftware identifies its shape and cross-references it with a database replete with other known military vehicle shapes and sizes.\n\nOther sensors analyze its movement, heat signature, and location. A model assesses the probability that it belongs to the enemy.\n\nIn the background, another system layer checks the surrounding area for civilians. Yet another determines which instrument of death is most suitable and immediately available.\n\nBy the time a soldier sees anything, the machine has already managed to convert a messy piece of reality into something a whole lot easier to process.\n\n**HOSTILE TARGET: 94% CONFIDENCE (shouldn’t it be 100%?)**\n\n**CIVILIAN RISK: LOW (whatever this means)**\n\n**ENGAGEMENT RECOMMENDED (by whom?!)**\n\n**APPROVE?**\n\nThe soldier clicks yes.\n\n*Technically*, there was a human in the loop. Just like Italian luxury fashion brands technically “assemble” their accessories in Italy (after having them shipped from Asia), or American car companies “assemble” their cars domestically and definitely not in Mexico. \n\nBut I am not sure I would describe that human as the person who *made* the decision.\n\nThe more of the process we automate, the stranger that distinction tends to become. And these cutting-edge technologies are *really* starting to make it easy to do something unforgivable with the unconscious click of a button.\n\nA person with a meager several seconds to approve an analysis assembled by systems they cannot possibly reproduce in their own head may have legal authority over the decision without possessing much practical agency over it.\n\nGive them fifty targets instead of one and the problem gets worse. Maybe make the required response time half a second?\n\nOr give them five hundred targets, and now we’re totally raining down chaos.\n\nEventually, \"human control\" starts looking a hell of a lot like one of those useless boxes confirming that you have indeed read the 345,000 terms and conditions for your iPhone update.\n\nYou haven't, and everyone, including Apple, knows you haven't.\n\nBut legally, congratulations, you’ve participated. Here’s your participation ribbon!\n\nThere is actually a name for part of this problem: **automation bias**. Humans have a tendency to place too much faith in recommendations generated by computers simply because a computer generated them. \n\nIn military decision-support systems, that creates the slightly horrifying possibility that the human supposedly exercising judgment becomes little more than a rubber stamp for whatever the machine recommends.\n\nWhich seems less than ideal when the rubber stamp launches a missile.\n\nThis is the fork in the road where the Geneva debate starts becoming much more interesting than the phrase \"killer robots\" suggests. And I think we can all agree that “killer robots” already sounds pretty interesting. Proven by the millions of Hollywood movies about killer robots and Terminators.\n\nThe reassuring version of human control likes to imagine a soldier looking through a camera, assessing a target, and in turn using their combined experience and wisdom to make an informed and conscious decision to fire.\n\nIn reality, relevant human judgment could and probably should occur much earlier.\n\nPeople can determine geographic boundaries, target categories, operating conditions, and rules of engagement governing when autonomous systems are permitted to act.\n\nThe human doesn't necessarily disappear but they do take a tremendous step into the background.\n\nFirst a person chooses the target >>> They approve a target selected by a machine >>> They approve a category or cluster of targets >>> They approve an entire area where a machine can attack that category of target.\n\nThis slippery slope eventually leads to a person designing or authorizing the rules governing the system that decides what does or doesn’t count as a target in the first place.\n\nBut there is still a human in the loop, despite them being further and further away from the corpse.\n\n## THE GENEVA CHECKBOX: WHAT DOES HUMAN CONTROL ACTUALLY MEAN?\n\nThis could happen without anyone ever formally deciding to surrender control to machines.\n\nImagine the progression.\n\nIn the first generation, an AI identifies a hostile vehicle and recommends a strike, and then a soldier approves it.\n\nThe next iteration, the system identifies ten targets simultaneously. The soldier reviews them quickly and approves the whole package.\n\nThen fifty. A hundred. A thousand. Ten thousand. Keep going.\n\nIt ain’t long before an autonomous swarm detects an incoming attack and calculates that it has 0.7 seconds to respond before losing part of the network.\n\nAt which point the system politely asking Gary for permission begins to seem less like a critical ethical safeguard feature and more like an inconvenient software bug.\n\nAnd this isn’t just some meaningless thought experiment…\n\nA [__U.S. Army Military Review article published this summer__](https://www.armyupress.army.mil/Journals/Military-Review/Online-Exclusive/2026-OLE/Algorithm-or-Order/?ref=gonzocapital.net) describes a growing **\"velocity gap\"** between machine-speed sensing and human-paced targeting procedures. AI systems can collect, combine, and analyze battlefield information at mind-blowing speeds we can’t comprehend. But action still proceeds according to human validation and approval. For now.\n\nMajor Brodka argues that potential adversaries are actively developing systems and doctrine intended to exploit that gap.\n\nThat phrase, *velocity gap*, is doing an extraordinary amount of legwork there.\n\nBecause once speed becomes an advantage, and by all accounts it already is, every country eventually faces a conundrum over whether it wants autonomous killing or not.\n\nImagine two different militaries.\n\nOne requires individual human approval before lethal action.\n\nThe other permits a machine to respond automatically once predetermined conditions are satisfied.\n\nThe second military gets faster, more tactically efficient, and gains a huge upper hand.\n\nThe first responds by automating slightly more of its own process.\n\nThen the second gets faster again. Ad infinitum.\n\nThis frantic spiral is basically a non-cooperative game: each side automates because the cost of being slower rises as the other side automates. Nobody has to want that equilibrium. Competitive pressure does the rest.\n\nModern military game theory defines this as an __optimization trap requiring constant automation for survival.__\n\nIn this strange series of events, nobody ever had to become a fanatical, overzealous advocate for robot murder. They only need to suspect that the other side has shaved another second or two off its murder decision loop.\n\nEventually morality encounters latency.\n\n## **WE HAVE SEEN THIS MOVIE BEFORE, WITH FEWER EXPLOSIONS**\n\nFinancial markets offer a strange but ultimately useful preview.\n\nLet’s take trading for example.\n\nHigh-frequency trading did not emerge because society held a grand philosophical debate and decided financial markets would function better if humans were replaced by computers fighting over fractions of mere microseconds.\n\nThere was an obvious advantage to being faster than the rest. Makes total sense, right?\n\nSooner or later, somebody exploited it, which in turn forced someone else to respond.\n\nIt wasn’t long before trading firm behemoths were spending small fortunes locating and utilizing computers physically closer in proximity to financial exchanges because tiny differences in transmission time mattered. Every tick on the clock presented an opportunity to grab an edge.\n\nNow this isn’t to suggest that human traders vanished from the system all of a sudden because intelligence started taking a backseat to speed.\n\nThey did, however, disappear after the realization arrived that intelligence taking several seconds had become far too slow to permit.\n\nNow turn around and make the stakes literal life-or-death situations. Way more on the line.\n\nBetween those systems IDing an object, classifying it, predicting behavior, calculating a response, and ultimately executing… There is no technological reason why that sequence needs to slow down or get less efficient. The need for it to happen at a speed compatible with human reflection is no longer there.\n\nQuite the opposite, in fact.\n\nIf autonomous weapons develop the way most technology develops, speed will become a feature. Not just any feature but the paramount feature.\n\nSomeone will *always* advertise lower latency. That’s how capitalism tends to work.\n\nSome other defense company will demonstrate that its swarm reacts 40 percent faster than the competition.\n\nSome overworked defense contractor will eventually have to create a PowerPoint slide titled **TIME-TO-KILL OPTIMIZATION** and everyone in the room will nod thoughtfully because that is their job.\n\nSuddenly we’re talking about warfare leaving humans behind altogether.\n\nThis is what worries me significantly more than the Hollywood version where an artificial superintelligence suddenly develops consciousness and decides humanity has had a good run while it lasted.\n\nThe biggest fallacy is that machines need hatred or consciousness or a villainous backstory to present a clear danger to us squishy, fleshy humanoids.\n\nThey don't even need much autonomy at first.\n\nThey simply need to keep getting faster, and that’ll be enough to surpass us and cut us out of the sequence.\n\n## **UNFORTUNATELY, THE ROBOTS HAVE A POINT**\n\nThis would all be easier if humans were skilled at war.\n\nEven after tens of thousands of years honing our craft, we simply aren’t.\n\nHumans are prone to panic. We get tired. We misidentify things constantly. We seek revenge in intergenerational conflicts. We “just follow orders.”\n\nWe misunderstand good ones. We rotate through a symphony of emotions, getting scared and angry and confused, occasionally all at the same time.\n\nAs far as I know, a machine does not yet become enraged because its friend was killed yesterday. It does not really need to sleep.\n\nIt can even theoretically evaluate several sensors simultaneously and apply the same targeting rules every time.\n\nUnder sufficiently constrained conditions and with the right training, an autonomous system might become much better than a person at identifying a legitimate military target. It could already be the case.\n\nThis creates a problem I don't think the simple \"always keep a human in the loop\" argument handles particularly well.\n\nSuppose an autonomous defensive system identifies incoming missiles correctly 99.9 percent of the time.\n\nSuppose humans under the same conditions manage 97 percent.\n\nDo we deliberately insert a worse, less efficient, less capable decision-maker because the act feels morally cleaner when a person clicks the confirm button?\n\nNow make it more complex. More morally ambiguous.\n\nImagine the machine is demonstrably better than a human soldier at distinguishing military vehicles from civilian ones.\n\nWould forcing a soldier to make the final decision actually protect civilians?\n\nOr would we be sacrificing real actual human beings to preserve a philosophical idea about human agency?\n\nI very clearly don't know where that line should be, and I am incredibly suspicious of anyone who claims they do.\n\nThe [__U.S. position has long emphasized that autonomy can, under some circumstances, improve precision and discrimination__](https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodd/300009p.PDF?ref=gonzocapital.net) in the use of force, while Defense Department policy requires systems to allow commanders and operators to exercise appropriate levels of human judgment.\n\nNeither this nor the opposite is an absurd position given what we currently know.\n\nThe UN Secretary-General and the president of the International Committee of the Red Cross [__warned on August 25__](https://www.ungeneva.org/en/news-media/press-release/2026/08/renewed-call-united-nations-secretary-general-and-president?ref=gonzocapital.net) that humanity is approaching a \"moral red line\" in allowing machines to autonomously target human beings. They both called for legally binding rules and restrictions on these weapons systems.\n\nBoth sides are really arguing about the same uncomfortable question underneath it all…\n\nIf a machine becomes better at making certain lethal decisions than we are, what exactly are we protecting by insisting a human make them?\n\nHuman judgment? Dignity? Accountability? Like humanity is some upstanding arbiter of morality without blemish?\n\nOr our own psychological need to believe that somewhere behind every act of organized violence there remains a person consciously choosing to do it?\n\nWhy are we compelled to think that way? I don't have a clean answer. And somehow that bothers me more than having an answer.\n\n## **THE MOST IMPORTANT LATENCY IN HISTORY**\n\nThere is, however, one fairly compelling historical argument for keeping the unreliable hairless ape somewhere in the kill system.\n\nHis name was Stanislav Petrov.\n\nOn September 26, 1983, Petrov was the Soviet lieutenant colonel on duty when an early-warning system informed him that the United States had launched a nuclear missile toward the Soviet Union.\n\nThen more appeared. The system was reporting five incoming American missiles headed right for the motherland.\n\nThis was not a particularly chill moment in human history, despite occurring in the *Cold War*.\n\nRelations between the Soviet Union and the United States were atrocious. Only a few weeks earlier, the [__Soviets had shot down Korean Air Lines Flight 007__](https://adst.org/2014/03/the-downing-of-kal-flight-007/?ref=gonzocapital.net), killing 269 people. Both countries had enormous nuclear arsenals pointed at each other and elaborate systems designed to make sure that if one civilization ended, the other one would have a fairly bad afternoon too.\n\nPetrov was ordered to report any warning up the chain of command.\n\nHis machine and information were all pointing to the conclusion that America had finally made the first move to launch a nuclear attack and eradicate its bitter foe.\n\nPetrov didn't believe it. The whole thing felt off.\n\nFive missiles made little sense as the opening salvo of an American first strike. Ground radar had not confirmed what the satellites were reporting. Petrov decided the computer was probably wrong and waited it out.\n\nHe was right, which is potentially why we’re still here today and not fighting for scraps in some dusty, irradiated, apocalyptic wasteland.\n\nThe Soviet satellite system had [__mistaken sunlight reflecting off high-altitude clouds__](https://migflug.com/afterburner/stanislav-petrov-1983-nuclear-false-alarm/?ref=gonzocapital.net) for American missile launches.\n\nThe machine looked at the sky and concluded the sky was falling.\n\nIt took a human looking at the machine and thinking, essentially:\n\n“Are you sure?”\n\nThat hesitation may have been one of the most valuable, if not the most valuable, pieces of human latency in history.\n\nPetrov never particularly liked being described as the man who saved the world. He insisted that he had simply been doing his job.\n\nBut years later he said something that, in the context of autonomous weapons, feels considerably more important.\n\n\"They were lucky it was me on shift that night.\"\n\nWhich is a pretty haunting sentence the more you think about it, especially in the context of what we’ve already been discussing.\n\nBecause Petrov represented almost everything a highly optimized automated military system is designed to eliminate.\n\nHe introduced a delay.\n\nHe distrusted the data given to him.\n\nHe relied partly on intuition.\n\nHe deviated from what the system was practically imploring him to do.\n\nHe inserted messy human uncertainty and judgment into a process built to detect an attack and respond as quickly as possible.\n\nFrom the perspective of pure system efficiency, Stanislav Petrov was latency.\n\nThank God for that.\n\n## **THE DISTANCE BETWEEN US AND THE DEAD**\n\nThere is another part of autonomous warfare that may matter even more than whether our machine friends make better targeting decisions.\n\nWar has always been politically expensive because humans fight it. Politicians expend tremendous amounts of goodwill and human capital to fight these needless wars, all financial costs aside.\n\nArmies require people. Those people have families, communities, and voters attached to them. Governments asking citizens to fight are asking them to risk becoming maimed, traumatized, broken, or resigned to oblivion altogether. For God and Country!\n\nThat ends up creating some friction. Unfortunately, not enough friction, clearly.\n\nHuman history is littered with an impressive amount of war.\n\nBut friction nonetheless…\n\nAutonomous warfare changes the political equation because it can increase the distance between a society and the violence carried out on its behalf.\n\nSort of like how most people nowadays happily eat cheeseburgers and buffalo chicken wings but never need to butcher a cow or slaughter a chicken to enjoy those creature comforts.\n\nA government that can project military force without exposing many or any of its own citizens to danger faces a different political calculation from one that must send thousands of young people overseas to die in strange lands they couldn’t find on a map.\n\nThe war in Iran right now is a timely example.\n\nIn fact, literally yesterday, Iran’s [__Revolutionary Guards said they captured an American autonomous underwater vehicle__](https://www.reuters.com/world/middle-east/iran-says-it-captured-us-submarine-drone-strait-hormuz-2026-09-08/?ref=gonzocapital.net) near the entrance to the Strait of Hormuz.\n\nThe Pentagon says the Anduril-built Dive-LD had malfunctioned while surveying regional waters.\n\nPalmer Luckey’s Anduril had an even more interesting description for it: an “attritable autonomous system.” Which is some wonderfully sterile defense-industry mumbo-jumbo jargon for a machine deliberately designed to go somewhere dangerous because losing the machine is an acceptable outcome.\n\nAnd that is sort of the entire point I’m trying to make here.\n\nYou can send the robot into the Strait instead of the sailor. If somebody captures it, shoots it, sinks it, or it simply decides to stop working, nobody’s mother gets a knock on the door and a neatly folded flag.\n\nThis is obviously a good thing for the sailor.\n\nBut zoom out one level and the political arithmetic starts changing considerably.\n\nThe easier it becomes to project force while insulating your own population from the consequences, the easier it potentially becomes to keep projecting force.\n\nWe are already beginning to replace the human cost of escalation with hardware costs.\n\nHardware is considerably easier to reorder.\n\nThis is usually discussed as a benefit of drones and autonomous systems, and from the perspective of the soldier who does not have to die, it sure as hell is!\n\nI would think that most people would rather attrit the robot than attrit somebody's son. The fact that the word \"attrit\" exists is another matter entirely.\n\nThe problem appears one step later.\n\nWhat happens to the threshold for using violence when violence becomes much cheaper for the country deciding to use it?\n\nNo more grieving families, fewer funerals on television, fewer homeless veterans with PTSD.\n\nLess political pressure to explain why any of this is happening at all.\n\nThe violence remains extremely physical for the person standing on the other end of the drone.\n\nIt just becomes increasingly abstract for everyone else.\n\nThis may be the more important divide in the coming age of autonomous warfare.\n\nNot human versus machine.\n\nPeople who remain exposed to war versus people for whom war has become another remotely administered system.\n\nOn September 7, [__UN human-rights chief Volker Türk condemned Russia's reported use of fully autonomous drones__](https://www.reuters.com/technology/ai-could-pose-existential-risk-humanity-un-rights-chief-warns-2026-09-07/?ref=gonzocapital.net) in Ukraine and renewed calls to prohibit lethal autonomous weapons.\n\nSo some version of the future we are already fighting about in the halls of Geneva may already be bleeding into the present.\n\nThat should probably make the discussions feel slightly less academic and a bit more pressing.\n\n## **HUMAN-IN-THE-LOOP**\n\nGoing back to the beginning, the agreement in Geneva definitely matters.\n\nInternational norms matter. Weapons treaties matter. Even the tedious process of 128 countries arguing until the middle of the night about individual phrases matters.\n\nPretending otherwise would be cheap and unnecessary cynicism.\n\nWhat worries me is that the language we are using may conceal the actual nature of the problem.\n\n\"Human in the loop\" sounds reassuring because the human is doing all the moral work in that there sentence.\n\nHuman means judgment and responsibility, and that we are still ultimately steering the ship.\n\nBut the loop itself is going to keep changing and getting faster, and adding more machines, and adding more targets, and incorporating more information that we could ever possibly hope to process or understand.\n\nIt will increasingly ask humans to supervise decisions rather than originate or inform them.\n\nAnd perhaps most importantly, militaries have enormous incentives to make that loop move faster than the enemy's. Oh, and defense companies and their investors have the financial incentive to give those militaries what they want.\n\nHumans are known to build technology to remove friction we are tired of facing.\n\nEvery so often, we discover some of the friction was unknowingly protecting us.\n\nHesitation or waiting for authorization before firing is inefficient.\n\nHuman uncertainty or asking another person to verify the target is not ideal.\n\nConscience might be the least scalable part of the entire system.\n\nThat does not mean we should go ahead and freeze military technology where it is, even if that were possible in our game-theory-informed system. Nor does it mean autonomous systems are intrinsically more or less ethical than humans.\n\nHistory has provided approximately zero evidence that humans possess some magical competence for applying violence responsibly. We’re awful at it.\n\nWhat it *does* mean is that the problem is way harder than banning Terminator killbots.\n\nThe danger may not be that one morning a general decides machines should be given control over life and death. No, that would be too convenient, and there would be a clear decision to oppose. Or a villain to blame. The reality is that it’s way more banal and mundane than any James Cameron film.\n\nWe’ll simply keep on improving our targeting software, shortening response times, automating more steps in the kill chain, and making bigger, badder battle bots for our bottom lines and petty squabbles.\n\nWe will scale up and give each operator more systems to supervise because the software makes them more productive. Then more and more and more. Like what Claude Design did for marketing teams or Codex did for product sprints…\n\nThe reports will ultimately still contain a human name neatly printed at the top and signed at the bottom.\n\nThere will still be rules, commanders, operating parameters, and accountability frameworks we pretend to adhere to and care about.\n\nAnd one distant day we may look closely at the process and discover something peculiar.\n\nThe person legally responsible for the killing will no longer really be deciding whether the target should die or not.\n\nThe machine has already identified him, evaluated him, classified him, selected its brutal instrument of death, calculated the odds and risk, and left nothing but a big, bright red button eager to be pushed by a permitting human hand.\n\nThe lawyers get a human. The generals get accountability. The diplomats get \"meaningful human control.\" Gary gets a precious 0.7 seconds.\n\nEveryone wins. Well, nearly everyone.\n\nAnd maybe 99.9 percent of the time, the machine will even be right.\n\nI, unsurprisingly, am writing about and haunted by what happens in the other 0.1 percent.\n\nAbout a Soviet lieutenant colonel staring at a screen in 1983 while his computers confidently informed him that nuclear missiles were crossing the sky. About the twenty-three minutes of irritating, inefficient human hesitation that followed.\n\nAbout whether the next Stanislav Petrov gets the same twenty-three minutes' grace period.\n\nOr three seconds. Two. Or none.\n\nMaybe we never remove the last human at all. Somewhere, a human will still be sitting there, technically in charge, with less time to think than Petrov had to blink.\n\nWe’ll keep him in the loop, but we won’t leave him enough time to be human.", "url": "https://wpnews.pro/news/the-human-is-becoming-latency", "canonical_source": "https://www.gonzocapital.net/the-human-is-becoming-latency/", "published_at": "2026-09-10 09:26:41+00:00", "updated_at": "2026-09-10 09:53:53.377411+00:00", "lang": "en", "topics": ["ai-policy", "ai-safety", "ai-ethics"], "entities": ["Geneva", "United States", "Russia", "Reuters"], "alternates": {"html": "https://wpnews.pro/news/the-human-is-becoming-latency", "markdown": "https://wpnews.pro/news/the-human-is-becoming-latency.md", "text": "https://wpnews.pro/news/the-human-is-becoming-latency.txt", "jsonld": "https://wpnews.pro/news/the-human-is-becoming-latency.jsonld"}}