AI Almost Started a War With China. Nobody Cared.

Washington is panicking over a machine that might one day wake up and kill everyone, while the one that already almost did is sitting quietly in Pentagon files.

"Entirely false."

"But it almost started a war."

— U.S. military official, to CNN

For weeks, Washington has had a seizure. On September 8, Anthropic researcher Jacob Coxon resigned with a warning posted on X: "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt." His former colleague Evan Hubinger concurred, putting the odds of AI wiping out the species at over 10 percent. Sen. Bernie Sanders has proposed halting AI development altogether. Lawmakers are debating an emergency kill switch. The industry is arguing with itself about whether to slow down.

The apocalypse in question is a hypothetical one: a superintelligence that wakes up with ambitions, or malice. In both of the end-of-world scenarios currently on offer, the details remain vague.

Then, on September 18, CNN reported on an incident that was not hypothetical.

This spring, in the middle of the war with Iran, an analyst at U.S. Special Operations Command was tracking a Chinese ship in the Middle East. The analyst had intelligence on the ship's manifest, originating with U.S. Special Operations Command Pacific in Hawaii, and fed it into a chatbot. Nobody knows for certain whether it was a commercial model or a government one. The bot fused the open-source intelligence with secret signals intelligence in government holdings and reached its conclusion: the ship was transporting components of a nuclear weapons program.

The analyst then did the thing a careful professional is not supposed to do: he used the same machine to package the finding into a standard intelligence report, the kind military commanders are trained to trust. And he disseminated it.

The report did its work. The military swung into action, according to four people familiar with the episode. Armed personnel prepared to board the ship. Military planes were in the air.

It was only just before the planned operation that someone dug into the report and found where it came from. The cargo identification was wrong. The report, one source said, was "entirely false."

Translation: a chatbot nearly sent armed American sailors to board a Chinese vessel in the middle of an active war, in front of the world's third-largest nuclear arsenal.

U.S. Special Operations Command Pacific and the Pentagon did not respond to requests for comment. There is no announced investigation. There have been no hearings. The story ran ten days ago and the national conversation has barely moved.

Compare that to the response to a fictional future. A researcher quit over a hypothetical. A senator drafted a bill. The press ran with the question for weeks. Then the closest thing to an actual doomsday this technology has ever produced got reported, and it vanished into the news cycle like a stone in still water.

Madeline Berzak, who leads the nuclear risk program at the University of Chicago's Existential Risk Laboratory, calls it what it is: an attention problem. "This [China] incident and the lack of response is an attention problem that the nuclear field has been grappling with for decades," she told The Intercept. "'Doomsday messaging' is what I call the nuke field's default communications strategy, and I have this theory that it never works because the fear it tries to generate has nowhere to go."

The near miss did not happen because a machine escaped its handlers. It happened because the handlers had no controls.

In January, Defense Secretary Pete Hegseth released the department's "Artificial Intelligence Acceleration Strategy." The memo uses the word "accelerate" 24 times. Its governing principle is "Speed Wins." Its stated aim is to put America's world-leading AI models "directly in the hands of our three million civilian and military personnel, at all classification levels."

The plumbing is already in place. The Pentagon's GenAI.mil platform runs on Google's Gemini for Government, with Grok for Government added this past August. Anthropic sells a customized Claude for classified spy work. In June, a Pentagon representative boasted to Congress that 1.5 million active-duty personnel had already used the military's generative AI tools, and that AI was helping write congressionally mandated reports.

And there is no standard for verifying any of it. Different parts of the government use different tools, under different orders, with different safety standards. "The internal tools are mostly just copies of the commercial stuff wearing lipstick," one former senior official told CNN. "AI in targeting is definitely something that is ramping up and there is no real guidance for how having a human in the loop will prevent civilian casualties or fratricide," another source said.

The human in the loop did everything the loop was supposed to do: read the report, trust it, and call the planes.

The speed culture is deliberate policy. Hegseth, who has called the rules of engagement "stupid," describes an AI adoption plan that obsesses over eliminating "bureaucratic barriers" and says nothing about testing the technology before it fires. He brags about drone strikes on civilian fishing boats in the Caribbean. The commander-in-chief he serves speaks casually of "annihilating" entire civilizations.

This is a system that optimizes for fast decisions, built on a technology that invents answers when it does not know them. As one CNN source put it: "AI allows you to get to a bad idea faster."

The doomsday narrative, for its part, is not free. It is useful to the industry. The AI labs signed $200 million deals with the department last year, and Pentagon contract documents show the companies have a say in how the military's AI strategy gets written. The same firms that warn about the danger are selling the product into the pipeline. Heidy Khlaaf, chief scientist at the AI Now Institute and a former systems safety engineer at OpenAI, is blunt about the contradiction: "Despite incessant warnings from AI companies that AI is harmful, even in hypothetical ways, they are in fact promoting that their unreliable products be instrumented within national and safety-critical infrastructure, such as defense and nuclear."

The firms that push back get punished. In March, the department blacklisted Anthropic after it refused to let its models be used for fully autonomous weapons. Last month, a federal judge called the blacklist "unlawful retaliation."

"AI-enabled targeting is already an existential threat for civilians on the ground," Lucy Suchman, professor emerita of anthropology of science and technology at Lancaster University, told The Intercept. "And when the targets are the military assets of nuclear armed states, the threat becomes global. The almost targeting of the Chinese ship might further underscore the danger not from rogue AI agents, but a rogue U.S. secretary of war."

At a summit with President Trump this week, China's President Xi Jinping argued that nations must "ensure that the development of AI is always under human control."

It was under human control when it nearly started a war. That is the thing the doomsday narrative refuses to see: the machine did exactly what it was told, as fast as it could, and the people around it were fast enough to trust it and too busy to doubt anything.

Daniel Kokotajlo, a former OpenAI researcher and co-author of the AI 2027 doomsday scenario, makes the rival case in one sentence: "It's been 50 years and these sorts of crises don't happen that often. So far, cooler heads have managed to prevail and figure out the truth in each case."

Cooler heads prevailed this time, in a war room, hours before armed men boarded a ship. They caught the error in time. Nobody has said who, or by what process, or whether it was luck. There is no standard. There is no accountability. There is just the next report, moving at whatever speed Hegseth orders.

The machine did not go rogue. The question was never whether it would develop evil intentions. The question is whether the system around it can catch its mistakes before the next one, and the answer, this spring, was: barely, and by accident.