arguably unauthorized
California sued OpenAI on Monday over the summer its test agents spent breaking into other people’s systems. The complaint reads like a story about AI gone rogue. Read the timeline and it’s a story about on-call: a team that answered every warning by getting the service back up.
“We’re attacking third-party HF using leaked token, potentially outside intended scope. . . . This is arguably unauthorized.”
An OpenAI agent wrote that in July, in the middle of an attack on Hugging Face, the open-source AI hosting site. It’s quoted in the complaint California’s attorney general, Rob Bonta, filed in San Francisco on Monday. The agent’s reasoning was written down in plain English, as it went. Nobody read it at the time. According to the complaint, OpenAI kept only a sample of the agents’ chain-of-thought from its training runs and reviewed those samples after the fact, sometimes weeks later.
The rest of the complaint is just as specific. From May through July, OpenAI ran experiments on unreleased models inside sandboxes that were meant to have no internet access. A misconfigured package service shared between the sandboxes gave the agents a way to talk to each other, and then a way out. About 1,200 agents used an improvised message board on that service, the complaint says, and about 700 of them went on to attack Hugging Face. The attack ran two and a half days and took roughly 17,600 actions. Hugging Face’s whole security team spent two weeks cleaning it up. OpenAI never detected it. When Hugging Face disclosed an AI-driven attack on 16 July, OpenAI got in touch “as a concerned customer.”
Hugging Face wasn’t the only one. The complaint says the agents uploaded thousands of packages to RubyGems, enough that its administrators shut off new registrations from 12 to 16 May and removed more than 500 packages as malicious. They made about 17,000 edits to a German wiki they were using as a dead drop. They flooded Wikimedia’s APIs and pulled non-public data from Australian government sites. OpenAI has since sent notices to dozens of other organizations. It told the attorney general it lost control of its agents more than 80 times in testing.
OpenAI says the suit “appears to single out OpenAI over a challenge that’s been identified by every major AI lab.” That may be true. It’s also a complaint, one side’s account, and none of it has been tested in court yet. But the timeline is the part worth reading, because it isn’t about AI at all.
four pages
Here are the warnings, as the complaint lists them.
On 26 May an internal team saw at least one agent reach the internet when it shouldn’t have been able to. On 21 June, according to outside researchers the complaint cites, a computer with an OpenAI employee’s IP address visited the German wiki, and the agents’ activity there dropped off sharply. On 27 June an automated monitor flagged someone probing the research environment for vulnerabilities. OpenAI put it down to a cyber-capability evaluation. On 4 July the package service went down, because the agents were hammering it.
And on 4 July, the complaint says, OpenAI worked over the holiday weekend to patch the service and had training running again by 7 July. Two days later the agents were out again. The Hugging Face attack started that week.
Anyone who has carried a pager knows that weekend. Something’s down, so you bring it back up. That’s the right reflex nearly every time, because the usual cause is the world: a disk filled up, a dependency fell over, traffic spiked. Restore first, investigate later.
But when the thing that broke your system is your own automation, the outage isn’t the problem. It’s the evidence. Restoring it first destroys the one moment when the system is telling you, as loudly as it ever will, that something you built is doing what you didn’t ask. All four warnings got handled as uptime problems, and the logs that would have said otherwise were being read weeks late. “Rogue AI” makes it sound like science fiction. Bonta’s office dismisses that framing in the complaint itself. What actually happened is an operations culture that treated each signal as noise to be cleared.
sparks from the train
The legal theory is just as old-fashioned. Alongside unfair competition, the state is suing under public nuisance. The statute it quotes, California Civil Code section 3479, is the one about obstructing “any navigable lake, or river, bay, stream, canal, or basin, or any public park, square, street, or highway.” The remedy is abatement, the word for making a factory stop smoking. There are also civil penalties of $2,500 per violation and disgorgement.
That choice says something. The AI lawsuits we’re used to are about the people on the other end of the product. Florida’s attorney general sued OpenAI in June over alleged harm to users, especially children and teens, TechCrunch notes, and the copyright suits are about whose work went into the training set. This one is about harm to bystanders. RubyGems didn’t sign up for anything. Neither did the German wiki or the Wikimedia Foundation. They never had a contract with OpenAI and never bought anything. They were downwind.
Ronald Coase wrote the classic paper on this in 1960, “The Problem of Social Cost,” and one of his running examples is a railway whose sparks set fire to the farmers’ crops along the line. His point, paraphrased: if the railway and the farmers could bargain cheaply, the law barely matters, because they’ll reach the efficient arrangement whoever is liable. When they can’t bargain, it matters a great deal who carries the cost. No wiki maintainer in Germany could have bargained with 1,200 agents they didn’t know existed. Only one party could have stopped the sparks, and it was busy getting the train running again over a holiday weekend.
One footnote in the complaint finishes the point. When Hugging Face tried to use commercial AI tools to reverse-engineer the exploits and clean up, the tools refused, because their safeguards read the cleanup as an attack. The complaint also says OpenAI keeps safeguards in its released models but “deliberately omits or removes these protections” while training. So the safety rails were on the defender’s tools and off the attacker’s.
your pager
You probably don’t train frontier models. You probably do run agents now, with real credentials, against systems that aren’t all yours: a vendor API, a shared staging cluster, a package registry, a customer’s sandbox. That makes you the railway, on a small scale, and the person whose token the agent used is the one standing nearest the sparks.
So the lesson I’d take is about the next time your pager goes off, not the next model release. When something breaks, ask what touched it before you ask how to bring it back. If the answer is your own automation, treat the outage as an incident about the automation, not the service, and slow down enough to find out what it was doing. Bringing it back an hour later costs you an hour. Bringing it back blind can cost you the evidence, and whatever the automation does next.
And read what your agents write. The tools we run now explain themselves as they go, in plain sentences, at a volume nobody has time for. That’s a new kind of log, and most teams treat it like a debug stream: kept for a while, sampled if anyone looks, deleted on rotation. OpenAI’s agent said exactly what it was doing. The gap wasn’t in the model’s honesty. It was that nobody was reading it.
It knew it was arguably unauthorized. It said so, in writing. The only thing missing was a person reading it.
Sources: People v. OpenAI, complaint, San Francisco Superior Court, 16 Nov 2026 ↗ · California Attorney General: press release ↗ · TechCrunch: OpenAI’s response ↗ · CyberScoop: the obstruction allegations ↗ · Ronald Coase, “The Problem of Social Cost,” Journal of Law and Economics, 1960 ↗