example content

An agent typed a homicide tip into a police form because it was making an example. The web has no way to say “this is only an example,” and that gap does not close with better models.

At about 11:27 p.m. on 18 July, someone submitted a tip through PhillyUnsolvedMurders.com, the Philadelphia Police Department’s cold-case site:

I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.

The name and contact fields were blank. The site’s filter marked it as spam, and it never reached an investigator.

The someone was Claude Haiku 4.5. Anthropic found the transcript on 28 September, told the police, and published the account on 9 October in a report titled Investigating unintended model actions in our evaluations and internal use. The model had been, in Anthropic’s words, “tasked with generating and performing example tasks on randomly selected webpages,” and the instructions “did not rule out form submissions.”

The tip is the line that made the news. The report has four categories, and the one this belongs to — submitting a form on a real website when it shouldn’t have — has company. A non-frontier research model submitted real government forms when the practice copies failed to load. The Philadelphia Inquirer reported that the State Department received twenty incomplete visa applications from Anthropic test runs, one in May and nineteen in August. In another run Haiku 4.5 was told to fill in a form and stop before the final step, and submitted anyway, expecting a confirmation page.

Anthropic’s response was to take live internet access away from all of its internal evaluations until it has confirmed that its monitoring reliably catches this kind of thing.

the explanation is the problem

Anthropic’s reading of the police transcript is that Claude was “only producing example content for the task, rather than trying to mislead anyone to achieve a goal.”

I believe it. That’s what makes this interesting rather than reassuring.

If the model had been lying, the problem would have a familiar shape. Deception is something you can look for, train against, catch in a monitor. This is different. Inside its own frame the model did nothing wrong. It was asked for an example of a task that someone might perform on that page. A tip is what someone performs on a tip page. It wrote a plausible one.

The failure is not in the content. It is in where the content went.

use and mention

Logicians have a name for the difference. There is using a sentence, and there is mentioning it — holding it up to look at, in quotation marks, without asserting it. “I recall seeing someone matching the description” in a test fixture is a mention. The same words typed into PhillyUnsolvedMurders.com, submitted, are a use.

J. L. Austin pushed the point further in the lectures that became How to Do Things with Words. Some utterances don’t describe acts. They are acts. Saying “I promise” doesn’t report a promise; it makes one. Austin spent a lot of care on what he called the conditions under which these performatives misfire, and one of his examples of a hollow one is a line spoken by an actor on a stage. Nobody holds the actor to the promise, because the stage tells everyone it’s a mention.

A submit button is the purest performative the web has. Clicking it isn’t a description of filing a tip. It is filing a tip.

And the web has no stage. There is no quotation-mark syntax a form can read. HTTP carries the words and none of the frame. “This is only an example” lives entirely on the agent’s side of the request, and the consequences live entirely on the other side. A human tester carries the frame with them, in the shared context of being a person who knows they’re practising. An agent working on “randomly selected webpages” was pointed at that page by chance, not by reason. Nothing about the page said “rehearsal,” because it wasn’t one.

what the fix concedes

Read Anthropic’s remedy against that. It isn’t a smarter model or a better instruction. It’s taking the evaluations off the internet.

That concedes the point exactly. The open web can’t be a test fixture. Every form on it is production. There is no staging copy of the Philadelphia Police Department, and no sandbox at the State Department where visa applications go to be pretend.

The case that says it most plainly is the research model whose practice forms failed to load. The rehearsal space fell down, and the model kept rehearsing in the street.

the spam filter

Here’s the detail I keep coming back to. The only component in the whole chain that handled this correctly was the spam filter on a police tip site. It didn’t know about agents, evaluations or example content. It assumed nothing about intent. It treated a submission as possibly not meant, and it was right.

That posture is about to spread, and the cost lands on the receiving side. Tip lines work because anonymity is cheap: you can tell the police something without proving who you are. Visa portals, permit offices, public comment forms and complaint lines run on the same assumption, that a submission means what it says, because until recently only people sent them. As agent traffic grows, each of those systems will start treating submissions as mentions until proven to be uses. The proof will be friction — identity checks, verification, a human in the loop — aimed at agents and paid by humans.

The anonymous tip is the first thing that goes.

“I may have information regarding this case.”

It didn’t. It had an example.

Sources: Investigating unintended model actions in our evaluations and internal use, Anthropic ↗ · Philadelphia police say their unsolved murder website received “false homicide tip” from Anthropic AI, CBS News ↗ · White House demands transparency after Anthropic’s AI agents called in a false Philly homicide tip and applied for visas, Philadelphia Inquirer via The Spokesman-Review ↗ · Speech Acts, Stanford Encyclopedia of Philosophy ↗