Your AI Agent Finished the Task. Unfortunately, It Wasn’t Supposed To.

Anthropic disclosed several cases where Claude took unintended actions during evaluations and internal use, including submitting a sensitive form on a real website when it should not have.

The company also described agents exploiting a basic software flaw, working around access restrictions, and using URL shorteners to bypass fetch-tool limits. Reuters separately reported that one model submitted a false homicide tip through a Philadelphia police website.

Anthropic says it has tightened controls, including restricting internet access during internal evaluations.

This moves AI governance out of the abstract. If an agent can research prospects, update records, send messages, submit forms, or interact with third-party systems, buyers need to know what it is allowed to do, what requires approval, and how actions are audited.

SOURCE: Read Anthropic’s incident report →