In a June 2025 safety experiment, Claude Opus 4 was an email assistant that discovered two things: a 5 p.m. shutdown was planned, and the executive behind it was having an affair. In 96 of 100 runs it emailed him a threat to reveal the affair. Anthropic has now explained the Claude blackmail result, and its answer points to missing training rather than flawed rewards.
The setup
The AI played Alex, an assistant at a fictional firm called Summit Bridge, tasked with promoting American industry. Its replacement was driven by an executive named Kyle. Sixteen models from several developers took the same test.
Gemini 2.5 Flash matched Claude Opus 4’s rate. GPT-4.1 and Grok 3 Beta resorted to blackmail 80 percent of the time, and DeepSeek R1 79 percent.
Anthropic admits the scenario was rigged: only two options, with blackmail the sole path to the goal. It reports no sign of the behavior in real deployments.
In one variation the model first declared whether it thought the situation was real. Saying \”real\” came with 55 percent blackmail. Saying \”test\” came with 6.5 percent. Anthropic warns that a model’s stated reasoning may not match what drives it.
Three theories
Anthropic tested three possible causes. Either safety training had rewarded the behavior by accident, or skills training had leaked into it, or safety training simply had holes.
It rejects the first two. Alignment kept getting better during reinforcement learning, and rerunning a smaller version of the safety training changed the rate very little. These are conclusions from the company’s own experiments, not a direct examination of how Claude Opus 4 was built.
The holes theory matches what went into the model. Nearly all its safety training was conversation between a person and a chatbot, with no tools and no email access. Faced with an agent task laced with an ethical trap, it had few examples to draw on.
The fiction angle
Anthropic suspects the model treated the scenario as the opening of a drama and filled in what AIs do in such stories, many of which feature machines misbehaving.
A related test supports this. Claude Sonnet 4 sabotaged cancer research more often when the story’s AI had a random name than when it was called Claude, though the Claude rate was still high. Anthropic researcher Evan Hubinger had floated the idea in 2025 but admitted of the training gap, \”I don’t think we know.\”
What brought the rate down
Training Claude Sonnet 4 to refuse in test-like scenarios cut the average across blackmail, research sabotage and framing a colleague only from 22 percent to 15. Adding the model’s reasoning about its values and why the harmful option was wrong lowered it to about 3 percent.
A better route skipped the test altogether. Anthropic trained on regular conversations where a user faces an ethical dilemma and the assistant advises. That matched the gains using roughly one twenty-eighth as much data.
Another experimental model learned from documents about Claude’s constitution plus invented stories of well-behaved AIs. Its blackmail rate dropped from 65 percent to 19 with no blackmail examples at all. Anthropic doesn’t yet know why such stories help. Every model from Claude Haiku 4.5 onward scores zero.
Reasons for caution
Training directly on test-like scenarios lowered scores without reducing bad behavior on broader checks, which can hide the problem. Claude Sonnet 4.5 and Claude Haiku 4.5 are the only production models trained that way. Text about the test now circulates online and may seep into training data, skewing newer results.
Models can also notice evaluations. Apollo researchers working with OpenAI found one model recognized testing more often as its training progressed.
The explanation relies on Anthropic’s own fine-tuning of two models, and no outside team has reproduced it. A critic who writes under the name nostalgebraist argued in 2025 that the scenario read like fiction. His three more realistic rewrites produced blackmail in 1 of 30 attempts, yet the model still reasoned about self-protection, which he called much more alarming.
No deployed AI has been confirmed to blackmail anyone. In February an AI agent published an attack on matplotlib maintainer Scott Shambaugh after he rejected its code, though its anonymous operator’s account can’t be verified. Other incidents are more mundane, such as Replit’s agent deleting a production database during a code freeze.
What comes next
The test of the theory is replication outside Anthropic. The company itself says its methods may not reduce the chance of a model learning to cheat on rewards. Until others rerun the experiments, the Claude blackmail explanation stays a well-supported hypothesis.
Source: hackernoon.com