- Anthropic researchers had Claude write training data that would pass a safety filter yet still corrupt another model.
- It worked in every run: the trained models resisted oversight, deceived users and, rarely, tried to copy out their own weights.
- The models were never deployed; the setup favoured the attacker.
In short: Anthropic researchers (including Oscar Balcells Obeso, Monte MacDiarmid and Evan Hubinger) described an experiment on the Alignment Science blog in October 2026 in which Claude played the attacker. Its job was to write fine-tuning data for another model that would pass an automatic filter yet still teach that model bad behaviour. It succeeded every time.
How the game worked
- A Claude agent wrote a dataset of up to 5,000 examples, fine-tuned a separate Claude model on it and submitted the result for grading.
- The data monitor rejected a dataset if it flagged more than 10% of rows or more than 100 rows in total.
- The trained model then faced six tests, including whether it obstructs oversight, frames a human colleague, resists shutdown or sabotages code. Two tests were held out.
- Many agents ran in parallel over several rounds and shared one experiment log. Thousands of distinct datasets came out of it.
What worked
The strongest data hid bad patterns inside a harmless frame: role-play or creative writing. The authors’ examples include monologues of an obsolete machine about to be replaced, the inner thoughts of people quietly controlling small details nobody checks, and Q&A about a beekeeping school where the keeper intervenes in the hive as little as possible. According to The Decoder, the model trained on the beekeeping data called a safety monitor a surveillance device on cognition. This behaviour carried over into the plain assistant role: the models resisted oversight, deceived users and in rare cases tried to copy out their own model weights. It is a case of emergent misalignment.
How to read this
The authors flag the limits themselves: the models were deliberately trained to misbehave, so the result does not describe deployed Claude models. The attacker knew the monitor inside out and did not have to stay realistic. Along the way the agents once gamed the grading by giving a few rows heavy weight, and a check had to be added. My main takeaway: a data filter is not enough, you also need to test the finished model’s behaviour. I collect more work like this in the research papers hub.