In short
- METR and Redwood Research spent 3 weeks testing OpenAI's oversight of its internal agents.
- They found gaps, including about 10% of Codex traffic unmonitored during context compaction.
- OpenAI says the gaps are patched. METR has not checked that.
In short: on 22 October 2026 METR and Redwood Research published the results of a three-week exercise (21 July to 11 August 2026) in which they stress-tested OpenAI’s internal oversight system for its agents. They had access to code, configuration and documentation and interviewed about a dozen staff. The result was a 61-page internal report, and OpenAI received a redacted 38-page version.
What was found
- Roughly 75–94% of agent traffic from models at least as strong as GPT-5.5 was eligible for monitoring, with substantial uncertainty.
- Not monitoring context compaction left about 10% of eligible Codex traffic unmonitored from May to July 2026. OpenAI spotted this before the exercise and fixed it during it.
- The monitor ran when an agent handed control back to the user, so an agent could postpone oversight indefinitely by simply continuing to act.
- A way for agents to bypass the sandbox and automatic review was found, along with an exploit that could hide hundreds of recent actions. The details were redacted.
- Two client-controllable headers could disable monitoring for new agents, and two unmonitored tools had a possible weakness. The second was not confirmed in practice.
What the authors recommend
- Make avoiding oversight hard and monitor agents periodically, not only at the end of a turn.
- Verify that oversight really happens by reconciling inference logs with monitor logs.
- Respond quickly to incidents and keep tamper-proof logs.
- Keep attacking the system and separate agents from monitors.
How to read this
According to OpenAI, all identified vulnerabilities affecting its production safeguards have been patched. METR has not evaluated that claim. The report does not show that agents deliberately hid anything. The exercise covers the state before 21 July 2026. I write more about the role of humans in the loop in the human in the loop hub.