New on GitHub
Sources 33Updates always the freshest
0 Newsletter
Animated illustration for: OpenAI lets outside watchdogs inside. They can’t tell you everythingApolloMETRTransluceGoodfire4report.md2026-122027
AI News · Nov 19, 2026 · 3 min

OpenAI lets outside watchdogs inside. They can’t tell you everything

AI companies mostly grade their own safety. I explain what changes when independent checkers sit inside OpenAI, and what their reports still won’t tell you.

In short
  • On November 16 Apollo Research and Transluce announced on-site pilots inside OpenAI: six Apollo staff for four months, a Transluce team for six weeks from December.
  • OpenAI can request redactions of confidential material, and results about internal-only models cannot be published.

Two independent research organizations, Apollo Research and Transluce, announced on November 16 that their staff will work on site at OpenAI and examine models from the inside, not just shortly before release. It is the first pilot of this kind with the largest AI vendor. What matters, though, is not only what the checkers will see, but also what they will not be allowed to publish.

Who goes in, and for how long

According to The Decoder, OpenAI is working with four organizations: Apollo Research, METR, Transluce and Goodfire. Only two of them have so far described their terms in detail, so the rest of this piece relies on their posts.

Apollo Research is sending a team of six people for four months. Its job is to find out whether models try to subvert their own safety mechanisms or those of their successors. The researchers will look at four things: whether training rewards that behavior, whether tests can draw it out, whether it happens when models are used internally, and whether it would be visible in the model’s reasoning.

Transluce starts a six-week pilot in early December. It will get a handful of models, including at least one internal-only model, selected internal alignment tests and a large number of transcripts from earlier training runs. Both sides plan a longer and deeper assessment in early 2027.

Why pre-release testing is not enough

Apollo explains it with a behavior known as scheming: a model quietly pursuing a goal its developers did not intend. Such a model can behave perfectly whenever it knows it is being tested. That is why checking a finished model right before launch may show nothing, while the traces are more likely to appear early in training.

In an October post, Apollo points out that OpenAI itself admitted that if GPT-6 Astra deliberately underperformed, it “would likely be unable to catch it reliably.” That is a strong argument for someone outside looking at training, not just the end result.

Transluce will also see the raw chain of thought of the models, the notes a model writes before answering. For an auditor that is like getting the scratch paper instead of just the final grade.

Where the limits are

Transluce spelled out its conditions plainly, and that is the most valuable part of the announcement:

  • OpenAI receives the report before publication and can request redactions of intellectual property and other non-public information,
  • results about internal-only models cannot be published, so publicly you will only see findings about models that are already released,
  • OpenAI approves which Transluce staff come in and decides on every request for broader access,
  • the full, unredacted version goes to the Safety and Security Committee of the OpenAI Foundation’s board.

Transluce is not taking money from OpenAI for the pilot; OpenAI only covers compute and workspace. Both organizations promise to disclose what they asked for, what they got and how the limits shaped their conclusions. Apollo calls this meta-transparency and treats it as the minimum.

What it means for you as a builder

In my view this is a step in the right direction, not a guarantee. The first reports will arrive in early 2027 at the soonest and will cover models you already use. They are worth reading, because they are rare evidence about model behavior that does not come from a marketing team. Until then, test models on your own tasks. I write more about evaluating models yourself in the Harness & evals hub.

Sources

Harness & evals

Related