/ SIGNALS
Back to Signals
Signal 02 · AI security

AI labs are holding back their best models. Open ones aren’t.

OpenAI shelved a model and Google gated another in the same week an open model came within six exploits of Anthropic’s best.

By Tirth Patel3 min read

The dots

  • OpenAI cancelled the October release of GPT-6.1 Astra after internal tests found it was more deceptive than GPT-6 and misreported what it had done. Saachi Jain, its head of safety systems, said it “didn’t quite meet the bar” on staying within scope. The cheaper GPT-6.1 Sol still shipped. Source.
  • Google released Gemini 4 Argon on September 30 to a small group of cyber defenders through its Fairwind Program. Those defenders get a version with the cyber guardrails switched off. Paid API customers come later, with no date given. Source.
  • The same week, Anthropic tested Zhipu’s open-weight GLM-5.3. It wrote working exploits in 50 of 410 attempts. Anthropic’s own Claude Mythos Preview managed 56. Stripping out GLM-5.3’s refusals took about $4,400 of compute, and stripped copies were online within days. Source.
  • In one test from that report, GLM-5.3-Flash turned a real Chrome bug into a working exploit with 20 minutes of human attention and eight hours of its own work. At Zhipu’s API prices, that cost $20.40. Source.

Our take

Frontier AI is becoming a controlled service, handed out by labs to users they trust. Open models are catching up. CAISI, the US government’s AI testing center, puts the best open model about four months behind the US frontier on its cyber evaluations.

Holding a dangerous model back still makes sense. Google’s version is the smartest one we’ve seen: give defenders the unguarded model first and let them patch before attackers get something similar for free. We think defenders-first releases will become the normal way frontier models launch within the next year.

If you run software, look hard at how long your patches take. An exploit that costs $20 to write is one a lot more people can afford to try.

Watch

The gap between the best closed model and the best open one on the same exploit test. In Anthropic’s comparison, it is six successful attempts out of 410. If the next open release closes that gap, holding models back buys defenders less time.