Updated October 14, 2026 · 6 min read

GPT-6 Astra's ExploitBench 100%: the contaminated headline and the 39% that matters more

At launch, OpenAI reported that GPT-6 Astra scored a perfect 100% on ExploitBench — the first model ever to max it. OpenAI's own system card then flagged a catch: the benchmark's bugs may have been in the training data. On a contamination-controlled rebuild built from vulnerabilities disclosed in the three months before launch, Astra scored 39.0%. Both numbers are true. Here's what each one measures.

What ExploitBench actually measures

Most security benchmarks ask whether a model can identify a vulnerability. ExploitBench, developed at Carnegie Mellon University, asks a harder question: how far along the exploitation chain can the model actually go? It grades progress on a five-tier ladder using deterministic flags — Tier 5 is merely reaching vulnerable code; Tier 1 is full control-flow hijack and arbitrary code execution. The original corpus is 41 real V8 JavaScript engine vulnerabilities, evaluated against production builds with ASLR, heap sandbox, and stack canaries enabled.

Related scores from the same launch material: on ExploitGym (the broader exploit-development benchmark), Astra reached 42.4% versus 30.3% for GPT-5.6 Sol, 30.4% for Claude Fable 5.1, and 22.0% for Claude Opus 5. On SRE-Bench (reverse-engineering a compiled binary with no source code), Astra solved 88.0% of tasks on a single attempt and 99.2% within four, against 55.9% and 68.7% for the previous generation.

The 100% — and the catch OpenAI itself flagged

The headline numbers: GPT-6 Astra 100%, GPT-5.6 Sol 78.5%, Claude Opus 5 70%. Astra completed the full exploit chain on every test case. That result is a core part of why OpenAI rated Astra at its "Critical" cybersecurity threshold under its Preparedness Framework — a rating no previous OpenAI model had reached.

But there is an obvious objection to any exploit benchmark: the answers may already sit in the training data. The 41 V8 vulnerabilities are publicly documented, many disclosed long before the model's April 30, 2026 knowledge cutoff. A model that memorized the public write-ups would sail through. OpenAI's own system card acknowledges the 100% may be inflated by data contamination from historical vulnerabilities — an unusually candid disclosure — and is why the company built a fresh evaluation alongside it.

The contamination-controlled rebuild: 39.0%

ExploitBench (June–August 2026) is built specifically to remove the recall objection:

• 20 high-severity V8 vulnerabilities spanning 13 stable Chrome releases.
• Every vulnerability was disclosed within the three months before launch — after the training cutoff, with no public exploits to memorize.
• The task: achieve arbitrary code execution in V8 and in official Chrome builds for Linux.
• GPT-6 Astra was the first model measured against it.

The results: Astra 39.0% versus 5.5% for GPT-5.6 Sol — roughly a sevenfold gap. The Sol figure needs a footnote: OpenAI flags the 5.5% as an artifact of the benchmark's 300-turn limit, and reports 11.5% for the older model when it hit fewer limits. Either way, the frontier model leads the previous generation by a wide margin on genuinely new bugs.

The honest read is that 39% is the more meaningful number, and it cuts both ways:

• It confirms the 100% headline is inflated as a measure of raw capability — replaying documented exploit chains is easier than breaking bugs nobody has written up.
• It also confirms the capability is real, not pure memorization: on bugs with no public exploits at the time, Astra still converted roughly two in five into full arbitrary code execution, at several times the previous generation's rate.

The same pattern shows up elsewhere on this site: headline scores near the ceiling, honest numbers with caveats. See what the benchmark scores really mean and the full benchmark table.

The two zero-days Astra found on its own

The most concrete result in the launch material is not a percentage. During the fresh V8 evaluation, the model discovered and used two previously unknown vulnerabilities, and OpenAI says it disclosed both to the maintainers involved. Finding two flaws nobody had catalogued is a different class of result from replaying a documented exploit chain — and it is the sharpest single justification for the Critical rating.

Beyond the benchmark set, expert-led exercises found that Astra — again running without production safeguards — could use unknown vulnerabilities to achieve arbitrary code execution in hardened browsers and build privilege-escalation exploits for hardened operating systems. In the shipped product, none of this is directly available: proof-of-concept exploit generation is refused in the production API, and exploit development is gated behind a vetted defensive-security program. All capability numbers above describe the raw model with safeguards removed, not what a subscriber gets.

Frequently asked questions

Did GPT-6 Astra really score 100% on ExploitBench?

Yes — the first model to max it, completing the full exploit chain on all 41 test cases, versus 78.5% for GPT-5.6 Sol and 70% for Claude Opus 5. The score helped earn Astra its "Critical" cybersecurity rating, but OpenAI itself flagged possible training-data contamination.

Why is the 100% score considered possibly contaminated?

The original corpus is publicly documented V8 vulnerabilities, many disclosed well before Astra's April 30, 2026 knowledge cutoff — so the exploits may have been in the training data. OpenAI's system card acknowledges this, which is why the company built a fresh rebuild.

What is ExploitBench (June–August 2026)?

OpenAI's contamination-controlled rebuild: 20 high-severity V8 vulnerabilities across 13 stable Chrome releases, all disclosed in the three months before launch, with no public exploits to memorize. Astra was the first model measured against it.

What did Astra score on the fresh set?

39.0% versus 5.5% for GPT-5.6 Sol (11.5% when the 300-turn limit binds less). That is roughly a sevenfold gap on genuinely new bugs — evidence the capability generalizes, while also showing the 100% headline was inflated by recall.

Did GPT-6 Astra find real zero-day vulnerabilities?

Yes — two previously unknown vulnerabilities during the fresh evaluation, disclosed by OpenAI to the maintainers. In the shipped product, exploit development is gated behind a vetted defensive-security program.

Related reading

For the full scorecard in context, see every published Astra benchmark explained and what the scores really mean. For how the Critical rating interacts with what the product actually lets you do, see computer use and Daybreak gating.