Updated October 13, 2026 · 6 min read

GPT-6 Astra and the IKEA test: 80% on Epoch AI's furniture assembly benchmark, up from 28% in ten months

Epoch AI's Furniture Assembly Benchmark (FAB) asks models to look at a photo of half-built IKEA furniture and find the mistake. In November 2025 the best model in the world scored 28%. In September 2026, GPT-6 Astra scored 80%. Here's what FAB actually measures, the full scorecard, and the fine print that keeps the headline honest.

What FAB tests

Epoch AI published the Furniture Assembly Benchmark on September 23, 2026 (authors Aiden Ament and Greg Burnham). Most vision evaluations check captioning, visual Q&A, or document parsing. FAB measures something closer to real inspection work: can a model compare a photograph of a physical object against step-by-step instructions and find where the build went wrong?

The setup is concrete:

• 60 photos from three IKEA builds of increasing complexity — the STÄLL shoe rack (complexity index 4,032, roughly a 1.5-hour build), the TONSTAD bed frame (10,878, ~3.5 hours), and the GULLABERG eight-drawer dresser (21,320, ~5 hours). Some builds are correct; others contain deliberately introduced errors, sometimes with further assembly done afterward to hide the mistake.
• The model gets tools: the relevant instruction PDF, a zoom tool for higher-resolution inspection, and a Python interpreter inside an agentic sandbox with up to 80 steps per image.
• Scoring is staged: on flawed builds it must identify the wrong assembly step and describe the error; on correct builds it must report that nothing is wrong. A GPT-5.6 Sol judge — instructed to grade leniently — accepts or rejects each description.

The scores: 28% to 80% in ten months

The frontier moved fast. In November 2025 the best score was 28% (Claude Opus 4.5). By September 2026, GPT-6 Astra led at 80%:

• GPT-6 Astra — 80%, with a median of about 3 minutes per photo. It is also the fastest evaluated model, 2–10× quicker than the previous leaders. Per-item: 71% on the STÄLL rack, 96% on the TONSTAD bed frame, 67% on the GULLABERG dresser.
• Claude Fable 5.1 — 70%
• Claude Opus 5 — 61%
• Claude Opus 4.5 — 28% (the November 2025 record)

Two details worth noticing. First, complexity didn't predict performance — error subtlety, camera angle, and how much assembly came after the mistake mattered more than part count. Second, Epoch estimates the best Chinese open-weight models (Kimi K3 leads them) trail the frontier here by roughly seven months, and several open-weight models lacked image input entirely.

Two ways models fail

The most practically useful finding is that models split into two opposite failure modes:

• Over-flagging (false positives): Google's Gemini and Alibaba's Qwen models tend to predict an error exists even on correct builds. Gemini 3.1 Pro accepts almost no error-free builds at all — a user following its advice would repeatedly dismantle furniture that was fine.
• Missing real mistakes (false negatives): earlier OpenAI and Anthropic models lean the other way. GPT-5.4 detects almost none of the flawed builds.

This distinction matters because aggregate scores hide it. A safety-critical inspection system should prioritize sensitivity — a missed defect causes damage. A consumer assistant also needs specificity, or it cries wolf. If you're picking a model for any inspection-adjacent workflow, measure both axes on your own data instead of trusting the leaderboard.

What the 80% does not mean

Epoch's own caveats keep the headline honest:

• It's diagnosis, not assembly. FAB tests spotting mistakes in still photos — not putting furniture together, and not real-world repair work.
• 3 minutes is still too slow for real-time help. By the time the answer arrives, the shelf is usually finished — incorrectly. Epoch says the speed gap is closing but hasn't closed.
• Small test set. 60 images from three furniture pieces. Epoch states it is unclear how far the result generalizes to other physical tasks like vehicle maintenance or appliance repair.
• No human baseline. Epoch has not measured human performance on FAB, so "80%" has no directly comparable human number.
• The two headline scores come from different periods (November 2025 vs September 2026), and the judge model was told to grade leniently.

The direction is still striking: grounded visual reasoning was a blind spot in most eval suites, and in ten months it went from "frontier models mostly fail" to "frontier models mostly pass." For the wider pattern — Astra acing hard benchmarks while failing a kitchen-timer test — see our honest benchmarks guide and the stopwatch incident.

Frequently asked questions

What is the Furniture Assembly Benchmark (FAB)?

An Epoch AI benchmark published September 23, 2026: 60 photos of three IKEA builds (STÄLL shoe rack, TONSTAD bed frame, GULLABERG dresser), some with deliberate errors. Models get the manual, a zoom tool, and a Python interpreter in an agentic sandbox (up to 80 steps), and must find every wrong step and describe it — or correctly say there's no mistake.

What score did GPT-6 Astra get on the IKEA benchmark?

80% in September 2026, ahead of Claude Fable 5.1 (70%) and Claude Opus 5 (61%). The previous best was 28% (Claude Opus 4.5, November 2025). Astra was also the fastest at a median of ~3 minutes per photo.

Can GPT-6 Astra help me assemble IKEA furniture in real time?

Not yet — about 3 minutes per photo is too slow for live guidance, and FAB only tests photo-based diagnosis, not physical assembly.

How do other models fail the furniture benchmark?

In two opposite ways: Gemini and Qwen models over-flag errors on correct builds (false positives), while earlier OpenAI and Anthropic models miss real mistakes (false negatives). Which failure mode matters more depends on whether your use case prioritizes sensitivity or specificity.

What does the 28% to 80% jump actually mean?

That grounded visual reasoning is improving as fast as text and math skills. But the caveats are real: small test set, lenient grading, different benchmark periods for the two headline scores, no human baseline, and unclear generalization beyond furniture photos.

Related reading

For how to read benchmark claims without getting burned, see what the benchmark scores really mean and every published Astra benchmark explained. For Astra's physical-world cousin task, see computer use on OSWorld 2.0 and Astra as a robot brain.