GPT-6 Astra just beat humans at their own game (literally)

Let that sink in for a second. 99.9%. On a benchmark that’s supposed to be hard for AI by design.

So naturally I went and read the actual ARC Prize writeup instead of just doomscrolling reaction threads, and it’s a lot more nuanced (and honestly more interesting) than the headline number suggests.

First, what is ARC-AGI-3 actually testing?

It’s not a trivia benchmark. It’s a set of novel, turn-based game environments where the model gets zero instructions. No rulebook, no “here’s your goal.” It has to explore the environment, figure out the mechanics on its own, set its own goal, and then plan a path to it, adjusting as it learns more. Basically the four things intelligence needs to do in the real world: explore, model, set goals, plan and execute.

Humans solve 100% of these environments. That’s the baseline. The question ARC-AGI-3 asks is: can a model do the same, efficiently, without being spoon-fed?

The two numbers actually mean two different things

Here’s the part most viral posts skipped. Astra was tested under two different harnesses, and they measure two different questions.

The Standard harness is the strict one. It gives the model everything it needs to solve the game, but the model has to decide for itself what notes to carry forward between steps. No hidden memory tricks, same interface for every provider. Under this harness, Astra (max reasoning) scored 62.7% for about $26K. That’s still state of the art, but nowhere near 99.9%.

The Provider Adapter harness lets Astra use OpenAI’s own context-management features under the hood, things like preserving opaque reasoning state between calls and compacting long conversations so it can reuse prior work instead of starting fresh. Under this setup, Astra (high reasoning) hit 99.9% for about $18.8K.

So the “99.9%” everyone’s sharing is real, but it’s Astra playing with its home-court advantages turned on. The apples-to-apples number, the one ARC Prize actually uses to compare across labs, is 62.7%.

The part that actually impressed me more than the score

Two things from the analysis stood out more than the headline percentage:

1. It builds its own shorthand. While playing, Astra started writing itself compact, code-like notes to track game state, like L8: hub q2 (8↓). Lengths: 14=1… — basically inventing its own algebraic notation on the fly to compress what it had learned about a game’s mechanics. Not a full programming language, but a working symbolic model of the world it was dropped into.
2. It’s more action-efficient than humans. ARC Prize ran roughly 500 untrained people through these games to set a human baseline for how many actions it takes to solve each level. Astra beat that human median on 96% of levels, using about 52% fewer actions on average. That’s arguably the bigger deal than the accuracy score, because it means Astra isn’t just brute-forcing its way to a solution, it’s actually learning the mechanics efficiently, closer to how a person figures out a new game.

So is this AGI?

ARC Prize’s own answer: no, and they’re pretty upfront about it. They built ARC-AGI-3 knowing that saturating it wouldn’t count as “proof of AGI.” The environments are still closed, deterministic, and bounded. Real life is messier than any of these games.

What they will say is that Astra represents a real step-function jump in capability, not just a benchmark-gaming trick. It cleared a bar that was specifically designed to separate “pattern matching” from “actually building a model of a new environment and reasoning inside it.”

I’m not deep enough into the ML weeds to say whether this is a genuine inflection point or just really good scaffolding around a strong base model. Probably both, honestly. But watching an LLM invent its own notation to track state in a game it’s never seen before, that’s the kind of thing that makes the “just autocomplete” argument feel a little thinner every year.