GDPx Benchmark Explained: 11 Domains Where the Best AI Scores 12%

SemiAnalysis's GDPx benchmark put 13 frontier models through 206 agentic tasks in 11 high-value domains. Claude Opus 5.5 led at 12%. The domains, the scores, the limits and what it means for agents.

This explainer is based on SemiAnalysis’s launch post for GDPx and independent coverage of its results. I have not run the benchmark, most of its tasks are private, and every score below is SemiAnalysis’s own published figure.

The GDPx benchmark has a blunt headline: the best AI model in the world scores 12%. SemiAnalysis published it on 2 November 2026, put 13 frontier models through 206 long agentic tasks in 11 high-value professional fields, and found that Claude Opus 5.5 led at 12.0%, with every other model below it.

That number is easy to misread in both directions. It does not mean AI agents fail 88% of real work. It also does not mean they are close to replacing a chip designer or a quant. What it does show, more clearly than any benchmark before it, is where autonomous agents break when the work gets long, specialised and messy. That is the useful part, and it is what this guide focuses on.

Table of Contents

GDPx benchmark scores chart for frontier models
A rising line is the hope. On the GDPx benchmark the best frontier model still finishes only 12% of long expert tasks.

What the GDPx Benchmark Is

SemiAnalysis, the semiconductor and AI research firm founded by Dylan Patel, built GDPx to measure how well AI agents do the work its analysts expect to generate the most AI revenue. The launch post was written by Max Kan, Andrew Megalaa and Dylan Patel, and it describes the aim plainly: the world’s best benchmark for general agentic capability.

Each task is long and runs inside a simulated work environment, rather than being a single question with a single answer. The tasks were built with 14 specialist data partners, mostly startups that work in each field. That design choice is the whole point. GDPx is meant to look like a week of real work, not a quiz.

Scoring is pass@1 averaged over three attempts, and each domain is weighted equally in the headline number. Tasks mix binary pass or fail checks with partial credit depending on the field. SemiAnalysis says the GDPx benchmark will evolve, with new versions every three to six months that add feedback and partners.

The 11 GDPx Benchmark Domains

Here is the full list of domains, partners and task counts, as compiled from SemiAnalysis’s domains page.

Domain Partner(s) Tasks
Chip design Aria, Arcverex, Ergodex 34
Finance Living Context 24
Materials science ValenceForge, Emulated 23
Hardware design Beautiful Machines 21
Legal Chakra 20
Cybersecurity Arithmetic 18
Alignment Fleet 18
Software engineering Datacurve 15
Biology LatchBio 13
Recursive self-improvement Extrapolate 10
Quant trading Third Moment 10

The list is deliberately narrow. SemiAnalysis says it chose to leave out industries such as government, real estate and insurance. So the GDPx benchmark is a bet on which fields matter most for frontier AI, not a sample of the whole economy weighted by jobs or GDP.

Notice too how uneven the task counts are. Chip design has 34 tasks and quant trading has 10, but because domains are weighted equally, each quant task moves the headline score more than three times as much as each chip task. Keep that in mind before you read too much into a one-point gap.

The GDPx Benchmark Leaderboard

These are the launch scores for all 13 models.

Rank Model GDPx score
1 Claude Opus 5.5 12.0%
2 Gemini 4 Carbon 10.9%
3 GPT-6 Astra 9.6%
4 Muse Wonder Full 9.0%
5 Claude Fable 5.1 8.5%
6 GPT 6.1 Sol 8.4%
7 Claude Sonnet 5.5 8.2%
8 Gemini 4 Argon 7.3%
9 Muse Wonder 6.8%
10 DeepSeek 4.1 Flash 1017 5.4%
11 GPT-6 Luna 5.1%
12 Grok 4.7 3.9%
13 GLM 5.3 2.7%

Two things jump out. Opus 5.5 beat Anthropic’s more expensive Fable 5.1, which is a reminder that the most powerful model on paper is not always the best agent in practice. And the spread from top to bottom is only about nine points, which tells you this is a hard test for everyone, not an easy one for a few.

Opus 5.5’s lead came at a price. Coverage of the results reports an average cost of $51.41 per task and around 548,000 output tokens. SemiAnalysis’s cost chart shows GPT-6 Luna and DeepSeek fighting for the cheap end, Opus at the expensive end and OpenAI’s models in the middle.

Claude Fable 5.2 arrived a week after the launch, so it is not on this table. If you are tracking it, our Claude Fable 5.1 guide covers the version that was tested.

What the GDPx Benchmark Says About Each Lab

Read lab by lab, the GDPx benchmark tells a more nuanced story than the top line.

Anthropic has three models in the top seven. Opus 5.5 leads, Fable 5.1 sits fifth at 8.5% and Sonnet 5.5 is seventh at 8.2%, only a fraction behind its far more expensive sibling. For teams already on Claude, that small gap between Sonnet and Fable is the most practical finding in the whole table. On long agentic work, the cheaper model gives up very little.

Google takes second with Gemini 4 Carbon at 10.9%, its strongest showing on any agent benchmark this year. The catch is availability. As of mid-November, release trackers reported no full public release or price for Carbon, so most developers cannot use the model that scored so well. Gemini 4 Argon, which you can buy, sits at 7.3%.

OpenAI lands in the middle. GPT-6 Astra is third at 9.6% and GPT 6.1 Sol sixth at 8.4%. GPT-6 Luna, at 5.1%, is the budget option and one of the cheapest models on the cost chart. If you are already paying for Astra, these results do not give you a strong reason to switch. Our GPT-6 Astra guide covers where it does lead.

Meta surprised a few people. Muse Wonder Full scored 9.0%, fourth overall and ahead of Fable 5.1. The standard Muse Wonder sits ninth at 6.8%.

Open-weight models trail, as expected on a test this hard. DeepSeek 4.1 Flash 1017 is the best of them at 5.4%, which is a decent result for a Flash-tier model you can download. GLM 5.3 is last at 2.7%. Kimi K3.1 Pro, the strongest downloadable model on other indices, was not in the launch table.

xAI had a poor debut. Grok 4.7 scored 3.9%, twelfth of thirteen, which matches the uneven picture we described in our Grok 4.7 timeline.

Why Scores Are So Low

Finance and legal explain a lot. SemiAnalysis said basically every model scores exactly zero on them under binary grading. On finance, every model fails every task outright, so the leaderboard uses partial credit there instead.

That is less damning than it sounds. A finance task in the GDPx benchmark is seeded from a real deal, and one public quality-of-earnings task called Bellwood-QoE has 11 separate checks. Getting ten right and one wrong is a fail under binary grading. In real life, a junior analyst who got ten of eleven checks right would get feedback, not fired.

Safety systems also pull scores down. Classifiers on some models, including Opus 5.5 and GPT-6 Astra, block parts of the cyber, biology and alignment tasks. SemiAnalysis publishes a second ranking that excludes those three domains for that reason, and anyone comparing models for commercial work should look at that version.

The deeper reason is length. These are long, multi-step tasks in specialised environments. Every additional step is another chance to make a small error that compounds. Agents that are 95% reliable per step still fail most twenty-step tasks.

Do the arithmetic and the low scores stop looking strange. If each step succeeds 95% of the time, the chance of getting twenty steps in a row right is about 36%. At thirty steps it falls to about 21%. Many GDPx tasks run far longer than that, through tools, files and simulated systems, and a single wrong assumption early on can quietly poison everything after it. That is also why the same model can look brilliant in a short chat and fragile on the GDPx benchmark: the test is measuring stamina as much as intelligence.

One more factor deserves a mention. The tasks were written by domain specialists who know where the traps are. A chip design task from people who design chips will include the edge case a generalist would never think to test, and models trained mostly on public text have seen very few examples of that kind of work.

GDPx Benchmark vs OpenAI’s GDPval

GDPx is partly a response to GDPval, the professional-work evaluation OpenAI released in September 2025, which SemiAnalysis says remains the most-reported general benchmark on model release cards. SemiAnalysis’s criticism is specific.

It argues that GDPval’s grading is entirely rubric-based and often noisy. Human expert agreement was just 71% on its gold subset, and the automated grader did worse. Its biggest complaint is that GDPval lacks domain-specific simulated environments, so tasks end up too tidy and scoring too soft.

GDPval (OpenAI) GDPx benchmark (SemiAnalysis)
Released September 2025 2 November 2026
Task style Professional deliverables graded by rubric Long agentic tasks in simulated environments
Grading Rubric-based Binary checks and partial credit by domain
Coverage Broad set of occupations 11 chosen high-value domains
Top score at launch Varies by version 12.0%

Both approaches have value. GDPval is better at telling you whether a model can produce a reasonable first draft. The GDPx benchmark is better at telling you whether an agent can finish a long job without supervision. Those are different questions, and most businesses need the answer to the first one far more often than the second.

Limitations of the GDPx Benchmark

SemiAnalysis is candid about some of these, and independent coverage has flagged the rest.

The task makers sell training data. Partners are companies that sell data to AI labs. That gives them expertise. It also gives them a commercial interest in showing that models need more of their data.

Most tasks are private. This prevents labs from tuning against the test, which is good. It also means outsiders cannot audit the tasks. Only 14 are public, with 168 downloadable transcripts.

Sample sizes are small. Several findings rest on just 12 to 25 runs per task.

Each model ran in its own scaffold. That makes the results a comparison of products as much as models. A different scaffold might move scores noticeably.

Benchmarks age. Labs will learn the shape of these tasks over time, which is why SemiAnalysis plans regular new versions.

None of this makes the GDPx benchmark useless. It does mean that the ranking order matters more than the exact percentages, and that a model one point ahead is not meaningfully better.

The One Lesson for Teams Building Agents

Here is the lesson I would take to any team building AI agents: design for checkpoints, not autonomy. The GDPx benchmark shows that today’s best models cannot carry a long professional task to the end on their own. It also shows, through partial credit, that they get a lot of the way there.

That combination points to one architecture. Break long jobs into stages. Let the agent do each stage, then have a person or a cheap verification step check the output before the next stage begins. A model that gets ten of eleven checks right is a poor autonomous worker and an excellent assistant.

It also tells you where to spend money. Paying four times more for the top model buys a few points on GDPx. Building good verification between steps usually buys far more. Our guide to building an AI agent from scratch shows how to add those checkpoints, and LLM evaluation tools covers how to measure whether they work.

How to Use the GDPx Benchmark in Model Decisions

If you are Start with Why Link
Choosing a model for long agent tasks The ranking that excludes cyber, bio and alignment Removes the effect of safety classifiers on commercial comparisons SemiAnalysis launch post
Working in one of the 11 domains Your domain’s scores, not the headline Equal weighting hides big differences by field Let’s Data Science breakdown
Watching costs The cost chart alongside the score The leader spent about $51 per task Cut LLM API costs
Building your own evals The 14 public tasks and transcripts Shows what a realistic long task looks like LLM evaluation tools

Read the public transcripts if you have an hour. Watching a frontier agent work through a real quality-of-earnings check, and seeing exactly where it slips, teaches you more about agent design than any summary statistic.

Related on this site: for more context, see Muse Wonder and Claude Opus 5.5.

FAQ About the GDPx Benchmark

What is the GDPx benchmark?
It is an agentic AI benchmark from SemiAnalysis, published on 2 November 2026, that tests models on 206 long tasks across 11 high-value professional domains, built with 14 specialist partners.

Which model leads the GDPx benchmark?
Claude Opus 5.5, at 12.0%. Gemini 4 Carbon is second at 10.9% and GPT-6 Astra third at 9.6%.

Does a 12% score mean AI fails 88% of real work?
No. The tasks are hand-picked to be hard and are not a random sample of jobs. Binary grading also marks a task as failed even when most of the work is right.

Why do models score zero on finance and legal?
Under strict binary grading, every model failed every finance task. The leaderboard reports partial credit for finance instead, which gives a fairer picture of how close models get.

Can I run the GDPx benchmark myself?
Only partly. Most tasks are private to stop labs tuning against them. Fourteen public tasks with 168 transcripts are available for anyone to inspect.

How often will the GDPx benchmark be updated?
SemiAnalysis plans new versions every three to six months, adding feedback from industry and new partner organisations.

Verdict

Treat the GDPx benchmark as the most honest stress test of autonomous agents published so far, and as a guide to where to put human checkpoints rather than a verdict on AI’s usefulness. Its order matters: Opus 5.5 leads, Gemini 4 Carbon and GPT-6 Astra follow, and the gap between any two neighbours is small.

The caveat is that the tasks are private, made by companies that sell training data, and scored on small samples with each model in its own scaffold. A one-point lead means little, and the headline number says nothing about how useful these models are with a person in the loop.

Your next step: pick the GDPx domain closest to your work, read its public transcripts, and note the first step where the agent goes wrong. Then put a verification check at that exact point in your own workflow.

Sources and Further Reading

For related reading here, compare the leading models in AI models compared, read our breakdowns of GPT-6 Astra and Grok 4.7, and see how to build reliable agents in build an AI agent from scratch.