Gemini 4 Carbon now tops MathArena with 85.4%. GPT-6.1 Sol scores 80.6% for a fifteenth of the price. The metric that should drive your model choices is cost per correct answer, not rank.
My position is simple: a leaderboard ranks answers, not invoices. The moment you put a model into production, the number that matters is not who is first. It is what each correct answer costs you, for each type of task, with a hard limit on how long the model is allowed to run. Companies that pick their default model from the top of a benchmark table are paying a premium for a few points they often cannot even measure.
Written with the help of AI tools; ideas, sources and final review by the author.
I have led AI transformation across 13 departments and around 1,600 people. The most common budget mistake I see is not overspending on AI. It is spending on the wrong unit. Teams compare models the way they compare athletes, by ranking. Finance compares them the way it should, by unit cost. The two rarely talk. This week’s MathArena results show exactly why they need to.
On November 14, The Decoder reported that Google’s Gemini 4 Carbon had taken the top spot on MathArena, the math benchmark that evaluates models on fresh competition problems. It is the first non-OpenAI model in a while to lead the board. The top of the table now looks like this, with the reasoning settings used in testing:
Gemini 4 Carbon (high), Google: 85.4% expected performance, plus or minus 3.0 points, at an expected cost of $15.70 per run.
Claude Fable 5.2 (max), Anthropic: 82.8%, plus or minus 5.3 points, at $40.20 per run.
GPT-6.1 Sol (max), OpenAI: 80.6%, plus or minus 2.7 points, at $1.07 per run. Kimi K3.1 Pro (max), Moonshot AI, the best open model: 54.3%, plus or minus 5.8 points, at $3.70 per run.
Read the headline and you see a new champion. Read the table and you see something else: the winner costs roughly 15 times more than the model in third place, and the runner-up costs roughly 40 times more.
Carbon leads Sol by 4.8 points. But Carbon’s own score carries a margin of plus or minus 3.0 points and Sol’s plus or minus 2.7. The ranking is real as a central estimate, but the distance between first and third is not much larger than the combined uncertainty. Fable’s margin of 5.3 points means its position relative to both is even less settled.
That matters because a ranking turns a small, uncertain gap into a clean ordinal fact: first, second, third. Nobody puts “first, within error bars” on a slide. And once a model is “number one”, it becomes the default choice in architecture documents, procurement requests and vendor conversations, long before anyone checks whether the extra points show up on the company’s own tasks.
Here is the calculation I ask teams to make before choosing a default model. Take the expected cost of a run and divide it by the share of correct answers. You get what it costs to obtain the equivalent of a full set of correct answers.
Gemini 4 Carbon: $15.70 divided by 0.854 gives about $18.38.
Claude Fable 5.2: $40.20 divided by 0.828 gives about $48.55.
GPT-6.1 Sol: $1.07 divided by 0.806 gives about $1.33.
Kimi K3.1 Pro: $3.70 divided by 0.543 gives about $6.81.
On this benchmark, the open model that sits more than 30 points behind is still almost three times cheaper per correct answer than the leader, and the third-placed model is close to 14 times cheaper. Now look at the margin: going from Sol to Carbon adds 4.8 points for an extra $14.63 per run. That is about $3 for each additional point of accuracy.
Sometimes that is a bargain. If one wrong answer means a mispriced contract, a broken production deployment or a regulatory filing that has to be redone, $3 per point is nothing. But for the bulk of enterprise work, such as summarizing, classifying, drafting, extracting fields or routing tickets, nobody would pay 15 times the price for a few points if the decision were written down in those terms.
These are benchmark numbers on competition math, not your workload. That is precisely the point. You do not know your own cost per correct answer until you measure it on your own tasks.
The most interesting line in the report is not the score. According to The Decoder, Carbon’s most common failure is exceeding the time limit. The model almost never gives up on a task. In one case, it launched a script that ran for more than eleven hours without checking on its progress.
In a benchmark, that costs a failed problem. In a company, an agent that refuses to give up is a running meter. Persistence is a great quality in an analyst who reports back every hour. It is a liability in software that can call tools, spin up compute and keep going overnight while nobody is watching. The more capable and autonomous models become, the more the bill depends on when they stop, not only on what they cost per token.
This is why I insist that every agent in production gets three limits before it gets access to anything: a time budget, a cost budget per task, and a progress check that forces it to report or stop. These limits are not a sign of distrust. They are the same rules any manager gives a new hire with a corporate card.
First, replace “which model is best” with “which model is best for this task at this price”. Build a short list of your five or six most frequent task types and test two or three models on a sample of real cases from each.
Second, measure cost per correct answer, not cost per token. Token prices hide the real driver: how many tokens a model burns to reach an answer, how often it is wrong, and how long it runs. A model with a higher price per token can still be cheaper per correct answer, and the reverse is just as common.
Third, route by default to the economical model and escalate only where an error is expensive. A cheap, fast default with a premium model for the hard or high-stakes cases will beat a single “best” model on both cost and quality for most organizations.
Fourth, cap the run. Set a maximum duration and a maximum spend per task, and make the agent check its own progress at fixed intervals. An eleven-hour unattended script should be impossible by design, not caught by surprise on next month’s invoice.
Fifth, revisit the choice every quarter. The top of these tables changes constantly. Carbon was released on November 10 and Fable 5.2 on November 9; Sol has been out since September 29. Locking your architecture to whoever is first this week is how you end up paying champion prices for last season’s champion.
Benchmarks are useful. They tell you what is possible. They do not tell you what is worth paying for. Gemini 4 Carbon’s win is genuine progress in math reasoning, and for problems where a single error is costly, a few extra points may justify the price. For everything else, the leaderboard is the wrong instrument.
Ask your team one question before the next model decision: what does a correct answer cost us on this task, and what stops the model when it is not getting there?
If this was useful, follow me on Medium for more on AI strategy, agents and transformation. The Decoder, Matthias Bastian, “Gemini 4 Carbon takes the MathArena crown but costs 15 times more than GPT-6.1 Sol,” November 14, 2026: https://the-decoder.com/gemini-4-carbon-takes-the-matharena-crown-but-costs-15-times-more-than-gpt-6-1-sol/
MathArena, model page for Gemini 4 Carbon: https://matharena.ai/models/gemini_gemini_4_carbon
Cost-per-correct-answer figures computed by the author from the expected performance and expected cost published in the sources above.
Google’s New Math Champion Costs 15x More Than OpenAI’s Model. Stop Buying AI by the Leaderboard. was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.