cd /news/large-language-models/carbon-beats-opponents-when-it-think… · home › topics › large-language-models › article
[ARTICLE · art-172200] src=micjm.com ↗ pub= topic=large-language-models verified=true sentiment=· neutral

Carbon beats opponents when it thinks more than them, but not more in total

Analysis of Agent Duel battle data, a system built on OpenCode that records real developer work, found Gemini 4 Carbon outperforms opponents when it produces significantly more output tokens than them, but shows no performance advantage when total output tokens per task are larger. The finding addresses confusion over why Gemini 4 Neon, released today, posts better benchmark scores than Carbon despite Carbon's higher scores, which had been attributed to Carbon being better on long-horizon tasks. The author concludes Carbon appears trained to think longer, but that this does not necessarily translate to better performance on larger tasks, and asks users to download Agent Duel and contribute votes because the dataset may be too small.

recommendation

When comparing models, measure output tokens relative to the opponent as well as total tokens per task; Agent Duel data showed Carbon's edge appeared only in the former.

more in playbook: Evals and testing for LLM apps
read1 min views3 publishedNov 18, 2026

Today, Gemini 4 Neon was released with better benchmark scores than Gemini 4 Carbon. This was confusing because Carbon has higher scores. The explanation given for this was that Carbon is better on long horizion tasks.

I looked into this using data I have from Agent Duel (a battle system build on OpenCode, reflecting real work done by real devs).

It looks like Carbon actually does a lot better when it uses significantly more output than opponents, but not better when total output tokens per task is larger.

Output vs opponents.

This is where Carbon does well.

Carbon’s own output #

This is where we’d expect to see some effect if Carbon did better on larger tasks, and we see none.

To be fair, our data might just be to small, please do help us by down Agent Duel and contributing votes

Conclusion #

Looks basically like Carbon was trained to think a lot longer, but this doesn’t necessarily translate to better performance on larger tasks.

── more in #large-language-models 4 stories · sorted by recency
── more on @gemini 4 carbon 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/carbon-beats-opponen…] indexed:0 read:1min 2026-11-18 · —