{"slug": "carbon-beats-opponents-when-it-thinks-more-than-them-but-not-more-in-total", "title": "Carbon beats opponents when it thinks more than them, but not more in total", "summary": "Analysis of Agent Duel battle data, a system built on OpenCode that records real developer work, found Gemini 4 Carbon outperforms opponents when it produces significantly more output tokens than them, but shows no performance advantage when total output tokens per task are larger. The finding addresses confusion over why Gemini 4 Neon, released today, posts better benchmark scores than Carbon despite Carbon's higher scores, which had been attributed to Carbon being better on long-horizon tasks. The author concludes Carbon appears trained to think longer, but that this does not necessarily translate to better performance on larger tasks, and asks users to download Agent Duel and contribute votes because the dataset may be too small.", "body_md": "# Carbon beats opponents when it thinks more than them, but not more in total\n\nToday, Gemini 4 Neon was released with better benchmark scores than Gemini 4 Carbon. This was confusing because Carbon has higher scores. The explanation given for this was that Carbon is better on long horizion tasks.\n\nI looked into this using data I have from Agent Duel (a battle system build on OpenCode, reflecting real work done by real devs).\n\nIt looks like Carbon actually does a lot better when it uses significantly more output than opponents, but not better when total output tokens per task is larger.\n\n### Output vs opponents.\n\nThis is where Carbon does well.\n\n## Carbon’s own output\n\nThis is where we’d expect to see some effect if Carbon did better on larger tasks, and we see none.\n\nTo be fair, our data might just be to small, please do help us by downloading [Agent Duel](https://github.com/bottomless/agent-duel) and contributing votes\n\n## Conclusion\n\nLooks basically like Carbon was trained to think a lot longer, but this doesn’t necessarily translate to better performance on larger tasks.", "url": "https://wpnews.pro/news/carbon-beats-opponents-when-it-thinks-more-than-them-but-not-more-in-total", "canonical_source": "http://micjm.com/blog/carbon-beats-opponents-when-it-thinks-more-than-them-but-not-more-in-total/", "published_at": "2026-11-18 23:18:00+00:00", "updated_at": "2026-11-18 23:42:48.285695+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-agents"], "entities": ["Gemini 4 Carbon", "Gemini 4 Neon", "Agent Duel", "OpenCode"], "also_reported_by": [], "takeaway": "In Agent Duel data, Gemini 4 Carbon beat opponents when it emitted significantly more output tokens than them, but showed no gain when total output tokens per task were larger, suggesting longer thinking does not by itself improve performance on bigger tasks.", "recommendation": {"advice": "When comparing models, measure output tokens relative to the opponent as well as total tokens per task; Agent Duel data showed Carbon's edge appeared only in the former.", "playbook": "evals-testing", "playbook_url": "https://wpnews.pro/playbooks/evals-testing"}, "alternates": {"html": "https://wpnews.pro/news/carbon-beats-opponents-when-it-thinks-more-than-them-but-not-more-in-total", "markdown": "https://wpnews.pro/news/carbon-beats-opponents-when-it-thinks-more-than-them-but-not-more-in-total.md", "text": "https://wpnews.pro/news/carbon-beats-opponents-when-it-thinks-more-than-them-but-not-more-in-total.txt", "jsonld": "https://wpnews.pro/news/carbon-beats-opponents-when-it-thinks-more-than-them-but-not-more-in-total.jsonld"}}