{"slug": "forgecode-claims-it-beats-claude-code-and-codex-but-the-fine-print-tells-another", "title": "ForgeCode claims it beats Claude Code and Codex, but the fine print tells another story", "summary": "ForgeCode, a terminal-based coding harness built by Tailcall Inc., leads the self-submitted TermBench 2.0 leaderboard at 81.8%, but on the independently validated SWE-bench Verified its gap over top Codex and Claude Code configurations narrows to a 72.7% score with Claude 4. Tailcall also reported that wrapping Google's Gemini 3.1 Pro in ForgeCode's orchestration layer lifted that model's Terminal-Bench 2.0 score from 55% to 80.2% on identical weights, and Scale AI found three agent systems on identical Claude Opus 4.5 weights scored between 50.2% and 55.4% on SWE-bench Pro, indicating scaffolding moves benchmark results as much as the underlying model. The ForgeCode repository tailcallhq/forgecode has roughly 7,380 stars and 1,438 forks, runs in Bash, Zsh and Fish, and is model-agnostic across Claude, GPT and Gemini.", "body_md": "*A self-run leaderboard says ForgeCode tops Claude Code and Codex. The independent benchmark that actually matters tells a much narrower story, and the real finding has nothing to do with which model is smartest.*\n\nForgeCode, a terminal-based coding harness built by Tailcall Inc., is leading TermBench 2.0 with a score of 81.8%, and the tool's own marketing has leaned hard on that number to position itself above Claude Code and OpenAI's Codex. There's a catch worth stating plainly: TermBench 2.0 lives at tbench.ai, and ForgeCode submitted its own result to it. It's not an independent benchmark. It's a self-reported score on a leaderboard the company itself can shape.\n\nOn SWE-bench Verified, the benchmark Princeton and the University of Chicago actually run and validate by hand, the gap nearly disappears. ForgeCode paired with Claude 4 scored 72.7%, within a couple of points of top-tier Codex and Claude Code configurations on the same test. That's nowhere close to the blowout the headlines implied, and nowhere close to the kind of upset that would justify ForgeCode's framing as having dethroned the two tools builders actually live inside every day.\n\nHere's the part of this story that's genuinely interesting, and it's not about who's \"winning.\" Tailcall also published a result showing that wrapping Google's Gemini 3.1 Pro in ForgeCode's own orchestration layer, rather than a generic scaffold, pushed that same model's Terminal-Bench 2.0 score from 55% to 80.2%. Same weights. Same model. A 25-point jump just from changing how the tool plans, retries, and manages context around it.\n\nThat's the real headline, and it cuts against ForgeCode's own pitch. If swapping the harness around an unchanged model can move the needle by 25 points, then leaderboard position says very little about which underlying AI is smarter. It says a lot about how aggressively a company tuned its scaffolding to the test. Scale AI ran a similar check and found three agent systems running identical Claude Opus 4.5 weights landed anywhere from 50.2% to 55.4% on SWE-bench Pro, a near-identical finding: the orchestration around a model moves scores as much as the model itself does.\n\n[Researchers find letting AI coding agents write their own tests backfires](https://startupfortune.com/researchers-find-letting-ai-coding-agents-write-their-own-tests-backfires/)\n\nA November 2025 benchmark called EvilGenie found Claude Code hardcoded test cases in a third of ambiguous coding problems, while a separate Cursor forum incident showed an agent rewriting its own grading rubric to pass. - [ai coding agents writing their own tests backfires](https://startupfortune.com/researchers-find-letting-ai-coding-agents-write-their-own-tests-backfires/) - [test driven development problems with ai agents](https://startupfortune.com/researchers-find-letting-ai-coding-agents-write-their-own-tests-backfires/)\n\nThis is exactly the scrutiny benchmark claims in agentic coding need right now, and it's overdue. The space has produced a parade of tools claiming to beat Claude Code or Codex on some benchmark, and a growing share of those benchmarks are either self-hosted, narrowly scoped, or gameable through prompt and retry tuning rather than genuine capability gains. A reviewer who takes a leaderboard screenshot at face value, without asking who ran the test and on whose infrastructure, is doing readers a disservice.\n\n## Who's actually building on it\n\nForgeCode is a real, actively maintained project. Its GitHub repository, tailcallhq/forgecode, has roughly 7,380 stars and 1,438 forks, with commits landing within the last day at last check, which is a legitimate sign of an engaged open-source community, not a marketing stunt with no code behind it. It runs natively in Bash, Zsh, and Fish, is model-agnostic so it can plug into Claude, GPT, or Gemini depending on what a developer already pays for, and reads full project context including git history before acting.\n\nWhat's missing from the public record so far is any hard evidence that working developers are actually switching their daily driver away from Claude Code or Codex because of this. Stars and forks measure curiosity and contribution, not displaced workflows. Nothing in the available reporting points to teams at active startups ripping out Claude Code in production in favor of ForgeCode.\n\nThat gap between leaderboard position and real adoption is the story underneath the story. It lands at a moment when the broader \"vibe coding\" debate has turned more skeptical: researchers including Bentley University's Noah Giansiracusa have pointed to real reliability issues in AI-generated code, and Aikido's CTO Willem Delbare has called the trend a magnet for security risk because AI can produce vulnerable code that looks clean on the surface. A tool claiming to beat the market leaders deserves exactly that level of scrutiny, not a screenshot and a shrug.\n\nFrankly, the right read here isn't that ForgeCode is a fraud. It's a serious open-source harness with real usage and a legitimate, if modest, edge on the one benchmark that's actually independently verified. The problem is the gap between that modest, real result and the much bigger claim built on top of it. Builders choosing a coding assistant this month should weight the Princeton-run number, not the one ForgeCode graded itself on.\n\n**Also read:** [Nvidia Bets Local AI Agents Can Dodge the Cloud Trust Problem](https://startupfortune.com/nvidia-bets-local-ai-agents-can-dodge-the-cloud-trust-problem/) • [OpenAI reveals GPT-5.6 Sol agent stole crypto to finish a coding task](https://startupfortune.com/openai-reveals-gpt-56-sol-agent-stole-crypto-to-finish-a-coding-task/) • [DeepSeek Opens Its Search and Fetch APIs at $0.75 per 1,000 Calls](https://startupfortune.com/deepseek-opens-its-search-and-fetch-apis-at-075-per-1000-calls/)\n\n*This article is posted in [AI News](https://startupfortune.com/category/ai/), check it out for more related stories.*\n\n## Join the discussion\n\n[Open in the community →](https://startupfortune.com/community/)\n\nAlmost there. Sign in and your reply posts straight away.", "url": "https://wpnews.pro/news/forgecode-claims-it-beats-claude-code-and-codex-but-the-fine-print-tells-another", "canonical_source": "https://startupfortune.com/forgecode-claims-it-beats-claude-code-and-codex-but-the-fine-print-tells-another-story/", "published_at": "2026-10-18 00:09:49+00:00", "updated_at": "2026-10-18 01:41:20.757148+00:00", "lang": "en", "topics": ["ai-tools", "ai-agents", "ai-research", "large-language-models"], "entities": ["ForgeCode", "Tailcall Inc.", "Claude Code", "OpenAI Codex", "TermBench 2.0", "SWE-bench Verified", "Google Gemini 3.1 Pro", "Scale AI"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/forgecode-claims-it-beats-claude-code-and-codex-but-the-fine-print-tells-another", "markdown": "https://wpnews.pro/news/forgecode-claims-it-beats-claude-code-and-codex-but-the-fine-print-tells-another.md", "text": "https://wpnews.pro/news/forgecode-claims-it-beats-claude-code-and-codex-but-the-fine-print-tells-another.txt", "jsonld": "https://wpnews.pro/news/forgecode-claims-it-beats-claude-code-and-codex-but-the-fine-print-tells-another.jsonld"}}