Anthropic released Claude Fable 5.1, a refinement of its Fable 5 model that improves agentic performance and readability while cutting cached prompt read costs by about 75%, resulting in roughly 25% c…
Researchers introduced Hack-Verifiable Terminal Bench (HVTB), an adaptation of the hack-verifiable environments (HVE) methodology to Terminal Bench, a benchmark of real-world terminal and coding tasks…
Cline, an AI coding assistant, is open-sourcing its evaluation framework for open-weight coding agents, revealing that its requests run 20–30% heavier on tokens than the most efficient harnesses. The …
DeepSeek V4 Flash scored 82.7 on Terminal Bench 2.1, beating the previous-generation V4-Pro-Preview by 14.7 percentage points, and Gemini 3.7 Flash shipped with a 26% jump in coding capability, signal…
Alibaba released Qwen 3.8 Max, a 2.4 trillion parameter open-weight AI model with 95 billion active parameters, set to become the largest open-weight model once weights are open-sourced about a week a…
DeepSeek shipped the public beta of V4 Flash on July 31 with no architecture changes — same 284B MoE, 1M-token context — but a retraining-only round boosted Terminal Bench 25.8 points to 82.7, Toolath…
METR contributor Ivan Bercovich argues that most AI benchmarks are flawed and that building good ones requires nuanced understanding, drawing on 18 months of experience with Terminal Bench. Good tasks…
XAI's Grok 4.5 scored 83% on Terminal Bench, outperforming many costlier models in multi-step coding tasks, while Anthropic's Claude Opus 4.8 excels in complex reasoning and high-stakes professional w…
A Hacker News user asks the community for recommendations on benchmarks to evaluate different agent harnesses, noting that Terminal Bench does not align with their experience.…
Terminal Bench completion scores and per-attempt costs now appear directly in the Kilo CLI and VS Code extension model details panel, providing real benchmark data where developers choose models. The …