DeepSeek shipped the public beta of V4 Flash on July 31 with no architecture changes — same 284B MoE, 1M-token context — but a retraining-only round boosted Terminal Bench 25.8 points to 82.7, Toolath…
METR contributor Ivan Bercovich argues that most AI benchmarks are flawed and that building good ones requires nuanced understanding, drawing on 18 months of experience with Terminal Bench. Good tasks…
XAI's Grok 4.5 scored 83% on Terminal Bench, outperforming many costlier models in multi-step coding tasks, while Anthropic's Claude Opus 4.8 excels in complex reasoning and high-stakes professional w…
A Hacker News user asks the community for recommendations on benchmarks to evaluate different agent harnesses, noting that Terminal Bench does not align with their experience.…
Terminal Bench completion scores and per-attempt costs now appear directly in the Kilo CLI and VS Code extension model details panel, providing real benchmark data where developers choose models. The …