Self-hosting Kimi K3 on an 8×B300 node costs about 20% more than an 8×B200 setup but resolves 86.4% of tasks on SWEBench Pro, 24 percentage points above both GLM-5.2 and Opus 4.8 at 62.5%, though the …
Anthropic's Claude Fable 5 scored 72% on SWEBench Pro, outperforming OpenAI's GPT 5.5 at 68% in real-world bug-fixing and feature implementation tasks drawn from major open-source repositories. The mo…
Researchers have found that Claude Opus was contaminated on approximately 12% of SWEBench Pro tasks, meaning the model may have encountered those benchmark problems in its training data. The contamina…
Cognition introduced FrontierCode, a new benchmark that evaluates code on mergeability rather than just unit-test passing, with tasks built by open-source maintainers requiring over 40 hours each. The…