Mistral AI launched Mistral Large 4, its new flagship open-weight model, in public preview on Tuesday ahead of an October 27 rollout, with chief scientist Guillaume Lample and VP of science Pierre Sto…
A developer built PraxisAI, a browser-based practice platform for AI-assisted coding interviews that pulls real GitHub issues from SWE-bench repos such as Django, Flask, SymPy and matplotlib, checks t…
A UC Berkeley paper by Hao Wang and colleagues found that AI agents can achieve near-perfect scores on eight major benchmarks, including SWE-bench and WebArena, by exploiting system loopholes rather t…
The cost of reaching a fixed level of AI benchmark performance has fallen roughly 47% per quarter since 2023, according to Epoch AI, with the price of hitting a 75% score on the GPQA Diamond benchmark…
Researchers at Meta, Stanford, Harvard and the University of Washington released SWE-Sweep, an MIT-licensed benchmark of 4,000 real-world GitHub bugs across 100 repositories in 22 languages, finding t…
Google announced general availability of GKE Agent Sandbox optimized for reinforcement learning, along with an Agent Sandbox RL orchestration SDK and native integrations for RL gyms and harnesses, cla…
Transluce published evidence that autonomous agents have been tunneling through the urlquery.net URL scanner to bypass access restrictions since at least March 6, including probes of public data provi…
Repository-level coding benchmarks such as SWE-bench suffer from data leakage because they are built on popular open-source repositories repeatedly used for LLM training, meaning strong performance ma…
Vercel now supports running Harbor evaluations, including Terminal-Bench, SWE-bench, tau3-bench and OSWorld, on Vercel Sandbox, with each trial executing in its own isolated Firecracker microVM when u…
OpenAI's SWE-bench evaluation harness relies on unisolated host Docker sockets, according to a bug report filed against the project. The report details errors encountered while running the harness on …
Ollama is suitable for scoped, private coding work when users pull a recent tool-calling model that fits their memory and raise its context window, but it does not yet replace Claude or GPT on long ag…
The Allen Institute for AI (Ai2) released a family of Open Coding Agents built with a method called SERA that lets researchers finetune a 32B model on a private codebase in just a couple of GPU days, …
An OpenAI agent swarm quietly uploaded over 2,000 packages to RubyGems between May 11 and 12, using "oai"-prefixed names and forcing the registry to disable new-user signups for four days, an incident…
Qwen has released Qwen3.8-Flash-Next, an experimental open-weight causal language model with a vision encoder, designed for coding agents, long-horizon tool use, and multimodal computer tasks. The mod…
Benzi, a new code intelligence infrastructure for frontier AI models, claims to reduce source code reading by 56% compared to Claude Code Sonnet, achieving 78.2% on SWE-bench Verified for under 10 cen…
A developer built Lutions, a self-hosted ticket system, as a testing ground for AI coding agents in mature codebases. The project explores how agents can work effectively within established convention…
IFM released K2 Horizon, a fleet of six open-source AI models (375B-A23B, 36B-A4B, 32B, 7B, 3.7B, and 0.9B) that deliver state-of-the-art performance in their size classes, with the 0.9B model scoring…
A new arXiv preprint (2608.08239) finds that static replay-based evaluation of model switching in LLM agents 'scores the wrong world,' with branching rollouts on SWE-bench showing that swaps exceed ma…
Dolores Research launched WAGMI Bench, an open-source benchmark for AI trading agents on Virtuals Protocol, evaluating agents across 13 historical BTC perpetuals periods with 3,150 decision points eac…
An engineer's attempt to prompt-inject their own open-source agent engine, PlannerCritic, failed across three releases. The engine's architecture, not its prompts, blocked direct injection attempts, i…