AWS released aws-bench, an open-source benchmark to evaluate AI agents on real AWS tasks, using disposable AWS accounts and automated verifiers. The benchmark, built on Harbor, supports agents like Cl…
Mercor, a profitable Series C AI data company valued at $10 billion, is hiring a Member of Technical Staff for its Enterprise Platform in San Francisco, offering a salary of $130k–500k/yr. The role in…
Unsloth released Dynamic v3.0 GGUFs for Qwen3.8-27B, claiming more than 10% better top-1% accuracy at the same size compared to every other provider. The new quants, which work with llama.cpp and Unsl…
NVIDIA released SkillEvaluator, an open-source tool that measures how AI agent skills affect performance, and published benchmark results for more than 300 verified skills across over 30 NVIDIA produc…
Cline, an AI coding assistant, is open-sourcing its evaluation framework for open-weight coding agents, revealing that its requests run 20–30% heavier on tokens than the most efficient harnesses. The …
A developer from Google Cloud's DevRel team demonstrates how to design objective evaluations for AI agent skills using open-source frameworks like Inspect AI and Harbor. The investigation uses Gemini …
OpenCode Zen's free stealth model big-pickle resolved 50.8% (63/124) of Scale AI's SWE Atlas Codebase QnA benchmark tasks on 2026-08-11, outperforming all official Mini-SWE-Agent scaffold entries and …
Mercor, a profitable Series C AI data company valued at $10 billion, is hiring a Member of Technical Staff for its Enterprise Evals Platform in San Francisco, offering a salary of $220,000–$425,000 pe…
Mupt AI released self-bench, an open-source tool that automatically converts private, completed pull requests into held-out coding-agent evals, generating validated Harbor datasets with a single CLI c…
JetBrains' PyCharm introduced the Agent Environment Coordinator skill, which provides AI agents with the correct Python interpreter information, boosting task success rates from 68% to 98% on average …
Antigma Labs released Ante, a self-contained coding agent in a single ~15MB Rust binary that runs offline with zero runtime dependencies, achieving 82.7% on Terminal-Bench 2.1 (368/445 trials) using D…
BoundaryBench, a new open-source benchmark from the developer community, measures how much capability coding agents lose when running inside enterprise/NIST-derived hardened sandboxes, with live resul…
A controlled test of the token-saving tool Headroom on 25 senior software engineering tasks using Anthropic's Claude found that while it cut input tokens by 39%, it did not reduce costs due to cache b…
JetBrains tested the Caveman token-compression skill on Claude Code 2.1.200 and found it saves only about 8.5% of output tokens on real agentic tasks, far below the advertised 65%. The skill, which fo…
Letta AI introduced the Trajectory package, a standardized data format for agent experience that reduces token counts by up to 5.6× compared to native session formats, enabling agents to learn across …
The team behind Terminal-Bench and Harbor released Frontier-Bench v0.1, a continuous benchmark with 74 tasks across 7 domains that measures agent abilities at the frontier, with the best models achiev…
Letta released the open-source trajectory package on July 24, 2026, a standard format for recording coding-agent sessions across harnesses including Claude Code, Codex, and Letta Code. The package red…
A developer built nano-harness, a coding agent in ~970 lines of Python that scored 59.6% (53/89) on the full Terminal-Bench 2.0 suite using Claude Opus 4.8. The harness uses only three tools—bash, rea…
IntelligenceLab requests Hugging Face to add Long-Horizon-Terminal-Bench (LHTB) to the Benchmark allow-list. LHTB is a 46-task benchmark that tests LLM agents on long-horizon terminal tasks using hidd…
JetBrains AI tested the rtk (Rust Token Killer) CLI proxy, which claims to cut agent tokens by 60–90%, and found that most agent bytes never touch the hook, with measured token savings far below adver…