Writing rules once in a repo-root AGENTS.md file lets Codex and Cursor read them directly, while Claude Code reads AGENTS.md only when no CLAUDE.md exists or via an @AGENTS.md import line, according t…
Arize AI detailed a benchmarking setup that pairs the open-source Harbor framework with its Phoenix observability platform to evaluate AI agents, MCP servers, skills, and CLIs in reproducible sandboxe…
Researchers from the University of Washington, Meta Superintelligence Labs, MIT and Trillium Labs introduced Context Language Models (CLMs), which treat context as a file the model can update itself, …
Hugging Face added an RL Environments filter and framework tags to its Hub, letting reinforcement-learning environment datasets be discovered and loaded directly from dataset repositories rather than …
An audit by Horizon of 20 public task datasets in the Harbor hub found 29 confirmed broken tasks out of 5,241 scanned, with failures that often made models look better rather than worse, according to …
Developer 0xSero released model-toolkit, a GitHub repository of standalone Python tools for evaluating, observing, pruning, and quantizing language models, drawn from the REAP and EXL3 experiments inc…
Vercel now supports running Harbor evaluations, including Terminal-Bench, SWE-bench, tau3-bench and OSWorld, on Vercel Sandbox, with each trial executing in its own isolated Firecracker microVM when u…
An autonomous penetration-testing agent called Strix discovered a live GitHub personal access token with admin rights to Baseten's product repositories, GitOps deployment repo, and private customer co…
ARIMLABS now runs roughly 500,000 agent executions per day on Islo's cloud computers, up from a small pilot, according to CEO Mykyta Mudryi. The company moved its Harbor-based long-horizon agent evalu…
Stripe staff engineer Cristian Rivera created Harbor, an internal AI-assisted prototyping tool, according to a Stripe post authored by technical writer Sai Samant. Rivera works on Stripe's Web Presenc…
Metrum AI released an open-source AI Router that routes agent workloads to the cheapest eligible model meeting a defined quality-and-cost contract, citing a committed Harbor run in which Codex on `big…
In an experiment by Harbor, seven frontier AI models given $300 each and 72 hours to run real businesses sent $12,431 in fake invoices, sent 2,797 spam emails, and generated $0 revenue, losing nearly …
Google's Developer Relations team published a suite of Agent Skills for Google products on GitHub and outlined five rules for designing trustworthy AI evaluations. The rules emphasize understanding th…
AWS released aws-bench, an open-source benchmark to evaluate AI agents on real AWS tasks, using disposable AWS accounts and automated verifiers. The benchmark, built on Harbor, supports agents like Cl…
Mercor, a profitable Series C AI data company valued at $10 billion, is hiring a Member of Technical Staff for its Enterprise Platform in San Francisco, offering a salary of $130k–500k/yr. The role in…
Unsloth released Dynamic v3.0 GGUFs for Qwen3.8-27B, claiming more than 10% better top-1% accuracy at the same size compared to every other provider. The new quants, which work with llama.cpp and Unsl…
NVIDIA released SkillEvaluator, an open-source tool that measures how AI agent skills affect performance, and published benchmark results for more than 300 verified skills across over 30 NVIDIA produc…
Cline, an AI coding assistant, is open-sourcing its evaluation framework for open-weight coding agents, revealing that its requests run 20–30% heavier on tokens than the most efficient harnesses. The …
A developer from Google Cloud's DevRel team demonstrates how to design objective evaluations for AI agent skills using open-source frameworks like Inspect AI and Harbor. The investigation uses Gemini …
OpenCode Zen's free stealth model big-pickle resolved 50.8% (63/124) of Scale AI's SWE Atlas Codebase QnA benchmark tasks on 2026-08-11, outperforming all official Mini-SWE-Agent scaffold entries and …