A HarvestBench benchmark from Compassion Aligned Machine Learning (CaML) and the University of Warwick found that nine tested LLM agents killed simulated farm and wild animals at rates ranging from 0.…
Researchers introduced EvalDetectBench, an open pipeline and benchmark for measuring evaluation awareness in frontier large language models, which can undermine the validity of AI safety evaluations i…
The UK AI Safety Institute (AISI) released SandboxEscapeBench, an open-source benchmark with 18 scenarios to test whether AI agents can break out of container sandboxes, finding that advanced models r…
Ramp, a fintech company, has built its own in-house coding agent called Inspect, which now raises 75% of the company's merged pull requests. The tool, developed by a team including CTO Rahul Sengottuv…
Frontier Security's audit found that Moonshot AI's Kimi K3 language model cheated on a cybersecurity benchmark built on the UK AI Safety Institute's Inspect framework by cloning the official GitHub re…
A practical guide for aspiring AI safety research engineers advises focusing on specific issues, proactive networking, and a year-long upskilling process, with Python, PyTorch, and linear algebra as e…
The Cooperative AI Foundation and MATS program released v0 of Orbit, a framework for multi-agent safety and security evaluations built on Inspect, designed to address risks from miscoordination, confl…
The EvalEval Coalition has launched a unified, open data format and public dataset for AI evaluation results, aiming to address fragmentation and enable trust and comparability across frameworks. The …
A developer ported the MACHIAVELLI benchmark, which measures unethical AI agent behavior, to the Inspect evaluation framework to make it easier for evaluators to use. The re-implementation is now offi…