Researchers introduced HSPR (Hierarchical Posterior Inference), a method for recovering software-agent benchmark scores from a small task subset, addressing the problem that subset evaluations preserve agent rankings but yield inaccurate scores. The work notes that historical responses provide useful structure, yet tasks with identical source responses can produce different outcomes for a new target agent.
Evaluating software agents on a full benchmark is expensive. A small task subset can preserve agent rankings while yielding inaccurate scores. Historical responses provide useful structure, but tasks with identical source responses can have different outcomes for a new target agent. We introduce Hie
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.