Researchers introduced FrontierCS 2, a benchmark of 48 open-ended tasks that tests whether LLM agents can contribute to frontier computer science R&D by taking a paper and its released code, deleting the paper's contribution, and using the deleted code as the human reference that sets the bar. The benchmark's design makes the authors' removed code the standard against which agent performance is measured.
Can LLM agents make contributions to frontier computer science R&D? We introduce FrontierCS 2, a benchmark of 48 open-ended tasks. We take a paper and the code its authors released and delete the paper's contribution. The deleted code becomes the human reference that sets the bar. Earlier open-ended
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.