A study titled "ImpossibleRubrics: Stress-Testing Generated Rubrics as Reward Signals" examines whether language model-generated rubrics used as reward signals for rubric-based reinforcement learning, LLM-as-a-judge evaluation, and automated grading reliably reward honest answers over adversarial answers optimized to exploit them. The research focuses on the robustness of these generated rubrics to such adversarial exploitation.
Language model-generated rubrics are increasingly used as reward signals for rubric-based reinforcement learning, LLM-as-a-judge evaluation, and automated grading. Such rubrics are reliable only if they reward honest answers over adversarial answers optimized to exploit them. Yet their robustness to
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.