Beyond Binary Verifiable Rewards: Behavioral Self-Rewarding for SWE Agents

Researchers introduced Behavioral Self-Rewarding (BSR), a reward-refinement method that lets a policy compare observable debugging behavior among trajectories sharing the same verifier outcome, converting pairwise preferences into bounded auxiliary rewards. On SWE-bench Verified and SWE-bench-Live, BSR produced consistent improvements across evaluated Qwen3-14B and Qwen3-32B configurations, per the arXiv paper 2610.12598v1. The method adds supervision from already-collected trajectories without new environment interactions, addressing the inability of binary verifiable rewards to distinguish trajectories with substantially different debugging quality.

arXiv:2610.12598v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards RLVR provides reliable supervision for software engineering agents through executable tests, but binary rewards fail to distinguish trajectories with substantially different debugging quality. We investigate whether the current policy can judge its own trajectories and use these judgments to improve task performance without a separately trained reward model. To this end, we introduce Behavioral Self-Rewarding BSR , a reward-refinement method that reuses the policy to compare observable debugging behavior among trajectories from the same task with the same verifier outcome. BSR aggregates pairwise preferences into task-local scores and converts them into bounded auxiliary rewards, introducing reward contrast while preserving the verifier-defined pass/fail ordering. This provides additional supervision from already-collected trajectories without requiring new environment interactions. Experiments on SWE-bench Verified and SWE-bench-Live show consistent improvements across the evaluated Qwen3-14B and Qwen3-32B configurations, supporting using the policy's own comparative judgments as a complementary learning signal for training software engineering agents.