OPRD: On-Policy Reminder Distillation for Long-Horizon Coding Agents

On-Policy Reminder Distillation (OPRD), a training method for long-horizon coding agents, improves the benchmark-averaged task resolution rate by 11.3 points over the open-source Qwen3.5-4B base model on SWE-bench Verified, SWE-bench Pro, and SWE-bench Multilingual, according to the arXiv paper 2610.21496v1. OPRD combines Multi-Scope Reminder supervision with outcome-based routing on reminder-free student rollouts, using autoregressive self-imitation for successful rollouts and Reminder-conditioned reverse-KL supervision from a teacher for failed ones. Under the same diagnostic protocol, OPRD cuts the aggregate error-event count from 446 to 222, a 50.2% reduction, and the student operates without Reminders at inference.

arXiv:2610.21496v1 Announce Type: new Abstract: Reliable repository-level coding requires agents not only to generate correct code but also to preserve constraints and respond to tool feedback across long interactions. However, our diagnostic analysis shows that agents often fail to act on relevant information already available in context, while final Pass/Fail outcomes provide only sparse process supervision. We introduce On-Policy Reminder Distillation OPRD , which combines Multi-Scope Reminder supervision with outcome-based routing on reminder-free student rollouts. Successful rollouts provide autoregressive self-imitation targets, whereas failed rollouts receive Reminder-conditioned reverse-KL supervision from a teacher that re-evaluates the same student-generated prefixes. A Global Reminder converts final-test diagnostics into trajectory-level guidance, while Turn-local Reminders apply only to the next decision following the relevant environment observation. The teacher and student share a two-stage initialization from Coding SFT and Instruction-Following IF SFT. At inference, the student operates without Reminders. On SWE-bench Verified, SWE-bench Pro, and SWE-bench Multilingual, OPRD improves the benchmark-averaged task resolution rate by 11.3 points over the open-source Qwen3.5-4B base model. Under the same diagnostic protocol, it reduces the aggregate error-event count from 446 to 222, a 50.2% reduction.