CLR-Bench finds six self-evolving agent methods gain at most 5 points
Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.
Standard self-evolving agents fail to abstract underlying rules from feedback, gaining at most five points over frozen baselines and performing worse than simply retaining raw interaction history in the prompt. In production, this means dynamic state-rewriting loops will degenerate into bloated, brittle exception lists rather than true system adaptation, requiring you to either inject explicit domain instructions to gain a fifteen-point boost or upgrade to frontier models running architectures like ACE.
Your self-evolving agents (ACE, Dynamic Cheatsheet, etc.) aren't actually inferring latent rules from feedback — they're just appending each correction as a memorized exception, and on this benchmark that list-growing behavior underperformed simply retaining raw practice history (excluding only 6-12% of unseen members vs. 20%), gaining at most 5 points over a frozen agent. The capability depends heavily on base model strength: a weak model never abstracts the rule no matter the memory scaffold, while a much stronger model learns it from feedback alone (78% accuracy), so if you're relying on persistent-state memory to generalize, expect bloated states and brittle pattern-matching unless your underlying model can already generalize — the scaffolding won't rescue a weak one.