# ALTK-Evolve: How learned memory tops the τ²-bench leaderboard on customer service tasks without a frontier model

> Source: <https://www.snipvote.com/story/cmvhsfpl10004yw9jo9kg650i>
> Published: 2026-10-21 07:40:28.055783+00:00

[Hugging Face](https://huggingface.co/blog/ibm-research/altk-evolve-bc)

### ALTK-Evolve: How learned memory tops the τ²-bench leaderboard on customer service tasks without a frontier model

Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.

ALTK-Evolve achieves 67.5% Pass^4 (solves tasks correctly four times consecutively) on τ²-bench retail using GPT-4.1 by distilling past attempts into behavioral rules, proving lightweight, in-context learning can match frontier-model performance for policy-bound tasks. This lets teams upgrade existing agents without costly fine-tuning or model swaps by dynamically generating and applying domain-specific guidelines—cutting deployment time and compute costs while maintaining strict compliance.

By automatically distilling its own conversation failures into system-prompt behavioral rules, a basic ReAct agent using the older GPT-4.1 model achieved 67.5% Pass^4 on the τ²-bench retail benchmark, outperforming all frontier entries without any weight updates. This proves that policy-bound customer service agents can be optimized for reliability and strict guardrails entirely through automated offline prompt evolution rather than costly fine-tuning or upgrading to reasoning models. For teams shipping agents, this enables a self-improving pipeline where models dynamically write their own domain playbooks, shifting the optimization bottleneck from model capability to system-prompt token length.
