{"slug": "hugging-face-fixes-trl-bug-that-let-rl-rollouts-reuse-kv-cache-from-older", "title": "Hugging Face fixes TRL bug that let RL rollouts reuse KV cache from older weights", "summary": "Hugging Face fixed a training-correctness bug in TRL, its open-source model-training library, that let asynchronous reinforcement-learning rollouts reuse vLLM prefix-cache blocks computed under older model weights. Pull request #8300 adds 'cache_salt=f\"v{model_version}\"' to affected rollout requests so each policy version gets its own prefix-cache namespace, and the change landed in TRL v1.18.0, whose release notes say the stale prefix KV lasted for the whole run before the fix. The bug affected AsyncGRPO and AsyncDistillation student rollouts, which could pair new weights with first-version prefix KV.", "body_md": "# Hugging Face fixes TRL bug that let RL rollouts reuse KV cache from older weights\n\n**TRL v1.18.0 adds a model-version cache salt so AsyncGRPO and AsyncDistillation student rollouts stop pairing new weights with first-version prefix KV.**\n\n        By [RuntimeWire Staff](https://runtimewire.com/author/runtimewire-staff)\n        · Published \n        · Updated \n\nHugging Face has fixed a training-correctness bug in TRL, its open-source model-training library, that let asynchronous reinforcement-learning rollouts reuse vLLM prefix-cache blocks computed under older model weights. The patch, pull request #8300, adds 'cache_salt=f\"v{model_version}\"' to affected rollout requests so each policy version gets its own prefix-cache namespace. The change landed in TRL v1.18.0, whose release notes describe the stale prefix KV as lasting for the whole run before the fix.…", "url": "https://wpnews.pro/news/hugging-face-fixes-trl-bug-that-let-rl-rollouts-reuse-kv-cache-from-older", "canonical_source": "https://runtimewire.com/article/hugging-face-trl-stale-kv-cache-asyncgrpo-fix", "published_at": "2026-11-13 21:09:10+00:00", "updated_at": "2026-11-13 22:22:28.423544+00:00", "lang": "en", "topics": ["machine-learning", "ai-research", "ai-tools", "developer-tools"], "entities": ["Hugging Face", "TRL", "vLLM", "AsyncGRPO", "AsyncDistillation", "TRL v1.18.0", "pull request #8300"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/hugging-face-fixes-trl-bug-that-let-rl-rollouts-reuse-kv-cache-from-older", "markdown": "https://wpnews.pro/news/hugging-face-fixes-trl-bug-that-let-rl-rollouts-reuse-kv-cache-from-older.md", "text": "https://wpnews.pro/news/hugging-face-fixes-trl-bug-that-let-rl-rollouts-reuse-kv-cache-from-older.txt", "jsonld": "https://wpnews.pro/news/hugging-face-fixes-trl-bug-that-let-rl-rollouts-reuse-kv-cache-from-older.jsonld"}}