# Google Open-Sources TPU Sync for KV Cache and Weight Transfer on TPUs > Source: > Published: 2026-11-12 10:06:00+00:00 Google Cloud has open-sourced [TPU Sync](https://cloud.google.com/blog/topics/ai-infrastructure/introducing-tpu-sync-fast-kv-cache-and-weight-transfer-on-tpu), a native C++ library that moves KV cache blocks and model weights between TPU HBM, host DRAM, shared storage, and the data center network. GCP also announced the [TPU 8tx and TPU 8i Max systems](https://blog.google/innovation-and-ai/infrastructure-and-cloud/global-network/tpu-8i-max-and-8tx/), the project targets two workloads where accelerators often wait on data: long-context and disaggregated LLM serving, and reinforcement learning loops that push updated weights from trainers to rollout samplers. For prefill/decode disaggregation, Google says median KV transfer latency for Qwen3 235B fell from several seconds to about 230 ms, with average TTFT reductions of 60% for that model and 90% for a 480B model at low concurrency. For prefix caching, it reports up to 90% lower average TTFT on 64K-context workloads. Developers get Python wheels for JAX and PyTorch, with KV cache manager, KV store, and weight synchronizer APIs. TPU Sync acts as the KV connector for vLLM-TorchTPU through vLLM's KVConnector interface, as the disaggregation and hierarchical-cache backend for SGLang-TorchTPU, and pairs with [llm-d](https://llm-d.ai/) 1.0 on GKE for prefix-cache-aware routing. The RL weight-sync module for Pathways and multi-controller JAX is experimental. The [GitHub repository](https://github.com/google/tpu-sync), which absorbed the earlier TPU Raiden codebase, warns that APIs are changing rapidly and that only development builds are on PyPI, and recommends pinning to its last-known-good commit. It is validated on TPU v7x and v6e, and the PyTorch wheel requires a matching torch_tpu build that is not on a public index. On GPUs, similar problems are handled by [NVIDIA Dynamo](https://github.com/ai-dynamo/dynamo) and its [NIXL](https://github.com/ai-dynamo/nixl) transfer library, [LMCache](https://github.com/LMCache/LMCache), which offloads KV cache to CPU memory, disk, or remote stores, [Mooncake](https://github.com/kvcache-ai/Mooncake), the KVCache-centric serving architecture behind Moonshot AI's Kimi, and vLLM's own [disaggregated prefill connectors](https://docs.vllm.ai/en/latest/features/disagg_prefill.html). TPU Sync differs mainly in being tied to PJRT and TPU tile layouts, and in covering RL weight transfer in the same library. Early reactions focused on data movement rather than the new chips, with developer [Anand Butani](https://x.com/AnandButani/status/2119830708516745520) sharing: TPU sync cutting kv transfer latency 90% matters more to me than the chip names, long-context serving mostly dies on moving cache between nodes. Another developer posting under the pseudonym [@lgoodbookd](https://x.com/lgoodbookd/status/2120081156624507050) ran rough numbers on why bandwidth alone does not solve the problem: 8i Max doubles HBM bandwidth to ~17.2TB/s, yet one 64K session on a 235B MoE still eats ~12.6GB of KV cache. That's why open-sourcing TPU Sync (host-DRAM KV tiering, ~223GB/s per v7x host) may matter as much as the new package. Bandwidth helps; capacity is still the wall. TPU Sync is available under the Apache 2.0 license, and its [repository](https://github.com/google/tpu-sync) includes architecture documentation, C++ and Python APIs, and examples for single-host and multi-host disaggregated serving and host KV offloading. Until a stable release is cut, the [JAX](https://pypi.org/project/tpu-sync-jax/) and [PyTorch](https://pypi.org/project/tpu-sync-torch/) packages are published only as development builds, while the storage tier and RL weight synchronization remain experimental