{"slug": "valkey-9-2-cuts-backup-memory-overhead-and-gives-gpu-fleets-a-shared-cache", "title": "Valkey 9.2 Cuts Backup Memory Overhead and Gives GPU Fleets a Shared Cache", "summary": "Valkey 9.2, now generally available, adds forkless snapshots that move backup writes to a background thread, targeting the 50% extra memory operators commonly reserve for snapshots down to 10% to 15%, according to maintainer and AWS principal engineer Madelyn Olson. The release also ships open-source distributed KV caching, which Valkey says lets inference engines serve about 25% more requests on the same GPUs and stream responses about 2× faster for open-weight models, plus compressed replication that cuts ongoing traffic roughly 50% to 65% and syncs new replicas about 35% faster at a cost of 2% to 4% of one CPU core per replica. Backup files shrink up to 80% with LZ4, and large sorted sets move from skiplists to B+ trees for a 36% to 43% reduction in per-member index overhead in a five-million-member test and up to 105% faster updates.", "body_md": "TL;DR — Key Takeaways\n\n- \n**Stop paying for RAM you barely use.** Valkey 9.2’s forkless snapshots could dramatically reduce memory reserved for backups, making smaller, cheaper instances more practical.\n- \n**More AI inference, fewer GPUs.** Distributed KV caching lets inference engines reuse previously computed work, potentially boosting throughput and reducing expensive GPU workloads.\n- \n**Less infrastructure waste, more performance.** Compressed replication, smaller backups and more efficient data structures promise savings that extend well beyond AI applications.\n\nValkey keeps getting better and better, and not just for AI workloads.\n\nThe most useful new feature in [Valkey 9.2](https://valkey.io/) doesn’t have anything to do with AI per se. The open-source in-memory datastore, now generally available, can take a snapshot without forking a separate process. If you’ve been paying for RAM that sits idle until backup time, that’s an improvement well worth looking at. And, let’s face it, if you’re running AI inference, you can use every byte of RAM you can get.\n\nWhat will matter directly for AI is Valkey’s key-value (KV) caching. According to the company, this enables you to serve about 25% more requests on the same GPUs and s[tream responses about 2× faster](https://valkey.io/blog/kv-caching-on-valkey/) for open-weight models. \n\nIn addition, Valley is releasing open-source distributed KV caching. This enhances AI workload efficiency by creating a large, shared cache across a GPU fleet. This enables your inference engines to reuse computed work rather than repeating the work. The common thread is straightforward: get more out of the hardware you already have.\n\n### **Less RAM for Backups, Fewer Bytes Between Regions**\n\nGetting a bit deeper into the new features, conventional Valkey snapshot forks a child process to write the database to disk. Forkless snapshots move this job to a background thread, avoiding that fork-related copy-on-write penalty.\n\nWhen I covered the 9.2 preview at ValkeyConf, maintainer and AWS principal engineer Madelyn Olson told me operators commonly reserve 50% extra memory for snapshots. The project’s [goal, she said, was to get that down to 10% to 15%.](https://techstrong.it/featured/valkey-9-2-targets-memory-overhead-with-ai-used-to-improve-its-code/) Google engineer and Valkey maintainer Jacob Murphy also flagged another trade-off in the interview: The initial pause could cause timeouts on older, slower CPUs. So, if you’re counting on the feature to move to smaller instances, test it under your own write load first.\n\nReplication gets a different trade-off. That is, you spend a little CPU time in exchange for fewer bytes on the wire. The project reports that continuous stream compression cut ongoing replication traffic by roughly 50% to 65% and let new replicas sync about 35% faster, at a cost of about 2% to 4% of one CPU core per replica. It also reports backup files up to 80% smaller with LZ4. The network savings are especially relevant if your replicas sit in another region and you pay for the traffic between them.\n\nLarge sorted sets, the structures behind gaming leaderboards and priority queues, switch from skiplists to B+ trees. By packing entries together rather than following pointers through a skiplist, Valkey can, again, do more with less memory. The project reports a 36% to 43% reduction in per-member index overhead in a five-million-member test and up to 105% faster updates in its testing. That memory figure applies to the index, not to your entire dataset.\n\n### **A Shared Cache for Work GPUs Has Already Been Done**\n\nAn AI agent can send the same system instructions, tool definitions, and conversation history to a model over and over. The model doesn’t have to recompute the attention state for every repeated prefix if that state is already cached. The trick is finding the longest matching prefix and getting its cached state to the inference server that needs it, even if that server didn’t compute it in the first place.\n\nThat’s where **Path Hash**, a new native data type contributed by Alibaba Cloud, comes in. It gives Valkey a way to find the longest matching prefix and look up where its cached state lives. Valkey can hold the index while the large tensors stay in GPU or host memory. The distributed caching capability arriving alongside 9.2 uses LMCache to connect self-managed inference deployments running engines such as vLLM and SGLang. Path Hash is part of Valkey 9.2. The shared inference cache is a separate integration.\n\nThe project claims up to 40% lower inference costs and up to 80% shorter time to first token. Those are best-case figures, not what you should plug into a budget. Multiverse Computing offered a more specific result: It says it handled 50% more requests on the same GPUs while staying within its one-second target for median time to first token. Its median fell 23% under normal traffic and 95% at peak. Those figures are from one deployment, as reported in the [release announcement](http://ppl-ai-file-upload.s3.amazonaws).\n\nThe GPU cache will get the attention, but you don’t need to run a model to get value from Valkey 9.2. For teams already running Valkey, the more immediate question is how much RAM they reserve for snapshots and how much they pay to move replica traffic between regions. With 9.2, both are worth measuring again.", "url": "https://wpnews.pro/news/valkey-9-2-cuts-backup-memory-overhead-and-gives-gpu-fleets-a-shared-cache", "canonical_source": "https://techstrong.it/ai/valkey-9-2-cuts-backup-memory-overhead-and-gives-gpu-fleets-a-shared-cache/", "published_at": "2026-11-19 18:00:58+00:00", "updated_at": "2026-11-19 18:14:50.425582+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-tools", "developer-tools", "mlops"], "entities": ["Valkey", "Madelyn Olson", "AWS", "Jacob Murphy", "Google", "LZ4"], "also_reported_by": [], "takeaway": "Valkey 9.2's forkless snapshots cut the memory reserved for backups from a typical 50% toward a 10%-15% target, and its distributed KV cache lets inference engines serve about 25% more requests on the same GPUs, though maintainers warn the initial pause can cause timeouts on older, slower CPUs.", "recommendation": null, "alternates": {"html": "https://wpnews.pro/news/valkey-9-2-cuts-backup-memory-overhead-and-gives-gpu-fleets-a-shared-cache", "markdown": "https://wpnews.pro/news/valkey-9-2-cuts-backup-memory-overhead-and-gives-gpu-fleets-a-shared-cache.md", "text": "https://wpnews.pro/news/valkey-9-2-cuts-backup-memory-overhead-and-gives-gpu-fleets-a-shared-cache.txt", "jsonld": "https://wpnews.pro/news/valkey-9-2-cuts-backup-memory-overhead-and-gives-gpu-fleets-a-shared-cache.jsonld"}}