# LowRAM Paper: Amortised Values and Low-Rank Keys for KV Compression > Source: > Published: 2026-11-09 00:00:00+00:00 **LowRAM** is a **test-time KV cache compression pipeline** from **Huawei Noah's Ark Lab** and **University College London** researchers, posted as [**arXiv:2611.07025**](https://arxiv.org/abs/2611.07025?ref=explainx) in early November 2026. Instead of treating keys and values as identical tensors to quantize or evict, LowRAM **amortises values** with per-head predictors trained on the current context and **compresses keys** with **cross-layer low-rank** structure borrowed from [**xKV**](https://arxiv.org/abs/2503.18893?ref=explainx). The authors report staying **near lossless at moderate compression** across three model families and reaching **26.9× compression** when stacked with **quantisation and eviction**, retaining **94.5%** of baseline **LongBench** performance on long contexts. If you already read explainx.ai's [**KV cache explainer**](https://explainx.ai/blog/what-is-kv-cache-llm-inference-explained-guide-2026), LowRAM is the next layer: **what to do when HBM is full but you cannot swap architectures**. ## The bottleneck LowRAM targets KV cache slots along a decoding channel where context length drives memory growth Autoregressive decoding stores every prior token's **keys and values** so each new token attends in linear time instead of recomputing the full prefix. That cache grows with **context length × layers × batch × head dimension**, eventually dominating activation memory on long chats, RAG dumps, and agent trajectories. Architectural fixes — **GQA**, **MLA**, linear attention, state-space hybrids — need training. Most deployed open models still ship classic caches, which is why post-training compression papers keep appearing alongside runtime tricks like [**speculative decoding**](https://explainx.ai/blog/what-is-speculative-decoding-llm-inference-guide-2026) and [** semantic caching**](https://explainx.ai/blog/semantic-caching-llm-apps-cost-latency-guide-2026). Prior post-training families, as LowRAM taxonomizes them: | Family | Idea | |---|---| | **Eviction** | Drop tokens deemed low utility (StreamingLLM-style sinks, head budgets) | | **Quantisation** | Store lower bit-width K/V with outlier-aware schemes | | **Low-rank** | Factorise K or V matrices per layer or across layers | LowRAM argues a **fourth axis** was underused: **within each head, values are largely predictable from their keys** because both projections share the same residual stream. ## Value cache amortisation in plain language Per-head predictors amortising value storage from keys during prefill During **prefill**, LowRAM fits a **small per-head MLP** mapping each token's **key vector** to its **value vector**. It scores reconstruction error per token and keeps **sparse residuals** only for the worst predictions. At decode time, values are **regenerated from keys** plus those residuals. That trades **one-time prefill compute** for **smaller DRAM footprint** — attractive when you serve **many queries on a shared long prefix** (legal RAG, support playbooks) if amortisation cost can be shared. The paper also offers **meta-learned initialisations** to cut prefill overhead and **selective reconstruction** so decode only touches a subset of KV pairs each step — reporting up to **5× peak decode throughput** at **4× compression** in their throughput experiments (see Section 1 and contributions in the arXiv PDF). ## Asymmetric key compression with xKV Values depend on keys, but attention still needs keys themselves. LowRAM applies **xKV-style cross-layer low-rank compression** to keys: adjacent layers share dominant subspaces, so a group of layers can store a **shared basis** plus small factors. The pipeline **inverts Rotary Position Embedding (RoPE)** before compression — following evidence that **pre-RoPE** tensors quantize and factorize more cleanly — then reapplies rotation during attention, consistent with other recent KV papers cited in Section 2. New decode tokens store **exact** KV pairs as they are generated; compression applies to the **prompt/cache body**, which is usually the bulk on agent workloads. ## Benchmarks and composability High-bandwidth memory pressure when KV caches scale with context length Figure 1 in the paper compares **LowRAM** and **LowRAM-EQ** (plus eviction and quant) on **Mistral-7B** for **LongBench** sequences beyond **10K tokens**, plotting **recovered performance vs compression ratio**. The headline composite: **26.9× compression at 94.5%** baseline performance when methods stack — a research peak, not a vendor SLA. LowRAM tests across **three model families** (named in the abstract) and emphasizes **composability**: teams can adopt amortised values without giving up **4-bit KV** or **eviction policies** they already tuned in [vLLM](https://explainx.ai/blog/vllm-vs-sglang-inference-server-comparison-2026) or [Samir's from-scratch inference tutorials](https://explainx.ai/blog/samir-vllm-build-llm-inference-from-scratch-cuda-guide-2026). ## How LowRAM differs from Amazon HOPE and architectural MLA Compression stories collide in readers' feeds: - **[Amazon HOPE](https://explainx.ai/blog/amazon-hope-moe-expert-pruning-second-order-awslabs-2026)** shrinks**MoE expert weights** , not activations. - **DeepSeek MLA** and similar architectures bake low-rank KV into training. - **LowRAM** keeps the**original checkpoint** and pays**test-time fitting** instead. Pick the lever that matches your constraint: **static VRAM for weights** versus **dynamic VRAM for context**. ## Practical takeaways for inference engineers 1. **Watch prefill latency** — amortisation is powerful when one long context serves many decodes; one-shot short prompts may not repay the fit cost. 2. **Validate on your eval suite** — LongBench mixes differ from your RAG corpus; reproduce the paper's eviction+quant stack before production. 3. **Track arXiv for code** — methods this fresh may ship implementations weeks after the PDF; pin versions if you experiment. 4. **Combine with policy-layer caching** — shrinking KV bytes does not remove need for[**rerankers in RAG**](https://explainx.ai/blog/how-to-add-reranker-rag-pipeline-latency-models-guide-2026) or tenant isolation. Huawei/UCL authorship also signals **European lab collaboration** on efficient inference — adjacent to [Apple's on-device speech research](https://explainx.ai/blog/apple-siri-agentic-dots-like-agent-ios-28-development-shake-up-gurman-2026) and [multimodal token budget guides](https://explainx.ai/blog/multimodal-vision-token-budget-context-window-guide-2026) in the "do more within a fixed envelope" theme. ## Limitations the authors acknowledge LowRAM adds **hyperparameters**: residual budgets, rank choices, meta-init versus scratch fit. Cross-layer key compression assumes **layer groups** share structure — brittle if you heavily fine-tune middle layers for a niche domain. And like all test-time schemes, it must survive **adversarial prompts** designed to break predictors; the paper focuses on standard benchmarks, not red-team contexts. ## Step-by-step mental model for one decoding session Think about a single long-context request: 1. **Prefill** runs on the full prompt. LowRAM inverts RoPE, fits**per-head MLPs** mapping keys to values, stores**sparse residuals** for bad fits, and compresses keys with**shared low-rank bases** across layer groups. 2. **Decode** generates new tokens. Fresh K/V pairs for new tokens stay**exact** in cache while compressed history is**reconstructed** on demand per the paper's selective reconstruction trick. 3. **Stacked compression** optionally applies**eviction** and**quant** on top (**LowRAM-EQ** ), chasing the**26.9×** headline ratio at**94.5%** LongBench retention in their long-context suite. Compare to **prefix caching** in [vLLM versus SGLang](https://explainx.ai/blog/vllm-vs-sglang-inference-server-comparison-2026): prefix caching avoids recomputing **identical prompts** across users; LowRAM shrinks **bytes per unique prompt**. They can coexist if your serving stack exposes both hooks — but you must profile **prefill latency** after amortisation. ## When to try LowRAM versus other KV tricks | Situation | Lean toward | |---|---| | Same 200K doc served to thousands of users | LowRAM-style amortisation if prefill cost amortizes | | Unique short chats | Quant or eviction only — fit cost may dominate | | You control model architecture | Train-time MLA/GQA instead of post-hoc compression | | MoE weight OOM | [HOPE expert pruning](https://explainx.ai/blog/amazon-hope-moe-expert-pruning-second-order-awslabs-2026) , not LowRAM | Huawei/UCL authorship signals **European + China lab collaboration** on efficient inference — relevant as **export and datacenter** politics tighten chip shipments ([Nvidia Korea forum](https://explainx.ai/blog/nvidia-korea-sovereign-ai-msit-nvlink-fusion-sk-telecom-rebellions-samsung-lg-kakao-openshell-2026)) and **Nordic power pauses** ([Norway grid story](https://explainx.ai/blog/norway-statnett-data-center-grid-pause-licensing-august-2027-2026)) constrain where you can run fat contexts. ## Reproducing paper numbers responsibly The arXiv PDF pins benchmarks to **LongBench** slices beyond **10K tokens** and multiple model families named in the abstract. Before claiming **26.9×** in a product brief: - Read **Appendix C** compression accounting — ratios combine methods and may not match your**vLLM metrics** . - Match **dtype, batch, and GPU** class if you replicate throughput claims. - Treat **94.5%** as**benchmark retention** , not**your** legal or medical RAG accuracy. ## Connection to AQUA-KV and prior predictors Section 2 of the paper places LowRAM beside **AQUA-KV** and other **predictive quant** schemes that exploit **cross-layer** dependencies. LowRAM's twist is **intra-head key-to-value amortisation** with **residual sparsity**, then **asymmetric** treatment: values predicted, keys low-ranked. If you maintain **RULER-style long-context evals** ([our benchmark guide](https://explainx.ai/blog/ruler-benchmark-long-context-evaluation-guide-2026)), add a **compressed-KV** configuration column once implementations ship. Regression there beats guessing from arXiv curves alone. For **local MLX or llama.cpp** users, none of this is wired in by default yet — watch Hugging Face and vLLM release notes for **plugin hooks**. Until then, lower `num_ctx` or raise hardware remains the pragmatic fix. ## Energy angle KV compression reduces **HBM traffic**, which can cut **watts per token** on bandwidth-bound decode. It does not shrink **FLOPs** on matmuls themselves. Datacenter operators chasing **[Envision wind-powered campuses](https://explainx.ai/blog/envision-galaxy-campus-wind-powered-ai-data-center-ulanqab-inner-mongolia-2026)** still care because **memory watts** dominate long-context serving nodes. ## Glossary tie-in If your team debates **KV cache** basics in standup, start with [/blog/what-is-kv-cache-llm-inference-explained-guide-2026](https://explainx.ai/blog/what-is-kv-cache-llm-inference-explained-guide-2026) before debating LowRAM — shared vocabulary prevents confusing **weight pruning** with **activation compression**. Paper authors list **Teresa R. Delgado**, **Hugo Hazard**, **Martin Benfeghoul**, **Jun Wang**, and **Zafeirios Fountas** across **Huawei Noah's Ark Lab** and **UCL**; citations should use **arXiv:2611.07025** rather than Hugging Face daily-paper blurbs. Serving teams should log **p95 prefill** before and after enabling any LowRAM integration — user-visible latency often hides in that phase, not decode tokens per second headlines. Pair those logs with **[semantic cache](https://explainx.ai/blog/semantic-caching-llm-apps-cost-latency-guide-2026)** hit rates so you know whether repeated prefixes, not novel compression, drove the win. Watch the arXiv page for a **code drop**; until then treat LowRAM as **research** you cite in design docs, not a production flag. The [arXiv abstract](https://arxiv.org/abs/2611.07025?ref=explainx) already lists composition with quantisation and eviction — the engineering story is **stacking knobs**, not picking one winner. Inference leads should prototype on **one long-document eval** before fleet rollout.