UL-SMF – Open-source linear-complexity KV-cache compression

Lawrence Architectures released UL-SMF, an open-source KV-cache compression fabric that reduces memory footprint by up to 384x with over 94% semantic retention, addressing the memory bottleneck in long-context Transformer inference. The dual-licensed software (AGPLv3 for non-commercial use, commercial license for proprietary deployments) dynamically maps any model hidden dimension using Finite Scalar Quantization and 16-dimensional latent mapping, achieving 99.7% memory savings per block.

The Unified Latent-State Memory Fabric UL-SMF is a hardware-software co-designed memory compression fabric that solves the memory bottleneck in long-context Transformer inference. By combining Finite Scalar Quantization FSQ with dynamic 16-dimensional latent mapping, UL-SMF compresses Key-Value KV cache tensors by up to 384x while maintaining 94% semantic retention . UL-SMF is dual-licensed: Open Source AGPLv3 : Free for non-commercial research, academic use, and open-source projects. Note: The AGPLv3 license requires any network-accessible service using this software to open-source its entire backend application code. Commercial Enterprise License: Required for proprietary commercial deployments, closed-source SaaS platforms, and enterprise data center infrastructure. Commercial licenses grant full rights without AGPLv3 copyleft restrictions, plus integration support. 📩 For Enterprise Licensing Inquiries: inquiries@lawrencearchitectures.com | Metric | Raw FP32 Cache | UL-SMF 16D Latent | Improvement | |---|---|---|---| VRAM Footprint 4096 tokens | 48.00 MB | 0.12 MB | 384x Reduction | VRAM Saved / Block | — | 47.88 MB | 99.7% Memory Saved | Semantic Retention | 100% | 94.15% - 95.84% | Cosine Similarity | Pipeline Latency | — | ~14.1 ms - 19.6 ms | CUDA Event Verified | UL-SMF dynamically maps any model hidden dimension Mistral, Llama, Qwen, etc. on-the-fly using orthogonal projection: python import torch from ul smf import UniversalLatentBridge 1. Load your compiled Aegis-KV oracle core binary device = torch.device "cuda" if torch.cuda.is available else "cpu" oracle core = torch.jit.load "aegis kv oracle core.pt", map location=device oracle core.eval 2. Wrap it with the Universal Dynamic Bridge auto-adapts to any model size ul smf bridge = UniversalLatentBridge core module=oracle core, core dim=3072 .to device 3. Seamlessly compress any model hidden dimension e.g., 4096 for Llama/Qwen kv cache tensor = torch.randn 1, 32, 4096, device=device reconstructed cache, compressed latents = ul smf bridge kv cache tensor print f"Compressed down to latent space: {compressed latents.shape}"