Researchers introduced RULER (Instance-aware Rubric Rewards for Reinforcement Learning), a method that converts each natural-language instruction into an instance-aware rubric of six items spanning se…
RBS-Attention, a training-free sparse-prefill method from a new arXiv paper (2609.20971v1), achieves 20.65x standalone prefill-attention speedup, 11.92x vLLM prefill-attention speedup, and 5.97x end-t…
Researchers introduced Fathom, a per-query read-depth key scan for sparse decoding over offloaded KV caches, which at one million tokens on Qwen3-8B runs a decode step 1.67x faster in GPU time than th…
Researchers introduced HeadWiseKV, a training-free framework that compresses residual global KV caches in hybrid long-context language models under a budget, preserving quality while reducing memory. …
Researchers have introduced Faster Flash Decoding (FFD), a hardware-algorithm co-design framework that achieves up to 11.6x kernel-level speedup and 2.37x end-to-end throughput improvement for long-co…
A new AI memory system achieved #1 on the BABILong benchmark at 10M tokens, with a gaming audit showing 24/24 accuracy at 1M tokens and 16/16 at 2M, 5M, and 10M tokens on RULER variable tracking. The …
Researchers propose PuzzleKV, a training- and calibration-free method for KV cache compression in large language models, which partitions per-head KV caches into fixed-length logical pages and applies…
Researchers introduced Attention-Aware Transform Coding (AATC), a KV cache compression method that reduces memory use by approximately 5.8x while maintaining near-lossless accuracy on models like Llam…
Researchers introduced LinearKV, a training-free framework enabling position-independent caching (PIC) for hybrid large language models (LLMs) that combine linear recurrences with full attention. The …
Pokee AI released Pokee-Isaac 28B, a 28B text-only foundation model with a 10M-token context window designed to run inside the customer boundary. The model scores 93.3% on RULER at 10M tokens, leads B…
Researchers introduced CoMem (Comprehension Memory), a method that leverages the depth division of labor in large language models to enable unbounded-context memory. In evaluations, CoMem achieved 97.…
LOCKS, a new method from arXiv, enables efficient long-context decoding by giving each page of the key-value cache its own compact spectral summary, reconstructing within-page logits, and attending on…
Researchers propose a value-of-information router for block-sparse attention that dynamically doubles the kept key blocks when the top-k selection is uncertain, improving recall on LongBench-v2 from 0…
Researchers propose Jet-Long, a tuning-free zero-shot method for extending LLM context windows that dynamically adjusts a rescaling factor based on sequence length, preserving short-context fidelity w…
A developer observed that after multiple context compactions in LLM sessions, output quality degrades non-linearly, with a brief improvement after the second compaction before declining. They built a …
Researchers introduced Still, a per-layer Perceiver model that compacts KV cache in a single forward pass, enabling efficient long-context language model deployment. On Qwen and Gemma models, Still ou…
Chroma tested 18 frontier AI models including GPT-4.1, Claude Opus 4, and Gemini 2.5 Pro, finding that every model's performance degrades as context length grows, contradicting vendor claims of 1M-2M …
A new analysis warns that large language model context windows degrade significantly beyond 100k tokens, making advertised sizes of 200k to 2M tokens misleading for practical use. Studies like RULER a…
Researchers introduced RULER, a set of representation-level verification metrics for machine unlearning, after finding that current output-level tests can pass even when models retain forgotten data i…