{"slug": "march-scaling-recurrent-memory-with-content-routed-state-anchors", "title": "MARCH: Scaling Recurrent Memory with Content-Routed State Anchors", "summary": "Researchers introduced Memory-Anchor Routing across Context History (MARCH), a recurrent architecture that scales state-space models beyond fixed-size memory by periodically caching state anchors with content-conditioned keys. In tests after standard pretraining, MARCH outperformed multiple linear attention variants on commonsense reasoning, LongBench, and in-context retrieval, offering a controllable trade-off between historical resolution and memory cost while maintaining efficient decoding.", "body_md": "arXiv:2608.12435v1 Announce Type: new\nAbstract: Transformers owe much of their strong long-context retrieval capability to a token-level memory that grows with context length. This flexibility, however, incurs a quadratic computation complexity during training and a key--value cache that grows linearly during autoregressive inference. Recurrent alternatives offer efficient decoding by compressing the entire history into a fixed-size state, but often underperform on recall-intensive tasks since earlier associations usually get overwritten by subsequent updates, and only the most recent contextual information is retained. In this paper, we introduce Memory-Anchor Routing across Context History (MARCH), a network architecture that effectively scales state-space models beyond a fixed-size dimension, while maintaining computational efficiency over long-sequences. MARCH periodically caches cumulative recurrent-state checkpoints as state anchors and associates each anchor with a compact, content-conditioned anchor key. This lets MARCH maintain a memory bank, which can grow as context length increases, providing a controllable trade-off between historical resolution and memory cost. At each token, MARCH produces an anchor query to attend all causally available state anchors, and the output is calculated as an attention-style aggregation over all historical anchors along the current state. We show that after standard pretraining, MARCH consistently outperforms multiple linear attention variants across commonsense reasoning, LongBench, and in-context retrieval. These results demonstrate that content-routed state caching substantially strengthens recurrent long-range memory while preserving its native computation path.", "url": "https://wpnews.pro/news/march-scaling-recurrent-memory-with-content-routed-state-anchors", "canonical_source": "https://arxiv.org/abs/2608.12435", "published_at": "2026-08-14 04:00:00+00:00", "updated_at": "2026-08-14 04:15:31.393182+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research"], "entities": ["Memory-Anchor Routing across Context History (MARCH)", "arXiv", "LongBench"], "alternates": {"html": "https://wpnews.pro/news/march-scaling-recurrent-memory-with-content-routed-state-anchors", "markdown": "https://wpnews.pro/news/march-scaling-recurrent-memory-with-content-routed-state-anchors.md", "text": "https://wpnews.pro/news/march-scaling-recurrent-memory-with-content-routed-state-anchors.txt", "jsonld": "https://wpnews.pro/news/march-scaling-recurrent-memory-with-content-routed-state-anchors.jsonld"}}