{"slug": "constant-memory-recall-learned-associations-in-a-fixed-matrix-state", "title": "Constant-Memory Recall: Learned Associations in a Fixed Matrix State", "summary": "A small DeltaNet variant with fixed token-specific key biases and 32 KiB of recurrent matrix state reached 99.95% mean recall accuracy across three training seeds when remembering 32 new key-value pairings per sequence, according to an arXiv paper (2610.00232v1). Recall stayed near perfect when filler extended the pre-query context to 1,798 tokens without adding pairings, while zeroing the first memory block removed the recall. Parameter-matched vector and Transformer baselines remained near chance, including the Transformer after additional training searches, which the authors say prevents a memory-efficiency comparison.", "body_md": "arXiv:2610.00232v1 Announce Type: new \nAbstract: Fixed-size recurrent memory limits storage growth during inference, but successful recall depends on the task and training. We study a small DeltaNet variant with fixed token-specific key biases, trained to remember 32 new key-value pairings per sequence. With 32 KiB of recurrent matrix state, it achieves 99.95% mean accuracy across three training seeds when choosing among the sequence's values. Recall remains near perfect when filler extends the pre-query context to 1,798 tokens without adding pairings. Zeroing the first memory block removes this recall. An exploratory 48-pair test remains near chance after one quarter of the primary training budget and does not locate a capacity limit. Parameter-matched vector and Transformer baselines remain near chance, including the Transformer after additional training searches. This unresolved baseline failure prevents a memory-efficiency comparison.", "url": "https://wpnews.pro/news/constant-memory-recall-learned-associations-in-a-fixed-matrix-state", "canonical_source": "https://arxiv.org/abs/2610.00232", "published_at": "2026-10-03 04:00:00+00:00", "updated_at": "2026-10-03 04:07:52.488013+00:00", "lang": "en", "topics": ["machine-learning", "neural-networks", "ai-research", "large-language-models"], "entities": ["DeltaNet", "arXiv", "Transformer"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/constant-memory-recall-learned-associations-in-a-fixed-matrix-state", "markdown": "https://wpnews.pro/news/constant-memory-recall-learned-associations-in-a-fixed-matrix-state.md", "text": "https://wpnews.pro/news/constant-memory-recall-learned-associations-in-a-fixed-matrix-state.txt", "jsonld": "https://wpnews.pro/news/constant-memory-recall-learned-associations-in-a-fixed-matrix-state.jsonld"}}