04:00
2026-10-12
machinebrief.com
large-language-models
STAPLE: Static Sparse Upcycling of Pretrained Checkpoints via Embedding Modules
STAPLE, a recipe built on STEM for integrating static sparse memory into pretrained LLMs, improves over continued dense training on OLMo-2 1B by 4.27 points on HumanEval, 2.80 on MBPP, 2.27 on GSM8K, …