Nested Sparsity for Scalable Pretraining and Continual Learning

Researchers at the Ma Lab at Berkeley introduced Nestor, a transformer architecture that adds a conditional-computation mechanism generalizing the attention operator, and reported it outperforms a conventional mixture-of-experts (MoE) transformer at equal compute and equal sparsity in language model pretraining. The sparsity Nestor induces nests hierarchically with MoE sparsity, and the team reported Nestor adapts to longer contexts more effectively than conventional MoEs and adapts to new domains more efficiently while forgetting less on old domains, with the benefits persisting as model size, dataset size and context length grow. Code and checkpoints are available on GitHub at github.com/Ma-Lab-Berkeley/Nestor.

arXiv:2610.31818v1 Announce Type: new Abstract: Mixture-of-experts MoE transformers, the standard backbone for modern large-scale deep learning, employ conditional computation: only a sparse subset of model weights is used in each forward pass. In MoE transformers, this sparsity arises in the MoE operator, which generalizes the multi-layer perceptron in the classical transformer. In this work, we explore an alternative mechanism for conditional computation that analogously generalizes the attention operator. The sparsity induced by this mechanism naturally nests i.e., hierarchically composes with the sparsity induced by the MoE operator. We show that a transformer employing this nested sparsity mechanism, which we call Nestor, performs better than a conventional MoE transformer at equal compute and equal sparsity in language model pretraining. We also show that Nestor is well-suited for continual pretraining: it can be adapted to longer contexts more effectively than conventional MoEs, as well as adapt more efficiently to new domains while forgetting less on old domains. Furthermore, we demonstrate that these benefits persist as model size, dataset size, and context length grow. Our results demonstrate the potential of alternative conditional computation mechanisms for deep neural network architectures. Code and checkpoints are available on GitHub at https://github.com/Ma-Lab-Berkeley/Nestor .