04:00
2026-08-14
arxiv.org
large-language-models
LoKiFormer: Locality-aware Attention with Decoupled Knowledge Memory for Efficient Large Language Model Pretraining
Researchers propose LoKiFormer, a novel large language model architecture that augments the standard decoder with Local Fusion Attention and a Knowledge Memory Module to address inefficiencies in pret…