04:00
2026-08-12
arxiv.org
artificial-intelligence
Cracks in the Foundation: Seemingly Minor Architectural Choices Impact Long Context Extension
A new arXiv preprint (arXiv:2608.10296v1) reveals that four minor architectural choices in dense transformer models—normalization, GQA, pretraining context length, and sliding window attention—can com…