The Allen Institute for AI released Olmo-core 3, an open training framework for large mixture-of-experts (MoE) models that scales MoE training into the trillion-parameter range while preserving comput…
A systematic study of six language models from four families and three sizes found that evaluation awareness—the ability to infer being tested and condition responses accordingly—is linearly decodable…
Ai2, the nonprofit Allen Institute for AI behind the Olmo models, told Lets Data Science in writing that releasing training data, code, and intermediate checkpoints enables research that open weights …
A new arXiv preprint (arXiv:2608.10296v1) reveals that four minor architectural choices in dense transformer models—normalization, GQA, pretraining context length, and sliding window attention—can com…