OLMoE

A new arXiv study (2608.02940v1) finds that a dense pruning score with 0.906 split-half reliability predicted a 16.1% gain but selected an endpoint 6.0% and 7.7% worse than two controls, modeling the …

Moonshot AI released the full technical architecture of its Kimi K3 model, a 2.8 trillion-parameter multimodal mixture-of-experts system with a one-million-token context window, on July 27. The 1.56 T…