Ai2 opens MoE training stack after testing a 1.2T-parameter setup
The Allen Institute for AI (Ai2) released Olmo-core 3 on October 1st, a redesigned open training stack for mixture-of-experts models that Ai2 says reached 52,000 tokens per second per GPU on a 47-bill…