Jane Street's three-player auction game MegaGem was used to train a 4B specialist model, with SFT raising Qwen3-4B-Instruct's benchmark rating from 551 to 1158, ahead of Claude Sonnet 4.6, Claude Opusβ¦
A team of researchers from Yale's Building AI Infrastructure class achieved a new speed record of 81.2 seconds on the modded-nanoGPT benchmark by introducing a learnable per-(layer, head) tanh(Ξ±) gateβ¦
DeepSeek researchers introduced manifold-constrained hyper-connections to restore the identity mapping property in transformer architectures, addressing training instability and scalability issues cauβ¦
Seven open-weight frontier models, including Hugging Face's SmolLM3, DeepSeek-R1, and OpenAI's gpt-oss-120b, were analyzed to distill common training methodologies for multi-billion parameter models, β¦