Watch a 535B parameter (23B active) LLM get trained live

The MARIN community is publicly training a 535B-parameter mixture-of-experts large language model with 23B active parameters on 18 trillion tokens, streaming the process live via Weights & Biases. The 'Hero Run' is part of a scaling ladder aimed at demonstrating frontier-scale training transparency.

Article URL: https://wandb.ai/marin-community/marin moe/reports/535B-A23B-18T-Token-Hero-Run-Scaling-Ladder--VmlldzoxNzc2MDM5Ng Comments URL: https://news.ycombinator.com/item?id=49386102 Points: 1 Comments: 0