19:51
2026-09-30
github.com
large-language-models
Coop: A small language model pretrained by volunteers
A volunteer-run project called Coop has moved to Stage 2, pretraining a ~145M-parameter decoder-only transformer from scratch on FineWeb-Edu using DiLoCo-style low-communication data parallelism, with…