{"slug": "strangers-pretrained-a-language-model-with-hf-prs-and-a-cron-job", "title": "Strangers pretrained a language model with HF PRs and a cron job", "summary": "A group of researchers pretrained a 15M-parameter language model from scratch using a GitHub Actions cron job as the training coordinator and pull requests as gradients, with no dedicated cluster or budget. The project, hosted on GitHub under commonsense-ai/coop, attracted a stranger who trained 23M tokens of the model, and applications are open until July 27.", "body_md": "| ||||||||||||\n1 point by |\nLast week a few of us pretrained a 15M-parameter language model from scratch —\npast its Chinchilla-optimal token budget — with no cluster, no server, and no\nbudget. The training coordinator is a GitHub Actions cron job. The gradients\nare pull requests. By the end, someone we'd never met had joined and trained\n23M tokens of it.\ngithub.com/commonsense-ai/coop — Model + card · huggingface.co/commonsense-ai/tinystories-15m — Gradient inbox — Leaderboard | |||||||||||\nApplications are open till July 27. |", "url": "https://wpnews.pro/news/strangers-pretrained-a-language-model-with-hf-prs-and-a-cron-job", "canonical_source": "https://news.ycombinator.com/item?id=49141174", "published_at": "2026-08-02 04:42:50+00:00", "updated_at": "2026-08-02 04:52:35.194687+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-infrastructure", "ai-tools"], "entities": ["GitHub Actions", "commonsense-ai", "Hugging Face"], "alternates": {"html": "https://wpnews.pro/news/strangers-pretrained-a-language-model-with-hf-prs-and-a-cron-job", "markdown": "https://wpnews.pro/news/strangers-pretrained-a-language-model-with-hf-prs-and-a-cron-job.md", "text": "https://wpnews.pro/news/strangers-pretrained-a-language-model-with-hf-prs-and-a-cron-job.txt", "jsonld": "https://wpnews.pro/news/strangers-pretrained-a-language-model-with-hf-prs-and-a-cron-job.jsonld"}}