Member-only story
The most-starred course on GitHub was never written. Here’s the full roadmap it points to, and where to learn every step. #
Read the article for free
[.]here Andrej Karpathy has a GitHub repo with 37,500 stars and not one line of code in it.
Two files sit in the repository, a README and an image. It has more stars than llm.c
(his GPT trainer written in raw C). What it is, is a list. 17 chapters for building your own language model from scratch, taking you from a bigram model to something that talks like ChatGPT. He posted it in the summer of 2024, and archived it that August.
So what are 37,500 people saving a bookmark to? Not code. The route. The 17 steps from a program that can barely spell to your own small ChatGPT, in the order that teaches you how the thing works, with nothing skipped and nothing out of place.
That route still holds. You can walk every step of it today, and most of the code got written after all, just not where the repo points you. Here it is.
The Map is an Arc, Not a List
A syllabus usually reads like a random assortment of topics. The LLM101n
outline works differently because it…