# LLMs From Scratch Reaches 100,000 GitHub Stars > Source: > Published: 2026-08-07 09:40:53+00:00 # LLMs From Scratch Reaches 100,000 GitHub Stars Just saw that the [LLMs-from-scratch repository](https://github.com/rasbt/LLMs-from-scratch) passed 100,000 stars on GitHub! This is super cool and motivating. I am really happy to see that this open-source repo has helped so many people. Thanks also to everyone who shared ideas and opened PRs with improvements! Of course, I plan to keep adding new material, including new attention variants and architectures (while bigger projects like RL and Reasoning From Scratch live in their separate repositories). I am also currently working on a larger applied custom “small” LLM project. It has been keeping me super busy this month, but I will share more on that soon in an upcoming Substack mega-article! It’s my longest one yet! If you are new to it, some of the highlights in the repo include - Of course, the complete code path from [tokenization](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch02/01_main-chapter-code/ch02.ipynb)and[attention](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch03/01_main-chapter-code/ch03.ipynb)to[pretraining](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch05/01_main-chapter-code/ch05.ipynb),[classification](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch06/01_main-chapter-code/ch06.ipynb), and[instruction fine-tuning](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch07/01_main-chapter-code/ch07.ipynb), etc. All of it FROM SCRATCH, of course! (RL lives in a companion repo.) - From-scratch implementations of [Llama](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch05/07_gpt_to_llama/standalone-llama32.ipynb),[Qwen](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch05/11_qwen3),[Gemma](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch05/12_gemma3), and[Olmo](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch05/13_olmo3)(smaller variants that run locally and can be plugged into the training scripts). - From-scratch implementations of attention alternatives and other architecture components, such as [GQA](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch04/04_gqa),[MLA](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch04/05_mla),[sliding-window attention](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch04/06_swa),[Gated DeltaNet](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch04/08_deltanet),[DeepSeek Sparse Attention](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch04/09_dsa),[cross-layer KV sharing](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch04/10_kv-sharing), and[mixture-of-experts](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch04/07_moe) - Materials on [KV caching](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch04/03_kv-cache),[training performance](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch05/10_llm-training-speed),[memory-efficient weight loading](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch05/08_memory_efficient_weight_loading/memory-efficient-state-dict.ipynb),[DPO](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch07/04_preference-tuning-with-dpo/dpo-from-scratch.ipynb),[evaluation](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch07/03_model-evaluation), and[LoRA](https://github.com/rasbt/LLMs-from-scratch/blob/main/appendix-E/01_main-chapter-code/appendix-E.ipynb) So, if you don’t have any weekend plans yet, happy tinkering! Source: website version of my [Substack note](https://substack.com/@rasbt/note/c-310007138). ## Read Next [Kimi K3 Architecture Notes Short architecture note on Kimi K3, including LatentMoE, Kimi Delta Attention, Attention Residuals, NoPE, multimodality, and inference-efficiency choices.](/blog/2026/kimi-k3-architecture-notes.html) [A Few Notable Open-Weight Models This Week Short note on the architectures of six new open-weight models, including Nanbeige 4.2, Laguna S 2.1, Motif-3-Beta, Solar Open 2, Antares 1B, and BTL-3.](/blog/2026/notable-open-weight-models-this-week.html) [Correction for Listing 6.5 in Build a Reasoning Model From Scratch Short correction note for the random seed in Listing 6.5 on page 198 of Build a Reasoning Model From Scratch.](/blog/2026/reasoning-model-listing-6-5-correction.html)