{"slug": "llms-from-scratch-reaches-100000-github-stars", "title": "LLMs From Scratch Reaches 100,000 GitHub Stars", "summary": "The LLMs-from-scratch repository by Sebastian Raschka surpassed 100,000 stars on GitHub, marking a milestone for the open-source project that provides from-scratch implementations of large language models. Raschka, the author, announced the achievement and outlined plans to add new attention variants and architectures, while also working on a larger applied custom 'small' LLM project to be detailed in an upcoming Substack article.", "body_md": "# LLMs From Scratch Reaches 100,000 GitHub Stars\n\nJust saw that the [LLMs-from-scratch repository](https://github.com/rasbt/LLMs-from-scratch) passed 100,000 stars on GitHub!\n\nThis is super cool and motivating. I am really happy to see that this open-source repo has helped so many people.\n\nThanks also to everyone who shared ideas and opened PRs with improvements!\n\nOf course, I plan to keep adding new material, including new attention variants and architectures (while bigger projects like RL and Reasoning From Scratch live in their separate repositories).\n\nI am also currently working on a larger applied custom “small” LLM project. It has been keeping me super busy this month, but I will share more on that soon in an upcoming Substack mega-article! It’s my longest one yet!\n\nIf you are new to it, some of the highlights in the repo include\n\n-\nOf course, the complete code path from\n\n[tokenization](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch02/01_main-chapter-code/ch02.ipynb)and[attention](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch03/01_main-chapter-code/ch03.ipynb)to[pretraining](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch05/01_main-chapter-code/ch05.ipynb),[classification](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch06/01_main-chapter-code/ch06.ipynb), and[instruction fine-tuning](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch07/01_main-chapter-code/ch07.ipynb), etc. All of it FROM SCRATCH, of course! (RL lives in a companion repo.) -\nFrom-scratch implementations of\n\n[Llama](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch05/07_gpt_to_llama/standalone-llama32.ipynb),[Qwen](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch05/11_qwen3),[Gemma](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch05/12_gemma3), and[Olmo](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch05/13_olmo3)(smaller variants that run locally and can be plugged into the training scripts). -\nFrom-scratch implementations of attention alternatives and other architecture components, such as\n\n[GQA](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch04/04_gqa),[MLA](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch04/05_mla),[sliding-window attention](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch04/06_swa),[Gated DeltaNet](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch04/08_deltanet),[DeepSeek Sparse Attention](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch04/09_dsa),[cross-layer KV sharing](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch04/10_kv-sharing), and[mixture-of-experts](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch04/07_moe) -\nMaterials on\n\n[KV caching](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch04/03_kv-cache),[training performance](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch05/10_llm-training-speed),[memory-efficient weight loading](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch05/08_memory_efficient_weight_loading/memory-efficient-state-dict.ipynb),[DPO](https://github.com/rasbt/LLMs-from-scratch/blob/main/ch07/04_preference-tuning-with-dpo/dpo-from-scratch.ipynb),[evaluation](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch07/03_model-evaluation), and[LoRA](https://github.com/rasbt/LLMs-from-scratch/blob/main/appendix-E/01_main-chapter-code/appendix-E.ipynb)\n\nSo, if you don’t have any weekend plans yet, happy tinkering!\n\nSource: website version of my [Substack note](https://substack.com/@rasbt/note/c-310007138).\n\n## Read Next\n\n[Kimi K3 Architecture Notes Short architecture note on Kimi K3, including LatentMoE, Kimi Delta Attention, Attention Residuals, NoPE, multimodality, and inference-efficiency choices.](/blog/2026/kimi-k3-architecture-notes.html)\n\n[A Few Notable Open-Weight Models This Week Short note on the architectures of six new open-weight models, including Nanbeige 4.2, Laguna S 2.1, Motif-3-Beta, Solar Open 2, Antares 1B, and BTL-3.](/blog/2026/notable-open-weight-models-this-week.html)\n\n[Correction for Listing 6.5 in Build a Reasoning Model From Scratch Short correction note for the random seed in Listing 6.5 on page 198 of Build a Reasoning Model From Scratch.](/blog/2026/reasoning-model-listing-6-5-correction.html)", "url": "https://wpnews.pro/news/llms-from-scratch-reaches-100000-github-stars", "canonical_source": "https://sebastianraschka.com/blog/2026/llms-from-scratch-reaches-100000-github-stars.html", "published_at": "2026-08-07 09:40:53+00:00", "updated_at": "2026-08-09 11:45:36.858935+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "developer-tools"], "entities": ["Sebastian Raschka", "GitHub", "LLMs-from-scratch", "Substack"], "alternates": {"html": "https://wpnews.pro/news/llms-from-scratch-reaches-100000-github-stars", "markdown": "https://wpnews.pro/news/llms-from-scratch-reaches-100000-github-stars.md", "text": "https://wpnews.pro/news/llms-from-scratch-reaches-100000-github-stars.txt", "jsonld": "https://wpnews.pro/news/llms-from-scratch-reaches-100000-github-stars.jsonld"}}