{"slug": "i-built-a-small-language-model-from-scratch-here-s-how-it-works", "title": "I built a small language model from scratch. Here's how it works.", "summary": "A developer has built Orin AI (AE-Assistant), a small decoder-only Transformer language model designed to run locally on ordinary laptop hardware without a dedicated GPU. The project incorporates custom BPE tokenization and experiments with recurrent-depth architectures and retrieval-based grounding, with code published on GitHub and a video walkthrough demonstrating the model running.", "body_md": "I've been building Orin AI (AE-Assistant), a small language-model project designed to run locally on ordinary laptop hardware without a dedicated GPU.\n\nThe project explores a decoder-only Transformer, custom BPE tokenization, and experiments with recurrent-depth architectures and retrieval-based grounding.\n\nI made a video walking through the project and showing the model running.\n\n**YouTube breakdown:**\n\n[https://www.youtube.com/watch?v=zraZKknc7LQ](https://www.youtube.com/watch?v=zraZKknc7LQ)\n\n**GitHub repository:**\n\n[https://github.com/Harper-16/Orin_AI](https://github.com/Harper-16/Orin_AI)\n\nI'm still developing it, and I'd appreciate constructive feedback from people interested in small language models and training models from scratch.\n\nWhat would you improve first?", "url": "https://wpnews.pro/news/i-built-a-small-language-model-from-scratch-here-s-how-it-works", "canonical_source": "https://dev.to/harper16_51/-i-built-a-small-language-model-from-scratch-heres-how-it-works-5c75", "published_at": "2026-10-17 07:53:09+00:00", "updated_at": "2026-10-17 08:01:52.261104+00:00", "lang": "en", "topics": ["large-language-models", "machine-learning", "ai-research", "natural-language-processing"], "entities": ["Orin AI", "AE-Assistant", "GitHub", "YouTube"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/i-built-a-small-language-model-from-scratch-here-s-how-it-works", "markdown": "https://wpnews.pro/news/i-built-a-small-language-model-from-scratch-here-s-how-it-works.md", "text": "https://wpnews.pro/news/i-built-a-small-language-model-from-scratch-here-s-how-it-works.txt", "jsonld": "https://wpnews.pro/news/i-built-a-small-language-model-from-scratch-here-s-how-it-works.jsonld"}}