# I built a small language model from scratch. Here's how it works.

> Source: <https://dev.to/harper16_51/-i-built-a-small-language-model-from-scratch-heres-how-it-works-5c75>
> Published: 2026-10-17 07:53:09+00:00

I've been building Orin AI (AE-Assistant), a small language-model project designed to run locally on ordinary laptop hardware without a dedicated GPU.

The project explores a decoder-only Transformer, custom BPE tokenization, and experiments with recurrent-depth architectures and retrieval-based grounding.

I made a video walking through the project and showing the model running.

**YouTube breakdown:**

[https://www.youtube.com/watch?v=zraZKknc7LQ](https://www.youtube.com/watch?v=zraZKknc7LQ)

**GitHub repository:**

[https://github.com/Harper-16/Orin_AI](https://github.com/Harper-16/Orin_AI)

I'm still developing it, and I'd appreciate constructive feedback from people interested in small language models and training models from scratch.

What would you improve first?