I built a small language model from scratch. Here's how it works.

A developer has built Orin AI (AE-Assistant), a small decoder-only Transformer language model designed to run locally on ordinary laptop hardware without a dedicated GPU. The project incorporates custom BPE tokenization and experiments with recurrent-depth architectures and retrieval-based grounding, with code published on GitHub and a video walkthrough demonstrating the model running.

I've been building Orin AI AE-Assistant , a small language-model project designed to run locally on ordinary laptop hardware without a dedicated GPU. The project explores a decoder-only Transformer, custom BPE tokenization, and experiments with recurrent-depth architectures and retrieval-based grounding. I made a video walking through the project and showing the model running. YouTube breakdown: https://www.youtube.com/watch?v=zraZKknc7LQ https://www.youtube.com/watch?v=zraZKknc7LQ GitHub repository: https://github.com/Harper-16/Orin AI https://github.com/Harper-16/Orin AI I'm still developing it, and I'd appreciate constructive feedback from people interested in small language models and training models from scratch. What would you improve first?