# Mini-AGI actually trains on 8GB VRAM without exploding > Source: > Published: 2026-09-21 13:46:17+00:00 # Mini-AGI actually trains on 8GB VRAM without exploding Calling a project "Mini-AGI" is a bold move that usually invites a shower of rocks, but this one actually has some technical meat on its bones. The core claim here is a dynamic continual learning model that manages to train on a measly 8GB of VRAM. For those of us tired of the "you need a H100 cluster to breathe" era of LLMs, this is a refreshing change of pace. The project is basically a rebellion against the fact that we can fine-tune 1B+ models on consumer gear, but we can't actually train them from scratch without selling a kidney. The goal was to get full control over the training data instead of just trusting whatever corporate slurry OpenAI or Google fed their models. ## How it handles the VRAM bottleneck The author managed to dodge the massive memory requirements using two specific architectural gambles: - **Dynamic MoE (Mixture of Experts):** Instead of a static wall of parameters, the model adds and prunes experts during training. Only a tiny subset of experts are active at any given moment. This effectively shifts the bottleneck from VRAM to disk space, as experts are loaded and unloaded on the fly. - **Batch 1 Training:** This is the real trick. By training on a single continuous stream of data, the model avoids the need to store massive randomized batches and their corresponding gradients, which is usually what kills your GPU memory. ## Current progress and training stats If you're expecting to download the weights today, stop. The model is currently chewing through a corpus of 7.8B characters. The training process involves reading interleaved passages of 32K characters each as a single, continuous stream. According to the project's current pace, the weights are still "cooking" and won't be ready for another couple of weeks. The author even shared a scaling law graph that they claim looks promising, though we'll see if that holds up once the weights actually hit the public. ## Getting it running The setup is straightforward if you want to watch the process yourself. Since it's designed for low-VRAM environments, you don't need a server farm to test the implementation. ``` git clone https://github.com/volotat/mini-AGI ``` The author admits to brainstorming the architecture with [Claude](https://promptcube3.com/en/tags/claude/), which proves that using AI to build AI is the only way we're actually getting anything done these days. It's a lean approach to training that prioritizes disk swapping and stream-processing over raw hardware brute force. Whether it actually achieves "AGI" (even a mini one) remains to be seen, but the 8GB VRAM footprint is a win for the home-lab crowd. [Next Opening an untrusted repository with an AI coding agent is riskier than just reading the files yourself →](https://promptcube3.com/en/threads/9521/)