cd /news/ai-agents/toaster-loop-a-self-improving-local-… · home › topics › ai-agents › article
[ARTICLE · art-173909] src=forum.level1techs.com ↗ pub= topic=ai-agents verified=true sentiment=↑ positive

Toaster Loop — a self-improving local AI agent for your home lab (+ Toaster Engine, the C++ MOE engine)

A developer published the first public post about Toaster Loop, a single Python tool that runs a full local self-improvement loop — task runs, trace buffer, curation, LoRA SFT/DPO/GRPO training, GGUF export, validation, and hot-swap serving — on one machine over loopback only, with no API keys or cloud calls. Toaster Loop binds to 127.0.0.1, requires token authentication on every API route with CSRF and DNS-rebinding defenses, and is Prometheus-scrapable; its "easy train" mode trains and hot-swaps a model after 50 good traces, and a failed validation leaves the previous model live. The same developer has begun coding Toaster Engine, a from-scratch C++ shared library behind a stable C ABI with a Vulkan-only backend intended to stream MOE experts from NVMe for models larger than VRAM plus RAM, upcycle dense models to MOE, and run on AMD, Intel and NVIDIA hardware; its design phase is complete across ten spec documents, with the first spike — measuring real per-expert sizes — planned for Devember.

key takeaway

Toaster Loop ties a downloaded GGUF and safetensor pair into one local loop that trains LoRA SFT/DPO/GRPO on curated traces, exports GGUF, validates, and hot-swaps only on a pass, so a failed run keeps the previous model live.

read3 min views1 publishedNov 21, 2026

AI Disclosure:

I love C++ and Python but AI helps me get more out and try things that I probably would not other wise do to time. local models were used to help make and test inside my program.

Toaster Loop — This is my first-ever public post about the project

This is the first time I’m showing this to anyone and is still a work in progress, so this first post is the “what/why/how”. I will try to keep this short as my readme has gotten to long and covers most of everything.

The problem I wanted gone

Every “local AI” stack I tried was a pile of half-connected parts: llama.cpp here, an agent framework there, fine-tuning scripts somewhere else entirely, and none of it remembered anything. Meanwhile the cloud stuff remembers everything but belongs to someone else and they are happy to sell it. Wendell hit a nerve: I wanted the curmudgeonly-home lab PC to run its own AI — one that gets better at my tasks over time without phoning home.

The goal

build Jarvis not Skynet, make it new user friendly and still have all the dials and knobs for the power users.

( here is a screenshot of me talking to an agent that made it’s own dataset about Toaster Loop and trained its self on it. then I asked it questions about the program with out letting it look at any files)

What Toaster Loop is

A single Python tool (toaster-loop) that runs the whole loop on one machine, over loopback only. Zero API keys, zero cloud calls. and sorry I know the frontend needs help

┌───────────────────────────── the loop ─────────────────────────────┐

run tasks ─▶ trace buffer ─▶ curate ─▶ train (LoRA SFT/DPO/GRPO) ─▶ export GGUF ─▶ validate

▲ │ pass

│ ▼

└───── serve --auto / hot-swap deploy ── active model ── promote ──────┘

(fail → previous model stays live; models rollback anytime) First time using

download a models gguf and safetensor version (these get tied together for running and training)

set up a serve instance with the model then setup an agent to tie to the instance, all settings are opt in but if you are new and don’t know all the options there is an enable all recommendations at the top of the options page to get a good base without setting everything,

there is more then one way to train the easiest is easy train, you can select an agent and it will set training base lines and when it has 50 good traces it will train and hot swap the model.

Home lab angle (localist) This is the “everyday useful thing” category in spirit: your box, your models, your data, your dashboards. Loopback-only by construction (bind address hardcoded to 127.0.0.1), token-auth on every API route with CSRF/DNS-rebinding defenses, and it’s Prometheus-scrapable so toaster-loop shows up next to Uptime Kuma and Home Assistant in your Grafana.

The side quest: Toaster Engine (C++, not public yet) llama.cpp is great, but I want things it fundamentally can’t do: run MOE models larger than VRAM+RAM by streaming experts at expert granularity from NVMe, locally upcycle dense models to MOE, and do it on AMD/Intel/NVIDIA with a Vulkan-only backend (no CUDA, no ROCm — cross-vendor or bust). So I’m writing the engine from scratch: one C++ shared library behind a stable C ABI, descriptor-driven model support (“support = data, not code”), CPU-tier computing cold experts concurrently with the GPU.

Status, honestly: design phase complete, and just started coding**.** Ten spec documents (reference-landscape autopsy of DeepSpeed/Muon/KTransformers/etc., Vulkan feasibility verdict, 22 MOE families matrix, memory-hierarchy math, upcycling staged plan, training stack, foundation/runtime/API architecture, roadmap with spikes S1–S8 and a risk register). Devember is where the first spike — measuring real per-expert sizes to settle the slot-sizing math — actually runs. I’ll post the specs’ honest performance projections (including the deliberately negative rows, like “Kimi-K3 on a 4070: correctness gate only, never a throughput gate”) as the build log starts.

Links

Feedback wanted any and all, especially: What home lab integration would make this actually earn its electricity in your lab?

── more in #ai-agents 4 stories · sorted by recency
── more on @toaster loop 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/toaster-loop-a-self-…] indexed:0 read:3min 2026-11-21 · —