LLMs Locally on Consumer Hardware — Part 10: How the Setup Evolved — A Timeline of What Changed and Why

A developer running a local multi-model LLM agent stack documented how the setup evolved from a single machine to three machines plus a cloud strategist, driven by concrete failures at each stage. The 120B coordinator model (~77 GB) and gemma workers (~20 GB) could not coexist in 80 GB of RAM, causing roughly 100 GB of swapping and 291 seconds just to say hello, so workers moved to a dedicated machine and the coordinator was shrunk to gpt-oss:20b (~14 GB). The final architecture uses a deterministic script to own the delegation loop, with the model only planning and evaluating, and repurposes a laptop without a discrete GPU as an SD 1.5 image node after it proved unusable as an LLM worker.

Part 10 of a series — a retrospective appendix. The narrative ran across Parts 1–9; this one lays the evolution out stage by stage. No part of the setup was chosen for its own sake. Every change — a new machine, a different coordinator, a smaller model — was forced by a specific failure in the stage before it. Below is each stage as a snapshot, with the arrow between snapshots showing what broke and what was added or removed to fix it. | Machine | What it runs | |---|---| | Main — 5950X · ~80 GB · RX 6900XT | Coordinator and all workers — everything on gemma4:26b , local | ⬇ Broke: gemma4:26b can't hold a coordination loop beyond ~1–2 delegations. Added: a search for a dedicated coordinator model workers stay on gemma . | Machine | What it runs | |---|---| | Main — ~80 GB · RX 6900XT | Coordinator: trialing gpt-oss:120b and groq-tool-use:8b , which fabricated sessions · Workers: gemma4:26b | ⬇ Broke: the 120B ~77 GB and the gemma workers ~20 GB can't both be resident in 80 GB RAM → ~100 GB swapped per hand-off; 291 s to say hello. Removed: the workers from the main machine · Added: a second machine 5600G + GTX 1060 . | Machine | What it runs | |---|---| | Main — ~80 GB · RX 6900XT | Coordinator: gpt-oss:120b resident; runs on CPU + RAM | | Worker .114 — 32 GB · GTX 1060 | Workers: gemma4:26b | ⬇ Noticed: with the 120B running on CPU and RAM, the main machine's GPU sits idle. Added: image generation ComfyUI + SDXL on the main 6900XT. | Machine | What it runs | |---|---| | Main — ~80 GB · RX 6900XT | Coordinator gpt-oss:120b ~77 GB RAM + SDXL image gen ~15 GB VRAM | | Worker .114 | gemma4:26b workers | ⬇ Broke: the 120B leaves only ~3 GB RAM free → SDXL generations time out. Removed: gpt-oss:120b · Added: gpt-oss:20b ~14 GB in its place. | Machine | What it runs | |---|---| | Main — ~80 GB · RX 6900XT | Coordinator gpt-oss:20b ~14 GB RAM + SDXL ~15 GB VRAM — ~60 GB RAM free | | Worker .114 | gemma4:26b workers | ⬇ Added over the same period: a cloud strategist Sage, on Gemini 3 Flash for the one role needing deep reasoning; the laptop as an SD 1.5 image node; and deterministic orchestration a script owns the delegation loop, the model only plans and evaluates . | Where | What it runs | |---|---| | Main — ~80 GB · RX 6900XT | Coordinator gpt-oss:20b + SDXL image generation | | Worker .114 — 32 GB · GTX 1060 | gemma4:26b specialist workers | | Laptop .247 — 32 GB DDR5 · 780M | SD 1.5 image node 512 px | | Cloud | Sage strategist — Gemini 3 Flash free tier , consulted when blocked | The 8840U laptop ran its own small arc inside the larger one. | Attempt | Result | |---|---| | Laptop 8840U / 780M as an LLM worker | CPU prefill ~25 tok/s → 13-minute stalls | ⬇ Why: no discrete GPU, so prefill is hopeless for large agent prompts Part 2 . Repurposed for the one job the hardware could do: | Repurposed | Result | |---|---| | Laptop as an image node | SDXL hangs on the 780M; SD 1.5 @ 512 px works ~37–42 s | | | Initial setup | Final setup | |---|---|---| | Machines | 1 main | 3 main, worker, laptop + 1 cloud model | | Coordinator | gemma4:26b , local | gpt-oss:20b , local | | Workers | gemma4:26b , same machine as the coordinator | gemma4:26b , dedicated worker machine .114 | | Strategist | — | Gemini 3 Flash, cloud consulted only when blocked | | Image generation | — | SDXL on the main 6900XT; SD 1.5 on the laptop node .247 | | Orchestration | model-driven loop | deterministic script owns the loop; model only plans and evaluates | Read top to bottom, the chain has a shape. Nearly every change was one of two moves: placement — which machine a thing runs on — or right-sizing — how large each model needs to be for its job. The workers moved for placement; the coordinator shrank for right-sizing; the strategist went to the cloud because that was the right placement for the one role that needed deep reasoning. Not once did the answer turn out to be "buy a bigger model and hope." The capability was almost always already there in the hardware on hand; the work was in arranging it so each piece had room to do the thing it was good at.