{"slug": "llms-locally-on-consumer-hardware-part-10-how-the-setup-evolved-a-timeline-of", "title": "LLMs Locally on Consumer Hardware — Part 10: How the Setup Evolved — A Timeline of What Changed and Why", "summary": "A developer running a local multi-model LLM agent stack documented how the setup evolved from a single machine to three machines plus a cloud strategist, driven by concrete failures at each stage. The 120B coordinator model (~77 GB) and gemma workers (~20 GB) could not coexist in 80 GB of RAM, causing roughly 100 GB of swapping and 291 seconds just to say hello, so workers moved to a dedicated machine and the coordinator was shrunk to gpt-oss:20b (~14 GB). The final architecture uses a deterministic script to own the delegation loop, with the model only planning and evaluating, and repurposes a laptop without a discrete GPU as an SD 1.5 image node after it proved unusable as an LLM worker.", "body_md": "*Part 10 of a series — a retrospective appendix. The narrative ran across Parts 1–9; this one lays the evolution out stage by stage.*\n\nNo part of the setup was chosen for its own sake. Every change — a new machine, a different coordinator, a smaller model — was forced by a specific failure in the stage before it. Below is each stage as a snapshot, with the arrow between snapshots showing what broke and what was added or removed to fix it.\n\n| Machine | What it runs | \n|---|---|\n| Main — 5950X · ~80 GB · RX 6900XT | Coordinator **and** all workers — everything on`gemma4:26b` , local | \n\n⬇ **Broke:** `gemma4:26b` can't hold a coordination loop beyond ~1–2 delegations.\n\n**Added:** a search for a dedicated coordinator model (workers stay on gemma).\n\n| Machine | What it runs | \n|---|---|\n| Main — ~80 GB · RX 6900XT | Coordinator: trialing `gpt-oss:120b` (and`groq-tool-use:8b` , which fabricated sessions) · Workers:`gemma4:26b` | \n\n⬇ **Broke:** the 120B (~77 GB) and the gemma workers (~20 GB) can't both be resident in 80 GB RAM → ~100 GB swapped per hand-off; 291 s to say hello.\n\n**Removed:** the workers from the main machine · **Added:** a second machine (5600G + GTX 1060).\n\n| Machine | What it runs | \n|---|---|\n| Main — ~80 GB · RX 6900XT | Coordinator: `gpt-oss:120b` (resident; runs on CPU + RAM) | \n| Worker (.114) — 32 GB · GTX 1060 | Workers: `gemma4:26b` | \n\n⬇ **Noticed:** with the 120B running on CPU and RAM, the main machine's GPU sits idle.\n\n**Added:** image generation (ComfyUI + SDXL) on the main 6900XT.\n\n| Machine | What it runs | \n|---|---|\n| Main — ~80 GB · RX 6900XT | Coordinator `gpt-oss:120b` (~77 GB RAM)**+** SDXL image gen (~15 GB VRAM) | \n| Worker (.114) | `gemma4:26b` workers | \n\n⬇ **Broke:** the 120B leaves only ~3 GB RAM free → SDXL generations time out.\n\n**Removed:** `gpt-oss:120b` · **Added:** `gpt-oss:20b` (~14 GB) in its place.\n\n| Machine | What it runs | \n|---|---|\n| Main — ~80 GB · RX 6900XT | Coordinator `gpt-oss:20b` (~14 GB RAM)**+** SDXL (~15 GB VRAM) —**~60 GB RAM free** | \n| Worker (.114) | `gemma4:26b` workers | \n\n⬇ **Added over the same period:** a cloud strategist (Sage, on Gemini 3 Flash) for the one role needing deep reasoning; the laptop as an SD 1.5 image node; and deterministic orchestration (a script owns the delegation loop, the model only plans and evaluates).\n\n| Where | What it runs | \n|---|---|\n| Main — ~80 GB · RX 6900XT | Coordinator `gpt-oss:20b` + SDXL image generation | \n| Worker (.114) — 32 GB · GTX 1060 | `gemma4:26b` specialist workers | \n| Laptop (.247) — 32 GB DDR5 · 780M | SD 1.5 image node (512 px) | \n| Cloud | Sage strategist — Gemini 3 Flash (free tier), consulted when blocked | \n\nThe 8840U laptop ran its own small arc inside the larger one.\n\n| Attempt | Result | \n|---|---|\n| Laptop 8840U / 780M as an **LLM worker** | CPU prefill ~25 tok/s → 13-minute stalls | \n\n⬇ **Why:** no discrete GPU, so prefill is hopeless for large agent prompts (Part 2).\n\n**Repurposed** for the one job the hardware could do:\n\n| Repurposed | Result | \n|---|---|\n| Laptop as an **image node** | SDXL hangs on the 780M; SD 1.5 @ 512 px works (~37–42 s) | \n\n|  | Initial setup | Final setup | \n|---|---|---|\n| **Machines** | 1 (main) | 3 (main, worker, laptop) + 1 cloud model | \n| **Coordinator** | `gemma4:26b` , local | `gpt-oss:20b` , local | \n| **Workers** | `gemma4:26b` , same machine as the coordinator | `gemma4:26b` , dedicated worker machine (.114) | \n| **Strategist** | — | Gemini 3 Flash, cloud (consulted only when blocked) | \n| **Image generation** | — | SDXL on the main 6900XT; SD 1.5 on the laptop node (.247) | \n| **Orchestration** | model-driven loop | deterministic script owns the loop; model only plans and evaluates | \n\nRead top to bottom, the chain has a shape. Nearly every change was one of two moves: **placement** — which machine a thing runs on — or **right-sizing** — how large each model needs to be for its job. The workers moved for placement; the coordinator shrank for right-sizing; the strategist went to the cloud because that was the right placement for the one role that needed deep reasoning. Not once did the answer turn out to be \"buy a bigger model and hope.\" The capability was almost always already there in the hardware on hand; the work was in arranging it so each piece had room to do the thing it was good at.", "url": "https://wpnews.pro/news/llms-locally-on-consumer-hardware-part-10-how-the-setup-evolved-a-timeline-of", "canonical_source": "https://dev.to/sven_welack/llms-locally-on-consumer-hardware-part-10-how-the-setup-evolved-a-timeline-of-what-changed-and-24f7", "published_at": "2026-11-11 07:15:58+00:00", "updated_at": "2026-11-11 07:22:12.696480+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "ai-infrastructure", "ai-tools"], "entities": ["gpt-oss", "gemma", "Gemini 3 Flash", "ComfyUI", "SDXL", "AMD RX 6900XT", "NVIDIA GTX 1060"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/llms-locally-on-consumer-hardware-part-10-how-the-setup-evolved-a-timeline-of", "markdown": "https://wpnews.pro/news/llms-locally-on-consumer-hardware-part-10-how-the-setup-evolved-a-timeline-of.md", "text": "https://wpnews.pro/news/llms-locally-on-consumer-hardware-part-10-how-the-setup-evolved-a-timeline-of.txt", "jsonld": "https://wpnews.pro/news/llms-locally-on-consumer-hardware-part-10-how-the-setup-evolved-a-timeline-of.jsonld"}}