# How I Built a Local AI Agent Stack for $0/month (Hermes + Windmill + NVIDIA + Groq) > Source: > Published: 2026-09-25 16:00:28+00:00 *Published: 2026-09-25 | ~2,200 words | 11 min read* I needed AI agents that could actually *do* things — run code, call APIs, browse the web, write files — not just chat. Cloud APIs got expensive fast, rate-limited unpredictably, and locked me into someone else's infrastructure. So I built a local stack on my Windows machine (i7-8700K, 32GB RAM, RTX 3080). Total monthly cost: **$0**. Everything runs locally or on free tiers. This article documents what actually works, what broke repeatedly, and the solutions that stuck. ``` ┌─────────────────────────────────────────────────────────────────┐ │ ORCHESTRATION │ │ Windmill (localhost:8000) — 3 containers │ │ - windmill-postgres, windmill-server, windmill-worker │ │ - Workspace: admins, NO_AUTH=true (dev only) │ │ - 13 scripts deployed: 7 pillars + 4 automations + 1 reviewer │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ MODEL LAYER │ │ Primary: nvidia/nemotron-3-super-120b-a12b │ │ Fallback 1: openai/gpt-oss-120b (Groq) │ │ Fallback 2: openai/gpt-oss-20b (Groq) │ │ Reviewer: nvidia/nemotron-3-ultra-550b-a55b (adversarial) │ │ │ │ Local Proxy: 127.0.0.1:8001 (OpenAI-compatible) │ │ - 10 NVIDIA API keys in rotation │ │ - 6-model cascade on failure │ │ - Tool calling + SSE streaming support │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ AGENT INTERFACE │ │ Hermes Agent (Nous Research) — Telegram + local CLI │ │ - Configured with custom provider → local proxy │ │ - Fallback providers: Groq (gpt-oss-120b → gpt-oss-20b) │ │ - Auxiliary model DISABLED (caused rate loops) │ │ - Bot Mode: 3 bots configured on Ultra 550B (see below) │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ TOOLS & AUTOMATION │ │ Webwright (Playwright) — browser automation │ │ NVIDIA Reviewer — adversarial code review via Windmill │ │ Edge-TTS — local voice synthesis (Microsoft, free) │ │ FFmpeg — video assembly │ │ Retry watcher — cron every 2 min, Telegram alerts │ └─────────────────────────────────────────────────────────────────┘ ``` NVIDIA NIM free tier: 40 RPM. That's it. One agent loop can burn 10 requests in seconds. Groq free tier is generous but has different models. OpenRouter credits exhausted. `nvidia_key_rotation.py`) ``` python # Simplified logic class KeyRotator: def __init__(self, keys: list[str]): self.keys = keys self.state = {k: {"fails": 0, "cooldown_until": 0} for k in keys} self.current_idx = 0 def get_key(self) -> str: now = time.time() for _ in range(len(self.keys)): key = self.keys[self.current_idx] if self.state[key]["cooldown_until"] <= now: return key self.current_idx = (self.current_idx + 1) % len(self.keys) raise Exception("All keys in cooldown") def report_result(self, key: str, status: int): if status == 429: self.state[key]["cooldown_until"] = time.time() + 30 self.state[key]["fails"] += 1 elif status in (401, 403, 410): self.state[key]["fails"] = 999 # permanent skip else: self.state[key]["fails"] = max(0, self.state[key]["fails"] - 1) ``` **Result**: 429 errors become 30-second cooldowns instead of hard stops. Keys that return 401/403/410 (EOL models) are permanently skipped. `nvidia_cascade.py`) When a key works but the model fails (timeout, 500, bad output), cascade to the next model: ``` ACTIVE_MODELS = [ "nvidia/nemotron-3.5-lightning-30b-a3b", # fastest, good for simple "openai/gpt-oss-20b", # Groq fallback via proxy "nvidia/nemotron-3-super-120b-a12b", # primary workhorse "openai/gpt-oss-120b", # Groq fallback via proxy "glm-5.2", # backup "nvidia/nemotron-3-ultra-550b-a55b", # heavy reasoning, reviewer ] ``` Each model tried in order. First success wins. Logs show which model actually responded. `nvidia_proxy.py`) This was the hardest part. Hermes sends tool calls; the proxy must forward them to NVIDIA and return `tool_calls` in the response. Also must stream SSE chunks correctly. **Key fixes that took days:** `tools` parameter in request body to NVIDIA`tool_calls` present, return as-is (don't wrap in text)`data: {json}\n\n`, end with `data: [DONE]\n\n` `finish_reason: tool_calls` correctly Without these, Hermes either got empty streams or couldn't execute tools. ``` # Hermes config.yaml (relevant section) model: "nvidia/nemotron-3-super-120b-a12b" provider: "custom" custom: base_url: "http://127.0.0.1:8001/v1" fallback_providers: - name: "groq" models: - "openai/gpt-oss-120b" - "openai/gpt-oss-20b" ``` Behavior: if primary (proxy) fails, Hermes automatically retries on Groq. Next turn, it tries primary again. | Pillar | Purpose | Status | |---|---|---| | 1 | Approval Gate (human-in-the-loop) | ✅ Working, tested end-to-end | | 2 | Alternative Supervision | ❌ Blocked — needs API keys | | 3 | Self-Healing | 📦 Archived — no Docker socket in Windmill CE | | 4 | Resilience/Queue | ✅ Restored via API (job 01a0d246) | | 5 | Watchdog/Resume | 📦 Archived — same as Pillar 3 | | 6 | Cleanup/Mutex | ⚠️ Partial — cleanup paths wrong, mutex OK | | 7 | Sandbox | ⚠️ Partial — shell escape possible | | 8, 9, 10 | — | ❌ Never deployed | **Reality**: Only Pillars 1 and 4 are production-ready. The rest need Docker socket access (Windmill EE feature) or significant rework. `retry_watcher.py` — cron every 2 min, checks failed jobs, retries, notifies Telegram`nvidia_reviewer.py` — adversarial code review (called via webhook or manually)`comando-gabriele` — receives signed Telegram commands, executes terminal/file ops ``` # ~/.config/hermes/config.yaml (key sections) model: "nvidia/nemotron-3-super-120b-a12b" provider: "custom" custom: base_url: "http://127.0.0.1:8001/v1" api_key: "not-needed" # proxy handles auth fallback_providers: - name: "groq" models: - "openai/gpt-oss-120b" - "openai/gpt-oss-20b" auxiliary: title_generation: enabled: false # critical: was causing rate loops ``` 3 bots, all on `nvidia/nemotron-3-ultra-550b-a55b` (strongest model via proxy): `@direttore` — strategic coordinator, custom SOUL.md`@analista` — market/technical analysis, custom SOUL.md `@critico` — adversarial review, custom SOUL.md Group chat "Progetto ReportForge" completed 3 rounds, converged on positioning. Cron on @direttore runs every 24h. **Limitation**: Bots share the same proxy/key pool. Heavy concurrent use hits rate limits. Serial execution (max 3 rounds) mitigates this. **Before**: 429 errors killed agent loops. Manual key switching. **After**: Automatic. 10 keys, 30s cooldown on 429, cascade across 6 models, Groq fallback. Uptime >99% on free tiers. **Problem**: Nemotron-Super-120b hallucinates function signatures, file paths, config options when context grows. **Fix**: `/new` frequently`nvidia_reviewer` (Ultra 550B, temp 0.1) **Problem**: Proxy returned text instead of `tool_calls`. Hermes waited forever. **Fix**: Proxy now passes `tools` param to NVIDIA, returns raw `tool_calls` array. Verified with Hermes tool execution. **Problem**: "Empty stream" errors. NVIDIA sends SSE; proxy wasn't forwarding chunks correctly. **Fix**: Proper chunk parsing, `data:` prefix, `[DONE]` terminator. Hermes now receives stream correctly. **Burned**: `meta/llama-3.1-*`, `meta/llama-3.3-*` (Aug 26, 2026), `openai/gpt-oss-120b` on NVIDIA (Sep 3, 2026). **Process**: Before using any model, `curl` test. 410 = remove from cascade. Current active list maintained in `nvidia_cascade.py`. **Problem**: PowerShell `Out-File -Encoding UTF8` adds BOM (EF BB BF). Python `json.loads` fails silently. **Fix**: Always write with `[System.IO.File]::WriteAllText($path, $content, $utf8NoBom)` where `$utf8NoBom = New-Object System.Text.UTF8Encoding $false`. **Mistake**: Direct psql inserts. Windmill cache didn't refresh. Scripts invisible in UI. **Rule**: Only official Windmill API (CLI or HTTP). If API fails, stop and ask. | Metric | Value | Verified | |---|---|---| | NVIDIA API keys | 10 | `API_key.txt` | | Active models in cascade | 6 | `nvidia_cascade.py` | | Fallback models (Groq) | 2 | Hermes config | | Windmill containers | 3 healthy | `docker ps` | | Deployed scripts | 13 | Windmill UI | | Pillar 1 test | PASS | Job 01a0d0cf | | Pillar 4 restored | PASS | Job 01a0d246 | | NVIDIA Reviewer test | PASS | Found bug in `sum(a,b): return a-b` | | Webwright + Reviewer integration | PASS | 10 issues found in git-filter-branch script | | Monthly cost | $0 | No paid services used | | Disk C: free | 36% | `df -h` | | Disk D: free | 35% | `df -h` | | Disk E: free | 66% | `df -h` | | Decision | Would Change To | |---|---| | Started with 10-pillar design | Start with 2-3 pillars that solve immediate pain | | Built custom proxy from scratch | Use existing OpenAI-compatible proxy if one supported tool calling + SSE | | Put all models on same key pool | Separate key pools per model tier | | Used Windmill CE for self-healing | Accept CE limits; build external watchers instead | | Tried to automate everything | Keep human-in-the-loop (Pillar 1) for critical gates | `docker-compose up` for Windmill`build.nvidia.com` → `API_key.txt` `pip install webwright` → skill for Hermes`pip install edge-tts` (Microsoft, free) This stack exists because I refused to pay $500+/month for agent infrastructure that randomly rate-limits or changes APIs. It's not magic — it's plumbing. Lots of plumbing. But it works, it's mine, and it costs nothing. If you build something similar: **test every model before trusting it**, **log every fallback**, and **keep sessions short**. The free tiers are generous but unforgiving. *Built on Windows 11, Docker Desktop, WSL2. Zero cloud dependencies. Zero subscriptions.*