Building Local: My 2026 Headless AI Server Journey

A developer reports that running Qwen 3.8 27B at Q5_K_M quantization on a dual AMD Radeon RX 7900 XT and 7800 XT setup achieves 20 tokens per second with a 256k context window, enabling autonomous multi-hour task execution. The 36GB VRAM pool and custom context manager prevent history bloat, and the developer is stress-testing the model to build a fully functional 3D CAD software by the end of the week.

It’s been a while since I last posted my hardware configuration, but local open-weight serving has come a long way. I wanted to share a quick update on how things are running with the Qwen 3.8 27B weights at Q5 K M quantization —it’s getting remarkably close to perfect for autonomous, long-horizon work. My original single-GPU setup had limitations when pushing heavy context lengths, so I recently added a second AMD Radeon RX 7900 XT to pair with my existing 7800 XT . This bumps the total VRAM pool up to a comfortable 36GB . That extra breathing room is an absolute game-changer, allowing me to comfortably host Qwen 3.8 27B at Q5 while opening up a 256k context window without running into memory walls. Inference Speed: Sitting right around 20 tokens per second . While it’s not blindingly fast compared to heavily speculative setups, it is entirely steady and reliable for automated execution. Thermals: Keeping dual AMD cards happy under continuous load takes some tuning. With the right fan profiles and power limits managed via rocm-smi , the 7900 XT memory temperatures stay stably locked between 80°C and 90°C even during extended loops. Running long instructions over massive context windows natively will eventually cause drift or degradation if left unmanaged. I spent a couple of days engineering a custom context manager that sits between the agent and the backend. It keeps the state aggressively clean, preventing the history bloat that usually kills multi-hour sessions. With this setup dialed in, the workflow has fundamentally shifted. Instead of interactive back-and-forth chat, I can feed the model a complex, multi-step list of instructions, walk away for a few hours, and come back to a completed task. Right now, I’m stress-testing the absolute limits of how far this model can go with agentic coding. The current target? Hoping to have a fully functional 3D CAD software completely built out through automated instruction loops by the end of the week. Curious to hear what kind of multi-GPU layer-splitting or context management strategies others in the community are using for the 27B class models on ROCm/llama.cpp right now