{"slug": "advice-on-a-local-ai-server-before-buying", "title": "Advice on a local AI server before buying", "summary": "A forum commenter running Qwen 3.8 Flash Next on dual AMD R9700 GPUs reported roughly 200 tokens per second decode and 6200 tokens per second prefill at 4-bit precision, and advised buyers to use an x8/x8 motherboard configuration because placing an R9700 in a chipset slot cost that card about 35-40% performance from added latency. The commenter also said llama.cpp is currently one of the slowest ways to run models given its problems with speculative decoding and concurrency.", "body_md": "Can you stretch the budget to go for the R9700 instead, if you drop to a used Ryzen 5x00 and an X570 board that’ll do x8/x8?\n\nFor what it’s worth, the radiance engine is currently running Qwen 3.8 Flash Next on my dual R9700s at ~200t/s decode, 6200t/s prefill (4-bit, with FP8 ngram table and activations, I think). I don’t know how it’d run with just one (I’m using mine right now, so I can’t check), but I’d hazard a guess that it’s significantly faster than a 3090.\n\nBTW, yes…if you want to go for a second card and TP, you’re going to want x8/x8. Layer split doesn’t need that, but at the very least you’re leaving a lot of performance on the table. With R9700s and one in a chipset slot, that card lost ~35-40% performance from the extra hop of latency.\n\nAlso…with the advent of all these architecture-specific inference engines, llama.cpp is currently one of the slowest ways to run models (also incredibly limited, given its problems with spec decoding and concurrency).", "url": "https://wpnews.pro/news/advice-on-a-local-ai-server-before-buying", "canonical_source": "https://forum.level1techs.com/t/advice-on-a-local-ai-server-before-buying/258001#post_2", "published_at": "2026-10-08 10:49:05+00:00", "updated_at": "2026-10-08 11:18:31.846319+00:00", "lang": "en", "topics": ["ai-infrastructure", "large-language-models", "ai-chips"], "entities": ["AMD R9700", "Qwen 3.8 Flash Next", "llama.cpp", "Ryzen 5x00", "X570"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/advice-on-a-local-ai-server-before-buying", "markdown": "https://wpnews.pro/news/advice-on-a-local-ai-server-before-buying.md", "text": "https://wpnews.pro/news/advice-on-a-local-ai-server-before-buying.txt", "jsonld": "https://wpnews.pro/news/advice-on-a-local-ai-server-before-buying.jsonld"}}