As of today, after watching prices go up and up over the last year of everything, heeding GN’s warnings last year that GPUs are becoming Un-obtainium, combined with the significant insanity in performance we are seeing with the R9700 under Radiance (see below)…
If I had 5k for a new build, I’d redo exactly what I have:
- 2x R9700
- AM5 Mobo w/true dual PCIe slots to the CPU ($330), that would be x8-x8 Gen5
- Any AM5 CPU found on a discount. $200-$300
- 64GB DDR5 (left kidney + $1000 ebay). This is needed to offload QFN to DRAM with 60 GB/s.
- 1TB M.2 Gen4
That’s around $5k today. Again, that’s if you have 5k to spend.
If you still have more monies to set on fire, the next level up would be to focus on increasing DRAM speeds to get faster MoE performance of QNF or even reach DeepSeek Flash level of hosting. I’ve clocked more than double AM5 DDR5 6000 memory bandwidth on my WRX80 ThreadRipper w/8-channels of DDR4 3200 ECC ram. Around 150 GB/s @ 1k random I/O (like LLMs would do). That’s about a 2.5x increase of DRAM over the AM5 DDR5.
Not to mention, the ability to run very high quality Q8 quants and much larger context windows, along with concurrency. Win-win-win all around, for more cost…
- Downgrade to Gen4 by going with an AMD ThreadRipper WRX80 build (stick with me). Motherboards are around $1k, CPU about $1.5k
- Same dual R9700s as before, around $4k if urgent/new, or $3.5k if patient/used (ebay)
- 256 GB of DDR4 - any speed, any type (RDIMM, UDIMM, LPRDIMM, ECC/non-ECC - the WRX80 chipset doesn’t care and takes all!). That will run $1.2 to $1.5k on STH forums.
That’s around $7-$8k. if you can stretch another $2k, I’d go with 8x 64GB DDR4 sticks for 512 GB. DeepSeek Flash @ Q4 fits nicely with a 1 million context on dual R9700s with that setup. The downside to an WRX80 platform is Gen4, which will hurt your PP a bit in exchange for running larger context windows and higher quality of Quants.
Here’s the info on Radiance, a custom LLM engine built specifically for the R9700:
If you are looking to stick to a cheaper budget build, and can afford $2k, then a single R9700 by itself (Radiance only supports AMD). Sell the 5070 to recoup costs and invest into 64GB of DDR5 (ebay $1000). That should yield a decent model to get real multi-agent performance, at the sacrifice of lots of churning and time (redos until all DoDs are met and satisfied).