With the hardware craze going on the past few years I’m forced to add extra compute to my current stack (after delaying it for more than a year). I’m torn between 2 builds, mainly for inference (think qwen 3.8 27b and deepseek v4 flash currently).
I also have a dgx spark which is currently struggling to keep up with the rising demands of my systems and orchestration layers so i would be thinking about actually selling it and help reducing the extra investment i’ll need to make.
The idea:
I have several systems with amd 7443 (24c/48t) and 128gb ddr4 ram. They’re bolted to an asrock rome motherboard with 7 pcie lanes. I could go the following setup:
but i can’t seem to see past the good value the amd gpus are bringing to the table and pick the nvidia ones over them. The nvidia ones allow me to have a denser setup in the case, they consume less (140watt vs 300watt per gpu).
I don’t mind having to fiddle with configurations for the initial setup. The nvidia dgx spark gave me weeks of issues as well so it’s not as “plug and play” as many on the interwebz are pretending it to be.
Any advice, gotchas, duh’s, roasts, ideas, brainfarts?