For an 8-GPU TP8 setup, focus less on the CPU and more on the PCIe topology. If the GPUs can communicate directly through a Gen5 switch without routing P2P traffic through the host, that should matter much more for tensor-parallel inference than having an expensive CPU. The important thing is to benchmark the actual topology rather than assume x16 per GPU will automatically translate into better vLLM performance. Check GPU-to-GPU bandwidth/latency, P2P support, NUMA effects, and how the workload scales from 4 → 8 GPUs. Also make sure the motherboard and BIOS expose the required PCIe lanes and ACS/IOMMU settings correctly.
For ~200 GB models, memory capacity and inter-GPU communication can easily become the bottleneck, so I’d validate the switch configuration with NCCL P2P tests before spending more on the host. A cheap CPU is reasonable if it doesn’t become the bottleneck for tokenization, scheduling, or data movement.