{"slug": "sanity-check-on-my-math-dgx-spark-vs-2x-radeon-pro-r9700-vs-mac-studio-m3-ultra", "title": "Sanity check on my math: DGX Spark vs 2x Radeon PRO R9700 vs Mac Studio (M3 Ultra) for multi-user office LLM serving, does reality match the numbers?", "summary": "A hardware-sizing analysis by an unnamed engineer claims that two AMD Radeon PRO R9700 GPUs ($1,300 each) and a Mac Studio M3 Ultra (256GB, $6k) could outperform NVIDIA's DGX Spark (128GB, 273GB/s) for multi-user office LLM serving, with theoretical throughput of 746 tok/s (37 concurrent users) and 478 tok/s (23 users) respectively versus 159 tok/s (7 users) for the Spark, but the author flags uncertainty about real-world scaling and software maturity, especially for Apple Silicon continuous batching.", "body_md": "**TL;DR**: Pricing out small on-prem LLM deployments for office clients (vLLM/continuous batching, ~20 tok/s/user target). My bandwidth-bound math says two of these options wildly outperform a DGX Spark for the same money, but I know the calc is idealized and real serving software doesn’t always hit it. Two things I specifically don’t trust: (1) dual-GPU PCIe scaling, (2) whether Mac Studio’s continuous-batching software is anywhere near mature enough to hit what the raw bandwidth numbers promise. Hoping someone’s actually run these and can tell me where I’m wrong.\n\n**What I’m trying to do**\n\nSizing hardware for small-office multi-user assistants (10-30 concurrent people, everyday office work + some coding). Budget lands around $8-9k USD equivalent. Options on the table:\n\n- 1x NVIDIA DGX Spark (128GB unified memory, 273GB/s bandwidth)\n\n- 2x AMD Radeon PRO R9700 (32GB GDDR6 each, 640GB/s each, ~$1,300/card), via vLLM `--pipeline-parallel-size 2`\n\n(no NVLink between them)\n\n- 1x Mac Studio, M3 Ultra, 256GB unified memory, 819GB/s bandwidth (~$6k config)\n\n**The math I’m running**\n\n```\naggregate_tok/s = bandwidth / (active_params_bytes * bytes_per_param)\nconcurrency = aggregate_tok/s / target_tok/s_per_user\n```\n\nFor a ~35B MoE model (3B active, ~20GB weights, 48 layers, 8 KV heads, 128 head dim):\n\n```\nSpark:        273e9 / (3e9*0.57) ≈ 159 tok/s → ~7 concurrent users\nDual R9700*:  1280e9 / (3e9*0.57) ≈ 746 tok/s → ~37 concurrent users\nMac Studio:   819e9 / (3e9*0.57) ≈ 478 tok/s → ~23 concurrent users\n```\n\nThe dual R9700 number assumes a naive linear combination of the two cards’ bandwidth. I know pipeline parallelism doesn’t give clean linear scaling in practice. There’s supposed to be pipeline-bubble overhead and vLLM’s own docs recommend pipeline parallelism over tensor parallelism specifically when there’s no NVLink, for exactly this multi-user-throughput use case. But I can’t find a real benchmark for this pairing, especially on AMD/ROCm rather than NVIDIA.\n\n**The number that really doesn’t sit right with me is the Mac Studio one at bigger model sizes.** Running the same formula against a ~120B-class model (5.1B active params, ~65GB weights) on the Mac Studio’s 819GB/s and 256GB memory gives:\n\n`aggregate = 819e9 / (5.1e9*0.54) ≈ 296 tok/s → ~14 concurrent users`\n\n14 concurrent users running a genuinely large (120B-class) model, on a single desktop box that isn’t even primarily marketed as an AI server? That’s the hardest number to believe out of this whole exercise.\n\n**Why I’m suspicious specifically of the Mac number**\n\nThe formula assumes vLLM-grade continuous batching, i.e. that the serving software can actually keep the bandwidth saturated across many concurrent users. That’s true and battle-tested for vLLM on CUDA. On Apple Silicon I can’t find anything that assures clarity: the most advanced continuous-batching project I could find for MLX reports something like 4.3x aggregate throughput at 16 concurrent requests, nowhere near the ~16x linear scaling my formula assumes. If that gap is representative, the real Mac Studio number could be way below 14, not just “a bit lower.”\n\n**What I actually want to know**\n\n-\nHas anyone run vLLM (or llama.cpp) with `--pipeline-parallel-size 2`\n\nacross two PCIe-only GPUs for concurrent multi-user serving (not single-stream)? What fraction of the naive linear-sum throughput did you actually get?\n\n-\nSpecifically for AMD/ROCm multi-GPU, is the R9700 (very new card) actually stable for this yet, or is ROCm multi-GPU still rough around the edges compared to CUDA?\n\n-\nHas anyone run mlx-lm, llama.cpp (Metal, `--cont-batching`\n\n), vLLM-metal, or vllm-mlx on an M3 Ultra Mac Studio under genuine multi-user concurrent load (not single-stream benchmarks)? What tok/s per user did you actually get at, say, 10+ simultaneous requests?\n\n-\nDoes anyone have real numbers running a 100B+ class model specifically on Mac (not just “it loads and runs” single-stream demos, which I’ve seen plenty of)?\n\n-\nIs continuous batching on Mac (any backend) actually production-viable today for a multi-user office deployment, or is this still “works great for one person, falls over past a handful of concurrent users” territory?\n\n-\nIs my ~20 tok/s/user target reasonable for “feels responsive,” or am I over/under-shooting what real users actually need, in a way that would change any of this?\n\nHappy to share more of my numbers/assumptions if useful, trying to get this right before I put real money behind a recommendation to clients.\n\nAlso very grateful that you took time out of your day to help me with my problem.", "url": "https://wpnews.pro/news/sanity-check-on-my-math-dgx-spark-vs-2x-radeon-pro-r9700-vs-mac-studio-m3-ultra", "canonical_source": "https://forum.level1techs.com/t/sanity-check-on-my-math-dgx-spark-vs-2x-radeon-pro-r9700-vs-mac-studio-m3-ultra-for-multi-user-office-llm-serving-does-reality-match-the-numbers/253436#post_2", "published_at": "2026-09-01 18:42:48+00:00", "updated_at": "2026-09-01 18:54:00.404983+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-infrastructure", "ai-tools"], "entities": ["NVIDIA DGX Spark", "AMD Radeon PRO R9700", "Mac Studio M3 Ultra", "vLLM", "ROCm", "MLX"], "alternates": {"html": "https://wpnews.pro/news/sanity-check-on-my-math-dgx-spark-vs-2x-radeon-pro-r9700-vs-mac-studio-m3-ultra", "markdown": "https://wpnews.pro/news/sanity-check-on-my-math-dgx-spark-vs-2x-radeon-pro-r9700-vs-mac-studio-m3-ultra.md", "text": "https://wpnews.pro/news/sanity-check-on-my-math-dgx-spark-vs-2x-radeon-pro-r9700-vs-mac-studio-m3-ultra.txt", "jsonld": "https://wpnews.pro/news/sanity-check-on-my-math-dgx-spark-vs-2x-radeon-pro-r9700-vs-mac-studio-m3-ultra.jsonld"}}