# Sanity check on my math: DGX Spark vs 2x Radeon PRO R9700 vs Mac Studio (M3 Ultra) for multi-user office LLM serving, does reality match the numbers? > Source: > Published: 2026-09-01 18:42:48+00:00 **TL;DR**: Pricing out small on-prem LLM deployments for office clients (vLLM/continuous batching, ~20 tok/s/user target). My bandwidth-bound math says two of these options wildly outperform a DGX Spark for the same money, but I know the calc is idealized and real serving software doesn’t always hit it. Two things I specifically don’t trust: (1) dual-GPU PCIe scaling, (2) whether Mac Studio’s continuous-batching software is anywhere near mature enough to hit what the raw bandwidth numbers promise. Hoping someone’s actually run these and can tell me where I’m wrong. **What I’m trying to do** Sizing hardware for small-office multi-user assistants (10-30 concurrent people, everyday office work + some coding). Budget lands around $8-9k USD equivalent. Options on the table: - 1x NVIDIA DGX Spark (128GB unified memory, 273GB/s bandwidth) - 2x AMD Radeon PRO R9700 (32GB GDDR6 each, 640GB/s each, ~$1,300/card), via vLLM `--pipeline-parallel-size 2` (no NVLink between them) - 1x Mac Studio, M3 Ultra, 256GB unified memory, 819GB/s bandwidth (~$6k config) **The math I’m running** ``` aggregate_tok/s = bandwidth / (active_params_bytes * bytes_per_param) concurrency = aggregate_tok/s / target_tok/s_per_user ``` For a ~35B MoE model (3B active, ~20GB weights, 48 layers, 8 KV heads, 128 head dim): ``` Spark: 273e9 / (3e9*0.57) ≈ 159 tok/s → ~7 concurrent users Dual R9700*: 1280e9 / (3e9*0.57) ≈ 746 tok/s → ~37 concurrent users Mac Studio: 819e9 / (3e9*0.57) ≈ 478 tok/s → ~23 concurrent users ``` The dual R9700 number assumes a naive linear combination of the two cards’ bandwidth. I know pipeline parallelism doesn’t give clean linear scaling in practice. There’s supposed to be pipeline-bubble overhead and vLLM’s own docs recommend pipeline parallelism over tensor parallelism specifically when there’s no NVLink, for exactly this multi-user-throughput use case. But I can’t find a real benchmark for this pairing, especially on AMD/ROCm rather than NVIDIA. **The number that really doesn’t sit right with me is the Mac Studio one at bigger model sizes.** Running the same formula against a ~120B-class model (5.1B active params, ~65GB weights) on the Mac Studio’s 819GB/s and 256GB memory gives: `aggregate = 819e9 / (5.1e9*0.54) ≈ 296 tok/s → ~14 concurrent users` 14 concurrent users running a genuinely large (120B-class) model, on a single desktop box that isn’t even primarily marketed as an AI server? That’s the hardest number to believe out of this whole exercise. **Why I’m suspicious specifically of the Mac number** The formula assumes vLLM-grade continuous batching, i.e. that the serving software can actually keep the bandwidth saturated across many concurrent users. That’s true and battle-tested for vLLM on CUDA. On Apple Silicon I can’t find anything that assures clarity: the most advanced continuous-batching project I could find for MLX reports something like 4.3x aggregate throughput at 16 concurrent requests, nowhere near the ~16x linear scaling my formula assumes. If that gap is representative, the real Mac Studio number could be way below 14, not just “a bit lower.” **What I actually want to know** - Has anyone run vLLM (or llama.cpp) with `--pipeline-parallel-size 2` across two PCIe-only GPUs for concurrent multi-user serving (not single-stream)? What fraction of the naive linear-sum throughput did you actually get? - Specifically for AMD/ROCm multi-GPU, is the R9700 (very new card) actually stable for this yet, or is ROCm multi-GPU still rough around the edges compared to CUDA? - Has anyone run mlx-lm, llama.cpp (Metal, `--cont-batching` ), vLLM-metal, or vllm-mlx on an M3 Ultra Mac Studio under genuine multi-user concurrent load (not single-stream benchmarks)? What tok/s per user did you actually get at, say, 10+ simultaneous requests? - Does anyone have real numbers running a 100B+ class model specifically on Mac (not just “it loads and runs” single-stream demos, which I’ve seen plenty of)? - Is continuous batching on Mac (any backend) actually production-viable today for a multi-user office deployment, or is this still “works great for one person, falls over past a handful of concurrent users” territory? - Is my ~20 tok/s/user target reasonable for “feels responsive,” or am I over/under-shooting what real users actually need, in a way that would change any of this? Happy to share more of my numbers/assumptions if useful, trying to get this right before I put real money behind a recommendation to clients. Also very grateful that you took time out of your day to help me with my problem.