How to Run Nemotron 3.5 Super VL Locally: You Need 128GB, Not a Blackwell GPU

NVIDIA released Nemotron 3.5 Super VL, a 120B vision-language MoE with 12B active parameters, on October 15, with community GGUF, MLX, FP8 and EXL3 conversions appearing a day later. According to Unsloth's planning figures, llama.cpp should run the model with vision on a 128 GB Mac or a 96 GB workstation card, while the RTX 5090 is left out. The model card's own NVFP4 benchmarks show large gains over Nemotron 3 Super on reasoning and agentic tasks but an 11-point drop on WebVoyager versus BF16 that the card does not flag outside its table.

Can you run Nemotron 3.5 Super VL locally without a Blackwell GPU? Yes, if you have roughly 128 GB of memory. The GPU generation matters much less than that number. NVIDIA released the model on October 15: a 120B vision-language MoE with 12B active parameters. The headline checkpoint is NVFP4, a 4-bit format that only Blackwell executes natively, and the model card https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 lists four deployment targets: DGX Spark, one H200, two H100s, or one GB200. A day later there are GGUF, MLX, FP8 and EXL3 conversions on Hugging Face, and going by Unsloth's numbers, llama.cpp should run it with vision on a 128 GB Mac or a 96 GB workstation card. The RTX 5090 is the one popular card that gets left out. I haven't run it. Everything below comes from the model card, the conversion repos and the Unsloth docs, with links throughout. From the card: off , on medium and on , set through the chat template I went looking for a separate text-only "Nemotron 3.5 Super" and there isn't one. The VL model is the 3.5 Super, and the card's benchmark section calls it that. The backbone was "initialized from Nemotron 3 Super Base", then distilled for 1T tokens from Nemotron 3 Ultra Base, extended to 1M context, given 67B tokens of software-engineering mid-training, and only then trained on vision. So the real predecessor is Nemotron 3 Super https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 , the text-only model from March 11, 2026. Both cards include NVFP4 scores, so this compares NVFP4 to NVFP4. Nemotron 3 Super was scored with NeMo Evaluator/NeMo Skills and 3.5 with NeMo Gym, though, so I'd treat the deltas as directional. | Benchmark NVFP4 | Nemotron 3 Super | Nemotron 3.5 Super VL | |---|---|---| | GPQA Diamond / GPQA no tools | 79.42 | 89.90 | | HLE, text-only | 17.42 | 31.44 | | LiveCodeBench v6 | 78.44 | 87.41 | | SciCode subtask | 40.83 | 46.61 | | tau2 airline | 54.75 | 75.75 | | tau2 telecom | 63.27 | 73.19 | | AA-LCR | 58.06 | 67.69 | | IFBench | 73.30 | 71.50 | | MMLU-ProX | 79.37 | 73.76 | On reasoning and agentic tasks it's a big jump. Multilingual knowledge MMLU-ProX and instruction following went down, and NVIDIA doesn't comment on either. On the vision side NVFP4 : MathVision 77.79, CharXiv reasoning 79.88, V 85.95, OCRBench v2 EN 68.34 / ZH 59.03, MMLongBench-Doc 62.82, OSWorld-Verified 52.3, Video-MME v2 32.54. For coding it posts SWE-bench Verified 71.07, SWE-bench Pro 51.30 and Terminal-Bench 2.1 62.78. NVFP4 isn't 4 bits across the board. The vision encoder, connector, Mamba convolutions and embeddings stay in BF16, and the MoE router stays in FP32. The card says inference "uses W4A4 on Blackwell and W4A16 on non-Blackwell platforms". In other words, a Hopper card still loads the small checkpoint but runs its matmuls with 16-bit activations. You get the memory savings without the FP4 tensor-core speed. The repo is 75.1 GB across 88 safetensors shards. The Hugging Face sidebar says "66B params", which I read as the U8-packed 4-bit weights being counted per byte, not as a different model. NVIDIA ran BF16 and NVFP4 with identical harnesses, and most rows are within noise. A few are not: | Benchmark | BF16 | NVFP4 | |---|---|---| | WebVoyager offline | 77.72 | 66.31 | | DeepSWE offline | 24.65 | 18.2 | | MMLU-ProX | 76.91 | 73.76 | | GDP.pdf offline | 6.6 | 6.0 | If you're building a browser agent, an 11-point drop on WebVoyager is a real problem, and the card doesn't mention it anywhere outside the table. The checkpoint and quant sizes below come from the NVIDIA, ggml-org https://huggingface.co/ggml-org/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF , Unsloth https://unsloth.ai/docs/models/nemotron-3.5-super-vl and rapid-mlx https://huggingface.co/rapid-mlx/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-MLX-4bit repos. The memory estimates are each source's own planning figures; I haven't measured any of them. | Hardware | Memory | Realistic option | |---|---|---| | DGX Spark GB10 | 128 GB unified | Official NVFP4 + vLLM + LiLiCorr drafter, --max-num-seqs 1 | | RTX Pro 6000 https://www.nvidia.com/en-us/products/workstations/professional-desktop-gpus/rtx-pro-6000-family/ | 96 GB | NVFP4 GGUF 79 GB, Unsloth estimates 96 GB or UD-Q3 K XL 65.8 GB | | Mac with 128 GB+ | unified | MLX 4-bit 69.5 GB, text only or llama.cpp Metal with UD-Q4 K XL 86.9 GB | | RTX 5090 https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/ | 32 GB | Doesn't fit. Even UD-IQ1 M is 46.1 GB. Offload to system RAM, or use Bolt | | 1× H200 https://www.nvidia.com/en-us/data-center/h200/ / 2× H100 | 141 GB HBM3e / two cards | Official NVFP4 at W4A16, or RedHatAI FP8 https://huggingface.co/RedHatAI/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-FP8 128.8 GB | So a single consumer card can't hold the 120B at any bit width. NVIDIA's answer is Nemotron 3.5 Bolt 74B-A9B 24GB https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF , compressed from Super VL with native 2-bit quantization, aimed at llama.cpp on 32 GB+ discrete GPUs Ada Lovelace and Blackwell . If you own a 5090, I'd start there before trying CPU offload on the 120B. For Mac owners, the rapid-mlx card reports a measured peak of 72.7 GB at 1K context on an M3 Ultra. It also says 96 GB Macs "are not yet verified" and that image input and MTP are not enabled. If you want vision on a Mac today, use llama.cpp with the mmproj file. On DGX Spark, this is the card's own Quick Start, using vllm/vllm-openai:v0.32.0 or the pinned nightly : export MODEL=nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 export DRAFT MODEL=nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4-LiLiCorr vllm serve $MODEL \ --served-model-name nemotron-3.5-super-vl \ --max-num-seqs 1 \ --trust-remote-code \ --gpu-memory-utilization 0.75 \ --max-num-batched-tokens 8192 \ --speculative config.model $DRAFT MODEL\ --speculative config.num speculative tokens 7 \ --speculative config.method dflash \ --mamba-backend flashinfer \ --mamba-ssm-cache-dtype float16 \ --enable-mamba-cache-stochastic-rounding \ --mamba-cache-philox-rounds 5 \ --enable-mamba-shared-prefix-checkpoint \ --video-pruning-rate 0.5 \ --media-io-kwargs '{"video": {"fps": 2, "num frames": 256}}' \ --block-size 64 \ --prefix-match-unit 64 \ --enable-auto-tool-choice \ --tool-call-parser qwen3 coder \ --reasoning-parser nemotron v3 If you copy the command from the card instead of from here, check the num speculative tokens 7 \ line. On the card it has a tab after the backslash, so bash ends the command there and then tries to run --speculative config.method as its own command. One more inconsistency: the card's own deployment table says DGX Spark uses "MTP with 5 draft tokens", but the command above uses LiLiCorr with 7. The card recommends LiLiCorr on Spark "in all scenarios", so I'd go with the command. Everywhere else, use llama.cpp. Unsloth's recipe after building with -DGGML CUDA=ON or OFF on Mac, where Metal is the default : hf download unsloth/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF \ --local-dir models/nemotron-super-vl \ --include " UD-Q4 K XL .gguf" " mmproj .gguf" ./llama.cpp/llama-server --model "$MODEL FILE" --mmproj "$MMPROJ FILE" \ --alias nemotron-super-vl --ctx-size 16384 --parallel 1 \ --temp 1.0 --top-p 0.95 --top-k 0 --min-p 0.0 \ --media-path "$PWD/media" --host 127.0.0.1 --port 8001 Start at 16K context, not 1M. KV cache, video frames and the 1.73 GB BF16 mmproj all come out of the same memory budget. For video you need ffmpeg and ffprobe on your PATH. Nemotron 3 Super shipped under the NVIDIA Nemotron Open Model License. Nemotron 3.5 Super VL ships under OpenMDW 1.1 https://openmdw.ai/license/1-1/ , which grants rights "to deal in the Model Materials without restriction". There are no field-of-use clauses and no restrictions on outputs. You have to keep the notices when you redistribute, and your rights end if you sue claiming the model infringes a patent or copyright. That's closer to MIT than to most "open" model licenses, and it matters more to me than the FP4 kernels do. NVFP4 is a good format attached to a sales pitch. The quality loss is small on most rows and large on a couple of agentic ones, and the speed advantage only exists on Blackwell, while the community had working GGUF and MLX builds within a day. If you already own a DGX Spark, the official path is the most polished option. If you don't, spend your money on memory, not on FP4 tensor cores. So, would you buy Blackwell hardware just to run NVFP4 models natively, or is a 128 GB Mac plus a GGUF good enough for you? And if you've run the 120B with offload on a 5090, post your tokens per second. I'd like to see them.